QLoRA

QLoRA is LoRA plus a quantized frozen base. The 7B or 70B weights sit in 4-bit NormalFloat (NF4). The adapters train in bf16 or fp16. Optimizer state covers only the adapters, not the full model. That split is why a 70B model can be fine-tuned on a single 48 GB GPU for many workloads.

The big idea

The base stays frozen in 4-bit, optionally with double quantization of constants. The optimizer sees adapter params (plus small fp32 compute paths as implemented). Throughput is lower than full-precision multi-GPU fine-tuning. The accessibility jump is the point: most open-source fine-tuning became possible on one box.

Most companies never full-fine-tune a 70B model. They QLoRA on proprietary tickets, SOPs and tone guides, evaluate on a fixed rubric, and export adapters.

Walking through the tooling

You rarely write a training loop from scratch:

• Hugging Face TRL — SFTTrainer, tight PEFT integration, same family as DPO/GRPO trainers.
• Axolotl — YAML-driven configs for many model families and packing tricks.
• Unsloth — optimized LoRA/QLoRA kernels for faster single-GPU and multi-GPU fine-tunes.

All of them sit on transformers + peft + bitsandbytes (for 4-bit). Read the model card's chat template and license before you ship.

from peft import LoraConfig
from trl import SFTTrainer

lora = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    bias="none",
    task_type="CAUSAL_LM",
)
print(lora.r)  # pass this config to SFTTrainer; mask prompt tokens there

Output:

16

At inference you either merge W' = W + scaled BA into a single weight file (simplest latency) or keep base plus adapter separate for hot-swap. Merging wins when one behavior ships to everyone. Separate adapters win when each tenant or workflow needs its own tune. Quantized bases plus LoRA need careful merge and quant toolchains. Test a few prompts after every export path.

Hyperparameters that matter:

Overfitting signs: training loss plunges while held-out loss rises; the model regurgitates demo phrasings; eval rubrics on novel prompts degrade. Fix with cleaner or more diverse data, fewer epochs, lower LR, dropout or smaller rank — not by blindly scaling steps.

Eval during SFT should include instruction-following suites, your golden product prompts, and a check that general capability did not collapse.

Common mistakes

• Assuming 4-bit is free. You pay some quality versus bf16. Measure the task, not only VRAM.
• Exporting a merged adapter without probing prompts. Quant toolchains silently break generation.
• Three epochs on 400 identical rows. Rewrite the intents instead.

Check yourself

1. What sits in 4-bit NF4 in QLoRA, and what trains in bf16/fp16? Answer: The frozen base is 4-bit. The LoRA adapters train in higher precision.
2. Why can a 70B QLoRA fit on one 48 GB GPU when Adam on 70B weights would not? Answer: Optimizer state covers adapters only. The 70B weights are 4-bit and frozen.

Key takeaways

• QLoRA keeps the base in 4-bit NF4 and trains LoRA adapters in higher precision.
• That is why large models fine-tune on one GPU.
• Use TRL, Axolotl or Unsloth rather than hand-rolling unless you are inventing methods.
• Watch LR, epochs and rank. Overfitting shows up as memorized demos and falling eval rubrics.

Further learning

Shaw Talebi's QLoRA talk builds the memory picture and walks Python for single-GPU fine-tuning:

QLoRA: Efficient Finetuning of Quantized LLMsWe present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. QLoRA backpropagates…arXiv.org

Hugging Face TRL documentationWe’re on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co