PEFT vs full fine-tuning

Full fine-tuning updates every parameter. It maximizes capacity to change behavior and is still used for major post-training stages at labs. Parameter-efficient fine-tuning (PEFT) freezes the base weights and trains a small set of adapters. LoRA is the usual PEFT method.

The big idea

You can pack many task adapters on one base and swap them at inference. See multi-tenant serving.

Default to LoRA/QLoRA for product fine-tunes and experiments. Use full fine-tuning when you have cluster budget, need maximum shift from the base (domain language, new special tokens at scale), or you are running a lab-scale SFT stage before RLHF. If a LoRA rank-64 run plateaus below requirements, try higher rank or more modules before jumping to full fine-tuning.

Walking through forgetting

Full fine-tuning on a narrow legal corpus can erase coding ability. LoRA on the same data usually keeps more general skill because the frozen W still computes the old features. A high learning-rate LoRA for many epochs on one task can still bend behavior far enough that general evals dip. Measure a small general capability suite every run, not only your task metric.

When you need multiple behaviors (support tone, SQL generator, internal jargon), prefer multiple adapters over one mega-adapter that averages conflicting styles.

Ship checklist: identical train/serve template, held-out golden prompts, regression on a general smoke set, adapter version pinned next to the base revision.

Cosine or linear decay after a short warmup is a fine schedule. Finetuning a classifier head on a frozen backbone is the same idea at smaller scale: a higher LR on the head, a much lower LR on the trunk, or freeze the trunk for one epoch. One LR for everything either underfits the head or wrecks pretrained features.

Common mistakes

• Jumping to full fine-tuning because LoRA rank 8 plateaued. Raise rank and adapt more modules first.
• Judging a run only on the task metric. A general smoke set catches forgetting.
• One adapter for conflicting voices. Split adapters, or you average them into mush.

Check yourself

1. You can LoRA a 7B on a laptop but full fine-tuning OOMs. What memory is LoRA avoiding? Answer: Adam state on 7B parameters, which is two extra tensors per weight.
2. When is full fine-tuning the right call? Answer: When you need a large shift from the base, have cluster budget, or you are running a lab-scale SFT stage.

Key takeaways

• Full fine-tuning updates every weight. PEFT trains a small adapter beside a frozen trunk.
• Default to LoRA/QLoRA for product work. Raise rank before going full.
• Forgetting is milder with a frozen trunk, but you still need a general eval suite.
• Multiple adapters beat one adapter that tries to be every voice.

Further learning

Back to the map: supervised fine-tuning and PEFT.