Full fine-tuning is expensive
Updating all of a large model's billions of weights needs huge GPU memory (you store the weights, gradients, and optimizer state) and produces a full-size copy per task. For most teams that's impractical — which is why parameter-efficient fine-tuning (PEFT) exists.
LoRA (Low-Rank Adaptation) — the workhorse
Instead of updating the full weight matrices, freeze the base model and train small low-rank adapter matrices injected into the layers. You update a tiny fraction (often <1%) of the parameters. Benefits:
- Far less memory and compute — fine-tune a large model on a single/few GPUs.
- Tiny artifacts — the adapter is megabytes, not gigabytes; you can keep many task-specific adapters and swap them on the same base model.
- Quality close to full fine-tuning for most tasks.
Key knob: the rank
r(adapter capacity) — higher = more capacity + more params.
QLoRA — LoRA on a quantized base
QLoRA quantizes the frozen base model to 4-bit and trains LoRA adapters on top. This slashes memory further — you can fine-tune very large models on a single consumer/pro GPU — with minimal quality loss. It's the standard way to fine-tune big open models on a budget.
Choosing
- Full fine-tuning — when you have the compute and need maximum adaptation (rare for most teams).
- LoRA — the default for efficient fine-tuning; great quality/cost balance, swappable adapters.
- QLoRA — when memory is the constraint (large model, limited GPU).
Instruction tuning & SFT
Most practical fine-tuning is supervised fine-tuning (SFT) on (prompt, ideal_response) pairs — teaching the model to follow your task/format. (RLHF/preference tuning aligns to human preferences and is mostly a model-provider concern.) For an app team, SFT + LoRA/QLoRA on a curated dataset is the common path.