The decision that saves you months
The most important fine-tuning skill is knowing when not to. The progression: prompt engineering → few-shot → RAG → fine-tuning as a last resort. Fine-tuning is expensive (compute, data, expertise), locks you to a model, and must be redone as models improve — many teams never fine-tune.
The mental model
- RAG is for facts; fine-tuning is for form. If the failure is missing knowledge, use RAG. If the failure is behavior — wrong format, tone, style, or a specialized skill the model can't do even with the right context — fine-tuning may help.
- Fine-tuning does not reliably teach new facts (it can even increase hallucination if you try). It teaches patterns and behavior.
When fine-tuning genuinely wins
- Consistent format/style you can't get reliably from prompting (a specific output structure, a brand voice, a domain's conventions).
- A narrow, well-defined task at high volume, where a small fine-tuned model beats a big general model on cost/latency while matching quality (classification, extraction, routing).
- A specialized skill or domain (legal drafting, a specific code style) where you have enough high-quality labeled examples.
- Distillation — fine-tune a small model on a large model's outputs to get most of the quality far cheaper.
When to avoid it
- You need knowledge that changes → RAG.
- You have little labeled data → prompting/few-shot.
- The task is broad/open-ended → a general model.
- You'd have to re-tune constantly as models improve.
The honest cost
Fine-tuning needs a curated dataset (hundreds to thousands of high-quality examples — quality > quantity), a training run, evaluation, and ongoing maintenance. Budget for the data curation, which is the hard part, not just the training.