Fine-Tuning

Training an LLM from scratch is apocalyptic expensive. GPT-3 cost an estimated 4.6 million dollars to train. GPT-4 cost significantly more. The compute, the data curation, the infrastructure. Only organizations with massive budgets and GPU clusters can do it. So the industry settled on a different approach: start with a base model trained on general internet text, then fine-tune it on your specific data.

Fine-tuning is transfer learning applied to language models. A base model has already learned general language patterns: grammar, facts, reasoning. You take that model and train it further on your domain-specific dataset. Maybe you have 10,000 customer support conversations. You fine-tune the base model on those conversations, and now it talks like your support team. Maybe you have 50,000 legal documents. You fine-tune, and the model understands legal terminology and reasoning.

The compute cost plummets compared to pretraining. Fine-tuning a model on 10,000 examples might cost 00 to ,000 depending on the model size and your infrastructure. Pretraining costs millions. But here's what often surprises teams: fine-tuning isn't a magic fix for every problem. The quality depends heavily on data quality, data quantity, and how different your domain is from the base model's training distribution.

You need enough data. How much is "enough"? It depends. For some tasks and models, 100 high-quality examples help. For others, you need 10,000. Generally, more data is better. But beyond a certain point, returns diminish and compute costs accumulate. We've seen teams fine-tune 100k examples when 5k would have solved the problem, wasting money chasing marginal improvements.

The data also needs to be representative. If you're fine-tuning on customer support conversations but your test set includes edge cases you never trained on, the fine-tuned model will fail. Or worse, it will succeed on your training data and fail in production because the distribution shifted. This is why careful evaluation and a held-out test set matter.

Fine-tuning also bakes behaviors into the model permanently. If you fine-tune on data that's biased, the model becomes biased. If you fine-tune on impolite conversations, the model becomes less helpful. You can't easily "undo" fine-tuning. You can fine-tune again on better data, but that doesn't erase the first fine-tuning. You're layering adaptations.

Different fine-tuning approaches exist. Full fine-tuning updates every parameter. LoRA (Low-Rank Adaptation) uses adapter layers so you only train a small fraction of parameters, making it cheaper and faster. Prompt engineering and RAG often outperform fine-tuning for knowledge-intensive tasks. Few-shot prompting (showing examples in the prompt) gives you fine-tuning-like behavior without actually fine-tuning. Many teams try fine-tuning when RAG would be better, or vice versa.

Fine-tuning is less common now than it was three years ago because prompt engineering and RAG improved so dramatically. A well-structured RAG system often beats a fine-tuned model that hallucinates. Few-shot prompting with GPT-4 often beats fine-tuning smaller models. But fine-tuning remains valuable for style/tone adaptation, domain-specific reasoning tasks, and scenarios where you need consistent format or behavior.

The other decision is whether to fine-tune a base model or a models provider's specialized offering. OpenAI offers fine-tuning for GPT-3.5 and some other models. Anthropic has fine-tuning options for Claude. Local open-source models (Llama, Mistral) are easier to fine-tune because you control the infrastructure. Each path has different economics.