Why Base Models Aren't Always Enough
A general-purpose base model like Llama-3-70B is trained on a broad slice of the internet. It can write code, answer questions, and summarize documents. But it doesn't know your company's internal terminology, it doesn't output data in your specific JSON schema, and it doesn't adopt your brand's communication style.
There are two ways to address this: prompting and fine-tuning.
Prompting (including RAG) is always the first thing to try. You can inject instructions, examples, and context directly into the prompt. This requires no training, is instantly reversible, and works well for most use cases.
Full Fine-Tuning is the traditional approach when prompting isn't enough. You take the base model and continue training it on your custom dataset, updating all billions of parameters to incorporate the new knowledge or behavior.
The Problems with Full Fine-Tuning
-
Compute Cost: Training a 70B model requires a cluster of high-end GPUs running for days or weeks. A single fine-tuning run can cost tens of thousands of dollars in cloud GPU time.
-
Storage Explosion: Each fine-tuned model is a full copy of the base model. Fine-tune for five different tasks and you're storing five 140 GB model files — 700 GB total, plus the original.
-
Catastrophic Forgetting: Neural networks have a well-documented failure mode where learning new information overwrites previously learned knowledge. A model fine-tuned heavily on legal documents may become worse at writing code. The model "forgets" in proportion to how aggressively it's trained on the new data.
-
Deployment Complexity: Serving five different fine-tuned models means running five separate model servers, each consuming full GPU resources.