Fine-Tuning vs RAG: Which One Does Your Problem Actually Need?

"Should we fine-tune a model?" is one of the most common — and most misunderstood — questions I get. Fine-tuning and retrieval (RAG) are both ways to make a general model useful for your specific problem, but they solve different things. Choosing the wrong one is an expensive way to learn the difference.

RAG changes what the model knows

If your problem is knowledge — the model needs facts from your documents, your policies, your product data — you almost always want RAG. It's cheaper, it updates instantly when your documents change, and it can cite its sources. Most "we want an AI that knows our business" requests are really RAG requests.

Fine-tuning changes how the model behaves

If your problem is behaviour — you need a consistent tone, a strict output format, a specialised style, or a task the base model just isn't reliable at — that's where fine-tuning earns its keep. It bakes a pattern into the model itself.

RAG for what the model should know. Fine-tuning for how the model should act. Many real systems use both.

A quick decision guide

• Answers from your docs? RAG.
• Facts change often? RAG (fine-tuning goes stale).
• Need a specific format or voice every time? Fine-tuning.
• Base model unreliable at a narrow task? Fine-tuning, with a proper evaluation set.

Don't fine-tune without measuring

The trap with fine-tuning is shipping a model that feels better but isn't. Every fine-tune should be judged against a held-out evaluation set that proves it beats what you had — otherwise you're guessing. That discipline is the core of good MLOps.

Not sure which your problem needs? Describe it in a line or two to b.a@live.co.uk and I'll tell you straight — including when the answer is "neither, just a better prompt".