Skip to content

Search across models, tools, services, companies, providers, APIs, plans, articles and news.

RAG & fine-tuningAdvanced 25 min

The complete LLM guide: RAG, fine-tuning and production evaluation

How to decide whether RAG is enough for your problem or fine-tuning is justified — and how to evaluate and operate an LLM system before and after launch.

The complete LLM guide: RAG, fine-tuning and production evaluation

Start by defining the problem precisely

Most LLM projects need neither a bigger model nor fine-tuning — they need a well-defined problem. Before picking a technique, answer three questions: what exactly should the output look like? What knowledge is required that a general model lacks? And what metric defines success? Without a success metric you can never tell whether RAG was enough.

When RAG is enough

When your problem is knowledge — the model must answer from your documents or up-to-date data — retrieval-augmented generation is the first and cheapest option. A simple embedding-based retriever, sensible chunking and a clean system prompt usually deliver 80% of the value. The key point: RAG quality depends far more on retrieval than on the model — if the right chunk never reaches the model, even the strongest model cannot answer correctly.

What fine-tuning actually solves

Fine-tuning changes behaviour, not knowledge. If the issue is response style, a strict output format, domain terminology or token cost, a lightweight adapter (LoRA) on an open model is usually enough. If the issue is "doesn't know", do not fine-tune — feed the knowledge with RAG. The common production recipe: RAG for knowledge plus light fine-tuning for behaviour.

Build evaluation before production

Without evaluation, every prompt or model change is a gamble. Build a small golden set (50–200 cases) with reference answers and run every version against it: accuracy, coverage, graceful refusal of unanswerable questions, and latency. Grow the set from real user failures — every new bug becomes a new golden case.

Operations: cost, latency and drift

Three things break in production: cost (control it with caching and smaller models for easy tasks), latency (stream responses and run retrieval in parallel) and silent quality drift (monitor, review response samples weekly, and be ready for provider behaviour changes). An LLM system is a living product, not a one-time delivery.