Generative AI and knowledge work
Fine-tuning or retrieval for LLM knowledge?
A comparison of model fine-tuning and retrieval finds different strengths when language models need facts they already know or have never seen.
When an organization needs a large language model (LLM) to use specialized facts, should it retrain the model or connect it to an information source? This study compares both routes and finds retrieval more consistent across the tested knowledge tasks.
Two ways to add information
In Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs, Oded Ovadia, Meni Brief, Moshik Mishaeli and Oren Elisha examine unsupervised fine-tuning and retrieval-augmented generation (RAG). Fine-tuning continues training a model on a domain-specific text collection. RAG searches an external collection when a question arrives, then supplies relevant passages to the model as context. The distinction matters when facts change or the model must explain where an answer came from.
The authors evaluate these approaches on knowledge-intensive tasks spanning established knowledge and newly introduced facts. Their comparisons include MMLU subject questions and a current-events task. Table 1 (PDF p. 5) reports the MMLU results; Table 2 (PDF p. 5) reports current-events performance. Across the reported settings, unsupervised fine-tuning improves some results, while RAG more consistently outperforms the fine-tuned alternatives. The authors also find that a model struggles to learn new facts through unsupervised fine-tuning. Repeating a fact in varied forms during training can ease that difficulty, according to their experiments (conclusion, PDF p. 8).
This is a comparison of particular training and retrieval implementations, rather than a universal ranking of every way to customize a model. RAG supplies evidence at answer time; fine-tuning changes model parameters. Retrieval performance therefore depends on the quality of the underlying collection and search, while model training carries different preparation and update costs. These are distinct operational properties, not outcomes directly measured as organizational cost in the paper.
What the comparison says about factual updates
For practitioners, a useful signal is the difference between helping a model recall facts and giving it access to new facts. Existing knowledge encountered during training and entirely new information did not respond equally to fine-tuning. In this study, retrieval provided the steadier route across the evaluated knowledge tasks. The result gives teams a reason to distinguish stable language behavior from frequently refreshed factual material when they assess system designs.
The findings are bounded by the selected models, datasets, training recipes and retrieval setup. The paper evaluates benchmark performance, not deployment outcomes such as answer freshness in a live enterprise knowledge base, user trust, or maintenance effort. Its limitations discussion (PDF p. 8) also frames the results as evidence about the tested knowledge-injection settings.
Possible implications for practice
The comparison suggests that retrieval could fit information needs where facts are changing or need to remain tied to supplied documents. Fine-tuning may still help with other objectives, but this paper does not establish a single best architecture for every organization. Its benchmark results instead sharpen a practical design question: should a model learn a pattern, or should it consult an updated source when the answer is needed?
Bibliography & sources
- Oded Ovadia; Meni Brief; Moshik Mishaeli; Oren Elisha (2024). “Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs.” Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 237–250. DOI: 10.18653/v1/2024.emnlp-main.15. Source paper