How Researchers Define and Test LLM Factuality

A language model can sound certain and still state something false. Measuring that problem is harder than comparing a multiple-choice answer with a key, especially when a model writes a long explanation. Yuxia Wang and colleagues survey research on factuality, including how studies define the problem, test outputs and try to reduce errors.

Factuality is not one metric

The survey distinguishes factuality from related concepts such as hallucination, relevance and trustworthiness. These terms often overlap in research, but ask different questions. A response can be relevant yet inaccurate; a statement can be supported by one source but not trustworthy in a broader sense. The paper organizes evaluation datasets by answer form: long open-ended text, yes/no answers, short answers or lists, and multiple-choice questions (Wang et al., 2024, pp. 19519–19522).

That structure explains why benchmarks vary. A fixed-choice test is comparatively easy to score automatically. A biography or open-ended answer may contain many separate claims, each needing evidence. FactScore-style methods break a response into atomic claims and check them against retrieved information. Human experts can help resolve ambiguous cases, though human ratings also depend on annotation rules (pp. 19521–19523).

The authors summarize error rates reported for several datasets. FactScore-Bio has a 42.6% human-annotated error rate, Factcheck-GPT 64.9%, and HaluEval 12.3% (Table 2, p. 19521). Those values describe different datasets, topics, sizes and evaluation methods; they are not a single factuality rate for current language models. The survey also cites a verifier using GPT-4 with Google Search that achieved an F1 score of 0.63 for identifying false claims against human labels (p. 19522). Even an automated fact-checker can make consequential errors.

Improving answers creates trade-offs

The review covers interventions at several stages: pretraining data curation, supervised fine-tuning, preference training, retrieval, prompting and post-generation checks. Retrieval can supply current or task-specific information, but it can also introduce inaccurate, biased or outdated documents. More elaborate verification can improve coverage while adding latency and sensitivity to prompts or model updates (pp. 19522–19527).

The paper is a literature survey, not a head-to-head test of all mitigation methods. The field changes quickly, so some studies may be missing. Its treatment of vision-language factuality is brief, and video and speech receive less attention (p. 19529). Its comparisons are best read as a map of methods and obstacles rather than a ranking of solutions.

Possible implications for practice

For teams using generative systems to summarize policies, answer customer questions or draft research, “factuality” should be tied to a defined task. Is the concern whether a statement is true, whether it follows from supplied documents, whether the source is current, or whether the model signals uncertainty? Different checks address different failure modes.

The survey suggests automated checks may help triage claims, while human review remains relevant for difficult or high-impact cases. A model that cites sources may still misread them, and an evaluator can be wrong too. Comparing outputs with evidence and tracking errors over time may provide a more meaningful view than relying on fluency or one aggregate benchmark score.

That framing can also help teams avoid building a dashboard that collapses different questions into one number. A system might be accurate on stable knowledge yet miss newly changed facts, or it might retrieve a useful source but fail to preserve its qualifications. Evaluations can make these conditions visible by separating factual correctness, grounding, recency and uncertainty rather than assuming one benchmark captures them all.

Bibliography & sources

1. Yuxia Wang; Minghan Wang; Muhammad Arslan Manzoor; Fei Liu; Georgi Nenkov Georgiev; Rocktim Jyoti Das; Preslav Nakov. 2024. “Factuality of Large Language Models: A Survey.” In *Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, 19519–19529. Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.emnlp-main.1088.