Can attention maps reveal language-model hallucinations?

A language model can receive the right document and still produce an answer that the document does not support. Lookback Lens asks whether the model's attention to supplied context can help detect that kind of error.

Measuring whether a response looks back

The researchers define contextual hallucination as a response that departs from facts available in the input. Their Lookback Lens measures the ratio between attention weights assigned to the supplied context and weights assigned to newly generated text. Attention weights are part of a transformer model's internal calculation of which tokens influence the next token. The authors compute a ratio for each attention head and train a linear classifier to distinguish supported and hallucinated spans.

The evaluation covers document summarization and question-answering tasks. The paper reports that a detector using these ratios performs about as well as richer detectors based on all hidden states or text-based entailment models. It also transfers across tasks and models: a detector trained with a 7-billion-parameter model can be applied to a 13-billion-parameter model without retraining (abstract; cross-model results in Table 4, PDF p. 6). For mitigation, a classifier-guided decoding method selects among candidate text chunks. In the XSum summarization task, the authors report a 9.6% reduction in hallucination (abstract; Table 3, PDF p. 5).

A useful distinction is that attention ratios serve as detection signals, not as a direct proof that each statement is true. A low ratio indicates that a model is attending less to the source relative to its own recent output; it does not independently verify facts. The system's value lies in using that signal to identify or reduce likely context drift.

Where the evidence is strongest

The results cover specific benchmarks, models and decoding choices. Tables 2–5 (PDF pp. 4–6) compare detection and transfer performance; the mitigation experiment uses eight candidates per chunk (Table 3, PDF p. 5). These are controlled tests, rather than evidence from a deployed retrieval or document workflow. The authors' limitations discussion (PDF p. 9) notes constraints in the decoding setup and the costs of generating multiple candidates. Access to model attention also depends on the model interface, which can limit use with closed systems.

For teams that evaluate document assistants, the study offers an example of a model-internal signal tied to whether generated text draws on its source. That can complement outcome checks, especially when a system already exposes attention information. Whether it adds reliable value in a particular setting depends on the document types, model and definition of an unacceptable error.

Possible implications for practice

Lookback Lens suggests that the model's relationship to its evidence can be measured alongside the answer itself. This could support more targeted review of summaries and document answers, while human or independent factual checks remain important for consequential uses. The paper demonstrates a benchmark method, not a ready-made quality guarantee.

Bibliography & sources

1. Yung-Sung Chuang; Linlu Qiu; Cheng-Yu Hsieh; Ranjay Krishna; Yoon Kim; James Glass (2024). “Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps.” Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 1419–1436. DOI: 10.18653/v1/2024.emnlp-main.84. Source paper