- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes GRA, a white-box method that detects whether a given answer is a hallucination by checking whether the model relies more on the prompt or on the generated tokens. Specifically, each time the model generates a token, GRA measures how much of the model's internal update is similar to the prompt tokens rather than to the generated tokens, and calls this proportion the grounding ratio. It then summarizes how the grounding ratio changes over the answer into features for a simple probe to detect hallucinations.
The paper claims three main contributions.
- A detection framework that measures whether the model relies on the prompt or on the generated tokens during generation.
- It shows that the mean and trend of the grounding ratio over the answer are an effective representation for hallucination detection.
- An evaluation across five open-weight LLMs and five QA datasets.
For validation, GRA reports the highest average AUROC among the compared methods on all five LLMs.
Critical strengths
None.
Critical weaknesses
- The performance advantage of the proposed method may still come from indirectly using the length of the LLM's answer. This is a big concern, since prior work shows that a simple detector using the length of response as feature alone can match complex hallucination detectors [1]. Although the paper groups the answers by length to show that GRA does not rely on length, two observations suggest that GRA uses length:
- The grounding ratio tends to decrease as the answer gets longer, because Eq. 3 and 4 use sums instead of averages over the prompt tokens and over the generated tokens. Lookback Lens [2] shares a similar idea with the paper but is not cited. It computes a similar ratio from attention weights, but it averages the attention weights over the prompt tokens and over the generated tokens, and thus avoids this dependence on the answer length.
- Table 5 in the supplementary splits the test answers into three length groups (fewer than 6, 6 to 15, and more than 15 tokens) and reports the AUROC within each group. For 17 of the 25 model–dataset pairs, the AUROC on all answers in Table 1 is higher than the AUROC within any length group. For example, for Qwen3-14B on HaluEval, the AUROC is 94.01 on all answers, but only 71.17, 83.02, and 63.15 within the three groups. This suggests that GRA is much less accurate among answers of similar length.
- The results in Table 1 are not reliable for two reasons:
- The test set is not held-out. The paper describes the 20% test set as held-out, but the released code get_test4.py shows it was also used to select the checkpoint and to adjust the learning rate.
- Table 1 uses a single seed. The five-seed average of GRA in Table 2 of the supplementary differs from its result in Table 1, and in 10 of the 25 model–dataset pairs this difference is larger than the gain of GRA over ICR Probe. So it is unclear whether these gains are stable.
[1] Janiak, Denis, Jakub Binkowski, Albert Sawczyn, Bogdan Gabrys, Ravid Shwartz-Ziv, and Tomasz Jan Kajdanowicz. "The illusion of progress: Re-evaluating hallucination detection in llms." In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.
[2] Chuang, Yung-Sung, Linlu Qiu, Cheng-Yu Hsieh, Ranjay Krishna, Yoon Kim, and James Glass. "Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps." In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.
Questions for authors
- How well does a classifier detect hallucinations if it uses only the prompt and answer lengths as input? Please compare it with GRA on the same test set. This needs no new computation, since get_test.py already saves the number of tokens in each answer.
- What are the AUROCs of GRA and ICR Probe when the checkpoint is selected on a separate validation set? Please report the mean and standard deviation over the five seeds.