acceptodds
Withdrawn from ICLR 2027

Confident Hallucinations Live in the Blind Spot of Entropy

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Test-time methods that detect or reduce hallucinations through entropy cannot handle confident hallucinations. These wrong answers are as low-entropy as correct ones and persist under resampling. We study where their confidence comes from in context-grounded QA. Context ablation splits an answer's log-probability into a parametric prior and the evidence added by the context. Entropy bounds only their sum and is therefore blind to how the confidence is split. The split reveals two opposite sources of confident hallucination. Context-lure errors copy a wrong span from the context and have abnormally low parametric support. Prior-dominated errors ignore the context and have abnormally high parametric support. We trace both errors to retrieval that either copies a wrong span from the context or finds no matching span. Because benchmarks mix the two types, pooled grounding signals flip sign across datasets and cancel within them. To address this problem, we propose Grounding-Profile Contrastive Decoding (GPCD). GPCD is a label-free test-time method that corrects confident hallucinations without updating any weights. It detects answers that lean too far on one source by scoring them within context-support strata. It re-decodes each flagged answer with contrastive decoding in the direction opposite to its failure. It accepts a change only when resampled candidates agree on an answer that the context supports. On five context-grounded QA benchmarks, the stratified score detects confident hallucinations better than pooled signals on every dataset, and GPCD gives the best average accuracy while global contrastive decoding and self-consistency lose up to 7 EM points on average.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
hallucination mitigation, decoding, test-time learning, multi-hop question answering, LLM reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_VEp3OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies uncertainty quantification for RAG-style use cases, where LLM generations should be faithful to the provided context. They propose two sources of confidence hallucinations: context-lure and prior-dominated errors. They suggest a method called Grounding Profile Contrastive Decoding, which does not require labels or model fine-tuning.

Critical strengths

1) Important problem: The paper studies an important problem. The research question (when does entropy provide meaningful signal for RAG vs. when does it produce confident hallucinations) is meaningful and of high consequence.

2) High-quality figures: The graphics in Figures 1 and 2 are well-made.

Critical weaknesses

1) Quality and clarity of writing: I found this paper incredibly difficult to read. Human-readable research should avoid "mannered prose", including coined jargon that is not explained/defined, forward references to not-yet defined terms, and flashy/handwaivy statements that try to grab reader attention at the expense of clarity and technical precision. Unfortunately, this paper is full of such mannered prose, which may be easy for an LLM to read, but very difficult for a human to read. Just the introduction took me a very long time to read. The purpose of research is to communicate scientitic ideas and concepts to humans, and this paper falls far short of that requirement. A non-exhuastive list of examples:

  • "Most label-free remedies treat hallucination as uncertainty" -> imprecise, these papers do not claim this. They use uncertainty signals as a predictor of hallucinations, but do not treat them as the same.
  • "These methods fail by construction on confident hallucinations." "such methods cannot see confident hallucinations" -> this is a tautology: confidence scores fail when they predict high confidence on incorrect answers. That is like saying: classifiers fail when they are wrong
  • "Detectors flag...test-time methods..." -> the "detector" methods cited are also test-time methods. This distinction is unclear and imprecise
  • "take the log-ratio of the two probabilities" -> what are the two probabilities? Unclear
  • "On 2WikiMultihopQA it reverses, and on MuSiQue the two groups coincide" -> what is meant by "reverses"? what is meant by "the two groups coincide" Unclear
  • "We show that the reversal" -> what "reversal" is this talking about?
  • "Entropy bound the full-context term of g (proposition 1)" -> this is a forward reference to a proposition that is not properly explained and cannot be understood in the context of the introduction
  • "it scores confident answers with fixed directions separately within the two context-support strata" -> what are "fixed directions"? "context-support strata" is coined jargon that is unexplained

This is just from the introduction. I will not do this for every section of the paper, but it is a persistent problem throughout.

2) Weak experiments:

  • Section 3 uses only one model in the experiments. This is far from sufficient to establish any meaningful empirical conclusions
  • There are zero baselines from other studies in the detection evaluation in Table 3. Standard UQ baselines can be implemented off-the-shelf in commonly used UQ toolkits for this purpose such as LM-Polygraph and UQLM.

3) Reproducibility:

  • The experiment setup description is minimal, including how F1 and EM are calculated in the context/for these datasets.
  • No reproducibility materials submitted, despite claiming in the reproducibility packet that they were submitted, worsens the transparency issues

4) Missing related work: The paper is missing important related work for RAG-specific UQ:

Questions for authors
  1. Can you improve the writing so it can be understood by human researchers?
  2. Why did you not include any baselines from other studies in the detection evaluation?
  3. Why did you not submit a reproducibility packet given the minimal explanation of how the experiments were conducted in the text?
Reviewer_7AeJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates confident hallucinations in context-grounded question answering, where incorrect answers show low predictive entropy comparable to the correct answers which makes entropy-based detection unreliable. The analysis identifies two mechanisms: context-lure hallucinations, where models confidently copy incorrect contextual information, and prior-dominated hallucinations, where models rely on parametric knowledge instead of the provided context. The authors propose a label-free, test-time method Grounding-Profile Contrastive Decoding (GPCD), which detects suspicious answers using context-support-stratified scores, and applies type-specific contrastive decoding. It accepts corrections only when candidates agree and satisfy the context-support criterion.

Critical strengths
  • The paper provides a good explanation for why grounding-based hallucination detection can produce inconsistent results when applied to various benchmarks. The authors show that, by identifying two opposing error patterns, context-lure and prior-dominated hallucinations, the errors can have opposite effects on grounding scores. Furthermore, when both types are combined, these effects can cancel each other out or change the direction of the overall signal depending on their relative proportions in a dataset.
  • Grounding-Profile Contrastive Decoding (GPCD) translates this insight into a method that neither requires labelled training data nor weight updates. Its agreement-based acceptance rule explicitly addresses the risk of changing correct answers.
Critical weaknesses
  • The experiments uses only one model, Llama-2-7B-Base, and focuses mainly on short-answer question answering. So, It remains unclear whether the two identified hallucination patterns and the effectiveness of GPCD would generalize to larger or instruction-tuned models. The method also relies on statistics calculated across a batch of test questions, which may limit its usefulness when processing individual queries.
Questions for authors

It would be helpful if the authors could provide confidence intervals or statistical significance tests for the EM and F1 improvements reported in Table 2. Given that GPCD improves average EM by only 0.1 points, this would help establish whether the improvements are meaningful.

Reviewer_PtEoOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a training-free decoding method called “Grounding-Profile Contrastive Decoding (GPCD)” for reducing confident hallucinations, defined as low-entropy incorrect responses, in context-grounded short-form QA. The core idea is based on a “grounding score,” defined as the difference between the log-probability of a generated answer under the full context and under a masked-context baseline. Two sources of confident hallucinations are identified: context-lure errors, where the provided context strongly supports an incorrect answer despite weak parametric support, and prior-dominated errors, where the model relies on its internal prior despite weak context support. GPCD identifies lure and prior candidates among high-confidence (low-entropy) generations: context-supported answers with low parametric support are treated as lure candidates, while unsupported answers with high parametric support are treated as prior-dominated candidates. These are then re-decoded using type-specific contrastive decoding, which reduces the contribution of the context for lure candidates and increases it for prior-dominated candidates. Experiments with Llama-2-7B across five benchmarks show that GPCD achieves slightly higher average scores than greedy decoding and the other decoding baselines considered. The proposed stratified score also detects confident hallucinations better than pooled grounding signals.

Critical strengths
  1. The paper provides an interesting hypothesis that confident hallucinations can be context-driven or prior-driven. Experiments show that these two error types can have opposite grounding patterns, motivating the need for different correction strategies in the two cases.
  2. The proposed method is training-free and label-free, which makes it lightweight and easy to apply at test time.
Critical weaknesses
  1. The scope of the paper is a bit narrow. The analysis and experiments focus on short-form QA where an explicit context is provided, and the model is prompted to give a direct answer without reasoning. The experiments are also limited to Llama-2-7B-Base, while prior work has shown that instruction-tuned variants can have less calibrated probabilities. These limitations make the practical applicability and generalizability of the approach less clear.
  2. The gains over greedy decoding are marginal despite requiring several additional decoding steps, making it unclear when this method would be preferred over standard decoding. The baseline comparison could also be stronger: the paper discusses many related approaches such as COIECD, ReDeEP, and Lookback Lens, but does not evaluate them. Alternative confidence signals, such as verbalized confidence, are also not considered.
  3. The writing could be improved as well. For example, in the introduction, the paper mentions several concepts before properly defining them, making the argument difficult to follow.
Questions for authors
  1. Can the authors provide results on additional models, particularly instruction tuned variants, and compare GPCD against additional baselines mentioned in the related work?
  2. Can the authors clarify whether the proposed approach is expected to generalize beyond direct-answer short-form QA, for example to settings where models generate longer responses, use explicit reasoning, or are allowed to abstain with responses such as “I don’t know”?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.