acceptodds
Under review as a conference paper at ICLR 2027

How Do Uncertainty Estimators Align with Hallucination Occurrence in LLMs?

Abstract

Large language models (LLMs) can hallucinate, but does model uncertainty reliably reveal when they do? Although uncertainty estimators are increasingly used to identify unreliable generations, their relationship with hallucination remains poorly understood and is often taken for granted. We systematically evaluate 46 uncertainty estimators across complementary hallucination settings, covering both intrinsic hallucinations, where outputs are unsupported by the provided context, and extrinsic hallucinations, where claims are unsupported by expected model knowledge. Across four benchmarks and multiple LLMs, we find that the association between uncertainty and hallucination varies widely and is often weak, depending on the hallucination setting and the model under evaluation. These results challenge the use of uncertainty as a direct signal of hallucination and clarify when it provides useful information.

Primary area:
foundation or frontier models, including LLMs
Keywords:
uncertainty quantification, large language models, evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZVFmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper conducts a benchmarking study of LLM uncertainty quantification (UQ) methods across four datasets with three models. The three models are open-weight models, and the datasets cover text generation in the context of RAG, short-form answers, and long-text answers. They focus on standard UQ methods (black-box, white-box, reflexive) and implement their experiments using LM-polygraph. They find mixed results across models, datasets, and UQ methods, similar to findings in prior studies.

Critical strengths
  1. Readability: The paper is very well-written. It is entirely human-readable, and I did not need to re-read any part of the paper due to lack of clarity of writing.

  2. Reproducibility: The paper implements experiments using an open-source library (LM-polygraph), open-weights models, and publicly available datasets. All necessary reproduction details are provided in the paper and easy to understand.

Critical weaknesses

The paper has several important weaknesses:

  1. Insufficient breadth for a benchmarking study: This study is not offering any new methods, and is primarily a benchmarking study. This is not itself a problem, but for an ICLR-caliber benchmarking study, one would expect far broader breadth and scope coverage than what this paper covers. This concern applies to several aspects of the paper's scope:
  • No evaluation of supervised ensembles: Although they do not evaluate supervised UQ ensembles, which have been shown to generally outperform individual scorer baselines (Bakman et al., ACL 2025; Bouchard et al., TMLR 2025) and have greater robustness across datasets and evaluation settings (Chauhan et al., AACL 2026)
  • Insufficient model and dataset breadth: The authors evaluate UQ on only four datasets and three models. They motivate this by claiming to require human annotations made available by the RAGTruth dataset with the three open-weight models they used. However, obtaining faithfulness-style ground-truth labels can be easily obtained with a competent LLM in the same way that long-form labels are obtained in LongWiki. Using an strong LLM to obtain RAGTruth labels and validating with human review on a reasonable sample would enable use of additional models, which is absolutely necessary for a benchmarking study.
  • Long-form setting missing well-established long-form scorers: Only one long-text adapted (claim-level) scorer (LUQ) is used on the long-text dataset, despite well-established literature on this, and it is used incorrectly. Per appendix B.6.2, as the method is designed to operate claim-wise or sentence-wise and check entailment of claims/sentences of original response against full sampled responses, but the authors instead compute full-response comparisons. The name of the scorer is Long-Text Uncertainty Quantification (LUQ), hence the claim/sentence-wise scoring, it was designed to compute. For other well-established methods designed for long-text UQ, see claim-level graph-based scorers (Jiang et al., NeurIPS 2024), long-form semantic entropy (Farquhar et al., Nature 2024), atomic calibration (Zhang et al., AACL 2025), and generalizations of these (Bouchard et al., TMLR 2026). Further, the authors use LUQ incorrectly,
  1. Insufficient motivation for the paper and lack of positioning relative to prior work: How is this paper different from other benchmarking studies such as Vashurin et al., TACL 2025; Bouchard et al., TMLR 2025; Bakman et al., ACL 2025? This appears to evaluate the same scorers as in LM-Polygraph (Vashurin et al., TACL 2025) but uses far fewer models and datasets than their TACL study. It also misses the claim-level and code generation settings in the recent benchmarking study evaluated by Chauhan et al. (AACL 2026).

  2. Main results missing from main text There is no results table in main text or exhaustive list of which specific scorers are used in the study, only scorer families.

References:

Note: some of the provided references are quite recent and provided only to help authors in future versions of the paper. I do not hold this against the authors for not including them in their evaluation.

Questions for authors
  1. why not present results table in main text?

  2. how do you distinguish this study from other benchmarking studies that have studied the same question with the same scorers?

  3. why not consider code generation, agentic settings, or more claim level scorers for long text settings?

  4. I would encourage the authors to position the paper relative to the papers mentioned in weaknesses to further clarify how their work differs and compare this work to those.

2 comments

We thank the reviewer for their constructive, detailed feedback and for pointing us to relevant studies. We address the four questions below, combining Q2 and Q4.

Q1. Why not present a results table in the main text?

We chose figures because our main message concerns variation across evaluation settings, rather than a ranking of estimators. Figure 1 shows each estimator's mean AUROC against its ranking variability, illustrating how average performance can hide substantial differences across settings. The complete results are available in Appendix D.1 (Tables 6–9). We also consider explicitly naming all evaluated estimators in §2, where we introduce representative estimators by family.

Q2 & Q4. How does our study differ from existing benchmarks?

We study how uncertainty estimators align with explicitly defined hallucination criteria, and how this relationship changes with the evaluation criterion. We agree that this focus should be positioned more clearly relative to previous benchmarks.

Bouchard and Chauhan (2025) and Chauhan et al. (2026) study supervised ensembles of uncertainty scores. Their focus is on combining estimators, whereas we evaluate individual methods. Ensembling is a natural extension, not our present objective.

Bouchard et al. (2026), Jiang et al. (2024), and Zhang et al. (2025) focus on fine-grained uncertainty estimation for long-form generation, particularly at the sentence or claim level. Our comparison uses query- and response-level estimators and response-level evaluation targets.

Bakman et al. (2025) investigate practical deployment conditions while keeping the correctness criterion fixed. They study sensitivity to decision thresholds, input transformations, long-form adaptation, and ensembling. In contrast, we focus on variation in the evaluation target, including changing the error definition on the same responses.

Vashurin et al. (2025, LM-Polygraph) are the closest to our study. We use their library and many of the same estimators, while their benchmark covers more tasks and models. The main distinction lies in the evaluation targets. Their benchmark primarily uses general response-quality measures: AlignScore against reference answers, accuracy, COMET, and ROUGE-L against reference summaries. It also evaluates hallucination detection through claim-level fact-checking of generated biographies. We do not claim to introduce hallucination detection into UQ benchmarking.

Our difference lies in how we operationalize and compare hallucination criteria. Following Bang et al. (2025), we consider responses unsupported by the supplied context (RAGTruth, with human span annotations) and responses unsupported by knowledge expected from pretraining (HalluLens). We examine short- and long-form generation, as well as failure to refuse false premises. We also investigate how rankings change with the error definition and severity while keeping responses fixed (Appendix D.6–D.7). Another difference is that they use base models for white-box estimation while we consider instruction-tuned models.

Thus, our contribution is not broader benchmark coverage, but a focused investigation of the relationship between uncertainty and hallucination under different evaluation criteria, using query- and response-level estimators rather than claim-level scorers or supervised ensembles.

We acknowledge the scope could have been more explicit.

Q3. Why not consider code generation, agentic settings, or more claim-level estimators?

Code generation and agentic settings involve different failure modes and evaluation requirements, such as execution correctness or multi-step decisions. They fall outside our present scope, which focuses on hallucinations in text generation.

Regarding LUQ, the reviewer is correct. We use the simplified LM-Polygraph implementation, which computes entailment between complete responses. The original LUQ (Zhang et al., 2024) instead evaluates the entailment of individual sentences against sampled responses. We may rename the methods and clearly distinguish it from the original method.

Reviewer_ZVFmOpenReview ↗

Narrow scope, no long-text scorers, insufficient positioning

Thank you for your replies. While I appreciate the author's clarifying comments on their work, I do not believe the breadth/scope is sufficient to meet the bar for an ICLR-caliber empirical/benchmarking paper.

I am choosing to keep my original score for the following reasons:

  • text generation only (no code generation or agent settings)
  • no supervised/ensembled uncertainty scorers, which has been shown in the literature to perform strongly across settings, in contrast to individual, unsupervised UQ scorers
  • no proper long-text scorer despite using long-text datasets and existence of a well-established long-text UQ literature
  • insufficient positioning relative to existing literature and general lack of novel contribution
Reviewer_xfi2OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies the relationship between uncertainty estimation methods for LLMs and hallucinations, i.e. the ability of the former to reliably detect the latter. It studies a broad set of uncertainty estimators, based on different sources of information, either based solely on the model or through external modules (either trained per task or pretrained). A set of four hallucination detection benchmarks serves to establish a weak link between the occurance of hallucinations and uncertainty estimators. Additional controls, such as response length or the impact of the judge are investigated.

Critical strengths
  • The paper assesses a comprehensive range of uncertainty estimators, based on different origins of their signal and various dependencies on training data, external modules, etc.
  • Looking at different controls (response length, judge model) is important to validate the results. While those controls have been studied in prior work already, its commendable they are investigated to rule them out as determining factors in the paper's findings.
Critical weaknesses
  • The major weakness of this work in my opinion is its scope and the observational nature of the experiments. Having written some papers in this particular area myself and having read a good share of related papers, its very obvious that the usefulness of uncertainty estimation methods hinges on the exact setting we want to use it for. I think it would make a fine EMNLP paper, combining a sound premise with careful and extensive experiments, but for ICLR I feel it adds too little new insights into the overall topic of uncertainty estimation / hallucination detection for LLMs. I really commend the authors for their work and getting all these methods to work across different models and tasks, but the findings are just not really informative for the field to move forward, nor for practitioners in pre-selecting methods they could try for their particular setting. Maybe it would make sense to try to design a benchmark for an interventional study, looking at how things change under different interventions that could inform practitioners? Like a flow diagram, e.g. "If training data is available in sufficient quantity, use MD based or probe based methods, otherwise if ..." that could help in practice. Akin to what there is in the second paragraph of the conclusion, but more comprehensive.
  • The evaluated models are just not relevant anymore, using models from 2023 (Llama 2 is even from early 2023), makes the findings less relevant to practitioners that are mostly operating on models of the latest generation or at least releases within the last year. I get that this was due to RAGTruth, but then one could relabel at least a smaller version of the dataset for new models and maybe show results on one or two older models where traces are available for completeness.
  • The authors did a good job in defining uncertainty and its scope for the purpose of this paper, but neglected to define hallucinations at the same level of detail. This is critical for a paper attempting to study the relationship between the two. I think the question of what exactly is an hallucination is a complicated one. Major works on llm uncertainty estimation (e.g. the Nature paper by Farquhar et al.) generally claim that the address confabulations - a subset of hallucinations caused by insufficient knowledge and the need to produce some answer.
Questions for authors
  • How exactly do you define "hallucinations" for the purpose of this paper? Is it only confabulations in the sense of prior work on uncertainty estimation in LLMs, or more broader? I see the reference to Bang et al., but the paper lacks context of how this taxonomy connects to the uncertainty estimation literature. If the authors define it more broad, why should we expect uncertainty estimation methods - designed for this particular set of hallucinations - to be reliable indicators for other types of hallucinations?
2 comments

We thank the reviewer for taking the time to write a constructive review, particularly for highlighting the importance of defining the errors used to evaluate uncertainty estimators.

Farquhar et al. (Nature, 2024) define confabulations as incorrect and arbitrary generations: their meaning can change with irrelevant variation, such as the sampling seed. They distinguish these from systematically incorrect generations, for example, errors consistently learned from unreliable training information. Arbitrariness describes observable behavior across generations; the authors also discuss possible causes, but that behavior alone does not identify the mechanism.

A useful distinction appears in their evaluation. Although confabulation motivates semantic entropy, their main AUROC measures whether it predicts incorrect answers, without independently verifying arbitrariness for each error. They explicitly acknowledge that their labels also include systematic mistakes, and present this as a lower-bound evaluation of confabulation detection. Defining arbitrariness using the same disagreement score under evaluation would risk circularity.

Our definition addresses a different dimension. Following Bang et al. (ACL, 2025), we evaluate whether a response is supported by the reference the model is expected to rely on: supplied context (intrinsic hallucination) or knowledge expected from pretraining (extrinsic hallucination, approximated in HalluLens using Wikipedia). This gives an explicit, observable evaluation criterion for each response; it does not require repeated generations or inferring why an error arose.

Stable across generationsVariable across generations
Supported by the referenceConsistently acceptableDifferent acceptable answers
Unsupported by the referenceConsistent errorVariable error, potentially confabulation

Reference support and generation behavior are related, but distinct dimensions. An unsupported statement may recur consistently; disagreement may occur among acceptable answers. Explaining why an error occurs is a further question that neither observation settles on its own.

This matters for the link with uncertainty. Sample-based semantic-entropy estimators use disagreement between meanings as a signal, not a direct test of reference support. Token probabilities, internal states, and self-evaluations measure other signals. We do not assume that every signal should detect every error. Our study asks how these estimators empirically align with specified hallucination criteria, including when we vary the error definition on fixed responses (§6, Appendix D). Explaining which generation mechanism each estimator detects would require other evidence.

Finally, the terminology itself is inconsistent: confabulation is used as a synonym for hallucination (Sui et al., ACL 2024), as a preferred alternative term, or as a subset (Farquhar et al., Nature 2024). Very recently, Kuramoto et al. (UncertainNLP 2026) likewise document substantial variation in operational definitions across 244 papers. This reinforces why we define our evaluation labels precisely. We will clarify these distinctions and their implications for interpreting the results.

Thank you for the detailed response on clarifying the definition of hallucinations in this work. Separating reference support from variability across generations is a helpful framing and should be highlighted in the revised paper.

However, the response does not address the identified weaknesses: the limited new insight for the field, the observational design, and the use of outdated models. I will thus maintain my score.

Reviewer_5NRpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a systematic empirical investigation of how estimated uncertainty of LLMs aligns with hallucination occurrences in LLMs. The authors evaluate 46 uncertainty quantification methods and estimators across four benchmarks covering context faithfulness, short-answer correctness, factual support in long-form generation, and abstention on false premises, using three open-weight LLMs. Their analysis considers discrimination, selective prediction, rank calibration, ranking stability, and redundancy among estimators. The main finding is that the relationship between uncertainty and hallucination varies substantially across tasks and models, with no universally reliable uncertainty estimator.
I found the study helpful, particularly because it challenges the common assumption that uncertainty is necessarily a reliable proxy for hallucination. The range of evaluated methods and detailed reproducibility information are significant strengths. However, the paper has several methodological weaknesses that limit the generality and significance of its conclusions.

Critical strengths
  1. The paper investigates an important and timely research question for LLMs: whether high predictive uncertainty correlates with hallucination or unreliability.

  2. The authors covered a broad range of uncertainty estimators (46 estimators). However, they missed some notable recent work.

  3. The authors go beyond hallucination-detection AUROC and use several complementary metrics.

  4. Well-documented and reproducibility details are provided in the Appendix.

Critical weaknesses
  1. my main concern is that it seems the hallucination type is confounded with task, response length, and evaluation protocol. One of the main conclusions is that uncertainty estimators align differently with intrinsic and extrinsic hallucination, and that the evaluation dataset is the primary source of variability in estimator performance. However, the experimental design does not sufficiently disentangle the effects of the hallucination definition from those of the underlying task. For example, the authors evaluate intrinsic hallucination using RAGTruth, which includes context-grounded summarization, qa, and dtt generation. Extrinsic hallucination is evaluated using three HalluLens tasks with substantially different response structures.

Therefore, the experimental comparisons vary multiple factors simultaneously. These effects cannot be separated by the current four-dataset experimental design. just as an example, lower uncertainty discrimination on RAGTruth might reflect intrinsic hallucination being intrinsically harder to detect. But it could also reflect longer responses, mixed subtasks, or a more demanding human-annotation criterion in which a single unsupported statement develops the entire response hallucinated. Due to this weakness the broader conclusion that hallucination setting determines estimator reliability needs more careful investigation and qualification.

  1. I also see a conceptual issue in the methodology on query-level uncertainty versus response-level hallucination. A query-level uncertainty estimator measures properties of the distribution of possible outputs, whereas response-level hallucination is a property of the actual observed generation. Therefore, even a perfectly estimated query-level uncertainty cannot necessarily distinguish the correct response from the hallucinated response for the same input.

I understand that the authors explicitly acknowledge that their evaluation associates uncertainty scores with observed response quality rather than estimating U^*(x) independently. But I believe this distinction needs greater emphasis in the analysis and interpretation. more specificaly, comparing query-level and response-level estimators under the same response-level target may intrinsically favour the latter.

  1. The authors use Llama-2-7B-chat, Llama-2-13B-chat, and Mistral-7B-Instruct-v0.2 with the explanation that RAGTruth provides human annotations for responses generated by these models, and retaining the original responses ensures consistency with those annotations. This is a legitimate constraint for the RAGTruth experiment. But it is much less compelling for the HalluLens experiments, where the authors generate or evaluate new responses and use an LLM judge. there are recent open-weight models that the authors could include in the HalluLens portion, even if they were not available in RAGTruth.
Questions for authors
  1. Disentangling hallucination from task effects (my main concern).

  2. Query-level versus response-level uncertainty (point #2)

  3. Model generalizability (point # 3)

3 comments

We thank the reviewer for taking the time to write such a careful and constructive review. The three questions go to the core of how our results should be interpreted, and answering them led us to run two new analyses that sharpen some of the paper's claims.

Q1. Hallucination type versus task and evaluation factors

In §5 Results, we state that rankings vary more across datasets than models. This is descriptive; we do not attribute the variation to hallucination type alone. Datasets also differ in task, length, and evaluation protocol.

This is why we've already examined in the appendix length (D.4), judge choice (D.5), RAGTruth error subtypes and severity (D.6), and LongWiki criteria and thresholds (D.7). Notably, D.6 distinguishes conflicting from unsupported content and requires 10% or 25% hallucinated tokens instead of a single unsupported span. These analyses vary the criterion on fixed responses.

HalluLens covers three tasks within the broad extrinsic setting, though their criteria differ. To examine task variation under a common criterion, we newly split RAGTruth into summarization, QA, and data-to-text. For each model, we ranked 45 estimators by AUROC and measured agreement (Kendall's τ). As a reference, we drew 200 random group pairs matched on correct and incorrect response counts.

ModelSubtask τ: median (range)Matched random τ: median
Llama-2-7B0.45 (0.25–0.47)0.73
Llama-2-13B0.32 (0.00–0.38)0.76
Mistral-7B0.30 (0.24–0.50)0.82

Three subtask pairs per model; all nine observed agreements are below their matched reference (one-sided permutation p ≤ 0.01).

We also matched correct and incorrect responses within five length bins and compared them with equally matched random groups. The gap remained in all nine pairs.

Task specifies what the model does; intrinsic/extrinsic specifies the reference used for assessment. One response could be judged against supplied context or external evidence expected from pretraining. Our sensitivity analyses vary criteria on fixed responses; evaluating the same responses against both references would more directly examine this distinction, but not the error mechanism.

Q2. Query-level versus response-level uncertainty

We agree this distinction deserves emphasis. The reviewer wonders whether individual-response labels favor the response-level estimator compared to query-level estimators.

We tested this by classifying estimators by their definitions: those depending on the evaluated response are response-level. Excluding token count, we compared median AUROC for 25 response-level and 21 query-level estimators across 12 configurations.

DatasetLlama-2-7BLlama-2-13BMistral-7B
PreciseWikiQA0.58 / 0.700.55 / 0.650.69 / 0.76
LongWiki0.57 / 0.570.57 / 0.600.52 / 0.55
RAGTruth0.46 / 0.460.43 / 0.460.51 / 0.32
NonExistentRefusal0.49 / 0.490.42 / 0.420.55 / 0.51

Response-level / query-level median AUROC. Bold marks the higher group median where exploratory Mann–Whitney p < 0.05.

There is no systematic advantage for response-level estimators: query-level medians are higher in 8/12 configurations, especially PreciseWikiQA, where differences are significant in their favor. When most answers to a query are wrong, query-level risk can reasonably predict one answer; aggregating samples may also reduce noise. However, method families differ between groups, so this does not isolate response conditioning.

Lin et al. (TMLR, 2024, Generating with Confidence) distinguish query uncertainty from response confidence; response-specific variants generally predict individual correctness better. A cleaner test would compare both variants of the same method, or evaluate query-level scores against average correctness over samples. This would have been indeed valuable to investigate further.

Q3. Newer models

We agree that recent models would strengthen generalizability, particularly on HalluLens, where RAGTruth's human-annotation constraint does not apply. New generations, uncertainty scores, and quality evaluations exceed our current rebuttal resources. We acknowledge this limitation.

Reviewer_5NRpOpenReview ↗

Helpful clarifications but not sufficient

I thank the authors for their efforts and time to respond to my comments and providing the additional experiments, particularly the RAGTruth subtask analysis in response to W1.

Re W1, the new results is helpful in a sense that it provides some useful evidence that estimator rankings vary substantially even within a common benchmark and broad hallucination definition. Also I think the length-matched comparisons is further strengthening this finding. I agree that the paper's primary conclusion is descriptive rather than causal. The revised manuscript distinguishes variation across tasks from variation attributable specifically to hallucination mechanisms, which is a good improvement to see. However, the additional results establish task-associated heterogeneity but do not fully isolate its underlying causes.

Re W2, the new comparison between query-level and response-level estimators is helpful and informative as it suggests that the evaluation does not systematically favour response-level estimators. However, the comparison involves different estimator families and therefore does not isolate the effect of response conditioning. As I conceptually stated in my comments, the query-level uncertainty and response-level hallucination correspond to different statistical targets, regardless of which group achieves higher AUROC. I appreciate that the authors acknowledged this distinction and their suggestion of a more controlled comparison. One more thing is that the reported table appears to show query-level superiority in 6 out of 12 configurations, with 4 ties, rather than the stated 8 of 12. This discrepancy should be clarified.

Re W3, the acknowledgment is appreciated. I understand the resource constraints during rebuttal, but the lack of more recent models remains a limitation on the generalizability of the findings.

Overall, with this clarification I have more confidence in the results, however, the broader concerns about generalizability and the interpretation of uncertainty still remain and I do not yet find sufficient grounds to change my overall recommendation.

I should add that as I went through the paper again, I see more fundamental limitations in the experimental design, particularly the dramatic class imbalance in several benchmarks. This raises additional concerns about the interpretability and practical significance of the reported results.

We thank the reviewer for their follow-up. We would like to clarify two points.

Regarding W2 (6/12 vs. 8/12): The reviewer is correct that the table, rounded to two decimal places, shows query-level estimators ahead in 6/12 configurations, with 4 apparent ties. The 8/12 count was based on three-decimal values we obtained internally but display only two decimals in our reply text for readability; two additional configurations show small differences that disappear after rounding to two decimals. The main lesson remains the same.

Regarding class imbalance: We acknowledge the substantial class imbalance in the HalluLens benchmarks, as reported in Appendix C.1 (Table 3) [for the minority class 33-43% for RAGTruth and 3.6-25.9% across the 3 HalluLens datasets, with the highest imbalances in NonExistentRefusal]. This is reflected in the bootstrap standard deviations reported in Appendix D.1 (Tables 6–9), which we take into account when interpreting the results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.