acceptodds
Withdrawn from ICLR 2027

What Does Semantic Uncertainty Reveal and What Does It Miss? A Large-Scale Study of Semantic Uncertainty in LLM Natural Language Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Semantic uncertainty quantification (SUQ) has become a central paradigm for detecting hallucinations in free-form language generation, as it measures uncertainty over meanings rather than surface forms. However, existing studies typically evaluate SUQ under limited tasks, models, and generation settings, making it difficult to understand when semantic uncertainty is reliable and where it fundamentally fails. We investigate both questions through SURE, a large-scale empirical study treating SUQ as both an object of evaluation and a lens on LLM behavior. We compare 19 measures and variants from six studies across four tasks, 19 English-output datasets, and 13 LLMs, complemented by 12-model translation experiments across 14 non-English-target directions. First, we analyze Semantic Uncertainty itself: method rankings vary with task and simple consistency measures remain competitive in several evaluated settings. We identify SURE-OC, diagnostic subsets of low-quality generations assigned confidence, to characterize where semantic agreement and quality diverge. Second, we use semantic uncertainty to analyze LLMs: higher-performing models tend to exhibit lower uncertainty, including on low-quality outputs, while QA temperature sweeps show that better uncertainty separability can accompany worse answer accuracy. Together, the two perspectives distinguish the effectiveness of an uncertainty estimator from the behavior of the generator, revealing why lower semantic uncertainty alone does not establish greater reliability. .

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Semantic Uncertainty, Large Language Model Evaluation, Black-box Uncertainty Estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_odmaOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper conducts a large-scale benchmarking study across 19 english-output datasets and 13 LLMs, as well as 12 model translation experiments. Their experiments explore several semantic uncertainty methods from the literature, compared against three other sampling-based baselines that do no use NLI-style scoring. They conduct descriptive analysis, including looking at false positives (instances where UQ method predicts correctness but actual answer is wrong or low-quality) as well as correlations of confidence vs. model size and correctness.

Critical strengths
  1. The paper is extremely thorough in its model and dataset breadth, covering 19 english-datasets and 13 LLMs.
  2. Quality of writing is generally very good. It is very readable prose and did not require any extra effort to understand the methods or setup.
  3. When UQ methods fail is an important research question
Critical weaknesses

1) The benchmarking study is not sufficiently distinguished from prior benchmarking studies (Vashurin et al., TACL 2025, Zhang et al., TMLR 202, Bouchard et al., TMLR 2025, Bakman et al., ACL 2025). In particular, the LM-Polygraph TACL study is of similar breadth, but investigates many more UQ methods. 2) The descriptive analysis is shallow and does not yield any meaningful insights. SURE-OC, the headline methodological contribution (which the authors honestly discuss as a diagnostic), is effectively just a coined term for false positives (instances where a UQ method predicts correctness but the model got it wrong / answered with low quality). Coining "overconfidence" is more tautological than insightful and is not itself a contribution. The correlations show that UQ scores correlate with correctness/quality, but this has shown by most previous studies on the topic. 3) Method correctness and/or clarity concerns:

  • LUQ-pair is used on math questions, but the claim/sentence-level decompositions do not make sense in that context
  • the authors appear to conflate discrete scoring as discrete in NLI judgements vs. discrete in cluster probabilities (discrete semantic entropy). The writing should be more clear about this 4) Incomplete experiment description in main text: The authors should clearly define which scorers they are using, instead of just citing the reference papers. The reader has to deduce purely from acronyms in the figure. The main text also does not mention which LLMs are used (reader again must deduce from figure). 5) Weak baselines used for long-text setting: UQ for long-text outputs has been well-established, but the authors only include a single method for this setting (LUQ-PAIR). From a benchmarking study, one would expect several more baselines that operate at the claim level, such as LUQ-atomic (Zhang et al., EMNLP 2024; Zhang et al., AACL 2025), Long-form semantic entropy (Farquhar et al., Nature 2024), claim-level graph-based methods (Jiang et al., NeurIPS 2024), and generalizations of these (Bouchard et al., TMLR 2026).

References:

Questions for authors
  1. Why did you only use LUQ-pair, despite having long-text outputs for some datasets?
  2. How does LUQ-pair work for math questions?
  3. Can you add the model names and scorer names to the main text?
  4. What does your benchmark study contribute beyond existing UQ benchmark studies?
Reviewer_HXiROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates Semantic Uncertainty Quantification for LLMs, providing a comprehensive benchmark (tasks and evaluation procedures) for this very prominent direction of uncertainty estimation/hallucination detection in LLMs. Its features significant task / model diversity and completeness of SUQ methods compared, giving a comprehensive overview. Furthermore there are also multilingual experiments, as well as translation experiments, which have been largely neglected by the community. The findings are somewhat unsurprising: depending on the task, there are often simple baselines outperforming SUQ (which was introduced mostly with QA in mind); continuous UQ signals are better than discrete ones; better models are mostly more confident; models of the same family more likely share overconfident wrong answers, etc.

Critical strengths
  • The investigation has a very broad scope, from QA and math reasoning to summarization and translation, evaluating SUQ in many different ways.
Critical weaknesses
  • The paper reads as a collection of findings, distilled from a very large-scale set of experiments. There is a lack of narrative or substantial finding that would move the field forward. For example Ielanskyi and Santilli et al. showed that one can not trust the standard correctness assessment and that UQ method ranking might be arbitrary under variations thereof - suggesting that mixture of judges or different tasks with better correctness signals result in more stable conclusions. A similar actionable outcome or at least meaningful insight is unfortunately lacking in this work in my opinion.
  • The paper never mentions how the answer that gets graded is selected (many prior works used a greedy decoded sequence for that). If it is just one of the sampled sequences, this conflates the question of which answer to use with the question of how to estimate uncertainty, e.g. through sampling at different temperatures as in Sec.5.2. Furthermore, many model providers suggest optimal sampling conditions for their model (e.g. certain temperatures, top-k, etc.) for generating high quality answers. This may not be optimal for UQ, which would actually be interesting to study.
  • The correctness metrics based on n-gram similarity (Rouge-L esp.) used in this work have been shown to be unreliable in multiple prior works, most prominently Ielanskyi et al. and Santilli et al. They suggest to use judges (better are mixtures of judges) to more reliably investigate UQ methods.
  • The focus of the study is somewhat hyper-specific. SUQ methods that can be applied in a black-box manner. Yet many of the conclusions are presented as rather general properties of UQ for LLMs.

Ielanskyi et al. (2026) Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation, ICLR

Santilli et al. (2024) On a Spurious Interaction between Uncertainty Scores and Answer Evaluation Metrics in Generative QA Tasks, NeurIPS GenAI WS

Questions for authors
  • How is the NLI model used on larger output sequences in summarization etc? This model is only trained on single sentence statements, its generally performing poorly for longer sequences. That could explain the bad performance of SUQ methods on those tasks. The way the DSE Nature paper got around that is splitting up the long sequence (they considered biographies) into individual claims. Also, what would it mean for a whole summarization to entail another summarization?

Remarks

  • In Figure 4, its hardly possible to distinguish different methods due to the similar colors. It would be good to highlight them more prominently, e.g. more differnet colors and linestyles.
  • Figure 6 labels should be bigger, they are not readable printed out as of now.
  • Same for Figure 7 and 8, they are hardly readable
  • AUROC is not independent of accuracy, making the conclusions from Figure 9 questionable. I generally think its not possible to disentangle correctness and UQ quality with any experimental setting available.
Reviewer_VUwEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper is a large-scale empirical study of black-box SUQ in LLM generation, asking when it works and where it fails. It's contribution is building SUREBench with a matched protocol, defines SURE-OC, and analyses model capability and sampling temperature. It evaluates with AUROC and a PRR-style score, average ranks and consistency baselines; only one comparison reports a confidence interval.

Critical strengths
  1. The comparison is matched and wide: the same samples and NLI evidence for all measures, 13 LLMs from 1.7B to 405B, four task families, with robustness checks on the quality metric, NLI backbone and output length.
  2. Code and an explorer are public, and the paper reports cases against its own story like SVAMP vs GSM8K where discrete best on the multilingual suite.
Critical weaknesses
  1. The main claims lack evidence of variability, and one metric is not standard. Each question has one draw of ten samples, no seeds are mentioned, Fig. 3 error bars are not defined, and only the family-gap comparison has a test or interval. The capability claim on a performance axis that mixes accuracy and ROUGE-L, which Appendix G says is "not a common unit". The PRR in the tables skips the random/oracle normalization of the standard PRR, so absolute values and gaps carry each model's and dataset's baseline quality.
  2. The contribution is a larger matched comparison, and its scale and numbers cannot be checked. LM-Polygraph and SNNE already compared these methods on QA, summarization, translation and math, and the paper reports ranks without explaining why methods differ. No saved outputs or logs are released, and nothing shows that the re-implementations match the original methods. Compute cost (about 500 A100-hours i think) is not stated.
Questions for authors
  1. What does a practitioner choose differently because of this paper, compared with LM-Polygraph and SNNE? Which findings were not already reported there?
  2. How many independent generation runs support each ranking and gap? Can you give 95% intervals for the Fig. 3 ranks and the main comparisons? 3.(suggestion) Report standard PRR next to your score, and say over which set quality is min-max scaled. 4.(suggestion) Check your re-implementation reproduce the original implementations’ scores on the original datasets, and report GPU-hours and cost.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.