acceptodds
Under review as a conference paper at ICLR 2027

Evidential Semantic Uncertainty for Autoregressive LLM Generation

Abstract

Uncertainty quantification (UQ) is essential for improving the reliability of autoregressive generation in large language models (LLMs). In open-ended generation, uncertainty can arise at lexical, syntactic, and semantic levels, among which semantic uncertainty is most indicative of answer correctness. A widely used approach, semantic entropy, estimates uncertainty by sampling multiple answers, clustering them into semantically consistent groups, and computing the entropy over group probabilities. However, this formulation assumes disjoint answer groups and fails to capture partial semantic overlap. Recent methods address this limitation by leveraging spectral clustering and kernel-based representations to model dispersion in semantic space. Despite these advances, an important failure mode remains unaddressed: cases where the model produces consistently incorrect yet semantically coherent answers. In this work, we observe that such incorrect generations are often associated with low evidential support, providing a complementary signal for uncertainty estimation. Building on this insight, we propose an evidential-theoretic framework that models answer groups using multinomial opinion and hyper-opinion. The proposed framework enables a principled decomposition of semantic uncertainty into: (i) lack of supporting evidence and (ii) conflict among competing answer groups. Our formulation explicitly captures inter-group semantic overlap and yields more expressive uncertainty measures aligned with the underlying semantic structure. Experiments across multiple question-answering benchmarks and LLMs demonstrate that our method consistently improves uncertainty estimation and enhances the detection of incorrect answers.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Evidential learning, semantic uncertainty

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_sF1MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an evidential framework for uncertainty estimation in autoregressive LLM generation. Sampled responses are grouped into semantic clusters, and transformed token logits are aggregated into cluster-level evidence. The uncertainty score combines vacuity, representing insufficient evidence, with dissonance, representing conflict among clusters. A hyper-opinion extension incorporates semantic overlap through NLI scores and spectral estimates of effective cluster count.

Relative to semantic entropy, the main additions are raw-logit evidence, the vacuity/dissonance decomposition, and overlap-aware scoring. The submission evaluates these components on four QA datasets and three models, with additional translation and sensitivity experiments.

Critical strengths
  1. Promising results against a substantial baseline set. Table 1 compares against semantic entropy, graph-based scores, KLE, CoCoA, Semantic Density, EVSE, and internal-confidence methods. For example, Hyper-Sum achieves 77.68 AUROC on Gemma/TriviaQA, compared with 69.44 for SE and 72.57 for the strongest reported baseline. The breadth of comparisons is a strength, although gains are smaller in some settings.

  2. Useful investigation of consistently incorrect generations. Appendix D.14 examines single-answer errors across sample counts and compares evidence distributions within the (K=1) subset. Together with ablations on sampling, NLI models, correctness evaluation, and evidence-only scores, this provides empirical support for supplementing semantic dispersion with another signal. Here you could cite [1] to make the claim stronger.

[1] Padhi, Trilok, Ramneet Kaur, Adam D. Cobb, Manoj Acharya, Anirban Roy, Colin Samplawski, Brian Matejek, Alexander M. Berenbeim, Nathaniel D. Bastian, and Susmit Jha. "Calibrating uncertainty quantification of multi-modal llms using grounding." arXiv preprint arXiv:2505.03788 (2025).

Critical weaknesses
  1. The participation-ratio formula does not satisfy its stated limiting behavior. Equation 7 applies participation ratio to eigenvalues of the normalized graph Laplacian. For complete overlap, taking A=11⊤A=\mathbf{1}\mathbf{1}^{\top} gives
L=I−1K11⊤,L=I-\frac{1}{K}\mathbf{1}\mathbf{1}^{\top},

with eigenvalues 0,1,…,10,1,\ldots,1. Consequently, WPRH=K−1W^H_{\mathrm{PR}}=K-1, rather than the claimed value of 1. For disjoint self-connected clusters, A=IA=I gives L=0L=0, making the ratio undefined. The singleton case is also undefined. These cases need explicit treatment because they are central to the effective-cluster interpretation.

  1. The overlap factor cancels in the two-cluster hyper-dissonance formula. For K=2K=2 and positive inter-group weights, Eq. 8 simplifies to
dissH=(b1H+b2H)Bal⁡(b1H,b2H).\mathrm{diss}^{H} =(b_1^H+b_2^H)\operatorname{Bal}(b_1^H,b_2^H).

At fixed beliefs, the overlap factor therefore has no effect. This needs reconciliation with Figure 12, which attributes a reduction from 0.08 to 0.000158 in a two-cluster example to the inter-group base rate. Effects through the separately changed prior weight should be distinguished from effects of overlap weighting.

  1. Raw-logit evidence needs stronger justification. Applying ReLU to raw logits is sensitive to a common additive shift of all vocabulary logits, although that shift leaves token probabilities unchanged. The resulting evidence is therefore not determined solely by the predictive distribution. Its empirical usefulness is interesting, but interpreting it as knowledge or evidential support requires further validation. Proposition 1 bounds assigned belief mass; it does not establish a relationship between that mass and actual correctness.

  2. The empirical superiority claims need uncertainty estimates and clearer evaluation details. Some gains over strong baselines are small: Hyper-PR improves AUROC from 73.58 to 73.80 over KLE on Qwen/NQ. Table 1 provides no confidence intervals or repeated-sampling variability. The manuscript also does not clearly specify evaluated dataset sizes or how the target answer is selected from the sampled responses. These details matter for interpreting the margins and reproducing the evaluation.

In summary, the paper offers a potentially useful combination of internal confidence and semantic structure, supported by broad baseline comparisons. My main reservation concerns the consistency of the mathematical formulation, rather than novelty alone. Clarifying whether the identified issues are manuscript errors or implementation issues, and verifying the resulting performance, would help me refine my assessment.

Questions for authors
  1. Does the implementation use Laplacian eigenvalues in Eq. 7? How are complete overlap, disjoint clusters, and (K=1) handled?

  2. How does Eq. 8 produce Figure 12’s two-cluster reduction when the inter-group weights cancel?

  3. How sensitive are the results to logit centering or other probability-preserving shifts?

  4. Could you provide dataset sizes, target-answer selection, and paired confidence intervals for the main comparisons?

Reviewer_2P43OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates to represent uncertainty in LLMs through an evidential framework, taking semantic equivalences of answer sequences into account. It introduces measures for lack of evidence and conflict of evidence, which shows suitable performance on standard evaluation settings. Furthermore, it accounts for partial overlap between different semantic clusters with an extended formulation of the evidential framework. Datasets, models and baseline methods are sufficient and relatively common for the this particular research community.

Critical strengths
  • The basic premise, capturing overlap between semantic clusters and disagreement between them better is a good idea given the often substantial overlaps (at least when using the NLI model as a ground truth).
  • The overall theme, attempting evidential modelling for LLMs, is clearly appealing, distinguishing honest IDK from being torn between answers.
Critical weaknesses
  • The derivation of the actually used scores is very complicated and a lot of the fundamental things (e.g. subjective logic) are just cited away and never discussed in detail. Also, at least from the current presentation, it seems its just introduced in an ad-hoc way to address certain issues, not in that a fundamental measure is derived first and certain assumptions are made to practially estimate it. Furthermore, there are countless assumptions and design choices that are never really discussed. To name a few: token selection through ReLU, clustering based on the NLI model (is it done in the same greedy fashion as done by Kuhn or pairwise with voting?), why not using neutral entailment in the actual score computation.
  • Presentation of the actual method needs to be improved, there are a lot of different symbols, defined in one way or the other with difference indices or brackets to denote different variants, making it very hard to keep track of them. I had to write everything on paper and substitute expressions into each other to make sense of the proposed framework. In a next version, I would advice to first state what should be achieved, maybe introducing evidential theory to a sufficient degree to understand the method and second discussing how those things are approximated in practice and why different aggregation methods have to be chosen together with their downsides.
Questions for authors
  • What is the reason for applying the ReLU function to token logits? As far as I can see, it just arbitrarily cuts off low-probability tokens, as training not really bounds them to any range. for some model, it could even be that all logits are negative, after softmax it just doesn't matter at what range logits are, just their relative size. Effectively it introduces a new hyperparameter (the cutoff for which tokens participate), that is not acknowledged or investigated.
  • From what I can see, the whole machinery is just a very complicated way of aggregating the contradiction and entailment score of the NLI model, combining it with token logits. Table 2 already shows that the u component (essentially a function of token logits) mostly captures the efficacy of the method. Why would we add all this extra complexity?
  • Is the question fed to the NLI model as well? The separateness of answer clusters hinges on the question that should be answered so I figure yes, but would like to have confirmation by the authors.
  • Why should we expect the model's logits to represent valid evidence? Work on evidential learning (e.g. Sensoy et al.) changed the training objective to foster that, why should we expect off-the-shelf LLMs to feature the desired properties?
Reviewer_RmPgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper focuses on the semantic uncertainty quantification of LLMs. It first points out an important failure mode: cases where the model produces consistently incorrect answers. This work observes that such generations are often associated with low evidential support, providing a complementary signal for uncertainty estimation. First, evidence is derived directly from the model’s token logits and converted into an evidential opinion distribution over classes (Hyper-opinion). Then the evidential distributions are converted into semantic uncertainties (either Multinomial Semantic Uncertainties or Hyper-opinion Semantic Uncertainties).

The experimental results on three models and four QA dataset demonstrate the effectiveness of this approach against multiple baselines. And the analysis compares the impact of different different uncertainty measurements.

Critical strengths
  1. The idea of convert the token logits into the evidential opinion distribution for semantic uncertainty is interesting.

  2. The proposed method is effective on QA dataset used for experimented in this work.

Critical weaknesses
  1. The writing needs improvement. The idea of this paper is hard to follow. There are many math notations are not explained very well, creating difficulties to understand the proposed method. For example, equation 4, the definition of dissonance, conflict (Bal) and belief mass are confusing and the intuition behind these definitions are not explained.

  2. Following the first point, the method proposed in this paper has many heuristic and ad-hoc designs. For example, why the conflict (Bal) is define like equation 4? What is inter-group base rate?

  3. The motivation is unclear. To my understanding, the core motivation is from line 84-89. But such bottleneck is not quantified.

  4. The experimental results is only on short-form QA datasets. The effectiveness is not evaluated on long text generation tasks.

Questions for authors
  1. How many times will Case II happen? (e.g., in the experimented QA datasets) Is the improvement mainly coming from fixing Case II?

  2. Will the evidential distribution still work for long text generation tasks?

  3. How is the semantic similarity matrix calculated? What model is used for semantic similarity? Will the choice of this model impact the performance of the evidential distribution as well as UQ?

  4. What is the relationship between uncertainty and correctness? Does this work mainly focus on "certain but incorrect cases"? (Case II in figure 1)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.