acceptodds
Under review as a conference paper at ICLR 2027

Stepwise Semantic Uncertainty in Long-Form Language Generation

Abstract

Semantic uncertainty of large language models (LLMs), i.e., their lack of confidence in the meaning of the generated output, is a well-established indicator of the reliability of short responses. However, in long-form generation, a single response-level estimate cannot distinguish reliable from unreliable units of meaning. Thus, we propose to represent and quantify the semantic uncertainty stepwise, i.e., induced by the next-token distribution at each step during generation, without additional post-hoc response decomposition or access to internal model states that existing approaches rely on. We introduce the concept of an attribute as the topic a unit of meaning refers to, which can be identified as soon as it emerges. By representing the alternative tokens at each step jointly by both their meaning and attribute, we can separate uncertainty due to variation across attributes from uncertainty due to contradictory meanings within attributes. Given that the latter primarily indicates potential factual errors, we leverage our representation to construct an attribute-conditional, kernel-based von Neumann entropy that specifically quantifies this kind of uncertainty. Empirically, we show this is more predictive of token- and unit-wise accuracy in long-form language generation than existing baselines. Since our representation is available before a token is committed, we further utilize it to steer decoding and improve the factual correctness of generated responses.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
LLMs, Long-Form Generation, Uncertainty Representation, Uncertainty Quantification, Semantic Uncertainty, Von Neumann Entropy

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LFmFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a stepwise semantic uncertainty estimation method for long-form LLM generation. It distinguishes uncertainty across different attributes from uncertainty about conflicting meanings within the same attribute. The authors introduce an attribute-conditional kernel-based von Neumann entropy and use attribute-conditional risk to guide beam search. Experiments show improvements in hallucination detection and factual correctness.

Critical strengths

[S1] Meaningful problem formulation and methodology.

The paper distinguishes uncertainty about which attribute to generate from uncertainty about the meaning within an attribute. It proposes a method to measure it at each generation step. I believe this is a meaningful contribution because it allows semantic uncertainty to be used during generation.

[S2] Consistent empirical improvements.

The proposed method achieves the best token-level performance across all 12 settings and the best entity-level performance in 8 out of 12 settings (Table 2). It also improves factual correctness during decoding compared to the evaluated baselines (Table 3). These results show the practical value of the proposed method.

Critical weaknesses

[W1] Insufficient validation of the meaning and attribute encoders.

I understood the proposed method relies heavily on two fine-tuned encoders to distinguish meaning and attribute similarity. Even though the paper suggests some ablation, it does not provide direct evaluation of how accurately these encoders distinguish the two types of similarity on actual next-token continuations. It makes difficult to verify whether the proposed method works as intended.

[W2] Missing ablation study for the decoding method.

The decoding experiments do not include a baseline that uses a meaning-only risk penalty without attribute conditioning. Therefore, it is unclear whether the improvements come specifically from attribute conditioning or simply from adding a semantic disagreement penalty. This comparison is important to support the main contribution of the proposed decoding method.

Questions for authors

[Q1] Evaluation of the meaning and attribute encoders (W1)

The paper does not report quantitative evaluation results. Could you provide the held-out performance of each encoder, especially on generation-derived token pairs? I believe this would help verify whether the encoders can reliably distinguish meaning and attribute similarity, which is fundamental to the method.

[Q2] Effect of attribute conditioning on decoding (W2)

Table 3 shows improvements in factual correctness, but I think it is unclear how much of the improvement comes from attribute conditioning. I'd like to ask you to test an ablation that uses only meaning similarity in the decoding risk, without attribute conditioning (e.g., setting the attribute kernel to a constant in Eq. 10). This would help determine whether attribute conditioning is necessary for the reported improvements.

3 comments

We thank the reviewer for their interest in our work. Below, we provide concise responses to each of the comments.

W1 and Q1: The fact that our uncertainty measure and our decoding method work in practice is already indirect evidence that the encoders work as intended. A direct evaluation is difficult in general, since there are no third-party datasets containing a ground truth for graded similarities on actual next-token continuations. However, we found this comment intriguing and decided to elaborate on it further. For this purpose, we use the test split of the labeled data that was constructed for fine-tuning the encoders (see Appendix C.1) to test whether the encoders work as intended. Ideally, the meaning (attribute) encoder should assign a high similarity to two alternatives with the same meaning (attribute) and a low similarity to two alternatives with different meanings (attributes). One caveat is that the labels in this data are binary (same or different meaning, same or different attribute) and not graded. However, this approximation should already be sufficient to see whether the encoders work as intended. We report the average similarity across pairs with the same and with different meaning (attribute) for both encoders in Table 8 and Table 9 of the updated manuscript. The results are in line with our expectations.

W2 and Q2: This is indeed also an interesting remark. To further highlight the importance of attribute conditioning, we added a baseline to our beam search experiments that uses a meaning-kernel risk without attribute conditioning (see Appendix A.2 for details). As shown in Table 3 of the updated manuscript, attribute-conditional beam search outperforms this meaning-only variant. This is in line with our expectations, since conceptually, a meaning-only risk also penalizes benign variation across attributes, which is exactly what the attribute kernel in Eq. 9 excludes. For further details, please refer to Section 5.2 and the corresponding Appendix A.2 of the updated manuscript.

Thank you for the rebuttal. I believe authors have adequately addressed my concerns. I will maintain my original score.

We thank the reviewer for engaging with our rebuttal and are glad that it addresses all raised concerns.

Reviewer_gbjPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework for estimating semantic uncertainty at each generation step in long-form language generation. It represents candidate next-token continuations through separate meaning and attribute encoders, then constructs an attribute-conditional kernel-based von Neumann entropy to distinguish disagreement within an attribute from variation across attributes. The authors also introduce a related candidate-level risk to guide beam search. Experiments evaluate hallucination detection at token and entity levels, alongside factuality improvements during generation.

Critical strengths
  • Separating uncertainty about which attribute to express from uncertainty about its value is a useful perspective for long-form generation. The prefix-based representation makes this distinction available before committing to a token, supporting both error detection and generation control.

  • The method consistently improves token-level detection across the evaluated settings, although entity-level gains are less uniform. The decoding results also show improvements in factual precision alongside competitive correct-entity counts, providing evidence beyond simply producing shorter responses.

Critical weaknesses
  • The methodological advance over closely related approaches needs stronger isolation. CCP already distinguishes uncertainty about claim values from variation in claim selection and wording, while KLE already uses graded semantic kernels and von Neumann entropy. The distinctive contribution therefore lies in the prefix-based attribute representation and its conditional formulation. Although the paper includes relevant ablations, it does not establish whether this formulation offers substantial benefits over a prefix-based CCP adaptation or a simpler disagreement measure using the same trained encoders.

  • The strongest results do not fully isolate factual discrimination from filtering non-entity positions. Token-level evaluation compares incorrect entity tokens against both correct entities and non-entity tokens, while the proposed method explicitly suppresses uncertainty at function-word and uncommitted positions. The smaller and less consistent entity-level gains suggest that part of the advantage comes from this filtering. Matched masking and encoder controls are needed to establish how much improvement specifically reflects better discrimination between correct and incorrect factual content.

  • The decoding evidence leaves the mechanism and broader utility uncertain. Local agreement does not establish factual correctness: a candidate with little competing mass within its attribute can receive low risk despite being incorrect. The reported correct-entity counts mitigate a simple avoidance explanation, but the biography-only evaluation does not establish whether improvements preserve information coverage and specificity more generally. The uncertainty-estimation cost analysis also does not resolve the end-to-end cost of the proposed beam-search procedure.

Questions for authors
  1. Do the detection gains persist when methods share the same function-word masking and are evaluated exclusively on factual entity tokens?

  2. Does the decoding method make factual assertions more reliable, or primarily steer the model toward attributes with fewer conflicting alternatives? What evidence distinguishes these explanations?

2 comments
AuthorsOpenReview ↗

Response 1/2

We would like to thank the reviewer for the engagement with our work. Below, we address each of the comments individually.

W1:

  • On isolating our contribution from related approaches: We fully agree that our approach shares similarities with existing approaches. This is natural, as tackling uncertainty in large language models requires some principles, e.g., a certain degree of semantic awareness, that every method should follow. We specifically tailor and extend these principles to the online, stepwise setting, thereby resulting in our three stated desiderata (token-level, semantic, structure-aware). By adhering to these desiderata, the presented representation yields a structure-aware, information-theoretic measure which can be obtained on the fly and in a computationally efficient manner, a combination that, to the best of our knowledge, no existing approach offers (see Table 1).

  • On a prefix-based CCP adaptation: The CCP baseline in our paper is indeed such a prefix-based adaptation.

  • On a simpler disagreement measure with the same encoders: We would like to note that the appeal of our measure lies in it being an information-theoretic quantity. As an entropy, it characterizes the entire next-token distribution through its top-N alternatives and allows us to condition on the attribute in a principled way (Eq. 6). In this realm of information-theoretic measures, VNE$_M$ is already a simpler measure on the same trained meaning encoder, and VNE$_{M|A}$ outperforms it w.r.t. AUPR in all 12 token-level settings. Beyond this, Reviewer Fs35 raised a related concern, namely that our advantage might stem solely from the fine-tuned encoders. To examine both points, we added several ablations in which the baselines, including CCP, are given our encoders and our fine-tuning procedure. Some baselines improve under these conditions, but our method remains the strongest w.r.t. AUPR in the majority of settings (Table 2). We refer to our response to Reviewer Fs35 for the details of these experiments.

AuthorsOpenReview ↗

Response 2/2

W2 and Q1: We found this comment very interesting and addressed it with several additional ablation studies on the effect of function-word masking.

  • First, we evaluated all methods, including ours, without our function-word masking, so that each method uses only its original handling (Table 5). The performance of VNE$_{M|A}$ decreases. However, it still performs best in 11 of 12 token-level settings, with our hard-clustering variant SE$_{M|A}$ ranking second in 10 of them.

  • Second, following the reviewer’s suggestion, we applied our function-word masking to baselines (Figures 8 and 9). Here, the effect differs systematically by method type. Structure-aware methods (e.g., CCP) and methods using fine-tuned encoders or the fine-tuned NLI model mostly benefit compared to their original formulation. In contrast, meaning-only methods with the original NLI model degrade. In these settings, without attribute conditioning, the added dissimilarity among function words is counted as semantic uncertainty, which produces false positives at benign positions. The suppression at uncommitted positions does come from attribute conditioning itself, whose ablation is VNE$_M$ in Table 2. Since entity positions are not known during generation, we consider this suppression a part of the task and not a confound.

W3 and Q2:

  • On the link between local agreement and factual correctness: We would like to note that this is an inherent property of almost all established UQ methods for LLMs and not a weakness specific to our approach. When we quantify the uncertainty of a model, we take the uncertainty encoded in its predictive (semantic) distribution and quantify it. If a model is overconfident while being wrong, the estimated uncertainty will be low, as this is the signal the model provides. Detecting and correcting such confidently wrong outputs is a matter of the calibration of the model itself, which is beyond the scope of this paper and of the baselines we compare to.

  • On the underlying mechanism of the proposed decoding: By the design of the attribute-conditional beam search, we expect two effects. First, the approach avoids attributes with a high number of contradicting candidates and thereby steers the LLM toward attributes with fewer conflicting alternatives. Second, within a single attribute, it selects an alternative that shows less contradiction with the other alternatives. The actual mechanism of the method is probably dominated by the first effect. We examine whether this assumption holds empirically in Appendix A.2, Table 6. The results indicate that the steering component dominates the increased factuality of the generated responses, whilst we still see an increase in correctness among attributes present in both our method and the baseline.

  • On the end-to-end cost of beam search: To further clarify this point, we have added a comparison of the computational cost of the different decoding methods to the updated manuscript in Figures 11 and 12 in Appendix A.2. As expected, our attribute-conditional beam search takes longer than the baselines that do not involve uncertainty, with the benefit of generating more factual responses.

Reviewer_Fs35OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Paper addresses long-form uncertainty in LLM by scoring uncertainty stepwise from the next-token distribution.

Contributions is the embedding the alternative tokens at each step by meaning and by attribute, using two EmbeddingGemma encoders fine-tuned with LoRA on WordNet-built pairs. It uses the conditional kernel von Neumann entropy H(M|A) to separate contradiction inside an attribute from benign variation across attributes, and an attribute-conditional risk to prune beams in decoding.

It is validated on 8 baselines, 3 LLMs, 4 datasets, 200 biographies for decoding, and no seeds or intervals.

Critical strengths
  1. A clear idea with built-in controls (VNE_M and SE_M|A) and stated limits.

  2. The method needs only the next-token distribution and two encoders, and its decoding variant gives the highest fraction of correct entities on Mistral.

Critical weaknesses
  1. Limited novelty, with prior work not cited. Each component has a published parent: embedding-kernel von Neumann entropy (semantic isotropy, Bhardwaj et al., arXiv 2510.21891, not cited, applied there to sampled responses), claim-type conditioning (CCP; with hard labels the risk in Eq. 10 reduces to 1 − CCP), and an uncertainty penalty in beam search (Xiao and Wang 2021, not cited). What remains is their combination at the token step with two trained encoders. The baselines do not get the same encoders, so it is unclear how much of the gain comes from the combination.

  2. The evidence is thin and partly unverifiable. No seeds or intervals are given, and at the entity level the lead is small or absent in several settings (best in 8 of 12). The supplement code hard-codes a number-token kernel rule not described in the paper, and its README names different encoders than the paper.

Questions for authors
  1. What does the method add beyond semantic isotropy (embedding-kernel VNE), CCP (claim-type conditioning) and Tomov et al. (Bayes risk), and beyond uncertainty-aware beam search (Xiao and Wang)? Can you show the improvement does not come only from the trained encoders, by giving CCP, KLE and the other baselines the same encoders?

  2. Which encoders produced the tables (EmbeddingGemma + LoRA as in App. C.1, or the MiniLM full finetune in the README), and on which data were the kernel scales, LoRA rank and function-word mode chosen?

  3. Can you give intervals over prompts and the number of independent runs for Table 2 and the decoding results.

3 comments
AuthorsOpenReview ↗

Response 1/2

We are thankful to the reviewer for taking the time to engage with our submission. Below, we try to address each of the comments individually.

W1 and Q1:

  • On the novelty: While our approach does share similarities with existing work, as elaborated in the paper, quantifying attribute-conditional semantic uncertainty in a stepwise manner that satisfies all three desiderata simultaneously (token-level, semantic, and structure-aware) poses challenges that neither naively combining existing approaches nor simply applying existing approaches to the token level can fully address. Existing entropy-based approaches (e.g., [1]) always lack structure awareness, whereas CCP, as the only structure-aware method, does not leverage the full underlying semantic distribution (even if applied in a stepwise manner, which is, of course, in contrast to its original post-hoc design). Thus, from a perspective of uncertainty representation and quantification, our contribution is a principled framework that, in contrast to existing works, satisfies all desiderata simultaneously, is theoretically sound, and is empirically effective.

  • On the missing references: We thank the reviewer for pointing out these two references, both of which have been cited in the updated manuscript. Regarding [1], it applies kernel-based VNE to sampled responses and, from this perspective, is closely related to the measure used in the Spectral uncertainty approach (Walha et al., 2025), which we already cite (see Table 1) and use as a baseline (VNE$_M$), though it employs a different kernel function. Regarding [2], it performs uncertainty-based beam search similar in spirit to our Section 4.3. However, the uncertainty signal used there is non-semantic, not structure-aware, and requires multiple models.

  • On the relation of Eq. 10 to CCP: It is true that for each potential candidate token, Eq. 10 reduces to 1- CCP_word (see Appendix B.1 for baseline details) when using hard cluster assignments. However, we see this connection as a strength of the framework, as our risk is a generalization that recovers CCP as a special case, while being a) graded (and thus arguably more realistic) and b) computationally more efficient than the 1-CCP analog. To further examine this connection, we added this CCP-based variant as an additional baseline during decoding. Both its lower fraction of correctness (Table 3) and its considerably higher computational cost (Appendix A.2, Figures 11+12) support our case for the above.

  • On the encoder advantage: The majority of baselines do not use encoders/embeddings in their original formulation (see Table 1 for an overview). Whenever a baseline does require an encoder comparable to ours, it receives exactly the same one as our method. For example, VNE$_M$, the localized version of Walha et al. (2025), uses our meaning encoder and thus ablates structure-awareness. The remaining kernel-based baselines (KLE, SD) derive similarities from NLI probabilities, and the clustering-based ones (CCP, SE$_M$, SE$_{M|A}$) rely on NLI labels.

    Nevertheless, we found this comment insightful and examined whether the baselines can also benefit from our encoders and our fine-tuning procedure through the following ablations:

    • Firstly, for all kernel-based baselines, we replaced the probability-based notion of similarity with an embedding-based distance approach using our meaning encoder.

    • Secondly, for methods that rely on hard clustering (CCP, SE$_M$, SE$_{M|A}$), we apply a thresholding strategy on top of the fine-tuned encoders to obtain an NLI-like signal, allowing NLI-based baselines to benefit from the same encoders as our approach.

    • Lastly, we fine-tune the original NLI model using the same training pairs and LoRA setup as for our encoder models.

    We run each of these variants in two settings. In the first, every baseline is used in its original form, meaning that CCP and STC keep their own function-word masking and all other baselines use none. In the second, baselines with a semantic similarity component use our function-word masking. We updated Table 2 with the best configuration for each baseline. Here, some baselines improve under these conditions (Table 2), but our method remains the strongest w.r.t. AUPR in the majority of settings. More specifically, VNE$_{M|A}$ is best in 18 of 24 settings in Table 2, and no baseline is best in more than 2.

[1] Dhrupad Bhardwaj et al., Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation

[2] Yijun Xiao, William Yang Wang, On Hallucination and Predictive Uncertainty in Conditional Language Generation

AuthorsOpenReview ↗

Response 2/2

W2 and Q3:

  • On seeds and intervals: If we understood the reviewer correctly, "intervals" refers to standard errors for AUROC or AUPR. Standard errors across seeds are not always reported in uncertainty quantification for LLMs (see Kuhn et al. (2023) and Walha et al. (2025)), and multiple independent runs are especially costly w.r.t. our very fine-grained evaluation. Nevertheless, to address this comment as well as the reviewer’s third question, we followed Nikitin et al. (2024) and added a bootstrapping approach. We resample the prompts with replacement 1000 times, recompute AUPR and AUROC on each resample, and report the standard error of those values. We sampled our prompts with a default seed (10 in our case), yielding a single independent run. Encoding and NLI models are deterministic.

  • On the entity-level results: We agree that the lead of VNE$_{M|A}$ on AUPR is smaller at the entity level, which we state in Section 5.1 and discuss in Appendix A.1. However, when interpreting these results, please note that entity spans are only identified post hoc by the evaluation pipeline and are not available during generation, so the token-level evaluation is the setting that matches the stepwise use case of our method. We nevertheless report both levels because, in an online setting, each reflects a requirement that an uncertainty measure should meet (see Section 5). At the token level, incorrect tokens have to be separated from both correct and non-entity tokens. Non-entity tokens tend to have high attribute uncertainty (Figure 4), so measures that are not attribute-aware report many false positives there, whereas VNE$_{M|A}$ suppresses this signal by design. The entity-level evaluation only compares entity tokens and thus removes exactly this source of false positives, which narrows the gap by construction. Even so, VNE$_{M|A}$ reports the highest AUPR in 7 of the 12 settings and is close to the best method in most of the remaining ones. Taken together, VNE$_{M|A}$ is the only method that meets both requirements consistently across datasets and models. The baselines that match or exceed it at the entity level fall clearly behind at the token level, thereby failing to suppress benign steps.

  • On the code: We thank the reviewer for the careful look at our supplementary material. It is worth mentioning that our code is a snapshot of the research codebase we developed throughout this project and therefore still contains options that we explored along the way, such as alternative masking rules and encoder models, not all of which are part of the final method. We will clean the code accordingly and align the README with the paper for the final version.

Q2: All reported tables were produced with the encoders described in Appendix C.1, i.e., EmbeddingGemma-300M with LoRA adapters for both the meaning and the attribute encoder. The LoRA rank (16 for both encoders) was selected on the test split of our fine-tuning data using the margin criterion described in Appendix C.1. This data is disjoint from all downstream evaluation data in Section 5. The kernel scales γM=1\gamma_M = 1 and γA=7\gamma_A = 7 are a single default shared across all models and datasets, which we set based on the sensitivity analysis on FS and LFo in Figure 8 (Appendix A.1). They are not tuned per setting. For example, at the token level, γM=0.1\gamma_M = 0.1 would be the better choice in 5 of the 6 panels. We note that our function-word masking (Appendix C) was not obtained through tuning. It was derived from the design of our attribute-aware representation (see Section 4.1 and Section 4.2). None of the reported runs contain the number-token kernel rule, which is thus in line with the components reported in the paper.

Dear reviewer,

As the discussion period ends tomorrow, we wanted to check whether our responses addressed your concerns. We are happy to answer any remaining questions. If our rebuttal addresses your concerns, we would greatly appreciate it if you would consider updating your assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank all reviewers for their careful and constructive feedback. We have carefully addressed all concerns raised by the reviewers, and wherever requested, we have added further experiments or additional results to elaborate on their questions and comments. All changes are included in the updated manuscript and are highlighted in blue. We summarize the additional results as follows:

To further improve baseline performance, we:

  • Provided a fine-tuned NLI model and our fine-tuned encoders to the baselines
  • Applied our function word masking across baselines

→\rightarrow Despite these improvements, our approach remains superior in the large majority of settings (18 of 24, vs. 20 of 24 originally), and no baseline is best in more than 2 settings (Table 2).

Furthermore, we added:

  • Bootstrapped standard errors for the reported results (Tables 2, 3, 4, 5; Figures 6, 11)
  • A direct evaluation of the meaning and attribute encoders on our test data, confirming the fine-tuned encoders work as intended (Table 8, Table 9)
  • A meaning-only risk and a CCP-based risk as additional decoding baselines, with our attribute-conditional approach still yielding the highest fraction of correct entities (Table 3)
  • A deep dive into the underlying mechanism of our beam search approach, shedding light on how it achieves higher factual correctness (Table 6)
  • A comparison of the computational cost of the decoding methods, to further clarify the cost-performance trade-off across methods (Figures 11, 12)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.