acceptodds
Under review as a conference paper at ICLR 2027

Demystifying and Resolving the Pathologies of Linguistic Uncertainty in Large Language Models

Abstract

Across diverse languages, speakers naturally convey uncertainty through epistemic hedges before asserting claims; however, uncertainty quantification (UQ) in large language models (LLMs) relies almost exclusively on post-hoc numerical scores. Can LLMs learn human-like, verbalized uncertainty? To investigate this, we systematically analyze model behavior along two primary axes: confidence placement (pre- vs. post-assertion) and modality (numeric scores vs. linguistic hedges). Our analysis uncovers two critical pathologies: (i) pre-assertion confidence acts as an ungrounded prior that is statistically decoupled from the downstream assertion; and (ii) linguistic hedging is acutely brittle under prompt paraphrasing. Consequently, naive epistemic hedging severely degrades model reliability. To remedy these deficiencies, we propose Stratified, a framework that decomposes generation into distinct epistemic and linguistic tiers by first committing to a statement and its numeric confidence before verbalizing the hedged assertion. Furthermore, to elicit fine-grained hedging across long-form responses, we construct WikiEU, a benchmark spanning diverse, factually verifiable queries, alongside an optimization recipe combining confidence-balanced supervised fine-tuning and reinforcement learning with a gated reward mechanism. Across three benchmarks and two models, our approach substantially improves verbalized uncertainty expression without sacrificing factual fidelity.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Uncertainty Quantification, Epistemic Hedging, Calibration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_K6RTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies confidence communicated before or after an answer, as numbers or verballly communicated. stratified first generates a claim and numerical confidence, then a hedged version of the claim. For responses with multiple claims, it introduces wikieu, confidence balanced SFT and gated statement level RL. Results show improved confidence ranking and prompt robustness over direct hedging, with mixed changes in accuracy and ECE.

Critical strengths

The AUROC improvements are good, around 10 to 12 points on wikieu after RL, with improvements on uncle for the models. results and analyses throughout the paper support these gains.

The controlled comparison of confidence position and modality is useful. I particularly liked the prompt perturbation experiment, where direct verbal hedging degrades substantially more than numeric confidence or stratified.

Critical weaknesses

1.The inference overhead is substantial. Table 5 reports up to 120%~ more generated tokens for the final RL models, roughly twice the baseline. This is a heavy price.

  1. I think it's plausible to ask for a comparison against the confidence estimation and rewriting approach of [Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs], also cited in this paper, since they seem like a very related method. I think both uncertainty performance and the number of generated tokens (a weakness of this work) should be compared.

3.the transfer tests measure shifts within factual QA, as the authors acknowledge. generalization to math for example, or other domains isn't clear. This limits what the results establish about the method's practicality outside factual QA.

  1. smaller point: I would soften Section 3.2's claim that reliable pre answer confidence must steer the answer distribution. I think I understand the desired causality here, but if the confidence is low, for example, I wouldn't want it to steer the model away from success: I would rather have a miscalibrated correct answer than a wrong calibrated one.
Questions for authors

a non-trained baseline: could you report accuracy, AUROC, ECE and Brier for the original instruction models with appropriate prompting but no additional training? Appendix D reports their hedging distributions, but not the main performance metrics. I think that's an important baseline.

5 comments

Dear Reviewer,

Thank you for your constructive feedback and thoughtful comments. Below, we address each of your points and provide the requested experimental comparisons.


  1. Inference overhead and comparison against the rewrite approach (W1 & W2)

We agree that the extra tokens from structured tags represent a practical trade-off, which we explicitly analyze in Section 6 and Appendix F (Table 5). Note that all XML tags are stripped prior to user presentation, so the user-facing response length remains concise (122 vs. 126 tokens on UNCLE) while delivering substantial calibration gains.

Regarding RLMF (Liu et al., 2026), it shares a similar high-level intuition by separating confidence estimation and verbal rewriting, whereas STRATIFIED integrates both into a single model and a single LLM call. We did not include RLMF in the main tables because it targets internal belief alignment (faithful confidence) rather than empirical correctness (factual confidence), as discussed in Section 5. Following your suggestion, we evaluate RLMF using its official codebase on the exact same WikiEU split and apply our stronger judge models (Appendix I) to all methods for a fair comparison. We also evaluate our HG-RL under PostNum and a two-step rewrite baseline. When testing rewriting with the trained 7B Qwen model itself, the output quality degraded severely, confirming that post-hoc rewriting requires a much stronger external model. Since Gemini-2.5-Flash-Lite used in RLMF is no longer available to new users, we use GPT-5.6-Luna consistently as the rewriter:

WikiEU

MethodAccAUROCECEBrier
RLMF (PostNum)0.5890.7490.1830.226
RLMF (PostNum+PreVerb Rewrite)0.5530.7450.1510.222
LoVeC (PostNum)0.5630.7500.1630.229
HG-RL (PostNum)0.6120.8110.0750.173
HG-RL (PostNum+PreVerb Rewrite)0.5860.7980.1550.203
STRATIFIED (HG-RL, Table 2)0.6010.8270.0780.164

UNCLE

MethodAccAUROCECEBrier
RLMF (PostNum)0.4050.7700.1800.226
RLMF (PostNum+PreVerb Rewrite)0.3860.7900.2050.226
LoVeC (PostNum)0.3930.7600.1090.196
HG-RL (PostNum)0.4170.8250.1380.188
HG-RL (PostNum+PreVerb Rewrite)0.3960.7990.2480.244
STRATIFIED (HG-RL, Table 2)0.4270.8040.1630.204

In terms of inference cost on UNCLE, RLMF requires 297 total tokens across two separate model calls (including a large external rewriter), which is comparable to or higher than STRATIFIED (277 tokens in a single 7B model call). Moreover, end-to-end STRATIFIED consistently outperforms two-step rewriting in both accuracy and calibration.

We also examine why the rewriting step in RLMF leads to a larger performance drop than expected. Specifically, RLMF provides a predefined list of candidate phrases for each confidence bin in its prompt, but several mappings do not align well with perceived certainty (for example, "just may be" is placed in the [0.05,0.1][0.05, 0.1] bin while our judge assigns it 0.35, whereas "I'm at a total loss" is placed in the [0.55,0.6][0.55, 0.6] bin while our judge assigns it 0.10). Thus, post-hoc rewriting remains sensitive to prompt design and requires careful calibration in future work. We will include these results and findings in the revision.

  1. Generalization to other domains and languages (W3)

To test transfer beyond English general-knowledge QA, we directly evaluate our TriviaQA-trained Qwen2.5-7B-Instruct models out-of-distribution on BioASQ-Factoid (English biomedical QA) and Chinese SimpleQA (Chinese prompts):

BioASQ-Factoid (English Biomedical QA)

MethodAccAUROCBrierECE
PRE-NUM0.4960.7630.2300.191
POST-NUM0.5040.8730.1420.064
PRE-VERB0.5060.8090.1700.057
POST-VERB0.5080.8210.1680.062
STRATIFIED NUM0.5100.8490.1610.085
STRATIFIED VERB0.5100.8350.1640.056

Chinese SimpleQA (Chinese Prompt)

MethodAccAUROCBrierECE
PRE-NUM0.2740.8030.2130.238
POST-NUM0.2480.8890.1390.190
PRE-VERB0.2700.7520.1980.177
POST-VERB0.2870.7690.2190.213
STRATIFIED NUM0.2640.8480.1420.147
STRATIFIED VERB0.2640.8230.1460.130

Across both new domains, STRATIFIED substantially outperforms direct verbal hedging (PRE-VERB and POST-VERB) in AUROC and Brier score while matching the perturbation stability of numeric confidence. Regarding domains such as math reasoning, our work focuses on expressing uncertainty for each individual claim within a response rather than assigning a single confidence score at the overall response level, which aligns naturally with long-form generation. In contrast, prior UQ studies on math evaluate uncertainty only at the final-answer level, while intermediate reasoning steps lack atomic ground-truth labels and a uniform correctness definition. Therefore, following recent long-form UQ work (Yang et al., 2025, Zhang et al., 2026), we focus on factually verifiable tasks where each individual statement can be rigorously evaluated.


  1. Claim in Section 3.2 (W4)

Thank you for your insightful comment. We share the same view on this phenomenon, which is also why Pre performs significantly worse than Post. Our intended point is that for pre-assertion confidence to be calibrated, the model would need to already determine its upcoming answer when selecting the confidence prefix so that the two are consistent, in which case the downstream answer probabilities would naturally shift across different confidence prefixes. We will follow your suggestion to soften and clarify this claim in Section 3.2.


  1. Non-trained baseline (Q1)

In our early exploration, we tested non-trained baselines. Although they can follow the formatting instructions, their uncertainty estimation is too poor to report meaningfully (with AUROC near 0.50). To draw solid conclusions about the pros and cons of each paradigm without confounding them with weak base capabilities, we optimize all formats throughout the paper. We choose RL over SFT because the RL calibration objective is unbiased and achieves superior performance. As you noted, we only analyze the hedge word distribution of non-trained models in Appendix D to show how SFT enriches expression diversity and provides a good starting point for RL.

Thank you for the additional experiments, these help, although I still have concerns

1.The inference cost is still significant. removing the XML before showing the response does not remove the cost of generating those tokens nor the repeated claim. I accept the comparison with the rewriting method, but 277 tokens compared with 126 for the direct hedging baseline is still a heavy price. Besides the comparison, you didn't quite address it (and I understand there's possibly nothing to be done or said about this cost, as it's the price for using the method).

2.The new comparison also makes the benefit of stratified over numerical confidence less clear to me. HG RL with postnum performs better on uncle in AUROC, ECE and Brier. I understand that stratified produces verbal hedges, but if the goal is uncertainty estimation, I am not convinced why I should prefer it over the numerical alternative. I think the gains from the training procedure and the benefit of verbalization should be discussed a bit more clearly: verbalization for users' understanding sake, and not for better uncertainty estimates. It's fine if this is the objective, all I'm saying is that it needs to be clear.

3.the new transfer experiments help, but still remain within factual QA. Math, coding or other types of tasks could have helped establish broader transfer. I think there is now reasonable evidence for transfer between similar tasks , but I am still not convinced about transfer to other domains.

4.I still think the original instruction models should be evaluated with the main metrics. It is only helpful for motivating your method to show that it's poor, so I don't understand why not to include it. Even if their performance is very poor, if you've already done the evaluations, they should be reported. Being too poor to report is not a convincing reason IMO.

I am raising my score to weak accept, but I feel barely above the acceptance threshold. The AUROC results and analyses are good, the idea is interesting, and the additional experiments strengthen the paper. I am not convinced about all the points above and still find the inference cost hard to justify for practical use. Nevertheless, I think there is enough useful work here to help move the field in the right direction, and your additional experiments had helped convince me of that. I still hope you would address my other points in the revision.

I also looked at other reviews, and was not convinced of any other important issues besides the ones I've described.

Reviewer_CFeTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies and evaluates linguistic uncertainty in LLMs across two dimensions: Pre- / post-asserted, and verbalized / numerical. It shows that pre-asserted numerical uncertainties are a largely irrelevant prior and that verbalized post-asserted uncertaintites are brittle to syntax. The authors develop a method that pre-asserts confidence together with a factual statement to mitigate these issues. To train their method they introduce confidence-balanced rewards and a gating for longer multi-factual generations and evaluate this on a newly curated benchmark.

Critical strengths
  • The paper attempts to solve a well-known limitation of verbalized confidence scores.
  • The papers makes many individual contributions: an insightful study of existing paradigms, a novel method, and a long-form benchmark.
Critical weaknesses
  • The paper's scope, and consequently its writing, are unfocused: It attempts many things at the same time (novel method, novel benchmark, CB-SFT, HG-RL) but keeps the corresponding evaluations and explorations relatively shallow: There are no baselines apart from the four corresponding prompt structures originating from the two axis (verbalized / numerical, pre- or post-faced). Hence, the paper lacks substantial context about the quality of the verbalized uncertainty in comparison to other uncertainty quantification approaches.
  • The novelty of the method STRATIFIED is limited: Instead of pre-asserting with numerical confidence alone, it also prefaces with a factual statement jointly (I have questions here, see below). Also, the significance of the other contributions is small: Reward-weighting to solve imbalance, and reward gating are both not novel ideas. The benchmark is useful, but it is unclear to me why it is needed in the first place (see below). Also the performance of STRATIFIED over the baseline (Table 2) is not convincing enough to assess the method as highly relevant (especially on Llama it seems closer to a tie than strict dominance of STRATIFIED). Overall, the paper only has many small contributions instead of a significant large advancement.
Questions for authors
  • For both Pre-Filling and STRATIFIED: How is this conditioning enforced? Do you have two separate LLM calls to first generate the condition (numerical score / (numerical score, fact)) or do you use a prompt to enforce the LLM to output in this format?
  • STRATIFIED: If the LLM is prompted to generate the tuple (numerical score, fact): Since generation is, again, sequential, aren't you again doing something very similar to pre-filling? After all, the tuple can not be truly generated simultaneously when sampling autoregressively, so you again first sample a numerical score, then sample a fact, then sample the free-form statement. I lack an intuition why this pre-filling-like approach actually should outperform vanilla pre-filling?
  • In Fig. 3 you show that pre-filling (both dimensions) largely do not affect the accuracy of the generated fact. This is an interesting finding. Later you claim that STRATIFIED solves this issue (Fig. 8), but do not show the average accuracy for each confidence bucket but only the sentence probability. I would argue that for the scope of the paper the accuracy is the far more relevant metric, how does this curve look like? Does STRATIFIED solve the issue of pre-filling? If so, why (see previous question)?
  • In general, why do we care about sentence probabilities at all in this work (Fig 3.)? Isn't accuracy the relevant metric as it aligns with downstream tasks and the RL metrics you introduce?
  • Fig 4., Table 2: What does AUCROC quantify here? Is it confidence predicting correctness AUCROC?
  • Fig 4.: To me it seems that PostNum actually performs very similar to STRATIFIED.
  • WikiEU: How are generated responses (by any method) decomposed into the individual facts? How are the individual statements aggregated to get the performance on a single entity?
  • I am sceptical as to why we need STRATIFIED to generate a free-form response in the first place. Isn't the tuple (confidence, fact) sufficient already or could we not just use any other auxiliary LM to sample on of the pre-defined valid verbalized hedges for the given confidence level to generate a free-form response? Particularly for multi-fact statements (Figure 7), the output is fragmented would need to be processed to be used further (I assume this is done anyway to score performance).
2 comments

Dear Reviewer,

Thank you for your thoughtful comments and detailed questions. We noticed a minor misunderstanding regarding the generation order in STRATIFIED, which we clarify below along with your other points. We will revise the manuscript accordingly to make these mechanisms clearer.


  1. The focus of this research and comparison with previous work (W1)

In this work, we study how LLMs express uncertainty in SVO languages (e.g., English). In Part I (Section 3), we analyze how linguistic expressions (typically PreVerb) perform compared with the standard paradigm in machine learning (PostNum) across the axes of modality and position. We propose STRATIFIED to resolve the failures of PreVerb, starting with single-claim responses in Part I and extending to multi-claim long-form responses in Part II (Section 4).

We have already compared several existing paradigms in our study, including PostNum (RLCR, Damani et al., 2026), SFT (using rollout agreement, Chaudhry et al., 2025, Eikema et al., 2025), and Global-RL (sequence-level reward optimization, Band et al., 2024, Zhang et al., 2026). Because prior studies target different objectives, we adapted them into a unified framework for factual linguistic uncertainty (discussed in Section 5). For direct comparisons with recent long-form methods, we provide new results below.


  1. WikiEU and Methods (W2)

WikiEU (Section 4.1, Appendix A) is built to enable verifiable training and evaluation for long-form generation. Evaluating multi-claim responses requires ground-truth labels for each atomic claim. Relying on web search engines is too costly for RL rollouts and introduces retrieval noise, whereas WikiEU avoids both issues by grounding queries in verified Wikidata triplets. Regarding CB-SFT (Section 4.2) and HG-RL (Section 4.3), prior long-form UQ methods overlook mid-band collapse and statement-level credit assignment, while our analyses (Figure 5 and Figure 6) show that both are essential.

As suggested by you and other reviewers, RLMF (Liu et al., 2026) and LoVeC (Zhang et al., 2026) are the most closely related baselines. Below, we report results using their official codebases on the same WikiEU split. We applied our stronger judge models (Appendix I) to all methods for a fair evaluation, as their original judges are less accurate. Note that RLMF originally aligns hedges with internal model beliefs (faithful confidence) rather than empirical correctness (factual confidence), as discussed in Section 5. We also evaluate HG-RL under PostNum to directly isolate the gain of our training algorithm:

WikiEU

MethodAccAUROCECEBrier
RLMF (PostNum)0.5890.7490.1830.226
RLMF (PostNum+PreVerb Rewrite)0.5530.7450.1510.222
LoVeC (PostNum)0.5630.7500.1630.229
HG-RL (PostNum)0.6120.8110.0750.173

UNCLE

MethodAccAUROCECEBrier
RLMF (PostNum)0.4050.7700.1800.226
RLMF (PostNum+PreVerb Rewrite)0.3860.7900.2050.226
LoVeC (PostNum)0.3930.7600.1090.196
HG-RL (PostNum)0.4170.8250.1380.188

  1. Results of Llama in Table 2 (W2)

Regarding the performance on Llama-3.1-8B in Table 2, we respectfully note that it is far from a tie on uncertainty calibration. STRATIFIED substantially outperforms BASELINE on confidence discrimination (AUROC improves by +0.116, from 0.672 to 0.788) and Brier score (0.171 vs. 0.195), and achieves consistent improvements across all calibration metrics on UNCLE (AUROC from 0.730 to 0.762, ECE from 0.189 to 0.169, and Brier from 0.242 to 0.225).


  1. Implementation of prefilling and STRATIFIED (Q1 & Q2)

In the prefilling experiments (Section 3.2, Appendix C), we programmatically insert the confidence prefix (or candidate answer) into the context and let the trained model complete the rest. For STRATIFIED, the model generates the entire response in a single call rather than separate calls, learning the format and hedge alignment via the reward in Eq. 3.

Crucially, the serialization order in STRATIFIED (Figure 1) is <fact statement="X" confidence="85"> I'm quite sure the answer is X </fact>. The model generates the factual statement X first, then the numeric confidence 85, and finally the fronted verbal hedge. While Part I shows that post-assertion numeric confidence is far more reliable, head-initial verbal hedges (PreVerb) are more natural in SVO languages. STRATIFIED bridges both by first committing to the claim and its posterior numeric confidence (effectively PostNum), and then verbalizing the hedged sentence (PreVerb), which is fundamentally different from prefilling.

  1. Difference between Fig. 3 and Fig. 8 (Q3 & Q4)

Note that prefilling is strictly an analytical probe rather than an inference method, as a deployed model generates the full response on its own. Importantly, the confidence-prefilling experiment in Figure 3 (Section 3.2) can only be conducted on PreNum and PreVerb because they generate confidence before the answer. Both Post models and STRATIFIED generate the factual statement first and the confidence afterward, making it impossible to prefill confidence prior to the answer. Instead, their corresponding causal intervention is shown in Figure 8 (Appendix C), where we prefill candidate answers with different empirical probabilities and measure the resulting shift in emitted confidence.

Figure 3 (Section 3.2) shows that prefilling confidence scores or hedge words has virtually no effect on downstream generation for PreNum and PreVerb, as both the answer probability and overall accuracy remain flat. Conversely, Figure 8 (Appendix C) shows that prefilling candidate answers with varying empirical probabilities causally shifts the subsequent confidence scores and hedges in PostNum, PostVerb, and STRATIFIED. Together, these results prove that under pre-assertion formats, confidence cc and answer yy lack a causal link, causing frequent mismatches. In contrast, a Post model conditions cc on the realized answer yy and lowers its confidence when yy is uncertain. We include sequence probability alongside accuracy in Figure 3 because probability provides a continuous, fine-grained signal of distributional shifts that binary accuracy can mask.


  1. The metric AUROC (Q5)

Yes, AUROC measures how well the extracted scalar confidence discriminates between factually correct and incorrect statements across decision thresholds, which is a standard metric in uncertainty quantification (Section 2.1). We will add explicit definitions of AUROC, ECE, and Brier score in Section 2.1.


  1. The difference between PostNum and STRATIFIED (Q6)

Our goal is to improve natural linguistic uncertainty (PreVerb) rather than raw numeric readouts (PostNum). Figure 4 (Section 3.3) shows that numeric confidence is much more stable than direct verbal hedging under prompt perturbations. By predicting the numeric confidence internally before rendering the verbal hedge, STRATIFIED inherits the perturbation stability of PostNum while outputting natural hedged language.


  1. The separation of individual claims and evaluation (Q7)

Following standard practice in long-form factuality evaluation, we evaluate at the atomic statement level (Section 4.1, Appendix I). Each WikiEU prompt corresponds to a set of ground-truth Wikidata triplets. After stripping all XML tags from STRATIFIED outputs, the judge matches each ground-truth triplet to its corresponding statement in the generated paragraph and extracts the implied confidence (Appendix I). All metrics (Acc, AUROC, ECE, and Brier) are computed across individual statements over the test set rather than aggregated per entity.


  1. Using two-step generation (Q8)

Generating the claim and numeric confidence first and then converting them into hedged text is indeed the core idea of STRATIFIED, except we execute it within one model and one LLM call. RLMF (Liu et al., 2026) takes the two-step approach you described by using one model to predict the answer and confidence, and a second LLM call to rewrite it into hedged prose (evaluated in Section 2 above). While conceptually similar, relying on two separate models or calls adds inference latency and deployment costs. For multi-fact outputs (Figure 7), the XML tags are deterministically stripped before presentation, leaving a fluent paragraph for end users.

Reviewer_1FnpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors in this papers look at how language models can express uncertainty in words, the way people do, instead of with a number. They use RL on TriviaQA and compare confidence placed before vs. after the answer, and expressed as a number vs. words. They report two problems as the confidence stated before the answer does not track whether the answer is correct, and word-based confidence breaks down when the prompt is reworded. Then they propose STRATIFIED, which first writes a claim with a numeric confidence and then turns it into a hedged sentence. They extend this to long answers with a new Wikidata benchmark, confidence-balanced SFT, and a gated RL reward.

Critical strengths
  • I think the problem is useful and in general people read hedges, not probabilities, so making hedges reliable is useful in practice.
  • The prompt-rewording result is the interesting and the authors show that the word-based confidence drops by while numeric confidence doesn't move.
  • The authors back up their claims with paired bootstrap tests, a judge swap and ablations for CB-SFT and the gated reward.
Critical weaknesses
  • Their method looks close to earlier work, for example Liu et al. (2026) already split numeric confidence from verbal hedging, and RL for verbal confidence in long answers has been explored before (e.g., Band et al., 2024, Zhang et al. 2026). The paper could explain more clearly what is new in comparison. Also if the model states its confidence before writing the answer, it has not seen the answer yet, so it makes sense that the confidence does not track whether that answer is right.

  • The comparisons in the paper are against the authors' baseline format and trained with their recipe. Prior method for verbal or long-form calibration in the results isn't there, and no standard confidence method such as answer agreement across samples. Without these how strong the numbers are cannot be explained.

  • Part I relies on a single model and a single dataset, which feels thin for claims presented as general "pathologies."

  • GPT-OSS-120B is used as judge and it converts hedges into numbers during RL training and again during evaluation. Also, the SFT data and the judge prompt both rely on the authors' own phrase-to-confidence tables. So the model could just be learning which phrases this particular judge likes, not hedges that people would interpret the same way? The human check covers 16 fixed phrases in single TriviaQA sentences, not the model's real long-form outputs.

  • The proposed method does not beat the post-answer formats in Part I, for example in Table 1, the AUROC is lower than both. Its seems the real advantage is robustness to prompt changes. In Part II, the baseline hedges directly, and the post-answer formats that did best in Part I are never tested on long answers. Because of this, the long-form gains can come from STRATIFIED or from putting confidence after the claim, one cannot tell.

  • The evaluation only covers short factual recall (TriviaQA, WikiEU, UNCLE, and closed-book HotpotQA), in English, with 7–8B models. I'm not sure the findings carry over to reasoning tasks, open-ended answers, or high-stakes areas like medical QA. The introduction talks about hedging across languages, but only English is tested. Also, the HotpotQA AUROC gains are small (at most 0.014), so I think generalizes robustly overstates the result.

Questions for authors
  • After training how often does the hedge disagree with the hidden numeric confidence? In those cases, which one is more accurate?
  • Since the tags and numbers are hidden from the user, is understanding the hedges better than understanding the number?
4 comments

Thank you for your thoughtful comments and constructive suggestions. Below, we address your concerns in detail and clarify our contributions.

  1. Focus of this research and comparison with existing work (W1 & W2 & W5)

We focus on how LLMs express uncertainty in SVO languages (e.g., English), which typically place hedges before claims (PreVerb), rather than post-hoc numeric scores (PostNum) commonly studied in machine learning. After uncovering the pathologies of pre-assertion and direct verbal hedging in Part I (Section 3), we propose STRATIFIED to first commit to a claim and its posterior numeric confidence (PostNum) before generating the hedged sentence (PreVerb).

Regarding W1, pre-assertion confidence could in principle still be calibrated if the model implicitly inferred the correlation between its planned answer and confidence to select a consistent pair, whereas Figure 3 (Section 3.2) shows that they remain decoupled in practice (∣r∣<0.1\vert{}r\vert{} < 0.1).

Compared with existing work, RLMF (Liu et al., 2026) adopts a related two-step idea by separating prediction and rewriting across two separate LLM calls (e.g., Qwen plus Gemini), without diagnosing why direct hedging fails. Moreover, RLMF focuses on aligning hedges with internal model beliefs (faithful confidence) rather than empirical correctness (factual confidence), as discussed in Section 5. In contrast, LoVeC (Zhang et al., 2026) and Band et al. (2024) target factual calibration in long-form generation, but they rely on standard SFT or monolithic sequence-level rewards and focus primarily on conventional post-hoc confidence rather than fronted verbal hedging (PreVerb). Since we focus on single-pass training methods rather than multi-sample inference, we already compared against SFT using rollout answer agreement as the proxy and Global-RL in Part II (Table 5 and Figure 6), showing that our CB-SFT and HG-RL design is essential to prevent mid-band collapse and resolve statement-level credit blurring.

Following your suggestion, we further evaluate RLMF (Liu et al., 2026) and LoVeC (Zhang et al., 2026) using their official codebases on the exact same WikiEU split. We apply our stronger judge models (Appendix I) to all methods for a fair evaluation, as their original judges are less accurate. We also report HG-RL under the PostNum format as well as a two-step rewrite baseline. While RLMF originally uses Gemini-2.5-Flash-Lite for rewriting, that model is no longer accessible to new users, so we use GPT-6-Luna consistently as the rewriter (we also tested using the trained 7B model itself as the rewriter, which performed poorly, confirming that post-hoc rewriting requires a much stronger external model):

WikiEU

MethodAccAUROCECEBrier
RLMF (PostNum)0.5890.7490.1830.226
RLMF (PostNum+PreVerb Rewrite)0.5530.7450.1510.222
LoVeC (PostNum)0.5630.7500.1630.229
HG-RL (PostNum)0.6120.8110.0750.173
HG-RL (PostNum+PreVerb Rewrite)0.5860.7980.1550.203
STRATIFIED (HG-RL, Table 2)0.6010.8270.0780.164

UNCLE

MethodAccAUROCECEBrier
RLMF (PostNum)0.4050.7700.1800.226
RLMF (PostNum+PreVerb Rewrite)0.3860.7900.2050.226
LoVeC (PostNum)0.3930.7600.1090.196
HG-RL (PostNum)0.4170.8250.1380.188
HG-RL (PostNum+PreVerb Rewrite)0.3960.7990.2480.244
STRATIFIED (HG-RL, Table 2)0.4270.8040.1630.204

As shown above, HG-RL substantially outperforms RLMF and LoVeC, confirming the advantage of our training recipe. Comparing STRATIFIED against HG-RL (PostNum) also directly addresses W5, confirming that committing to PostNum prior to verbal expression is crucial for reliable long-form hedging. Even when pairing HG-RL (PostNum) with a strong external rewriter, accuracy and calibration still degrade compared with end-to-end STRATIFIED.

We also examine why the rewriting step in RLMF leads to a larger performance drop than expected. Specifically, RLMF provides a predefined list of candidate phrases for each confidence bin in its prompt, but several mappings do not align well with perceived certainty (for example, "just may be" is placed in the [0.05,0.1][0.05, 0.1] bin while our judge assigns it 0.35, whereas "I'm at a total loss" is placed in the [0.55,0.6][0.55, 0.6] bin while our judge assigns it 0.10). Thus, post-hoc rewriting remains sensitive to prompt design and requires careful calibration in future work. We will include these results and findings in the revision.

  1. Extension of Part I to more models, languages, and domains (W3 & W6)

Thank you for this suggestion. We have extended Part I to Llama-3.1-8B-Instruct on TriviaQA, and also evaluated our trained Qwen2.5-7B-Instruct checkpoints out-of-distribution on BioASQ-Factoid (medical QA in English) and Chinese SimpleQA (Chinese prompt):

Llama-3.1-8B-Instruct (TriviaQA)

MethodAccAUROCBrierECE
PRE-NUM0.7980.8130.1210.019
POST-NUM0.7920.9000.1050.092
PRE-VERB0.7990.7460.1320.057
POST-VERB0.8050.8690.0950.036
STRATIFIED NUM0.8080.8850.0920.026
STRATIFIED VERB0.8080.8510.0950.025

BioASQ-Factoid (English Medical QA)

MethodAccAUROCBrierECE
PRE-NUM0.4960.7630.2300.191
POST-NUM0.5040.8730.1420.064
PRE-VERB0.5060.8090.1700.057
POST-VERB0.5080.8210.1680.062
STRATIFIED NUM0.5100.8490.1610.085
STRATIFIED VERB0.5100.8350.1640.056

Chinese SimpleQA (Chinese Prompt)

MethodAccAUROCBrierECE
PRE-NUM0.2740.8030.2130.238
POST-NUM0.2480.8890.1390.190
PRE-VERB0.2700.7520.1980.177
POST-VERB0.2870.7690.2190.213
STRATIFIED NUM0.2640.8480.1420.147
STRATIFIED VERB0.2640.8230.1460.130

Furthermore, our two core findings hold consistently across both BioASQ and Chinese SimpleQA. First, pre-assertion confidence remains decoupled from downstream generation, with all prefill correlations across numeric and verbal prefixes satisfying ∣r∣≤0.072\vert{}r\vert{} \le 0.072. Second, direct verbal hedging remains brittle under prompt paraphrasing, whereas STRATIFIED matches numeric robustness: POST-VERB suffers average AUROC drops of 0.081 on BioASQ and 0.155 on Chinese SimpleQA, compared with only 0.027 and 0.021 for STRATIFIED (and 0.007 and 0.023 for POST-NUM).

Regarding HotpotQA (Table 3), we agree that the AUROC gains are modest and will soften our wording in Section 4.4 to highlight that the primary improvement on multi-hop transfer is in calibration error (ECE and Brier score). As for broader open-ended long-form tasks, verifying isolated intermediate claims remains a well-known challenge, which is why we follow recent long-form UQ literature (Yang et al., 2025, Zhang et al., 2026) in focusing on factually verifiable claims.


  1. Judge circularity and human alignment (W4)

To ensure our results do not overfit to GPT-OSS-120B, we conducted judge-swap evaluations in Appendix G (Tables 6 and 7) using an independent model family (GPT-5.6-Luna), confirming that all trends remain consistent. Our human alignment validation (Section 2.2) follows established protocols from prior work (Yona et al., 2024, Liu et al., 2025, Liu et al., 2026), and 15 of the 16 human-annotated phrases (all except "likely") are never given to the judge model as in-context examples (Appendix I), supporting its ability to generalize to diverse natural expressions.


  1. Alignment between numeric confidence and verbal expression (Q1 & Q2)

Because Eq. 3 directly rewards consistency between the internal numeric confidence cmc_m and the emitted hedge uu, RL reduces their mean absolute discrepancy ∣fϕ(u)−cm∣\vert{}f_\phi(u) - c_m\vert{} from 0.075 (CB-SFT) to 0.050. When a minor discrepancy occurs, the numeric confidence cmc_m is slightly more accurate because it directly optimizes the continuous Brier objective without lexical quantization loss. Unlike Liu et al. (2026), which requires an extra model call to rewrite outputs, STRATIFIED handles both steps in a single forward pass (comparison results are shown above).

Regarding the user-facing modality (Q2), we believe that natural linguistic hedges are more intuitive and readable than raw percentages in everyday human-AI communication, whereas raw numeric scores can be surfaced in specialized high-stakes interfaces where exact probabilities are preferred.

Thank you for these clarifications and additional experiments. They address my concerns and I have no further questions, I'll maintain my score based on the results being somewhat on par in most cases with numerical methods.

Thank you for your reply and for confirming that our responses have addressed your concerns. We would like to briefly clarify the positioning of our work: our primary focus is on improving natural linguistic uncertainty expression (PreVerb) rather than conventional post-hoc numeric confidence (PostNum), with our core contribution lying in uncovering why PreVerb substantially underperforms PostNum and resolving these pathologies. Compared with prior two-stage approaches that first predict PostNum and then rely on an external model (such as Gemini) to rewrite it into PreVerb, STRATIFIED achieves reliable linguistic uncertainty end-to-end within a single model. Moreover, our proposed HG-RL algorithm also generalizes directly to PostNum, significantly outperforming existing methods.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.