- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies confidence communicated before or after an answer, as numbers or verballly communicated. stratified first generates a claim and numerical confidence, then a hedged version of the claim. For responses with multiple claims, it introduces wikieu, confidence balanced SFT and gated statement level RL. Results show improved confidence ranking and prompt robustness over direct hedging, with mixed changes in accuracy and ECE.
Critical strengths
The AUROC improvements are good, around 10 to 12 points on wikieu after RL, with improvements on uncle for the models. results and analyses throughout the paper support these gains.
The controlled comparison of confidence position and modality is useful. I particularly liked the prompt perturbation experiment, where direct verbal hedging degrades substantially more than numeric confidence or stratified.
Critical weaknesses
1.The inference overhead is substantial. Table 5 reports up to 120%~ more generated tokens for the final RL models, roughly twice the baseline. This is a heavy price.
- I think it's plausible to ask for a comparison against the confidence estimation and rewriting approach of [Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs], also cited in this paper, since they seem like a very related method. I think both uncertainty performance and the number of generated tokens (a weakness of this work) should be compared.
3.the transfer tests measure shifts within factual QA, as the authors acknowledge. generalization to math for example, or other domains isn't clear. This limits what the results establish about the method's practicality outside factual QA.
- smaller point: I would soften Section 3.2's claim that reliable pre answer confidence must steer the answer distribution. I think I understand the desired causality here, but if the confidence is low, for example, I wouldn't want it to steer the model away from success: I would rather have a miscalibrated correct answer than a wrong calibrated one.
Questions for authors
a non-trained baseline: could you report accuracy, AUROC, ECE and Brier for the original instruction models with appropriate prompting but no additional training? Appendix D reports their hedging distributions, but not the main performance metrics. I think that's an important baseline.
5 comments
- Generalization to other domains and languages (W3)
To test transfer beyond English general-knowledge QA, we directly evaluate our TriviaQA-trained Qwen2.5-7B-Instruct models out-of-distribution on BioASQ-Factoid (English biomedical QA) and Chinese SimpleQA (Chinese prompts):
BioASQ-Factoid (English Biomedical QA)
| Method | Acc | AUROC | Brier | ECE |
|---|---|---|---|---|
| PRE-NUM | 0.496 | 0.763 | 0.230 | 0.191 |
| POST-NUM | 0.504 | 0.873 | 0.142 | 0.064 |
| PRE-VERB | 0.506 | 0.809 | 0.170 | 0.057 |
| POST-VERB | 0.508 | 0.821 | 0.168 | 0.062 |
| STRATIFIED NUM | 0.510 | 0.849 | 0.161 | 0.085 |
| STRATIFIED VERB | 0.510 | 0.835 | 0.164 | 0.056 |
Chinese SimpleQA (Chinese Prompt)
| Method | Acc | AUROC | Brier | ECE |
|---|---|---|---|---|
| PRE-NUM | 0.274 | 0.803 | 0.213 | 0.238 |
| POST-NUM | 0.248 | 0.889 | 0.139 | 0.190 |
| PRE-VERB | 0.270 | 0.752 | 0.198 | 0.177 |
| POST-VERB | 0.287 | 0.769 | 0.219 | 0.213 |
| STRATIFIED NUM | 0.264 | 0.848 | 0.142 | 0.147 |
| STRATIFIED VERB | 0.264 | 0.823 | 0.146 | 0.130 |
Across both new domains, STRATIFIED substantially outperforms direct verbal hedging (PRE-VERB and POST-VERB) in AUROC and Brier score while matching the perturbation stability of numeric confidence. Regarding domains such as math reasoning, our work focuses on expressing uncertainty for each individual claim within a response rather than assigning a single confidence score at the overall response level, which aligns naturally with long-form generation. In contrast, prior UQ studies on math evaluate uncertainty only at the final-answer level, while intermediate reasoning steps lack atomic ground-truth labels and a uniform correctness definition. Therefore, following recent long-form UQ work (Yang et al., 2025, Zhang et al., 2026), we focus on factually verifiable tasks where each individual statement can be rigorously evaluated.
- Claim in Section 3.2 (W4)
Thank you for your insightful comment. We share the same view on this phenomenon, which is also why Pre performs significantly worse than Post. Our intended point is that for pre-assertion confidence to be calibrated, the model would need to already determine its upcoming answer when selecting the confidence prefix so that the two are consistent, in which case the downstream answer probabilities would naturally shift across different confidence prefixes. We will follow your suggestion to soften and clarify this claim in Section 3.2.
- Non-trained baseline (Q1)
In our early exploration, we tested non-trained baselines. Although they can follow the formatting instructions, their uncertainty estimation is too poor to report meaningfully (with AUROC near 0.50). To draw solid conclusions about the pros and cons of each paradigm without confounding them with weak base capabilities, we optimize all formats throughout the paper. We choose RL over SFT because the RL calibration objective is unbiased and achieves superior performance. As you noted, we only analyze the hedge word distribution of non-trained models in Appendix D to show how SFT enriches expression diversity and provides a good starting point for RL.
Thank you for the additional experiments, these help, although I still have concerns
1.The inference cost is still significant. removing the XML before showing the response does not remove the cost of generating those tokens nor the repeated claim. I accept the comparison with the rewriting method, but 277 tokens compared with 126 for the direct hedging baseline is still a heavy price. Besides the comparison, you didn't quite address it (and I understand there's possibly nothing to be done or said about this cost, as it's the price for using the method).
2.The new comparison also makes the benefit of stratified over numerical confidence less clear to me. HG RL with postnum performs better on uncle in AUROC, ECE and Brier. I understand that stratified produces verbal hedges, but if the goal is uncertainty estimation, I am not convinced why I should prefer it over the numerical alternative. I think the gains from the training procedure and the benefit of verbalization should be discussed a bit more clearly: verbalization for users' understanding sake, and not for better uncertainty estimates. It's fine if this is the objective, all I'm saying is that it needs to be clear.
3.the new transfer experiments help, but still remain within factual QA. Math, coding or other types of tasks could have helped establish broader transfer. I think there is now reasonable evidence for transfer between similar tasks , but I am still not convinced about transfer to other domains.
4.I still think the original instruction models should be evaluated with the main metrics. It is only helpful for motivating your method to show that it's poor, so I don't understand why not to include it. Even if their performance is very poor, if you've already done the evaluations, they should be reported. Being too poor to report is not a convincing reason IMO.
I am raising my score to weak accept, but I feel barely above the acceptance threshold. The AUROC results and analyses are good, the idea is interesting, and the additional experiments strengthen the paper. I am not convinced about all the points above and still find the inference cost hard to justify for practical use. Nevertheless, I think there is enough useful work here to help move the field in the right direction, and your additional experiments had helped convince me of that. I still hope you would address my other points in the revision.
I also looked at other reviews, and was not convinced of any other important issues besides the ones I've described.
Dear Reviewer,
Thank you for your constructive feedback and thoughtful comments. Below, we address each of your points and provide the requested experimental comparisons.
We agree that the extra tokens from structured tags represent a practical trade-off, which we explicitly analyze in Section 6 and Appendix F (Table 5). Note that all XML tags are stripped prior to user presentation, so the user-facing response length remains concise (122 vs. 126 tokens on UNCLE) while delivering substantial calibration gains.
Regarding RLMF (Liu et al., 2026), it shares a similar high-level intuition by separating confidence estimation and verbal rewriting, whereas STRATIFIED integrates both into a single model and a single LLM call. We did not include RLMF in the main tables because it targets internal belief alignment (faithful confidence) rather than empirical correctness (factual confidence), as discussed in Section 5. Following your suggestion, we evaluate RLMF using its official codebase on the exact same WikiEU split and apply our stronger judge models (Appendix I) to all methods for a fair comparison. We also evaluate our HG-RL under PostNum and a two-step rewrite baseline. When testing rewriting with the trained 7B Qwen model itself, the output quality degraded severely, confirming that post-hoc rewriting requires a much stronger external model. Since Gemini-2.5-Flash-Lite used in RLMF is no longer available to new users, we use GPT-5.6-Luna consistently as the rewriter:
WikiEU
UNCLE
In terms of inference cost on UNCLE, RLMF requires 297 total tokens across two separate model calls (including a large external rewriter), which is comparable to or higher than STRATIFIED (277 tokens in a single 7B model call). Moreover, end-to-end STRATIFIED consistently outperforms two-step rewriting in both accuracy and calibration.
We also examine why the rewriting step in RLMF leads to a larger performance drop than expected. Specifically, RLMF provides a predefined list of candidate phrases for each confidence bin in its prompt, but several mappings do not align well with perceived certainty (for example, "just may be" is placed in the [0.05,0.1] bin while our judge assigns it 0.35, whereas "I'm at a total loss" is placed in the [0.55,0.6] bin while our judge assigns it 0.10). Thus, post-hoc rewriting remains sensitive to prompt design and requires careful calibration in future work. We will include these results and findings in the revision.