- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the calibration of verbalized confidence in reasoning models. The authors first show that linear probes trained on hidden states provide better-calibrated confidence estimates than zero-shot verbalized confidence. They then propose Probe-SD, which uses these probe predictions to relabel confidence scores in self-generated trajectories and finetunes a base model on the resulting data. The method improves calibration on several factual QA benchmarks compared to popular baselines such as verbalized confidence, post-hoc recalibration, and self-consistency-based distillation.
Critical strengths
-
The observation that hidden-state probes produce significantly better-calibrated confidence estimates than verbalized confidence (for models where calibration has not been trained using RL) is interesting, although not totally surprising.
-
Results on factual QA are strong, showing significant improvements over the baselines used.
Critical weaknesses
-
Writing and presentation: The writing and presentation are extremely poor and do not meet the standard expected for publication. Important concepts are not introduced clearly, the writing is often informal, and the overall narrative is difficult to follow. A lot of the results sections simply repeat numerical values from tables without providing additional insights. This makes the paper extremely difficult to read through.
-
Missing and weak baselines: Several important baselines are missing. These include existing supervised calibration methods such as ConfTuner [1], RL-based calibration methods [2,3], and stronger classifier-based approaches. The authors acknowledge the absence of RL comparisons, but this does not justify their omission. Without these comparisons, it is difficult to assess the actual benefits of the proposed method over existing approaches.
-
Limited evaluation domains: All experiments are conducted on relatively narrow factual QA tasks. It is unclear how well the proposed approach would generalize to more complex reasoning domains such as mathematics, coding, and STEM problems. Given that the paper specifically targets reasoning models, this is a significant limitation.
-
Biased evaluation: The probe hyperparameters are selected using validation data from EntityQuestions and NQ-open, two of the three datasets subsequently reported as out-of-domain. Although the test sets are held out, this is not a clean evaluation of generalization to unseen domains.
-
Limited novelty: It is already well established that hidden-state probes contain information predictive of answer correctness [4]. The main contribution of this work is therefore distilling this signal into verbalized confidence, which is relatively narrow. There is already substantial prior work on training verbalized confidence using RL and SFT with empirical-accuracy-based supervision.
-
Motivation for answer-selection experiments: I do not understand the motivation for the paper's emphasis on demonstrating that the probe is a poor answer selector. The main contribution is distilling probe-based confidence into verbalized confidence, and it is unclear how answer-selection contributes to the story.
[1]: Li, Y., Xiong, M., Wu, J., & Hooi, B. (2026). Conftuner: Training large language models to express their confidence verbally. Advances in Neural Information Processing Systems, 38, 53484-53513.
[2]: Damani, M., Puri, I., Slocum, S., Shenfeld, I., Choshen, L., Kim, Y., & Andreas, J. (2026, April). Beyond binary rewards: Training lms to reason about their uncertainty. In International conference on learning representations (Vol. 2026, pp. 59456-59493).
[3]: Bani-Harouni, D., Pellegrini, C., Stangel, P., Özsoy, E., Zaripova, K., Navab, N., & Keicher, M. (2026, April). Rewarding doubt: A reinforcement learning approach to calibrated confidence expression of large language models. In International Conference on Learning Representations (Vol. 2026, pp. 117927-117943).
[4]: Zhang, A., Chen, Y., Pan, J., Zhao, C., Panda, A., Li, J., & He, H. (2025). Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification. arXiv preprint arXiv:2504.05419.
Questions for authors
-
Comparison to existing calibration methods: Why are important methods such as ConfTuner and RL-based calibration training omitted? How does Probe-SD compare to these approaches?
-
Classifier baselines: How does the proposed approach compare against stronger correctness classifiers trained with the same supervision, including nonlinear probes or fully trained classifier heads?
-
Answer selection: Why is demonstrating poor answer-selection performance a central focus of the paper?
-
Generalization across domains: Can the authors evaluate the proposed method on reasoning-intensive benchmarks in mathematics, coding, or STEM?
-
Novelty relative to prior work: Please clarify the novelty of the work given the existence of closely related papers (see weaknesses)