acceptodds
Under review as a conference paper at ICLR 2027

Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs

10 views

Abstract

As the chain-of-thought reasoning capabilities of large language models improve, evaluating and calibrating their reasoning confidence is becoming increasingly important for quantifying the uncertainty of their answers. Current methods for estimating the confidence of large language models are generally based on probabilities of selected key tokens, but the underlying mechanism remains unclear. Our pilot study finds that replacing selected token probabilities with coarse substitutes can also improve calibration, motivating us to further explore effective signals of model confidence. We introduce Divergent Token Confidence (DTC), a framework that estimates confidence by counting tokens at which two models strongly disagree during decoding. DTC identifies these divergent tokens using the Jensen–Shannon divergence between next-token distributions evaluated along the same reasoning trajectory. We find that their count is almost negatively associated with answer accuracy, thereby can serve as a simple yet effective signal for uncertainty quantification. DTC supports both white-box and black-box evaluation using auxiliary models, without explicit training and affecting the generation process. Experiments across multiple model families and six mathematical benchmarks demonstrate improved calibration over the considered probability-based and verbalized-based baselines. Under white-box evaluation, the count-only estimator achieves an average expected calibration error of 13.0%, compared with 32.7%–42.4% for standard full-sequence confidence methods. In black-box settings, it also improves calibration over the original verbalized scores. For example, mean expected calibration error falls from 32.1%–40.2% to 13.7%–16.3% on DeepSeek-V3.2. These findings provide new insights for improving reasoning uncertainty quantification in large language models. We release the code at https://anonymous.4open.science/r/DTC-8564/anonymous repository

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, uncertainty quantification, confidence calibration, chain-of-thought reasoning, inter-model disagreement

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3hD5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the uncertainty estimation during LLMs’ reasoning. Specifically, the authors claim the uncertainty should be estimated not only from token probabilities but also from the token counts. The authors use an auxiliary model to identify which tokens are important or uncertain and use token counts to get the model confidence. However, there are some important problems, such as a lack of accurate positioning, the motivation is not grounded, and the assumptions are too strong without sufficient evidence or illustration.

Critical strengths
  1. Good presentations and paper writing make it easy to follow.
  2. The experiments have a broad coverage.
Critical weaknesses
  1. The positioning of this method is not so clear. For example, in Line 129, the authors state that existing cross-model signal methods primarily focus on response-level uncertainty or ranking performance, implicitly positioning calibration as a gap that this work addresses. However, the use of cross-model signals does not determine whether a method supports or evaluates calibration. Even if some prior studies emphasize ranking, this does not establish a methodological gap in calibration. Moreover, existing cross-model methods already address calibration, including BaseCal (see figure 1 of the BaseCal paper), which the authors cite in the same paragraph and use as a baseline.

  2. The motivation in sec 3.3 and 3.4 is insufficiently grounded in prior works. The authors perform an ablation study on UQAC to demonstrate the importance of token count as an uncertainty signal, which is also the source of the core idea and claim of this paper. However, this general observation is already established [1~3].

  3. Too strong assumptions without sufficient supportive evidence. See Line 212 and Line 229. Especially for the dual-auxiliary assumption of a closed-source model, which is a very strong assumption and directly relevant to the deployment of this method in real applications, the author didn’t present any supporting experimental or even intuitive evidence.

  4. The design choice of two DTC confidence functions lacks justification. For DTClinDTC_{\text{lin}}, why does the confidence start from 0.95 and decrease linearly and reach the 0.05 floor finally? Similarly, for DTCprodDTC_{\text{prod}}, the choice of the offset k=4 is not sufficiently motivated. The authors should clarify how these numbers were determined and discuss whether they vary across different experimental conditions and data settings.

  5. Lack of cost analysis and trade-off analysis. The DTC needs 1~2 extra model forwards. But the author didn’t explicitly discuss the relationship between the performance gains and the additional cost.

  6. DTC’s confidence estimates depend on the chosen auxiliary model pair, but this dependence is insufficiently justified. Disagreement may reflect differences in training domains and/or knowledge rather than unreliable reasoning.

[1] Trace Length is a Simple Uncertainty Signal in Reasoning Models

[2] Lexical hints of accuracy in LLM reasoning chains

[3] Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens

Questions for authors
  1. Does DTC also provide good AUROC ranking signals? Did the author compare the AUROC performance with several baselines, for example, the methods mentioned in related work?

  2. DeepSeek v3.2 is still an open-source model. I understand it may not be easy, but it will be better if the authors can test DTC on closed-source models, such as ChatGPT, Claude, or Gemini.

3 comments
AuthorsOpenReview ↗

Response 1

Thanks for your constructive comments and the time you dedicated to reviewing our work. Below we answer your questions first, then we reply to the weaknesses.

Q1: Lack of baselines and AUROC of DTC.

The main paper uses ECE as the main metric (calibration is a core measure in uncertainty quantification [1], and it is also the main metric of UQAC). So the experiments use calibration-oriented methods such as UQAC as baselines. We did not include methods that mainly target AUROC / ranking. We agree that those methods should still be compared on AUROC.

We add three baselines that can score existing traces directly: Self-Certainty, DeepConf, and negative generation length. As shown in Figure 14, when nn grows, the best ECE stays almost the same, while the best AUROC goes up with nn. So this AUROC comparison uses n=20n=20, and we re-select θ\theta on the same validation set. AUROC on four datasets and Qwen2.5-Instruct 7B / 14B / 32B is:

DatasetModelSelf-CertaintyDeepConf−-LengthCmeanC_{\mathrm{mean}}DTClin\mathrm{DTC}_{lin}
MATH-5007B75.074.479.465.886.0
MATH-50014B69.972.381.163.787.1
MATH-50032B76.078.580.666.987.6
AMC237B75.080.573.369.183.4
AMC2314B72.577.972.166.581.8
AMC2332B75.673.672.965.880.1
AIME247B79.079.157.675.481.6
AIME2414B67.871.958.768.675.4
AIME2432B71.379.355.872.275.4
AIME257B79.980.347.078.671.0
AIME2514B72.374.048.972.365.9
AIME2532B76.372.539.875.862.0
Avg.74.276.263.970.178.1

DTC can provide a useful AUROC ranking signal: it is competitive with CmeanC_{\mathrm{mean}} and better than the other baselines.

Q2: Test the method on closed-source models.

We are sorry that we cannot access the models you mentioned from our country. DeepSeek is already comparable in ability to many closed-source models, so we believe our results are still reliable.

AuthorsOpenReview ↗

Response 2

W1: The positioning of this method is not so clear.

ECE is widely seen as a core metric of calibration in uncertainty quantification [1]. Existing cross-model methods rarely treat it as the goal to improve, and they often do not even report ECE. So it is reasonable to treat calibration as a gap in this line of work. At the same time, we agree that Related Work is not clear enough: it does not separate BaseCal from DTC. We will add this in the revision.

W2: The motivation in Sec. 3.3 and 3.4 is insufficiently grounded in prior works.

We agree that the papers you mention already discuss token counts. The starting point is different. Those papers mainly look at token counts through AUROC / ranking. We start from calibration, and we argue that a count signal that reflects how reliable a reasoning path is can help reduce ECE.

W3: Cost analysis.

DTC does not regenerate traces with the auxiliary model. In the white-box setting, we fix one smaller auxiliary model and run teacher forcing once to compute JSD. This extra cost is small compared with generating the trace. On 1× L40 GPU, using the Transformers package with Qwen2.5-7B-Instruct, the mean time per sample is:

Mean time per sample (s)
Trace generation21.86
Auxiliary teacher-forcing + JSD0.16

Teacher forcing on the auxiliary model plus JSD is much cheaper than trace generation, and the absolute time is also low.

W4: Disagreement may reflect training or knowledge differences rather than unreliable reasoning.

We agree that looking at the types of divergent tokens helps clarify what the signal is. On Qwen2.5-7B-Instruct and MATH-500, we use the same θ\theta as in the paper. We then group tokens that mainly reflect writing style or notation — discourse markers, reasoning verbs, task words, LaTeX, and punctuation / space — into one style class. This class is 10.2% of all tokens, but only 5.3% of divergent tokens. Most divergent tokens are content words and numbers (48.4% of the divergent set, vs 34.0% of all tokens). So DTC does not mainly select tokens that differ by model-specific reasoning style.

Refs:

[1] On Calibration of Modern Neural Networks, ICML 2017

Thank you for the response. The rebuttal leaves some of my original concerns unanswered, particularly W4 on the choice of confidence functions. I also find the responses on the paper’s positioning and the interpretation of cross-model disagreement insufficient. I therefore maintain my original rating of 1.

Reviewer_E7wiOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper looks at ways of estimating the probability of a reasoning chain being “correct”. They start off by considering prior work, that looks at the per-token probabilities, and the probabilities at certain “selected tokens” (the UQAC paper). Loosely, they find that what matters isn’t the probabilities themselves, rather the number of “selected tokens”.

In effect, the author’s method is a different way of selecting tokens. They look at points of disagreement between different models (based on J-S divergence across models, with model-specific thresholds). They then convert the number of selected tokens into a probability, either via linear mapping (DTC-lin), or as an exponent of the mean probability of the chain (DTC-prod).

It seems that both the DTC and UQAC methods are going to be highly sensitive to thresholding (I.e., theta). I suspect that this impacts AUROC and ECE in slightly different ways. For both, changing theta is changing a measure of sensitivity (too high a threshold misses things, too low a threshold muddies the water). However, for ECE, it is also also a way of indirectly recalibrating the probabilities. We see this in Figure 14: Changing n and theta shift the probabilities.

Given the authors are effectively applying a calibration to their probabilities using a validation set, it seems fair to see how well the comparison methods compare under a similar calibration (eg, temperature scaling). In particular, it seems that verbalized confidence has high AUROC but low ECE, suggesting that while it is overconfident, it is still discriminative and could likely get a better ECE if we were to calibrate. I am not sure whether UQAC has a similar validation set procedure, that does implicit calibration.

The proposed approach uses multiple models to assess calibration. This is unlikely to be done in practice, as it would have very high costs. I would like to see some qualitative discussion/exploration of how the set of tokens identified by DTC compare with the set of tokens identified by UQAC. Is one a noisy subset of the other, or are there systemic differences? Along that note, I’d love to see how the J-S divergences across models correlate with per-token entropy within models — can the LLM identify the disagreement tokens itself (see the “forking tokens” literature).

More conceptually: In the framing, the authors rather gloss over AUROC and measures of separability of answers, in favor of ECE. Why is ECE the thing we most care about here? The “probabilities” we are looking at don’t have, in most cases, a natural interpretation as a probability - for DTC-lin they are just a normalization of a count.

Overall, I think there is an interesting nugget here: what matters is the number of selected or divergent tokens, and the actual per-token probabilities don’t really matter beyond identifying those tokens. However, this has to an extent been explored: Devic et al show that the number of “forking tokens” is a good proxy for uncertainty, although they look at AUROC rather than ECE. That paper also shows that overall token length is a good proxy for uncertainty, which is a confounder here (longer responses are likely to have more divergent tokens).

I still think this could be sharpened into a stand-alone paper, but it would need more careful consideration of what exactly the token identification process is finding, whether the benefit can be realized using simpler proxies such as length (or can be fully explained using these proxies), and how much of the benefit persists under validation-set recalibration of all estimators.

Devic, Siddartha, et al. "Trace length is a simple uncertainty signal in reasoning models." arXiv preprint arXiv:2510.10409 (2025).

Critical strengths
  • The method does seem to improve ECE over alternative approaches (although I have some concerns that this is due to implicit recalibration that isn't carried out by comparison methods, see main review)
  • The paper is clear
Critical weaknesses
  • It is not clear to me how much the other methods could be improved by calibrating on the validation set used by this approach (eg, via temperature scaling).
  • This method would be prohibitively expensive in most realistic use cases. I would like to see some qualitative analysis of the tokens identified, and whether the use of multiple LLMs is truly necessary to identify them (vs, e.g., refinements of UQAC, looking at entropy).
Questions for authors
  • Could you please explain in more detail how parameters are selected for both your model and UQAC?
  • Can you address my concerns about calibration scaling?
3 comments
AuthorsOpenReview ↗

Response 1

Thanks for your constructive comments and the time you dedicated to reviewing our work. Below we answer your questions first, then we reply to the weaknesses.

Q1: Please explain in more detail how parameters are selected for both your model and UQAC.

θ\theta: On 100 validation samples randomly drawn from MATH, for each model family we use the smallest evaluated model in that family (for example, Qwen2.5-7B-Instruct). We scan θ\theta with step 0.05, compute ECE and AUROC of DTClin\mathrm{DTC}_{lin}, pick the θ\theta that minimizes ECE − AUROC, and then freeze this θ\theta for the other models in the same family. As shown in Figure 5, when nn is fixed, the θ\theta that is best for ECE almost matches the θ\theta that is best for AUROC. This θ\theta is also largely stable across datasets and models. So minimizing ECE − AUROC is a reasonable way to choose θ\theta.

nn: As shown in Figure 14, for n∈[10,25]n\in[10,25], ECE under the best θ\theta stays almost the same, while AUROC changes with nn. The main paper focuses on calibration, so we fix n=10n=10 to keep the method simple.

UQAC: It has many hyperparameters (CC, KK, LtgtL_{\mathrm{tgt}}, θ\theta, Lattn′L'_{\mathrm{attn}}, and so on). Table 3 in that paper shows that changing these hyperparameters has a limited effect on AUROC and ECE. When we implement the method, we still search the hyperparameter grid from Table 3 on the same validation dataset.

Q2: Concerns about calibration scaling.

We agree that baselines should get the same validation budget for post-hoc calibration as DTC. Following LM-Polygraph [1], we apply temperature scaling to baseline confidence scores on the same validation set. For UQAC, we already search hyperparameters on this validation set, and the original method is itself designed to improve ECE, so we do not apply temperature scaling to it. After calibration, results on four datasets and three models (Qwen2.5-Instruct 7B / 14B / 32B) are:

DatasetModelCNSLC_{\mathrm{NSL}}CmeanC_{\mathrm{mean}}Verb.UQAC$_{\mathrm{attn}}$DTClin\mathrm{DTC}_{lin}DTCprod\mathrm{DTC}_{prod}
MATH-5007B24.024.126.726.813.420.2
MATH-50014B28.128.332.126.89.915.8
MATH-50032B30.130.226.526.16.617.2
AMC237B24.624.522.824.76.715.7
AMC2314B27.627.629.325.39.510.3
AMC2332B28.728.817.524.910.89.9
AIME247B21.221.325.829.511.513.5
AIME2414B26.426.623.525.116.914.0
AIME2432B25.625.724.323.920.410.3
AIME257B19.920.818.326.620.59.9
AIME2514B27.026.822.027.525.410.4
AIME2532B27.327.418.730.628.39.9
Avg.25.926.024.026.515.013.1

Even after we calibrate the baselines on the same validation set, their ECE is still high. DTC does not get the same calibration on the validation set. We only keep the θ\theta selection from the paper. This shows that the ECE gain does not come from calibrating DTC only.

AuthorsOpenReview ↗

Response 2

We now reply to the weaknesses.

W1: Concerns about calibration scaling.

Please see Q2.

W2: Concern about cost.

DTC does not regenerate traces with the auxiliary model. In the white-box setting, we fix one smaller auxiliary model and run teacher forcing once to compute JSD. This extra cost is small compared with generating the trace. On 1× L40 GPU, using the Transformers package with Qwen2.5-7B-Instruct, the mean time per sample is:

Mean time per sample (s)
Trace generation21.86
Auxiliary teacher-forcing + JSD0.16

Teacher forcing on the auxiliary model plus JSD is much cheaper than trace generation, and the absolute time is also low.

W3: Qualitative analysis of the identified tokens.

We agree that looking at the types of divergent tokens helps clarify what the signal is. On Qwen2.5-7B-Instruct and MATH-500, we use the same θ\theta as in the paper. We then group tokens that mainly reflect writing style or notation — discourse markers, reasoning verbs, task words, LaTeX, and punctuation / space — into one style class. This class is 10.2% of all tokens, but only 5.3% of divergent tokens. Most divergent tokens are content words and numbers (48.4% of the divergent set, vs 34.0% of all tokens). So DTC does not mainly select tokens that differ by model-specific reasoning style.

Refs:

[1] Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph, TACL 2025

Thank you. Adding calibration does improve my opinion of the paper, but not enough to raise my score.

Reviewer_ykxJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the confidence of LLMs in their CoT responses to math reasoning problems. It introduces Divergent Token Confidence (DTC), a method that estimates confidence by counting the number of token positions in a generated reasoning trajectory at which two models exhibit disagreement in their next-token probability distributions. Specifically, the models are evaluated through teacher forcing on the same generated trajectory, and a token is classified as divergent if the Jensen–Shannon divergence between their next-token distributions exceeds a model-family-specific threshold. The authors propose two confidence estimators: DTC-lin, which maps the divergent-token count directly to a confidence score, and DTC-prod, which combines this count with a CoT trajectory-level probability score. The method supports both white-box evaluation, using the generating model and an auxiliary model, and black-box evaluation, using the probability distributions of two auxiliary models evaluated on the trajectory generated by the black-box model. Across several model families and mathematical benchmarks, the authors report that DTC reduces expected calibration error compared to the considered probability-based and verbalized-confidence baselines, while also achieving competitive AUROC.

Critical strengths

The paper's key strengths are summarized below:

  • Simple and interesting approach to uncertainty quantification. The idea of using the number of tokens at which models disagree as a confidence signal is intuitive and interesting. The paper also provides a useful analysis of UQAC (Section 3.3), showing that its calibration improvements may be driven more by the number of selected tokens than by their specific probabilities. This is an interesting observation that motivates the proposed method.
  • Applicable to both white-box and black-box settings. A strength of DTC is that it can estimate confidence even when the generating model's token probabilities are unavailable. The method also does not require additional training or modifications to the generation process, making it relatively flexible.
  • Promising empirical results across models and mathematical benchmarks. The paper evaluates DTC across several model families and six mathematical benchmarks, covering a range of model sizes and problem difficulties. The reported improvements in ECE (Table 1) are promising, and DTCprod\text{DTC}_{\text{prod}} also achieves strong AUROC in the white-box experiments (Table 5). The results further suggest that the divergent-token signal can improve the calibration of existing methods, including UQAC and verbalized confidence (Table 3, 8 and 9).
Critical weaknesses

A list of the paper's main weaknesses can be found below:

  • Important related work and baselines are missing. The paper motivates its baseline selection by referring to the "limited prior work on calibrated reasoning uncertainty" (Section 5.1). However, several relevant approaches to assessing reasoning-trace confidence or uncertainty are not sufficiently considered. These include DeepConf (Fu et al. 2025), Self-Certainty (Kang et al. 2025), CoT-UQ (Zhang & Zhang, 2025), CER (Razghandi et al. 2025), and EDIS (Zhu et al. 2026). Some of these methods are designed primarily for reasoning-path selection or uncertainty diagnostics rather than direct probability calibration, but their underlying confidence signals are nevertheless relevant comparisons. CoT-UQ, for example, is already discussed in the related work but is not included in the experiments. The authors should more comprehensively position DTC relative to these approaches.
  • It is unclear whether the main improvements reflect a better uncertainty signal or primarily better calibration. This is my main concern with the experimental evaluation. Several existing baselines already achieve reasonably strong AUROC but poor ECE. For instance, Table 5 shows that standard sequence-level confidence methods achieve average AUROC values of approximately 73%, whereas Table 1 reports substantially worse ECE. This suggests that these methods already contain useful information for discriminating correct from incorrect responses, but their scores are not properly calibrated. Importantly, DTC uses a held-out validation set to select the disagreement threshold θ\theta (Section 5.1), whereas the competing methods are largely evaluated without comparable post-hoc calibration. This raises the question of whether the ECE advantage comes from the divergent-token signal itself or from the calibration procedure. The results in Section 6, where transforming verbalized confidence scores substantially improves ECE while largely preserving AUROC, further highlight this concern. I would strongly encourage the authors to calibrate the baseline confidence scores using the same validation budget and then compare ECE again. This would provide a much fairer assessment of the benefits of DTC.
  • Missing control for raw reasoning length. Since DTC counts the absolute number of divergent tokens, longer reasoning trajectories have more opportunities to accumulate divergent tokens. This raises the possibility that the proposed method partly captures the already established relationship between reasoning length and answer correctness, rather than a distinct uncertainty signal. The authors acknowledge this concern in Appendix C.3 and analyze the ratio of divergent tokens to reasoning length, which is useful. However, this does not replace a direct comparison against reasoning length itself. A simple but important baseline would be to use the raw number of generated reasoning tokens as a confidence predictor and report its AUROC. The authors should also report the correlation between reasoning length and divergent-token count and investigate whether DTC provides additional predictive information after controlling for response length.
Questions for authors

A list of questions and suggestions can be found below:

  • Interpretation of overconfidence: In Section 4, the authors state that Section 3.3 shows that replacing selected-token probabilities with the trajectory mean and multiplying them reduces overconfidence. However, Section 3.3 primarily reports improvements in ECE, which measures the absolute calibration gap and does not distinguish overconfidence from underconfidence. How is the specific claim of reduced overconfidence established? Similar claims appear elsewhere in the paper (e.g., Section 6). Reporting signed calibration errors or additional reliability analyses would help supporting these statements.
  • Treatment of truncated responses: Why are truncated responses included when computing answer accuracy but excluded from uncertainty estimation when no answer can be extracted? In particular, how does this affect the interpretation of ECE in Equation (2), given that the evaluated set of predictions may differ from the set used for accuracy? Reporting the exclusion rate for each model and benchmark would help.
  • It is unclear whether divergent tokens reflect semantic uncertainty or differences in model-specific reasoning styles. DTC identifies divergent tokens based on the JSD between two models' next-token distributions under teacher forcing. However, a high JSD does not necessarily indicate uncertainty about the validity of the reasoning. Two models may strongly disagree about the next token because they prefer different formulations, mathematical notation, or reasoning styles, even if both continuations are logically correct. The paper would benefit from an analysis of the types of tokens classified as divergent and whether they correspond to meaningful reasoning decisions.
  • Evaluation is restricted to the mathematical domain. Although the paper evaluates six distinct benchmarks, all of them concern mathematical reasoning. It is therefore unclear whether the proposed signal generalizes to other types of reasoning. Why evaluate on six mathematical benchmarks rather than including at least one benchmark from another reasoning domain, such as logical reasoning or multi-hop question answering? Evaluating DTC on a qualitatively different reasoning task would strengthen the paper's generalization claims.
8 comments
AuthorsOpenReview ↗

Response 1

Thanks for your constructive comments and the time you dedicated to reviewing our work. Below we answer your questions first, then we reply to the main weaknesses.

Q1: Interpretation of overconfidence.

We agree that ECE only measures the absolute calibration error. It does not tell overconfidence from underconfidence. Our claim that CmeanC_{\mathrm{mean}} is overconfident, and that we reduce this overconfidence, is based on two points: (1) prior work CoT-UQ [1] already reports that CmeanC_{\mathrm{mean}} is overconfident; (2) in the calibration plot in Figure 3 on page 7, following the definition in [2], each point is one confidence bin, and a point below the dashed line means overconfidence. CmeanC_{\mathrm{mean}} lies below the dashed line in almost every bin, which supports that CmeanC_{\mathrm{mean}} itself is overconfident.

We further report NCE (Net Calibration Error) [3] for CmeanC_{\mathrm{mean}} and DTCprod\mathrm{DTC}_{prod} on four math reasoning datasets, using Qwen2.5-Instruct 7B / 14B / 32B. NCE <0< 0 means overconfidence. NCE >0> 0 means underconfidence.

ModelMATH-500MATH-500AMC23AMC23AIME24AIME24AIME25AIME25
CmeanC_{\mathrm{mean}}DTCprod\mathrm{DTC}_{prod}CmeanC_{\mathrm{mean}}DTCprod\mathrm{DTC}_{prod}CmeanC_{\mathrm{mean}}DTCprod\mathrm{DTC}_{prod}CmeanC_{\mathrm{mean}}DTCprod\mathrm{DTC}_{prod}
Qwen2.5-7B−45.4−20.1−45.4−15.3−44.7−9.6−44.4−4.5
Qwen2.5-14B−44.9−15.8−44.7−8.9−44.1−2.3−44.5−0.9
Qwen2.5-32B−45.4−17.1−45.1−8.4−44.6−3.3−44.8−1.8

On the Qwen2.5-Instruct family and these four datasets, NCE of CmeanC_{\mathrm{mean}} is strongly negative, while NCE of DTCprod\mathrm{DTC}_{prod} is still negative but much closer to zero.

This shows that DTClin\mathrm{DTC}_{lin} reduces the overconfidence.

Q2: Treatment of truncated responses.

Overall, we choose the sample sets for accuracy and for uncertainty estimation mainly by following common practice, so that we report results honestly and keep a fair comparison. For accuracy: in math reasoning evaluation, a truncated sample (no complete answer) is usually counted as wrong. This is a common choice [4]. For uncertainty estimation: we follow UQAC Appendix C.1. UQAC needs to trace attention back from the final answer. If a sample is truncated and no answer can be extracted, that method cannot be computed. To use the same samples for all baselines and keep a fair comparison, we drop truncated samples from uncertainty estimation.

We agree that reporting the drop rate helps check the effect of this choice. Under the Table 1 setting, the truncation rates (%) are:

ModelMATH-500AMC23AIME24AIME25
Qwen2.5-7B0.41.11.52.3
Qwen2.5-14B0.30.81.32.1
Qwen2.5-32B0.30.92.11.8
Qwen3-8B1.23.06.34.4
Qwen3-14B0.50.95.63.2
Qwen3-32B0.51.24.61.9
Gemma3-12B0.20.00.31.1
Gemma3-27B0.10.00.50.2

Almost all truncation rates are below 6%. So ECE is computed on a filtered subset, which is not exactly the same set used for full-set accuracy. But the drop rate is low, and it does not change the overall conclusion of uncertainty estimation by throwing away a large number of long traces.

Q3: It is unclear whether divergent tokens reflect semantic uncertainty or differences in model-specific reasoning styles.

We agree that looking at the types of divergent tokens helps clarify what the signal is. On Qwen2.5-7B-Instruct and MATH-500, we use the same θ\theta as in the paper. We then group tokens that mainly reflect writing style or notation — discourse markers, reasoning verbs, task words, LaTeX, and punctuation / space — into one style class. This class is 10.2% of all tokens, but only 5.3% of divergent tokens. Most divergent tokens are content words and numbers (48.4% of the divergent set, vs 34.0% of all tokens). So DTC does not mainly select tokens that differ by model-specific reasoning style.

AuthorsOpenReview ↗

Response 2

Q4: Evaluation is restricted to the mathematical domain.

We agree that a non-math evaluation is needed. We therefore test DTC on the logical reasoning benchmark ZebraLogic [5]. Baselines are CmeanC_{\mathrm{mean}}, CNSLC_{\mathrm{NSL}}, and verbalized confidence. Models are Qwen2.5-Instruct 7B / 14B / 32B. DTC uses the same hyperparameters as in the main tables. We do not retune them on this dataset. We also calibrate the baselines on the same validation set as in the paper.

ECE (%):

ModelCNSLC_{\mathrm{NSL}}CmeanC_{\mathrm{mean}}Verb.DTClin\mathrm{DTC}_{lin}
Qwen2.5-7B27.525.922.112.7
Qwen2.5-14B23.226.020.214.9
Qwen2.5-32B26.029.026.520.9

AUROC (%):

ModelCNSLC_{\mathrm{NSL}}CmeanC_{\mathrm{mean}}Verb.DTClin\mathrm{DTC}_{lin}
Qwen2.5-7B82.382.883.185.0
Qwen2.5-14B80.681.583.686.7
Qwen2.5-32B84.882.582.485.5

DTC is clearly better than these baselines on both AUROC and ECE. This shows that the method still works on a logical reasoning task outside math.

AuthorsOpenReview ↗

Response 3

We now reply to the main weaknesses.

W1: Lack of baselines.

We use ECE as the main metric of the paper (calibration error is a core measure in uncertainty quantification [2], and it is also the main metric of UQAC). So the main experiments use calibration-oriented methods such as UQAC as baselines. We did not include methods that mainly target AUROC / ranking, or that were not designed for calibration. We agree that those methods can still be compared on AUROC.

We therefore add three baselines that can score existing traces directly: Self-Certainty, DeepConf, and negative generation length (a shorter trace gets higher confidence). CoT-UQ [1] needs a different prompt protocol and new sampled traces. It is not a drop-in method on our current samples, so we do not include it in this comparison. As shown in Figure 14, when nn grows in a certain range, the best ECE stays almost the same, while the best AUROC goes up with nn. So this AUROC comparison uses n=20n=20, and we re-select θ\theta on the same validation set. AUROC of these three baselines, CmeanC_{\mathrm{mean}}, and DTC on four reasoning datasets and three models (Qwen2.5-Instruct 7B / 14B / 32B) is:

DatasetModelSelf-CertaintyDeepConf−-LengthCmeanC_{\mathrm{mean}}DTClin\mathrm{DTC}_{lin}
MATH-5007B75.074.479.465.886.0
MATH-50014B69.972.381.163.787.1
MATH-50032B76.078.580.666.987.6
AMC237B75.080.573.369.183.4
AMC2314B72.577.972.166.581.8
AMC2332B75.673.672.965.880.1
AIME247B79.079.157.675.481.6
AIME2414B67.871.958.768.675.4
AIME2432B71.379.355.872.275.4
AIME257B79.980.347.078.671.0
AIME2514B72.374.048.972.365.9
AIME2532B76.372.539.875.862.0
Avg.74.276.263.970.178.1

DTC is competitive with CmeanC_{\mathrm{mean}} on AUROC, and better than the other baselines.

W2: Use the same validation set to calibrate baselines.

We agree that baselines should get the same validation budget for post-hoc calibration as DTC. Following LM-Polygraph [6], we apply temperature scaling to baseline confidence scores on the same validation set. For UQAC, we already search hyperparameters on this validation set, and the original method is itself designed to improve ECE, so we do not apply temperature scaling to it. After calibration, results on four datasets and three models (Qwen2.5-Instruct 7B / 14B / 32B) are:

DatasetModelCNSLC_{\mathrm{NSL}}CmeanC_{\mathrm{mean}}Verb.UQAC$_{\mathrm{attn}}$DTClin\mathrm{DTC}_{lin}DTCprod\mathrm{DTC}_{prod}
MATH-5007B24.024.126.726.813.420.2
MATH-50014B28.128.332.126.89.915.8
MATH-50032B30.130.226.526.16.617.2
AMC237B24.624.522.824.76.715.7
AMC2314B27.627.629.325.39.510.3
AMC2332B28.728.817.524.910.89.9
AIME247B21.221.325.829.511.513.5
AIME2414B26.426.623.525.116.914.0
AIME2432B25.625.724.323.920.410.3
AIME257B19.920.818.326.620.59.9
AIME2514B27.026.822.027.525.410.4
AIME2532B27.327.418.730.628.39.9
Avg.25.926.024.026.515.013.1

Even after we calibrate the baselines on the same validation set, their ECE is still high. DTC does not get the same calibration on the validation set. We only keep the θ\theta selection from the paper. This shows that the ECE gain does not come from calibrating DTC only.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.