acceptodds
Withdrawn from ICLR 2027

Tracing Uncertainty in Language Model Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language model (LM) reasoning, commonly described as Chain-of-Thought or test-time scaling, often improves benchmark performance, but the dynamics underlying this process remain poorly understood. We study these dynamics through the lens of uncertainty quantification by treating the reasoning traces, the intermediate token sequences generated by LMs, as evolving model states. We summarize each trace by an uncertainty trace profile: a small set of features describing the shape of the uncertainty signal over its trace, such as its slope and linearity. We find that across eight LMs from three different families evaluated on eight datasets on mathematical and logical reasoning, these profiles predict whether a trace yields a correct final answer with AUROC up to , improving markedly on recent related work. We reach AUROC using only the first few hundred tokens of full traces, suggesting that errors can be detected early in the generation. A detailed comparison of correct and incorrect traces further reveals qualitatively distinct uncertainty profiles, with correct traces showing a steeper and less linear decline in uncertainty. Together, the results suggest that our method, grounded in decision-making under uncertainty, provides a principled lens for studying the generative process underlying underlying LM reasoning.

Primary area:
generative models
Keywords:
Language modeling, uncertainty quantification, reasoning, generative models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_st6eOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The title, method, and experiments of this paper seem to focus on the uncertainty estimation in LLM, which I think is probably the main scope of this work. However, the whole introduction is all about whether the “reasoning,” a term used in LLMs, is an accurate definition. From the introduction, it should be a paper discussing the philosophy of LLMs. I was trying to understand, but the introduction really confused me. Regarding the method, the authors combined several uncertainty signals to train a binary classifier to assess the correctness of reasoning.

Critical strengths
  1. Good perspective to model uncertainty from time series.
  2. This field of uncertainty estimation of LLM is interesting.
Critical weaknesses
  1. The motivation and problem formulation are not clear. The introduction is very confusing. The authors use a lot of space to discuss the philosophical definition of “reasoning” in LLM. I can not see the necessity to have some discussions. I also don’t know how this discussion motivates the study of uncertainty. Instead of that, I didn’t see the motivation for “tracking uncertainty”.
  2. Many statements and conclusions are inconsistent or even contradict each other. For example, line 22-25, the abstract states that correct traces exhibit a "steeper and less linear decline in uncertainty." But, in lines 435-439, the authors said that a "steeper and more linear slope" indicates an incorrect trace. Furthermore, lines 359-369 suggest that the relationship between linearity and correctness differs across model families, making the generalized conclusions in the abstract and discussion questionable. These inconsistencies make it difficult to understand what the actual empirical findings are.
  3. The paper does not establish its contribution relative to previous uncertainty-based correctness prediction methods. The main novelty seems to be temporal features from reasoning traces. However, the paper does not isolate the benefits of temporal modeling from those of supervised learning.
  4. The proposed method relies heavily on handcrafted features and several seemingly arbitrary design choices, without sufficient theoretical or empirical evidence. Why are these particular uncertainty measures selected? Why are the uncertainty trajectories summarized using these signals? More specifically, Appendix C (Table 3) divides each reasoning trace into 25%–50%–25% segments, but the authors provide no explanation for these particular proportions.
  5. The presentation and writing really need to be improved.
Questions for authors

See the weaknesses

3 comments

Thank you for taking the time to review our paper and providing some relevant critiques. We hope our response can alleviate some of your concerns regarding the paper, and that you, with our response in mind, would be willing to provide some concrete suggestions for how to improve the paper.

On your raised points:

  1. First of all we want to acknowledge that the introduction does spend perhaps too long discussing the notion of reasoning in language models. This research started from discussions of reasoning, but in our next revision we will significantly limit this and truncate large parts of the introduction discussing this. With that being said, we believe that stating that the entire introduction concerns reasoning is a slight overstatement. Language model reasoning is primarily discussed from ll. 31 - 36 and ll. 41 - 47 in the introduction which continues to l. 92.

  2. On the first point, this is a mistake on our part, the abstract contains the correct formulation. We will fix the mixup in the discussion section in the next revision. On the second point, as we discuss in ll. 368 - 377, this change in relationship between linearity and correctness is a function of the extent of the reasoning training. We argue that this is not an inconsistency, but a result of the model training regime.

  3. We’re unsure what this refers to with respect to supervised fine-tuning in the context of our paper as all of the correctness prediction methods, including the baselines such as DeepConf, are implemented using a supervised classifier (i.e. the correctness prediction with e.g. DeepConf works by generating a score for each sample and then training supervised classifiers to predict the sample’s correctness using these scores). With respect to the novelty of the contribution we refer to ll. 85-92 (Novelty & Contribution) where we explicitly describe the novelty of the approach with respect to methods such as DeepConf and Self-Certainty. With respect to why the specific uncertainty measures are selected please see Section 2.2. In summary we choose these measures to get a granular picture of the uncertainty dynamics, as only looking at either aleatoric or epistemic will often obfuscate the picture.

  4. The features are selected for the sake of simplicity and interpretability. We have also studied additional features of a much more complex nature (such as skewness, kurtosis, change-point detection, volatility, autocorrelation, cyclicality via FFT among others), but the simple features performed just as well. We therefore made the decision to use simpler features to improve the interpretability of our results.

  5. We are sorry to hear that you found the paper hard to understand and will work on the presentation, including regarding the first point of critique. The other reviewers seemed to have an easier time understanding the paper, but we would welcome any concrete suggestions for improvements.

Thanks to the authors for their efforts to address my concerns.

Regarding concerns about motivation and the inconsistent conclusions, the authors acknowledged that the problems exist.

Regarding W3, I'm also "unsure" how reviewers are expected to understand the detailed experimental settings of the baselines when these details are not provided in the paper. The authors didn’t mention that they use DeepConf to train a classifier. As I understand it, DeepConf was originally proposed as a training-free method. The authors should include these details in the paper.

Regarding W4, the authors claim to have conducted additional experiments with more complex features. However, no corresponding results are provided in either the paper or the rebuttal to support this claim. Moreover, the response does not address my specific question regarding the choice of the 25%–50%–25% segmentation in Appendix C.

Regarding W5, I find the authors' response particularly disappointing. Comparing my reading experience with those of other reviewers is neither constructive nor appropriate. I would expect the authors to address concerns about their writing rather than question a reviewer's ability to understand the paper.

Overall, the authors didn’t address most of my concerns, and the paper needs a significant revision. So I keep my original score unchanged.

I also noticed that the submission does not include an AI Use Statement, which is mandatory under the ICLR 2027 Author Guidelines.

We thank the reviewer for the swift reply.

Regarding W3, we realize that we have not adequately communicated the experimental setup and will amend this in future revisions.

Regarding W4, the initial experiments with more complex features were performed as initial experiments and thus not included in the final version of the paper. Below we show the results of correctness prediction on GSM8K and ProntoQA for the five smaller models with all the initial features that we considered included. As the results indicate, using a more complicated feature-set doesn’t provide much in terms of correctness prediction. We realize that this analysis should have been included in the paper.

ModelGSM8KProntoQA
Llama 3.10.7640.795
Llama 3.20.7680.504
Qwen 2.50.8020.559
DeepSeek R10.7930.708
Qwen 30.7220.688

Regarding W5, we apologize for our previous response and acknowledge that the comparison with other reviewers was inappropriate. We did not intend to cause offense, but rather hoped to identify specific ways to improve the clarity of the paper. We appreciate the reviewer’s feedback and will take it into account in our revisions.

Based on the reviewers’ comments, we have decided to withdraw the paper to work on a significant revision (see general comment above). We thank the reviewer for the feedback and suggestions.

Reviewer_GcLJOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents an empirical study of uncertainty dynamics in language-model reasoning. It constructs uncertainty trajectories using entropy, Bernoulli variance, and a gradient-norm proxy for epistemic uncertainty, with trace-level and retrospective answer-level channels. These trajectories are summarized by segment means, slope, and linear-fit quality, and used to train classifiers that predict final-answer correctness. Using greedy-generated traces across eight models and eight mathematical and logical reasoning datasets, the authors evaluate classifiers separately for each model-dataset combination with five-fold cross-validation and compare against confidence and length baselines. The paper reports full-trace and early-prefix prediction results and analyzes differences in uncertainty profiles between correct and incorrect generations.

Critical strengths
  1. A compact and inspectable diagnostic representation. The uncertainty trace profile summarizes several uncertainty signals through a small set of level and trend features, making their associations with correctness straightforward to examine.

  2. Meaningful predictive gains in several settings, including a lower-cost variant. The evaluation covers multiple model families and mathematical and logical reasoning tasks. Some reported gains are sizeable: Qwen 2.5/GSM8K improves from 0.725 for the strongest listed baseline to 0.807, and Qwen 3.5/Omni-Math improves from 0.799 to 0.899 (Tables 1-2). The latter uses only trace-level aleatoric features, showing that useful predictive signal can be obtained without backward passes or retrospective answer conditioning.

  3. An interesting cross-channel observation. Incorrect generations are reported to exhibit generally higher trace-level but lower answer-level gradient-based epistemic scores (Section 4.2). This contrast is an interesting descriptive finding that motivates examining local and answer-directed signals separately, although its interpretation in terms of training-data support remains unvalidated.

Critical weaknesses
  1. Unclear incremental value beyond existing local confidence methods. DeepConf already uses local confidence signals to improve answer aggregation and inference efficiency. Although the proposed uncertainty profiles achieve higher correctness-prediction AUROC in several settings, it remains unclear whether these gains translate into better downstream decisions, such as improved final-answer accuracy through candidate filtering or answer aggregation under a matched compute budget. If the contribution is instead primarily diagnostic, the paper should clarify what additional, well-supported understanding its richer profiles provide beyond existing local confidence signals.

  2. The contribution of temporal dynamics is not isolated. Tables 5-6 ablate uncertainty types and channels, but do not show whether the gains come from temporal structure rather than richer features and supervised training. A comparison against order-independent confidence statistics using the same classifier and training data would clarify the central methodological contribution.

  3. Cross-task generalization is not established. Classifiers are trained and cross-validated separately for each model-dataset pair (Appendix D). This demonstrates within-dataset predictability, not transfer of a fixed predictor. Performance on unseen tasks without new correctness labels, retraining, or recalibration is not reported.

Questions for authors
  1. Task-level comparison with DeepConf: Could you compare the proposed profile-based scores with DeepConf confidence scores for candidate filtering or answer aggregation, using the same model, evaluation questions, candidate traces, and downstream decision rule? Please report final-answer accuracy under a matched compute budget that includes uncertainty-estimation costs, with the correctness predictor trained on separate questions. An offline comparison would suffice to test this downstream benefit without requiring a new online generation-control system.

  2. Temporal contribution: Under the same classifier, training labels, and CV splits, how much do segment means and slope/linearity add beyond order-independent confidence statistics and trace length?

  3. Targeted transfer analysis: Could you report cross-dataset performance with the language model and correctness predictor fixed, without target-task labels, retraining, or recalibration?

2 comments

Thank you for emphasizing the interesting findings presented in our paper. We’re happy that you found the paper clear and easy to follow, and that you got a deeper understanding of the question we’re tackling. We hope the following response can alleviate any concerns you have regarding the soundness of the paper.

On your raised weaknesses:

  1. You correctly highlight that DeepConf does answer aggregation to improve inference efficient. But this is done by sampling a large set of generated traces. Once you have the trained classifier (ours), you no longer have to sample multiple generations. Whether our method could also improve in candidate filtering or answer-aggregation setups is an interesting questions that we will attempt to pursue.

It is also noted that we should clarify what additional understanding the richer profiles provide beyond the existing signals, which is a fair point to raise. We are working on incorporating this analysis into the analysis in Section 4.

  1. We emphasize that all the baselines we compute are used as features for supervised correctness classifiers, and thus that this isn’t only applied to the temporal dynamics. We also note that even the “static” features (the means) are also temporal in the sense that they’re also modeling how much e.g. the entropy drops from the beginning of the trace to the end.

  2. We indeed do not study cross-task generalization, but this is certainly an interesting point which we will pursue in future work related to this research. That being said, the epistemic uncertainty estimates cannot be compared across models (see Grünefeld et al. 2026) and different models are trained on different data in different ways and thus naturally uncertain in different ways. Further, different tasks can be expected to have different computational profiles. So we do not expect cross-task generalization to be simple.

Regarding your questions:

  1. Our uncertainty trace profiles (UTP) aren’t scores in the same sense that DeepConf provides a score for each generated trace. We only use the UTP features as input to a classifier that is trained to predict correctness. Thus our method allows us to forego the sampling that is inherent to a method like DeepConf or Self-Certainty, and replace it with a trained classifier, but it doesn’t enable us to score and rank generations in the manner that DeepConf does. Our approach is thus an alternative to methods such as majority voting and complementary to them.

  2. Table 1 of the paper showcases the predictive accuracy of correctness classifiers trained solely on the numbers of tokens (#Tok) and the mean entropy (Ent.). For GSM8K, ProntoQA, ZebraLogic, and Omni-MATH the UTP is better both simple comparisons across (almost) all models (except for R1 on ProntoQA).

  3. As noted above, this is a very interesting question for future work that we plan to pursue in a follow-up paper.

Thank you for your clarifications. I understand the distinction about "once you have the trained classifier, you no longer have to sample multiple generations." But how would this prediction be used to improve performance on the underlying reasoning task? Therefore, my main concerns remain unresolved: the response describes potential downstream benefits, but leaves their task-level validation to future work.

On generalization, does the method require a separately trained correctness classifier for each task, or can a classifier trained on one task be reused on others? If so, what is the adaptation cost, and how does this affect the method’s practical applicability? The current experiments train classifiers separately for each model–dataset pair, so this remains unclear.

Reviewer_gXLZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper utilizes token uncertainty over reasoning traces and trains small classifiers to predict final answer correctness. Evaluation across eight models and eight math and logic datasets shows AUROC gains in some settings. It also studies prediction from partial traces and differences between correct and incorrect generations

Critical strengths

The AUROC improvements on gsm8k and synthetic arithmetic are substantial for several models, despite the simple features and classifiers

I also liked the cheap variant's result on omni math with qwen 3.5. This shows that useful gains are possible on harder problems without the expensive gradient computations

Critical weaknesses

1.The classifier is trained seperately for every model and dataset, with no experiments showing transfer without retraining. This is a major practical limitation: the reported performance relies on labeled target data, and usefulness on new distributions without this supervision is not established. While showing transferability between datasets is critical (e.g., training on GSM8K and evaluating on omnimath), evaluating transferability between domains is very important too (i.e., not only math->math)

  1. The classifier receives correctness supervision training, but the baselines do not. I do not find this a fair comparison without learned baselines. some relevant examples are “Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuits” and “Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification”.

3.according to my understanding, most results use the full profile, with timings imply roughly 15~ times standard lllm inference in the measured setup. IMO this is completely impractical for most settings, and therefore less interesting. The cheap variant helps, but it has narrower evaluation and is not the focus in the paper.

Questions for authors

1.The 2048 token budget seems restrictive for longer math reasoning like AIME (I know AIME wasn't evaluated here, but my concern is about longer generations in general, not AIME specifically). Do the findings hold with substantially larger budgets? Please also report the number of generations hitting the limit, receiving a forced thinking stop, and being discarded, separately for each model and dataset.

2.I wonder about the utility of the tmeporal features and think there might be an ablation to show if this actually helps. for example, how does the same classifier perform using only trace means and variances?

3.How much of the full profile's performance is retained using only the cheap features on the same models and datasets?

2 comments

Thank you for the review, and for emphasizing the strength of our results and their applicability while also raising relevant points for improvement. We hope our response can alleviate some of the concerns you have regarding the paper.

On your raised weaknesses:

  1. We indeed do not study cross-task generalization, but this is certainly an interesting point which we will pursue in future work related to this research. That being said, the epistemic uncertainty estimates cannot be compared across models (see Grünefeld et al. 2026) and different models are trained on different data in different ways and thus naturally uncertain in different ways. Further, different tasks can be expected to have different computational profiles.

  2. We emphasize that all the baselines also receive correctness supervision training, and that this critique thus is not accurate. In the case of the baselines we use e.g. the DeepConf values as input to a simple classifier (just as for our uncertainty trace profile) and train the classifier to predict correctness based on the DeepConf metric. So the baselines are in that sense also learned.

  3. Thank you for raising this point. It is a fair critique, but it really depends on the aim of the application you want to use the method for. As we argue in the paper, if you just want correctness prediction, the cheaper profile is sufficient. If you want to learn something about your trace, to analyze it, then you can compute the full profiles. While we focus primarily on the full profiles to study the uncertainty dynamics in our paper, we provide the simple profile, which has very low additional compute overhead, as a cheap approach which is much more feasible.

Regarding your questions:

  1. Our experiments using larger models on more difficult datasets (ZebraLogic and Omni-Math) use a 8192 token budget rather than the 2048 and the performance holds up, so we would argue that the limit is not a significant confounder.

  2. The following table contains the mean correctness prediction performance using different feature types of our UTP:

Feature(s)Mean AUROC
Slope0.640
R²0.633
Slope + R²0.656
Early/mid/late means0.667
Curated profile0.687

That being said we want to emphasize that even the level features have a temporal aspect to them, as we divide into early, middle, and late level, so they, too, encode a temporal dynamic.

  1. The table posted in the following comment contains the results of the aleatoric trace level profile in the same settings as out table 1. The mean penalty of using the aleatoric trace only instead of the full profile is 0.051 AUROC, with the median penalty being 0.028 AUROC.
BenchmarkModelUTP (ours)SCDCEnt.Perp.
GSM8KLlama 3.10.7210.4910.5320.5710.511
Llama 3.20.6900.5660.6090.6270.558
Qwen 2.50.7900.6890.7040.7250.533
DeepSeek R10.7500.7030.6780.6060.627
Qwen 30.7400.7280.7190.7230.615
Qwen 3.5 9b0.7690.6920.7040.7570.581
ProntoQALlama 3.10.7930.5660.7000.7370.663
Llama 3.20.4560.5490.5570.5380.544
Qwen 2.50.5110.5650.5640.5670.556
DeepSeek R10.6700.6150.6710.6570.567
Qwen 30.6840.6110.6050.5400.418
Qwen 3.5 9b0.6090.7670.7570.5460.582
FOLIOLlama 3.10.6150.4560.4490.4870.535
Llama 3.20.5170.5090.5110.5290.510
Qwen 2.50.4940.5410.5380.4830.482
DeepSeek R10.5560.6260.6180.4600.534
Qwen 30.5790.5690.5660.5670.558
Qwen 3.5 9b0.6780.6360.5530.5420.632
BHHLlama 3.10.6380.5420.5920.5600.525
Llama 3.20.6080.5260.5570.5960.612
Qwen 2.50.5960.5200.5900.6130.630
DeepSeek R10.6820.6210.6650.6310.665
Qwen 30.6170.4840.4920.4460.451
Qwen 3.5 9b0.7810.8800.8350.7700.465
CRV-BoolLlama 3.20.5220.6160.6490.5720.534
Qwen 2.50.5450.5330.5530.5880.591
DeepSeek R10.6540.6270.6050.6140.426
Qwen 30.4600.5350.5090.4630.487
Qwen 3.5 9b0.5330.3100.2910.3350.523
CRV-Arit.Llama 3.10.9670.6370.7250.7290.576
Llama 3.20.7750.7200.7100.7130.705
Qwen 2.50.7470.6180.6600.4560.611
DeepSeek R10.6500.7110.6530.5350.572
Qwen 30.7360.5550.5440.4610.577
Qwen 3.5 9b0.8190.7220.5650.7160.640

Comments on the paper

1 comment

Dear reviewers,

Thank you for your reviews. Please read the authors’ response and follow up in the discussion, briefly indicating whether your main concerns have been addressed and highlighting any remaining questions. Please also update your review and rating where appropriate.

Many thanks for your time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.