- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper studies uncertainty localization in LLM reasoning trajectories: instead of scoring only the final answer, it identifies intermediate steps or tokens where uncertainty arises. The authors argue that NLL and token entropy are confounded by “benign variability” (e.g., synonym choices or alternative valid phrasings), and propose scores based on the sensitivity of generated tokens to perturbations of preceding token embeddings:
-
Random Perturbation (
Rand. Pert.): Gaussian noise is added to all token embeddings, and the score is the variance of the token log-probability over samples, . Proposition 3.3 relates this variance to the probability of a “less-supportive neighboring condition.” -
Adversarial Perturbation (
Adv. Pert.): Embeddings are perturbed along , where is the response-level log-likelihood, and the score is the drop in token log-probability, . Proposition 3.4 interprets this as directional alignment between local token and global sequence gradients. With causal masking,Adv. Pert.needs one backward and one forward pass for all positions.
The authors segment reasoning responses into steps using rule-based sentence boundaries and aggregate token-level scores within each sentence by taking the maximum (max). The method is evaluated against two observable proxies for uncertainty: (1) localizing the "First Wrong Step" (evaluated on MATH and BBH using Llama-3.1-8B, Qwen-2.5-7B, and Mistral-7B, and on GPQA using gpt-oss-20b and Qwen3.5-27B); and (2) identifying "Fragile Correct Steps" prone to errors upon regeneration within responses whose intermediate steps are all correct but exhibit low overall self-consistency (). Furthermore, the paper demonstrates the practical utility of these signals in guiding targeted external feedback and revision to improve reasoning accuracy, verifies runtime and memory efficiency on a single GH200 GPU, and candidly discusses limitations regarding factual hallucination detection on FactScore, white-box access requirements, and the scope of relative within-trajectory ranking versus cross-problem calibration.
Critical strengths
-
Well-targeted problem formulation with a simple and computationally efficient method.
The paper advances LLM reasoning uncertainty from coarse-grained response-level estimation to fine-grained intermediate-step localization, addressing the practical challenges that external Process Reward Models (PRMs) are often unavailable in open-domain reasoning and that raw probability/entropy metrics suffer from benign lexical variability. By exploiting autoregressive causal masking, the proposed approach avoids expensive token-by-token regeneration required by forking tokens as well as architecture-specific low-rank weight perturbation designs required by TokUR. It also achieves substantial efficiency gains: compared with 12.79s for original response generation,
Rand. Pert.() adds only 0.90s andAdv. Pert.adds only 0.28s (with ~3 GB extra GPU memory), which is substantially faster than TokUR (3.47s) and the multi-sampling method Graph Uncertainty (49.55s). -
Consistent empirical performance across multi-faceted proxy evaluations and transparent boundary analysis.
Beyond evaluating "first wrong step" localization, the paper designs a "fragile correct step" regeneration-risk evaluation to distinguish genuine uncertainty detection from mere error detection. In the 12 reported model–dataset–cutoff settings in Table 1,
Adv. Pert.achieves higher values than all non-proposed baselines (e.g., 0.45 on MATH with Llama at the 10% cutoff, compared with 0.27 for TokUR-max). Appendix B further supports evaluation reliability by reporting a 93.9% human alignment rate on 180 "first wrong step" annotations and a 97.5% alignment rate on 80 regenerated-step correctness judgments. Moreover, Section 5.2 and Figure 11 transparently report weaker performance on FactScore and a failure case involving confident reasoning errors, helping clarify the method's operational boundaries.
Critical weaknesses
-
The theoretical and empirical link between "embedding sensitivity" and "reasoning uncertainty" is only partially established, with potential confounding from position and step-length biases.
Lemma A.1 and Proposition 3.4 use first-order Taylor expansions to show that the proposed metrics essentially approximate the gradient norm of local log-probability with respect to preceding embeddings or its directional alignment with the sequence-level gradient. While this clarifies their mathematical meaning as embedding sensitivity measures, it does not rigorously establish their equivalence to reasoning uncertainty. The claim that perturbed prefixes form "valid neighboring conditions" relies primarily on qualitative coherence in a few regenerated examples (Figures 2, 6, and 7); however, surface fluency does not guarantee that key mathematical or logical premises and constraints in the prefix remain intact after perturbation. Furthermore, as shown in the Qwen failure case in Figure 11, the method fails when the model makes confident errors. When aggregating token-level scores into step-level scores, the paper only reports maximum aggregation (
max) for the proposed methods, whereas in Table 1 bothmaxand mean (avg) aggregations are reported for NLL, Entropy, and TokUR. Because rule-segmented natural sentences vary substantially in length, taking themaxmay systematically bias selection toward longer steps containing more tokens. Meanwhile, as the token position advances, the perturbed prefix embedding dimension grows linearly, and the response-level objective inAdv. Pert.accumulates more downstream gradient terms for earlier tokens; the paper lacks a quantitative check on whether token position induces systematic score drift. -
Dependence on proxy ground truths and automated GPT-5.6 annotations, alongside statistical and selection biases in parts of the evaluation, limits the strength of the conclusions.
The "first wrong step" is a proxy for incorrectness rather than intrinsic model uncertainty. While the authors complement this with "fragile correct steps" (Table 3), the regeneration risk is estimated using only samples and computed exclusively on the top-10%/20% steps flagged by each method before taking the maximum (
max). Not only does this introduce candidate selection bias, but taking the maximum over noisy small-sample estimates also introduces an upward statistical bias. Additionally, human verification covers 180 first-error cases, 30 additional cases checked for absence of errors, and 80 regeneration judgments on MATH and BBH, omitting the more challenging GPQA benchmark (Table 2, which also excludes TokUR and Graph Uncertainty). Regarding statistical uncertainty, Tables 1 and 7 only report approximate confidence intervals for a single proportion rather than paired intervals for differences between methods; Table 3 reports a uniform interval whose construction is unexplained; and Table 2 reports no intervals at all. Some differences are small (e.g., the margins in Table 1 for Qwen/MATH at 20% betweenAdv. Pert.0.52 and CCP 0.50, and in Table 2 for Qwen3.5 at 20% between 0.50 and TokenSAR 0.49). Finally, the revision pipeline in Table 4 relies on GPT-5.6 to judge errors and provide revision suggestions but lacks a "Random Step Selection" baseline (randomly selecting 10%/20% of steps for GPT-5.6 inspection), making it difficult to isolate the localization signal's contribution from GPT-5.6's strong inherent error-correction capability.
Questions for authors
-
Under , how does the direct analytic gradient norm/inner product compare with random samples or a finite-step adversarial perturbation? In Eq. 6, when (e.g.,
'Now'rises from 0.173 to 0.268 in Fig. 6(E)), what is the interpretation, and how are negative values handled in aggregation/ranking? -
Beyond Figures 2, 6, and 7, can you quantitatively verify on a representative subset that continuations from perturbed prefixes at preserve the mathematical/logical premises and constraints, ruling out confounding from disrupted premise representations?
-
In Table 1 and Table 7, NLL/Entropy/TokUR report both
maxandavg, whileRand. Pert./Adv. Pert.report onlymax; Table 2 includes NLL/Entropy but not TokUR. What are the coverage rates forRand. Pert.-avg/Adv. Pert.-avg? Are top-10%/20% steps selected bymaxsignificantly longer than average steps? -
In long GPQA responses (~2,000 tokens), does drift systematically with position ? How does performance change if scores are normalized by , or if only a fixed recent window is perturbed?
-
Tables 1 and 7 give only a single-proportion interval (). Given narrow margins, can you report paired confidence intervals or McNemar -values against the strongest baseline per setting? What is the human alignment rate for GPT-5.6 first-wrong-step annotations on GPQA?
-
How was Table 3’s derived, and does it include binomial variance from ? Are regenerations/judgments reused when methods select the same step? Does the advantage hold using mean risk over top-10%/20% steps, or re-evaluating the highest-risk step with fresh samples?
-
In Table 4, what is the accuracy of a “randomly select 10%/20% of steps for GPT-5.6 inspection and feedback” baseline? Among flagged steps, what fraction is judged by GPT-5.6 as introducing a new error and triggering feedback?
5 comments
Response to Reviewer Aq5M (Section 2)
Q1. Analytic gradient norm & whether adversarial perturbation sensitivity
Calculating analytic gradient norm as a substitution of random perturbation could be inefficient, as it requires conducting backward pass for each token. In our test, we check 10 cases on Llama Math dataset using analytic gradient norm and random perturbation, and find that they flag almost the same sentences. In Figure 6 (E) (now Figure 7(E) in the new paper), adversarial perturbation is applied to decrease the probability of token ="At". This suggests . For the token "now", it is an alternative of ="At", which is not the generated token.
Q2. Continuations from perturbed prefixes preserve premise representations.
We follow the reviewer's suggestion to verify that: continuations from perturbed prefix preserve premise representations. In MATH and BBH dataset, we sample 300 responses using Llama. For each sample, we randomly select steps and regenerate the subsequent sentence under perturbed prefix. The average semantic similarity (cosine similarity embedded by "all-MiniLM-L6-v2") between the original and regenerated steps is 0.9873 / 0.9772 (MATH / BBH) under random perturbation and 0.9826 / 0.9766 under adversarial perturbation. Their similarity indicates that our perturbations preserve the mathematical and logical premises and constraints.
Q3. Perturbation score in average-aggregation; GPQA does not include TokUR
Since our method identifies uncertainty at the token level, maximum aggregation is more appropriate than average aggregation, as other tokens within the same sentence may have very low uncertainty. Please refer to our response to Reviewer SBx7 (Point 6) for the full results under average aggregation, which further demonstrate the necessity of maximum aggregation. Meanwhile, the sentences identified by our method are not significantly longer than the overall average. We provide detailed sentence-length statistics in our response to Reviewer 4eMg (Point 4). Finally, we exclude TokUR from our GPQA experiment because its weight perturbation approach requires substantial model-specific reimplementation efforts for more advanced models (see the response to Reviewer 4eMg (Point 3).
Q4. Does perturbation scores drift systematically with position in GPQA?
Plz refer to our response to Weakness 1 (ii).
Response to Reviewer Aq5M (Section 3)
Q5. Pairwise CI for Section 4.1; GPQA annotation
The table below reports the paired 95% CI between our method and TokUR-max under the setting of Table 1 in our paper. The table below reports the performance improvement with 95% paired confidence intervals. Our method outperforms strong baseline TokUR in most cases. Besides, GPQA consists of graduate-level hard science questions, for which human annotations by the authors may be less reliable than those provided by GPT-5.6, given the specialized domain knowledge required.
| Math | Llama | Qwen | Mistral |
|---|---|---|---|
| Rand. Pert.-TokUR CI (top-10%) | +0.13 0.05 | +0.07 0.05 | +0.03 0.04 |
| Rand. Pert.-TokUR CI (top-20%) | +0.07 0.05 | +0.05 0.05 | +0.06 0.04 |
| Adv. Pert.-TokUR CI (top-10%) | +0.18 0.04 | +0.09 0.06 | +0.06 0.04 |
| Adv. Pert.-TokUR CI (top-20%) | +0.12 0.05 | +0.07 0.05 | +0.06 0.05 |
| Logic | Llama | Qwen | Mistral |
|---|---|---|---|
| Rand. Pert.-TokUR CI (top-10%) | +0.06 0.04 | +0.09 0.06 | +0.04 0.05 |
| Rand. Pert.-TokUR CI (top-20%) | +0.08 0.05 | +0.08 0.06 | +0.02 0.04 |
| Adv. Pert.-TokUR CI (top-10%) | +0.14 0.05 | +0.11 0.06 | +0.11 0.05 |
| Adv. Pert.-TokUR CI (top-20%) | +0.16 0.05 | +0.10 0.05 | +0.05 0.04 |
Q6. How to calculate CI for Table 3; 4.2 experimental details
Table 3's 95% CI of is calculated using the sample standard deviation, without assuming a specific distribution for the observed data. In Section 4.2, if a step is flagged by multiple methods, its regeneration risk estimate (including the underlying regenerations and judgments) is reused. In Section 4.2, reporting the mean regeneration risk across selected steps would not adequately reflect a method's ability to identify highly fragile correct steps, as such steps may be sparse within a reasoning trajectory. Therefore, we use the maximum risk to evaluate whether a method can successfully identify these steps.
Q7. Questions related to 4.3 experiment details
We provide random baseline result for Section 4.3 Table 4 in our updated paper. The reviewer can also refer to our response to reviewer 4eMg (Point 5) for the results. Meanwhile, among the responses that our methods provide flagged steps, about 40% to 50% are judged by GPT-5.6 as introducing a new error and triggering feedback. For example, in Llama Math (Precalculus), given top-10% flagged steps, 44% of random perturbation and 48% of adversarial perturbation cases are judged to contain error. As a comparison, there are 22% of cases judged as containing error provided by NLL localization.
Response to Reviewer Aq5M (Section 1)
We thank the reviewer for the comments. We first provide clarifications to the major weakness raised by reviewers. Then, we address other questions.
W1. Theoretical link to reasoning uncertainty. Is our method biased to token position or step length?
(i) Theoretical link to reasoning uncertainty. We agree with the reviewer that our original theoretical analysis may not have sufficiently established the connection to reasoning uncertainty. This concern was also raised by Reviewers 4eMg and R5XN. To address it, we have provided a new theoretical analysis in Section 3.3 of our revised paper. We kindly refer the reviewer to our response to Reviewer 4eMg and the revised manuscript. The key insight of our new theory is that our perturbation sensitivity score can help identify tokens where the model has a strong tendency to favor "conflicting alternatives". Such alternatives may lead to contradictory reasoning continuations, thereby reflecting reasoning uncertainty. In our Section 3.3, we mathematically characterize these conflicting alternatives and establish their connection to our perturbation sensitivity score.
(ii) Is our method biased toward token position? We find our perturbation scores do not exhibit systematic drift with token position. To show this, we compute the Spearman rank correlation between uncertainty scores and absolute token positions on the GPQA (longest) dataset. For Rand. Pert., the correlations are -0.062 (p=0.443) and -0.064 (p=0.377) for gpt-oss-20b and Qwen3.5-27B, respectively. For Adv. Pert., the corresponding values are -0.012 (p=0.687) and 0.002 (p=0.838). This suggests that our perturbation scores do not exhibit systematic positional bias. One possible explanation is that, due to the attention mechanism, each token may primarily attend to a relatively small subset of preceding tokens, rather than being equally influenced by all preceding tokens.
(iii) Is our method biased toward step length? Our method's performance is not driven by step length. Please refer to our detailed response to Reviewer 4eMg (Point 4), where we show that the sentences flagged by our method are not significantly longer than the overall average. Notably, our method estimates uncertainty at the token level, and even within the same sentence, some tokens may have very low uncertainty. Therefore, maximum aggregation is more appropriate than average aggregation, as it better captures highly uncertain tokens without diluting their scores.
W2. Experiment Setting Validation.
(i) The use of proxy ground truth. We acknowledge that a model's intrinsic uncertainty can be hard to directly observe, making proxy ground truths such as "first wrong step" and "fragile correct step" necessary. In fact, evaluations of UQ methods generally rely on observable proxies. For example, to assess whether an uncertainty metric can effectively predict errors in model outputs.
(ii) If identifying fragile correct steps helps demonstrate uncertainty, does our design in Section 4.2 has statistical or selection bias? First, we acknowledge that 10 regeneration samples may not exactly estimate the probability in Eq. (10). Indeed, estimating regeneration risk is computationally expensive, as it requires regeneration at different positions across different responses. Nevertheless, we argue that reporting the maximum risk among selected steps is reasonable, since fragile correct steps is inherently sparse, and our goal is to identify these highly fragile steps rather than assess their average risk.
(iii) GPT Annotation Examination in GPQA and Confidence Internals. GPQA benchmark is a graduate-level scientific benchmark, which is beyond our annotators' expertise, thus we could not provide reliable human verification. We provide more clarification on pairwise Confidence Interval (CI) in Q5, Random Selection baselines in Q7, and report CI for Table 2 in our revised paper.