acceptodds
Under review as a conference paper at ICLR 2027

Uncertainty Localization in LLM Reasoning via Embedding Perturbations

Abstract

Large Language Models (LLMs) have achieved significant breakthroughs across various domains, but they can still produce unreliable or misleading outputs. For responsible LLM applications, uncertainty quantification techniques are used to estimate a model’s uncertainty about its outputs, indicating the likelihood that those outputs may be problematic. For LLM reasoning tasks, it is essential to estimate uncertainty not only in the final answer but also in the intermediate reasoning process, particularly to identify where uncertainty arises. Such information may enable more fine-grained and targeted interventions during inference. In this study, we investigate which metrics can effectively localize uncertain places within an LLM reasoning trajectory. Our study reveals that uncertain intermediate continuations are more likely to occur at tokens that are highly sensitive to perturbations in the embeddings of preceding tokens. In our experiments, we show that such perturbation-based metrics achieve stronger performance in localizing uncertain intermediate steps than baseline methods, including probability-based, sampling-based, and Bayesian-based approaches. Meanwhile, our proposed metrics also enjoy good simplicity and efficiency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model, Reasoning, Uncertainty Quantification, Embedding Perturbation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.40 over 5 reviews, above 68% of the others.
Reviewer_Aq5MOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper studies uncertainty localization in LLM reasoning trajectories: instead of scoring only the final answer, it identifies intermediate steps or tokens where uncertainty arises. The authors argue that NLL and token entropy are confounded by “benign variability” (e.g., synonym choices or alternative valid phrasings), and propose scores based on the sensitivity of generated tokens to perturbations of preceding token embeddings:

  1. Random Perturbation (Rand. Pert.): Gaussian noise ϵ∼N(0,σ2I)\epsilon \sim \mathcal{N}(0, \sigma^2 I) is added to all token embeddings, and the score is the variance of the token log-probability over ll samples, Pert.(xt)=Varϵ[log⁡Pr⁡(xt∣H~1:t−1)]\text{Pert.}(x_t) = \text{Var}_\epsilon[\log \Pr(x_t \vert{} \tilde{H}_{1:t-1})]. Proposition 3.3 relates this variance to the probability of a “less-supportive neighboring condition.”

  2. Adversarial Perturbation (Adv. Pert.): Embeddings are perturbed along H^=H−αsgn(∇HF(H))\hat{H} = H - \alpha \text{sgn}(\nabla_H \mathcal{F}(H)), where F(H)\mathcal{F}(H) is the response-level log-likelihood, and the score is the drop in token log-probability, Pert.(xt)=log⁡Pr⁡(xt∣H1:t−1)−log⁡Pr⁡(xt∣H^1:t−1)\text{Pert.}(x_t) = \log \Pr(x_t \vert{} H_{1:t-1}) - \log \Pr(x_t \vert{} \hat{H}_{1:t-1}). Proposition 3.4 interprets this as directional alignment between local token and global sequence gradients. With causal masking, Adv. Pert. needs one backward and one forward pass for all positions.

The authors segment reasoning responses into steps using rule-based sentence boundaries and aggregate token-level scores within each sentence by taking the maximum (max). The method is evaluated against two observable proxies for uncertainty: (1) localizing the "First Wrong Step" (evaluated on MATH and BBH using Llama-3.1-8B, Qwen-2.5-7B, and Mistral-7B, and on GPQA using gpt-oss-20b and Qwen3.5-27B); and (2) identifying "Fragile Correct Steps" prone to errors upon regeneration within responses whose intermediate steps are all correct but exhibit low overall self-consistency (≤0.7\le 0.7). Furthermore, the paper demonstrates the practical utility of these signals in guiding targeted external feedback and revision to improve reasoning accuracy, verifies runtime and memory efficiency on a single GH200 GPU, and candidly discusses limitations regarding factual hallucination detection on FactScore, white-box access requirements, and the scope of relative within-trajectory ranking versus cross-problem calibration.

Critical strengths
  1. Well-targeted problem formulation with a simple and computationally efficient method.

    The paper advances LLM reasoning uncertainty from coarse-grained response-level estimation to fine-grained intermediate-step localization, addressing the practical challenges that external Process Reward Models (PRMs) are often unavailable in open-domain reasoning and that raw probability/entropy metrics suffer from benign lexical variability. By exploiting autoregressive causal masking, the proposed approach avoids expensive token-by-token regeneration required by forking tokens as well as architecture-specific low-rank weight perturbation designs required by TokUR. It also achieves substantial efficiency gains: compared with 12.79s for original response generation, Rand. Pert. (l=20l=20) adds only 0.90s and Adv. Pert. adds only 0.28s (with ~3 GB extra GPU memory), which is substantially faster than TokUR (3.47s) and the multi-sampling method Graph Uncertainty (49.55s).

  2. Consistent empirical performance across multi-faceted proxy evaluations and transparent boundary analysis.

    Beyond evaluating "first wrong step" localization, the paper designs a "fragile correct step" regeneration-risk evaluation to distinguish genuine uncertainty detection from mere error detection. In the 12 reported model–dataset–cutoff settings in Table 1, Adv. Pert. achieves higher values than all non-proposed baselines (e.g., 0.45 on MATH with Llama at the 10% cutoff, compared with 0.27 for TokUR-max). Appendix B further supports evaluation reliability by reporting a 93.9% human alignment rate on 180 "first wrong step" annotations and a 97.5% alignment rate on 80 regenerated-step correctness judgments. Moreover, Section 5.2 and Figure 11 transparently report weaker performance on FactScore and a failure case involving confident reasoning errors, helping clarify the method's operational boundaries.

Critical weaknesses
  1. The theoretical and empirical link between "embedding sensitivity" and "reasoning uncertainty" is only partially established, with potential confounding from position and step-length biases.

    Lemma A.1 and Proposition 3.4 use first-order Taylor expansions to show that the proposed metrics essentially approximate the gradient norm of local log-probability with respect to preceding embeddings or its directional alignment with the sequence-level gradient. While this clarifies their mathematical meaning as embedding sensitivity measures, it does not rigorously establish their equivalence to reasoning uncertainty. The claim that perturbed prefixes H~1:t−1\tilde{H}_{1:t-1} form "valid neighboring conditions" relies primarily on qualitative coherence in a few regenerated examples (Figures 2, 6, and 7); however, surface fluency does not guarantee that key mathematical or logical premises and constraints in the prefix remain intact after perturbation. Furthermore, as shown in the Qwen failure case in Figure 11, the method fails when the model makes confident errors. When aggregating token-level scores into step-level scores, the paper only reports maximum aggregation (max) for the proposed methods, whereas in Table 1 both max and mean (avg) aggregations are reported for NLL, Entropy, and TokUR. Because rule-segmented natural sentences vary substantially in length, taking the max may systematically bias selection toward longer steps containing more tokens. Meanwhile, as the token position tt advances, the perturbed prefix embedding dimension (t−1)×d(t-1) \times d grows linearly, and the response-level objective F(H)\mathcal{F}(H) in Adv. Pert. accumulates more downstream gradient terms for earlier tokens; the paper lacks a quantitative check on whether token position induces systematic score drift.

  2. Dependence on proxy ground truths and automated GPT-5.6 annotations, alongside statistical and selection biases in parts of the evaluation, limits the strength of the conclusions.

    The "first wrong step" is a proxy for incorrectness rather than intrinsic model uncertainty. While the authors complement this with "fragile correct steps" (Table 3), the regeneration risk is estimated using only n=10n=10 samples and computed exclusively on the top-10%/20% steps flagged by each method before taking the maximum (max). Not only does this introduce candidate selection bias, but taking the maximum over noisy small-sample estimates also introduces an upward statistical bias. Additionally, human verification covers 180 first-error cases, 30 additional cases checked for absence of errors, and 80 regeneration judgments on MATH and BBH, omitting the more challenging GPQA benchmark (Table 2, which also excludes TokUR and Graph Uncertainty). Regarding statistical uncertainty, Tables 1 and 7 only report approximate confidence intervals for a single proportion rather than paired intervals for differences between methods; Table 3 reports a uniform ±0.049\pm 0.049 interval whose construction is unexplained; and Table 2 reports no intervals at all. Some differences are small (e.g., the margins in Table 1 for Qwen/MATH at 20% between Adv. Pert. 0.52 and CCP 0.50, and in Table 2 for Qwen3.5 at 20% between 0.50 and TokenSAR 0.49). Finally, the revision pipeline in Table 4 relies on GPT-5.6 to judge errors and provide revision suggestions but lacks a "Random Step Selection" baseline (randomly selecting 10%/20% of steps for GPT-5.6 inspection), making it difficult to isolate the localization signal's contribution from GPT-5.6's strong inherent error-correction capability.

Questions for authors
  1. Under σ=0.001,α=0.0001\sigma=0.001,\alpha=0.0001, how does the direct analytic gradient norm/inner product compare with l=20l=20 random samples or a finite-step adversarial perturbation? In Eq. 6, when Pert.(xt)<0\text{Pert.}(x_t)<0 (e.g., 'Now' rises from 0.173 to 0.268 in Fig. 6(E)), what is the interpretation, and how are negative values handled in aggregation/ranking?

  2. Beyond Figures 2, 6, and 7, can you quantitatively verify on a representative subset that continuations from perturbed prefixes at σ=0.001,α=0.0001\sigma=0.001,\alpha=0.0001 preserve the mathematical/logical premises and constraints, ruling out confounding from disrupted premise representations?

  3. In Table 1 and Table 7, NLL/Entropy/TokUR report both max and avg, while Rand. Pert./Adv. Pert. report only max; Table 2 includes NLL/Entropy but not TokUR. What are the coverage rates for Rand. Pert.-avg/Adv. Pert.-avg? Are top-10%/20% steps selected by max significantly longer than average steps?

  4. In long GPQA responses (~2,000 tokens), does Pert.(xt)\text{Pert.}(x_t) drift systematically with position tt? How does performance change if scores are normalized by t−1t-1, or if only a fixed recent window is perturbed?

  5. Tables 1 and 7 give only a single-proportion interval (±0.057\pm 0.057). Given narrow margins, can you report paired confidence intervals or McNemar pp-values against the strongest baseline per setting? What is the human alignment rate for GPT-5.6 first-wrong-step annotations on GPQA?

  6. How was Table 3’s ±0.049\pm 0.049 derived, and does it include binomial variance from n=10n=10? Are regenerations/judgments reused when methods select the same step? Does the advantage hold using mean risk over top-10%/20% steps, or re-evaluating the highest-risk step with fresh samples?

  7. In Table 4, what is the accuracy of a “randomly select 10%/20% of steps for GPT-5.6 inspection and feedback” baseline? Among flagged steps, what fraction is judged by GPT-5.6 as introducing a new error and triggering feedback?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Aq5M (Section 1)

We thank the reviewer for the comments. We first provide clarifications to the major weakness raised by reviewers. Then, we address other questions.

W1. Theoretical link to reasoning uncertainty. Is our method biased to token position or step length?

  • (i) Theoretical link to reasoning uncertainty. We agree with the reviewer that our original theoretical analysis may not have sufficiently established the connection to reasoning uncertainty. This concern was also raised by Reviewers 4eMg and R5XN. To address it, we have provided a new theoretical analysis in Section 3.3 of our revised paper. We kindly refer the reviewer to our response to Reviewer 4eMg and the revised manuscript. The key insight of our new theory is that our perturbation sensitivity score can help identify tokens where the model has a strong tendency to favor "conflicting alternatives". Such alternatives may lead to contradictory reasoning continuations, thereby reflecting reasoning uncertainty. In our Section 3.3, we mathematically characterize these conflicting alternatives and establish their connection to our perturbation sensitivity score.

  • (ii) Is our method biased toward token position? We find our perturbation scores do not exhibit systematic drift with token position. To show this, we compute the Spearman rank correlation between uncertainty scores and absolute token positions on the GPQA (longest) dataset. For Rand. Pert., the correlations are -0.062 (p=0.443p=0.443) and -0.064 (p=0.377p=0.377) for gpt-oss-20b and Qwen3.5-27B, respectively. For Adv. Pert., the corresponding values are -0.012 (p=0.687p=0.687) and 0.002 (p=0.838p=0.838). This suggests that our perturbation scores do not exhibit systematic positional bias. One possible explanation is that, due to the attention mechanism, each token may primarily attend to a relatively small subset of preceding tokens, rather than being equally influenced by all preceding tokens.

  • (iii) Is our method biased toward step length? Our method's performance is not driven by step length. Please refer to our detailed response to Reviewer 4eMg (Point 4), where we show that the sentences flagged by our method are not significantly longer than the overall average. Notably, our method estimates uncertainty at the token level, and even within the same sentence, some tokens may have very low uncertainty. Therefore, maximum aggregation is more appropriate than average aggregation, as it better captures highly uncertain tokens without diluting their scores.

W2. Experiment Setting Validation.

  • (i) The use of proxy ground truth. We acknowledge that a model's intrinsic uncertainty can be hard to directly observe, making proxy ground truths such as "first wrong step" and "fragile correct step" necessary. In fact, evaluations of UQ methods generally rely on observable proxies. For example, to assess whether an uncertainty metric can effectively predict errors in model outputs.

  • (ii) If identifying fragile correct steps helps demonstrate uncertainty, does our design in Section 4.2 has statistical or selection bias? First, we acknowledge that 10 regeneration samples may not exactly estimate the probability in Eq. (10). Indeed, estimating regeneration risk is computationally expensive, as it requires regeneration at different positions across different responses. Nevertheless, we argue that reporting the maximum risk among selected steps is reasonable, since fragile correct steps is inherently sparse, and our goal is to identify these highly fragile steps rather than assess their average risk.

  • (iii) GPT Annotation Examination in GPQA and Confidence Internals. GPQA benchmark is a graduate-level scientific benchmark, which is beyond our annotators' expertise, thus we could not provide reliable human verification. We provide more clarification on pairwise Confidence Interval (CI) in Q5, Random Selection baselines in Q7, and report CI for Table 2 in our revised paper.

AuthorsOpenReview ↗

Response to Reviewer Aq5M (Section 2)

Q1. Analytic gradient norm & whether adversarial perturbation sensitivity <0< 0

Calculating analytic gradient norm as a substitution of random perturbation could be inefficient, as it requires conducting backward pass for each token. In our test, we check 10 cases on Llama Math dataset using analytic gradient norm and random perturbation, and find that they flag almost the same sentences. In Figure 6 (E) (now Figure 7(E) in the new paper), adversarial perturbation is applied to decrease the probability of token xtx_t="At". This suggests Pert(xt)>0\mathrm{Pert}(x_t) > 0. For the token "now", it is an alternative of xtx_t="At", which is not the generated token.

Q2. Continuations from perturbed prefixes preserve premise representations.

We follow the reviewer's suggestion to verify that: continuations from perturbed prefix preserve premise representations. In MATH and BBH dataset, we sample 300 responses using Llama. For each sample, we randomly select steps and regenerate the subsequent sentence under perturbed prefix. The average semantic similarity (cosine similarity embedded by "all-MiniLM-L6-v2") between the original and regenerated steps is 0.9873 / 0.9772 (MATH / BBH) under random perturbation and 0.9826 / 0.9766 under adversarial perturbation. Their similarity indicates that our perturbations preserve the mathematical and logical premises and constraints.

Q3. Perturbation score in average-aggregation; GPQA does not include TokUR

Since our method identifies uncertainty at the token level, maximum aggregation is more appropriate than average aggregation, as other tokens within the same sentence may have very low uncertainty. Please refer to our response to Reviewer SBx7 (Point 6) for the full results under average aggregation, which further demonstrate the necessity of maximum aggregation. Meanwhile, the sentences identified by our method are not significantly longer than the overall average. We provide detailed sentence-length statistics in our response to Reviewer 4eMg (Point 4). Finally, we exclude TokUR from our GPQA experiment because its weight perturbation approach requires substantial model-specific reimplementation efforts for more advanced models (see the response to Reviewer 4eMg (Point 3).

Q4. Does perturbation scores drift systematically with position in GPQA?

Plz refer to our response to Weakness 1 (ii).

AuthorsOpenReview ↗

Response to Reviewer Aq5M (Section 3)

Q5. Pairwise CI for Section 4.1; GPQA annotation

The table below reports the paired 95% CI between our method and TokUR-max under the setting of Table 1 in our paper. The table below reports the performance improvement with 95% paired confidence intervals. Our method outperforms strong baseline TokUR in most cases. Besides, GPQA consists of graduate-level hard science questions, for which human annotations by the authors may be less reliable than those provided by GPT-5.6, given the specialized domain knowledge required.

MathLlamaQwenMistral
Rand. Pert.-TokUR CI (top-10%)+0.13 ±\pm 0.05+0.07 ±\pm 0.05+0.03 ±\pm 0.04
Rand. Pert.-TokUR CI (top-20%)+0.07 ±\pm 0.05+0.05 ±\pm 0.05+0.06 ±\pm 0.04
Adv. Pert.-TokUR CI (top-10%)+0.18 ±\pm 0.04+0.09 ±\pm 0.06+0.06 ±\pm 0.04
Adv. Pert.-TokUR CI (top-20%)+0.12 ±\pm 0.05+0.07 ±\pm 0.05+0.06 ±\pm 0.05
LogicLlamaQwenMistral
Rand. Pert.-TokUR CI (top-10%)+0.06 ±\pm 0.04+0.09 ±\pm 0.06+0.04 ±\pm 0.05
Rand. Pert.-TokUR CI (top-20%)+0.08 ±\pm 0.05+0.08 ±\pm 0.06+0.02 ±\pm 0.04
Adv. Pert.-TokUR CI (top-10%)+0.14 ±\pm 0.05+0.11 ±\pm 0.06+0.11 ±\pm 0.05
Adv. Pert.-TokUR CI (top-20%)+0.16 ±\pm 0.05+0.10 ±\pm 0.05+0.05 ±\pm 0.04

Q6. How to calculate CI for Table 3; 4.2 experimental details

Table 3's 95% CI of ±0.049\pm 0.049 is calculated using the sample standard deviation, without assuming a specific distribution for the observed data. In Section 4.2, if a step is flagged by multiple methods, its regeneration risk estimate (including the underlying regenerations and judgments) is reused. In Section 4.2, reporting the mean regeneration risk across selected steps would not adequately reflect a method's ability to identify highly fragile correct steps, as such steps may be sparse within a reasoning trajectory. Therefore, we use the maximum risk to evaluate whether a method can successfully identify these steps.

Q7. Questions related to 4.3 experiment details

We provide random baseline result for Section 4.3 Table 4 in our updated paper. The reviewer can also refer to our response to reviewer 4eMg (Point 5) for the results. Meanwhile, among the responses that our methods provide flagged steps, about 40% to 50% are judged by GPT-5.6 as introducing a new error and triggering feedback. For example, in Llama Math (Precalculus), given top-10% flagged steps, 44% of random perturbation and 48% of adversarial perturbation cases are judged to contain error. As a comparison, there are 22% of cases judged as containing error provided by NLL localization.

Reviewer_SBx7OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper tries to find which steps in an LLM's reasoning are uncertain. The idea is to add small noise to the input embeddings and see how much each token's probability changes. They use i) random noise (variance over 20 runs) and ii) an adversarial version (one sign-gradient step). Then they check if the flagged sentences contain the first wrong step, on other datasets.

Critical strengths
  • The problem statement is useful. Knowing where the reasoning goes wrong is more helpful than one score for the whole answer.
  • The method is simple and fast and the gains on the two models tested, Llama and Mistral in Table 1 are large (e.g., 0.45 vs 0.27 on Llama MATH).
Critical weaknesses
  • Random input noise and perturbation are already used for uncertainty in vision, and for LLMs. TokUR does perturbation-based token uncertainty for reasoning. This paper perturbs embeddings instead of weights. The related work only mentions adversarial robustness and TokUR, and does not discuss earlier work that uses perturbation to measure uncertainty.
  • There is no random baseline, i.e., picking 10% or 20% of the steps by chance without any score, especially since at least one step is always picked, so chance can be above 10% for short responses. Simple rules like flagging the longest sentences, or sentences at a certain position, would be helpful..
  • The results are not statistically strong enough. Many gains are small and within the given confidence intervals, and Tables 2, 4, 5, 8, 9 and Figure 5 have no error bars. Since all methods are scored on the same responses, paired significance tests should be reported. Section 4.3 involves random generation from both the base model and GPT-5.6, so it should be repeated over runs and reported as mean ± std.
  • Section 4.3 uses GPT-5.6 to simulate a human checking the flagged steps, yet GPT-5.6 is a strong model that can likely find and fix errors by itself, which goes opposite to the idea that verifiers are not available as discussed int he paper. A control where GPT-5.6 checks random steps is needed to show the gain comes from the flagged steps and not from GPT-5.6.
  • The evaluation is rather narrow (mostly math and logic puzzles), and the ground truth comes from GPT-5.6 with a small human check. The method also does worse than NLL on FactScore for 2 of 3 models, so I'm not sure it works for knowledge-heavy reasoning like medical or physics QA. Using datasets with human step labels (e.g., PRM800K, ProcessBench) could make it stronger. I think the finding is interesting, but the paper is not ready yet. It needs more datasets, stronger baselines, and proper significance testing to support the claims.
Questions for authors
  • You use the maximum score within each sentence for your method, but test both maximum and average for the baselines. Why only the maximum for yours, and how does the average perform?
  • Each sentence is treated as one reasoning step. How much do the results change if steps are split differently, for example by line breaks or equations?
  • The worst-case version uses the whole response, including the text after each word. Can the score be computed while the model is still writing, as the abstract suggests?
  • In Figure 6, the worst-case noise changes "left" to "right," which is a large change. Does the math in the propositions, which assumes small changes, still apply here?
  • Can you report a ranking metric like per-response AUROC? You already use AUROC in Table 5.

Minor: there are grammar mistakes (e.g., line 44 "it is potential to use", line 90 "While, ...").

5 comments
AuthorsOpenReview ↗

Response to Reviewer SBx7 (Section 1)

We thank the reviewers for the comments and questions. We are happy to provide clarifications about each of them.

1. Relation to existing works that also use perturbation to measure uncertainty. (W1)

We agree with the reviewer that several prior works have explored using perturbations to measure uncertainty in LLMs. For example, the work [a] (also suggested by reviewer Wdjz) perturbs LLM inputs through paraphrasing and measures the consistency of generated outputs. However, these works focus on response-level UQ, which is distinct from our objective of uncertainty localization, i.e., identifying which positions within a given LLM reasoning trajectory are relatively more uncertain. Therefore, although these works also employ perturbations, their uncertainty measurement approaches cannot be directly applied to our problem. Moreover, in our response to reviewer Wdjz, we further demonstrate the performance limitation of alternative perturbation strategies, including the one used in [a].

This is why we discuss TokUR [b] as a particularly relevant work, since it also measures token-level uncertainty through perturbations. Even compared with TokUR, our method has clear methodological distinctions and novelty. For example, we propose an adversarial perturbation strategy that captures each token's uncertainty in relation to the overall reasoning trajectory. Meanwhile, by perturbing embeddings rather than model weights, our method is much simpler to implement and more computationally efficient, while achieving stronger performance.

Importantly, to the best of our knowledge, our work is the first to systematically evaluate uncertainty localization accuracy. For example, although previous work such as TokUR also studies intermediate uncertainty, it does not systematically evaluate localization accuracy at the step or token level. We will expand the related work discussion to better acknowledge prior perturbation-based UQ studies.

[a] Spuq: Perturbation-based uncertainty quantification for large language models.

[b] Tokur: Token-level uncertainty estimation for large language model reasoning.

2. Random baselines for main experiment (W2)

Following the reviewer's suggestion, we additionally evaluate two simple random baselines under the same experimental setting of Table 1: (1) Random, which randomly selects 10% or 20% of the reasoning steps without using uncertainty scores; and (2) Step Length, which selects the 10% or 20% longest steps. As shown in the table below, both baselines perform substantially worse than our methods. Notably, the random baseline achieves only around 10–12% accuracy at the 10% cutoff, confirming that the effect of rounding is minimal.

MethodMath LlamaMath QwenMath MistralLogic LlamaLogic QwenLogic Mistral
10% 20%10% 20%10% 20%10% 20%10% 20%10% 20%
Random0.11 0.200.10 0.200.12 0.210.10 0.200.11 0.200.10 0.20
Step Length0.18 0.280.17 0.320.16 0.270.20 0.280.17 0.360.14 0.31
Rand. Pert.0.40 0.540.33 0.500.31 0.500.33 0.500.38 0.550.28 0.46
Adv. Pert.0.45 0.590.35 0.520.34 0.500.42 0.580.40 0.570.35 0.50
AuthorsOpenReview ↗

Response to Reviewer SBx7 (Section 2)

3. Statistical significance of our experiments (W3)

We respectfully disagree that our performance gains are small, and we have added the necessary error bars in our revised paper. As shown in Tables 1 and 2, our methods demonstrate significant improvements over existing baselines across most experimental settings, while keeping consistently competitive with the strongest baselines in the few remaining settings.

To further address the reviewer's concern, we conduct paired statistical comparisons against TokUR, one of our strongest baselines, following the setup of our main result in Table 1. The table below reports the performance improvement with 95% paired confidence intervals. As shown, the improvements are statistically significant in the vast majority of settings, further supporting the effectiveness of our methods.

MathLlamaQwenMistral
Rand. Pert.-TokUR CI (top-10%)+0.13 ±\pm 0.05+0.07 ±\pm 0.05+0.03 ±\pm 0.04
Rand. Pert.-TokUR CI (top-20%)+0.07 ±\pm 0.05+0.05 ±\pm 0.05+0.06 ±\pm 0.04
Adv. Pert.-TokUR CI (top-10%)+0.18 ±\pm 0.04+0.09 ±\pm 0.06+0.06 ±\pm 0.04
Adv. Pert.-TokUR CI (top-20%)+0.12 ±\pm 0.05+0.07 ±\pm 0.05+0.06 ±\pm 0.05
LogicLlamaQwenMistral
Rand. Pert.-TokUR CI (top-10%)+0.06 ±\pm 0.04+0.09 ±\pm 0.06+0.04 ±\pm 0.05
Rand. Pert.-TokUR CI (top-20%)+0.08 ±\pm 0.05+0.08 ±\pm 0.06+0.02 ±\pm 0.04
Adv. Pert.-TokUR CI (top-10%)+0.14 ±\pm 0.05+0.11 ±\pm 0.06+0.11 ±\pm 0.05
Adv. Pert.-TokUR CI (top-20%)+0.16 ±\pm 0.05+0.10 ±\pm 0.05+0.05 ±\pm 0.04

Regarding Section 4.3, all methods are evaluated on the same 100 questions and, more importantly, the same initially generated responses. Therefore, randomness in the initial response generation does not introduce differences between methods. Moreover, we would like to clarify that Table 4 reports the overall accuracy across all evaluated questions, including both high- and low-uncertainty responses. For low-uncertainty responses, our uncertainty localization method is not invoked, as no additional inspection is performed. Therefore, the overall accuracy may dilute the improvements brought by uncertainty localization. To further illustrate these differences, we provide additional results below focusing on high-uncertainty responses and our methods achieve more significant improvements (under Logic dataset). We provide the complete results in Table 10 in our revised paper.

MethodLlamaQwen
10%20%10%20%
Uniform0.620.690.600.70
NLL-max0.680.760.660.78
Entropy-max0.750.800.670.75
TokUR-max0.700.770.690.79
Rand. Pert.0.820.890.760.80
Adv. Pert.0.890.950.790.87

4. Random baseline for experiment in Section 4.3 (W4)

To address the reviewer's concern and further isolate the contribution of our uncertainty localization methods in Section 4.3 (Table 4), we introduce an additional baseline called Uniform, which uniformly samples 10% or 20% of the reasoning steps for examination by GPT-5.6. As shown in the table below, our methods consistently outperform this baseline, demonstrating that the improvements are not merely due to the use of GPT-5.6. In this table, Math1 means the subset "Precalculus", Math2 means "Cnt. & Prob."

MethodMath1 LlamaMath1 QwenMath2 LlamaMath2 QwenLogic LlamaLogic Qwen
10% 20%10% 20%10% 20%10% 20%10% 20%10% 20%
Uniform0.41 0.480.69 0.700.58 0.610.83 0.840.89 0.900.89 0.89
Rand. Pert.0.47 0.560.75 0.760.66 0.750.87 0.880.98 0.980.93 0.94
Adv. Pert.0.51 0.540.78 0.790.70 0.770.88 0.890.98 0.970.94 0.95

5. Evaluation scope and ground-truth reliability (W5)

We respectfully disagree with the reviewer's assessment that our evaluation scope is narrow. In fact, our evaluation has been recognized as comprehensive or promising by other reviewers, including 4eMg, R5XN, and Aq5M.

  • (1) Our evaluation is not limited to math and logic. We also consider GPQA (Table 2), a challenging knowledge-intensive scientific reasoning benchmark, where our methods continue to show strong performance.

  • (2) As shown in Appendix B, we manually examined 180 reasoning responses and observed 93.9% agreement with GPT-5.6's annotations, representing a substantial human validation effort.

  • (3) PRM800K and ProcessBench cannot be directly used for our evaluation, because their step-level annotations are tied to pre-existing reasoning trajectories rather than those generated by our evaluated models. Therefore, we still need to annotate the newly generated reasoning trajectories ourselves.

AuthorsOpenReview ↗

Response to Reviewer SBx7 (Section 3)

6. Why use maximum-aggregation for our method? (Q1)

Our method assumes only a few critical tokens may exhibit high uncertainty, while the remaining tokens can be very certain. Therefore, averaging token-level scores in sentences would dilute these important uncertainty signals and degrade our method's performance. For the baselines, we report both maximum and average aggregation to ensure a fair comparison and avoid disadvantaging them. We also report the average-aggregation results of our method in the table below, which further confirm that maximum aggregation consistently performs better.

MethodMath LlamaMath QwenMath MistralLogic LlamaLogic QwenLogic Mistral
10% 20%10% 20%10% 20%10% 20%10% 20%10% 20%
Rand. Pert.-avg0.37 0.470.29 0.480.30 0.460.28 0.460.32 0.520.24 0.38
Adv. Pert.-avg0.40 0.550.34 0.490.29 0.490.36 0.520.34 0.520.30 0.45
Rand. Pert.-max0.40 0.540.33 0.500.31 0.500.33 0.500.38 0.550.28 0.46
Adv. Pert.-max0.45 0.590.35 0.520.34 0.500.42 0.580.40 0.570.35 0.50

7. Step segmentation strategy. (Q2)

We would like to clarify that our step segmentation already considers line breaks or equations, rather than simply splitting by sentences. As described in Appendix C, we use punctuation marks and line breaks as step boundaries. In mathematical reasoning, standalone equations are typically written on separate lines and are therefore naturally treated as individual steps. Thus, the suggested segmentation strategies are already largely covered by our current evaluation.

8. Can the score be computed while the model is writing? (Q3)

Adversarial perturbation method requires access to the full response and therefore cannot be applied online during generation. However, we have not claimed online uncertainty estimation in our paper. As emphasized throughout our paper, our method is designed for uncertainty localization after a complete response has been generated.

9. Small embedding perturbations and great changes in generated token probability (Q4)

We do not assume that changes in token probabilities must be small. The small-perturbation assumption refers only to the magnitude of embedding perturbations, rather than the resulting changes in token probabilities. Instead, substantial change in the model's prediction is precisely what our method aims to capture. In Figure 6 (now Figure 7), although the adversarial perturbation is very small (α=0.0001)\alpha=0.0001), it can lead to a substantial change in the model's prediction (e.g., generating "right" instead of "left"), which reflecting model uncertainty.

10. Per-response AUROC and Per-step AUROC (Q5)

We believe AUROC is not a suitable evaluation metric for our task. As discussed in Section 4, once an error occurs, subsequent reasoning steps may be confidently generated based on the earlier mistake. Therefore, it is unclear whether these subsequent steps should be considered uncertain or certain, making step-level AUROC not meaningful. Furthermore, aggregating AUROC to the response level is even less appropriate, as our method produces localization scores rather than response-level uncertainty scores. In contrast, for factual hallucination detection in Table 5, hallucinated claims may occur independently, making AUROC more appropriate for evaluating claim-level detection performance.

Reviewer_R5XNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the problem of empirical uncertainty quantification without additional training. The paper proposes token scores based on perturbing the prefix embeddings and see how the logits of each token change (variance or difference); The perturbations are based on random perturbations or adversarial perturbations. Then, rank each sentence in response by the maximum token score in each sentence. Sentence with large score (e.g., top 10%) is suspicious and may contain error. Surprisingly, the authors show that this simple sensitivity score beats many popular baselines such as NLL, entropy, and specialized methods such as Tok UR and Token SAR. Experiments were conducted on multiple 7B - 8B models and two datasets (MATH and BBH).

Critical strengths

The problem is solving a good and practical problem. Especially, localizing errors inside long CoT is a challenging and very useful problem.

I like the “fragile correct step” idea, which sounds original and interesting. It can separate uncertainty from error, by regenerating from the prefix and check if a correct step is stable.

The paper is generally easy to understand and the Figures in the paper present the idea very well. Experiments are relatively comprehensive.

Critical weaknesses

The “theoretical” results in this paper are largely well-known results (e.g., Prop. 3.4 is basically a Taylor expansion, Lemma A.1 is also a standard identity). The propositions (e.g., 3.3) also does not connect sensitivity to correctness, so from a theoretical point of view, it is unclear how this method works. The theory does not tell us that a high score means the token is likely wrong.

The responses on the tested benchmarks are usually short; there is no experiment on long chain-of-though traces, where the argument of “inspect 10% of steps” actually matters.

Although the paper claims to outperform baselines overall, after I carefully look at the numbers, the margins are quite small - for example in Table 3, the difference between NLL-max and Rand. Pert. is sometimes just 0.01 or 0.02, but the caption says “95% conf. interval is ±0.049”. Table 4 has similar results, so I am less confident how much better this method can do in practice (considering most papers tune their own methods with more efforts).

Other experimental issues (see questions below)

Questions for authors

The random perturbation essentially measures the gradient norm, but the gradient norm can also be simply obtained via a backward pass; why not use the gradient norm directly? A comparison to gradient norm will be essential.

The adversarial score seems to depend all tokens (including future tokens). Typically, baselines like NLL or entropy are causal, so the comparison may not be fair as the gain may come from global information rather than local uncertainty. Additional ablation studies are needed.

It is unclear how to calibrate the score. In a real deployment, how do you select thresholds? Can we first run some validation responses and choose a fixed threshold for all future inputs (e.g., top 10%)?

Please cite relevant papers in the gradient-saliency literature which also uses gradients to inspect the behavior of neural networks. One early work is Simonyan et al., “Deep Inside Convolutional Networks”, arXiv 1312.6034 and many works follow.

4 comments
AuthorsOpenReview ↗

Response to Reviewer R5XN (Section 1)

We thank the reviewer for raising the questions, and we are happy to provide clarifications.

1. Theoretical insights into why our method works. (W1)

We agree with the reviewer that our original theoretical analysis (e.g., Proposition 3.3 in the original version) may not provide sufficient intuition and explanation. To address this concern, we have developed a new theoretical analysis in Section 3.3 in our revised paper. Below, we summarize its key intuition and kindly invite the reviewer to refer to Section 3.3 for details.

In our new theory, we consider that a generated token in the LLM response may have "conflicting alternatives" in the vocabulary. For example, the token "left" in our Figure 2 has a conflicting alternative "right". Similarly, for a simple reasoning step 1+1=2, the token "3" can be considered a conflicting alternative to "2". If the LLM generates such alternatives, it will lead to reasoning continuations that contradict the original generation.

For our metric, a high sensitivity score of the token's log probability captures a key phenomenon: when a small embedding perturbation reduces the model's logit support for the originally generated token xtx_t, the model may substantially increase its logit support for certain alternative tokens, thereby amplifying the change in the generation probability of xtx_t. Notably, we use logits here to ensure that the increase is not simply caused by softmax normalization (when calculating token probability from logits). Instead, we argue that such an increase is more likely to come from conflicting alternatives. This is because conflicting alternatives tend to "compete" with the original token xtx_t, so weakening the support for xtx_t may instead increase the support for one or some of its conflicting alternatives. In this way, our method can capture the model's tendency to favor conflicting alternatives, thereby reflecting reasoning uncertainty.

In our response to reviewer 4eMg, we also provide further discussion of why our method is less confounded by multiple benign continuations. These results together provide insights into why perturbation sensitivity is effective at identifying reasoning uncertainty.

2. Evaluation on long Chain-of-Thought reasoning (W2)

Our evaluation indeed includes long chain-of-thought reasoning tasks, such as the experiment in GPQA dataset (Section 4.1, Table 2). It is a graduate-level scientific reasoning benchmark that involves substantially longer reasoning trajectories. On average, responses contain 71.94 steps (2,175.74 tokens) for gpt-oss-20b model and 73.89 steps (2,419.18 tokens) for Qwen3.5-27B. Our method remains effective in this challenging setting.

3. Why are some margins in Table 3 and Table 4 small? (W3)

We agree with the reviewer that the performance gaps between our methods and the baselines are relatively small in some certain settings in Table 3 and Table 4.

  • Regarding Table 3, one possible explanation is that even when a method successfully locates a fragile correct step, the model may still have a high probability of reproducing the original correct reasoning, resulting in a relatively low probability of generating an incorrect continuation. This may limit the performance differences between uncertainty localization methods. Additionally, fragile correct steps may be inherently sparse and difficult to locate within a reasoning trajectory. Nevertheless, our methods, particularly Adv. Pert., show more pronounced improvements when inspecting only the top 10% of steps. These results provide evidence of our methods' effectiveness in identifying fragile correct steps.

  • Regarding Table 4, we would like to clarify that it reports the overall accuracy across all evaluated questions, including both high-uncertainty and low-uncertainty responses. However, for low-uncertainty responses, our uncertainty localization method is not invoked, as these responses do not require additional inspection or correction. Therefore, the overall results may not fully reveal the differences among uncertainty localization methods. To further illustrate these differences, we provide additional results below focusing on high-uncertainty responses that require intervention (under Logic dataset). As shown in the table, our methods achieve more significant improvements over the baselines. We also provide the complete results for other datasets in Table 10 in our revised paper.

MethodLlamaQwen
10%20%10%20%
Uniform0.620.690.600.70
NLL-max0.680.760.660.78
Entropy-max0.750.800.670.75
TokUR-max0.700.770.690.79
Rand. Pert.0.820.890.760.80
Adv. Pert.0.890.950.790.87
AuthorsOpenReview ↗

Response to Reviewer R5XN (Section 2)

4. Why not calculate the norm of the gradient ∥∇HtF(H)∥\lVert \nabla_{\mathbf{H_t}} \mathcal{F}(\mathbf{H}) \rVert? (Q1)

We clarify that our random perturbation approximates a gradient norm that cannot be computed for all tokens in a single backward pass. Following the notations of our paper, we define the log-probability of each generated token xtx_t as ft(H1:t−1)=log⁡Pr⁡(xt∣H1:t−1)f_t(\mathbf{H}_{1:t-1}) = \log \Pr(x_t \mid \mathbf{H}_{1:t-1}), and the total log-probability of the whole response as F(H)=∑i=m+1m+nfi(H1:i−1)\mathcal{F}(\mathbf{H}) = \sum_{i=m+1}^{m+n} f_i(\mathbf{H}_{1:i-1}). Under a first-order Taylor approximation, our random perturbation score satisfies Pert.(xt)≈σ2∥∇H1:t−1ft(H1:t−1)∥22\mathrm{Pert.}(x_t) \approx \sigma^2 \lVert \nabla_{\mathbf{H}_{1:t-1}} f_t(\mathbf{H}_{1:t-1})\rVert _2^2. This corresponds to the gradient norm of the current token's log-probability with respect to its preceding token embeddings. It cannot be obtained for all generated tokens through a single standard backward pass. In contrast, the reviewer may be referring to the gradient norm ∥∇HtF(H)∥2\lVert \nabla_{\mathbf{H}_t}\mathcal{F}(\mathbf{H})\rVert_2, which measures the gradient norm of the total response log-probability with respect to each token's embedding. This is different from the token-level gradient norm of our method.

We additionally compare our method with this gradient-norm ∥∇HtF(H)∥\lVert \nabla_{\mathbf{H_t}} \mathcal{F}(\mathbf{H}) \rVert as a baseline under the same experimental setting as Table 1. As shown in the table below, using this gradient norm is less effective than our perturbation-based methods for uncertainty localization.

MethodMath LlamaMath QwenMath MistralLogic LlamaLogic QwenLogic Mistral
10% 20%10% 20%10% 20%10% 20%10% 20%10% 20%
grad norm-max0.28 0.420.25 0.440.30 0.440.25 0.390.25 0.450.24 0.37
grad norm-avg0.22 0.380.22 0.390.27 0.410.23 0.360.21 0.390.22 0.36
Rand. Pert.0.40 0.540.33 0.500.31 0.500.33 0.500.38 0.550.28 0.46
Adv. Pert.0.45 0.590.35 0.520.34 0.500.42 0.580.40 0.570.35 0.50

5. Use of future-token information in adversarial perturbation. (Q2)

We agree with the reviewer that our adversarial perturbation score is influenced by future tokens, as the perturbation direction is determined using the entire response. Importantly, our goal is to identify the relatively more uncertain steps within an already generated reasoning trajectory, rather than estimate uncertainty online during generation. Therefore, using information from the complete response can be reasonable and does not make the comparison unfair. Moreover, our random perturbation method does not use future-token information to determine perturbations, but it also outperforms the baselines, suggesting that our improvements are not solely due to the use of global information.

6. Do we need to calibrate the score? (Q3)

We would like to point out that our problem does not necessarily require a calibrated threshold. Unlike traditional uncertainty quantification, our uncertainty localization problem aims to answer: "Once a response is considered uncertain, at which places in the reasoning trajectory does the uncertainty arise?" Thus, our goal is to identify the relatively more uncertain steps within a given response, and rank these steps by their uncertainty scores. In practice, an inspector can examine steps in descending order of uncertainty scores until identifying a step that may need correction. Indeed, Section 4.3 illustrates this practical application, where we prioritize uncertain steps for external inspection and correction, starting with the top 10% and potentially expanding the inspection range as needed.

7. Citing more related work. (Q4)

We thank the reviewer for the suggestion and have added citations to relevant work in the gradient-based saliency literature, particularly the work of Simonyan et al.

Thank you for the response and detailed explanations. I really appreciate the efforts the authors made. The new theoretical insights and the clarification of the gradient norm are helpful. However several key weaknesses still remains: the "theoretical results" look more like a post-hoc explanation rather than coming from first-principle; the authors also admit the improvement compared baselines are sometimes small; the long CoT experiments have about ~2000 tokens only (much lower than today's reasoning model can practically use - 2k reasoning token is a really low number in today's standard); no ablation study on the future-token information use in adversarial perturbation; and no comparison to the baseline of the true gradient norm on H1:t−1H_{1:t-1} the random perturbation is approximating (the true gradient needs more computation, but it shows the ideal behavior; whether the proposed method is approximating the ideal behavior is a key question); and how accurately the random perturbation can actually approximate the gradient norm (typically, random perturbation is a very high variance estimation, so I doubt whether it actually works). In addition, I agree with other reviewers random perturbations have been previously widely studied in different scenarios and the novel contribution of this work is limited. Thus I believe the paper is borderline and I cannot strongly support its acceptance.

We thank the reviewer for acknowledging our efforts and the additional theoretical insights provided in our revision.

In this response, we would like to address the new question raised by the reviewer: Can the sensitivity score obtained from random perturbations approximate the squared gradient norm, and how large is the estimation error? This analysis may help explain why our method works.

Following our previous notation, let gt=∇H1:t−1ft(H1:t−1)g_t=\nabla_{\mathbf{H}_{1:t-1}}f_t(\mathbf{H}_{1:t-1}). Under a first-order Taylor approximation, we have ft(H1:t−1+ϵ)−ft(H1:t−1)≈gt⊤ϵ∼N(0,σ2∥gt∥22).f_t(\mathbf{H}_{1:t-1}+\epsilon)-f_t(\mathbf{H}_{1:t-1})\approx g_t^\top\epsilon\sim\mathcal{N}(0,\sigma^2\lVert g_t\rVert_2^2). For ll independent perturbations, let St2S_t^2 denote our estimated sensitivity score (the variance of log-probability change). Then, it satisfies (l−1)St2σ2∥gt∥22∼χl−12.\frac{(l-1)S_t^2}{\sigma^2\lVert g_t\rVert_2^2}\sim\chi^2_{l-1}. Therefore, the relative standard deviation of the estimated sensitivity score is: Std(St2)E[St2]=2l−1≈0.324(l=20).\frac{\mathrm{Std}(S_t^2)}{\mathbb{E}[S_t^2]}=\sqrt{\frac{2}{l-1}}\approx 0.324\quad(l=20).

This means that, with 20 perturbation samples, the standard deviation of the estimated sensitivity score is approximately 32.4% of its expected value, σ2∥gt∥22\sigma^2\lVert g_t\rVert_2^2. However, such sampling variability can be outweighed by differences in sensitivity magnitudes across tokens. In practice, as illustrated by the un-normalized scores for uncertain responses (red curves) in Figure 9, most tokens exhibit relatively small sensitivity scores, while a few critical tokens have substantially larger scores, often several times larger than those of other tokens. Thus, our method can still distinguish tokens with larger gradient norms from those with smaller ones. Empirically, we further compare random perturbation sensitivity with the exact squared gradient norm (scaled by σ2\sigma^2) on the MATH dataset using Llama, following the experimental settings in Table 1. The results show that both methods achieve comparable uncertainty localization performance.

MethodMath Llama
10%20%
σ2∥gt∥22\sigma^2\lVert g_t\rVert_2^20.390.52
Rand. Pert.0.400.54

Besides, we would also like to respectfully clarify several points where our views differ from the reviewer's assessment. (1) Regarding long CoT, our GPQA experiments already consider more challenging reasoning tasks than prior work on intermediate uncertainty, such as the experiments in TokUR. (2) Although our performance improvements are relatively small in a few settings, this does not necessarily indicate our method is ineffective, as explained in Point 3 of our rebuttal. (3) Regarding novelty, to the best of our knowledge, our work is the first to systematically evaluate uncertainty localization accuracy and demonstrate the effectiveness of embedding perturbations for this task. (4) Regarding the suggested ablation on future tokens, our task is offline uncertainty localization rather than online uncertainty estimation. Therefore, using future tokens does not introduce an unfair advantage, as the full reasoning trajectory is available in this setting.

Reviewer_WdjzOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses how to localize uncertainty in LLM reasoning trajectories through a measurement of sensitivity to input embedding perturbations. It introduces 2 types of perturbation scores at the token level, i.e., random and adversarial perturbations, aggregated into sentence-level scores by max pooling. The experimental validation evaluates how well they capture the first error in a response, and identify correct steps that vulnerably become incorrect under regeneration. Experiments on MATH, BBH, and GPQA show promising localization performance.

Critical strengths
  1. The evaluation thoughtfully distinguishes error origins within the lengthy responses. Sec. 4.1 measures whether the method's predicted most-uncertain steps can cover the first incorrect one. Sec. 4.2 finds correct steps that often become wrong when regenerated. Together, these provide practical but distinct evaluation of uncertainty localization. These experiments specifically target the step-level UQ.

  2. The method is simple and fast. Meanwhile, it consistently improves first-error detection on MATH and BBH while adding little computational overhead.

Critical weaknesses
  1. There is already a great number of perturbation-based UQ methods [a-c]. A comparison with them is necessary but lacking now for the proposed method to be fully established. For instance, [a] estimates uncertainty through input perturbations and output consistency, while SemGrad [b] measures how sensitive a model's predictions are w.r.t. small changes in semantic representations. Importantly, how does the proposed method differ and improve from them?

[a] Gao, X., Zhang, J., Mouatadid, L., & Das, K. (2024, March). Spuq: Perturbation-based uncertainty quantification for large language models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 2336-2346).

[b] Li, M., Lv, R., Li, X., Zhang, W., & Liu, T. (2026). Gradients with Respect to Semantics Preserving Embeddings Tell the Uncertainty of Large Language Models. arXiv preprint arXiv:2605.04638.

[c] Luo, H., Tao, L., Gui, J., Chen, X., Xu, C., Guo, J., & Dong, M. (2026). Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking. arXiv preprint arXiv:2610.02887.

Questions for authors
  1. How does the method compare with other perturbation-based UQ methods, in UQ and localization accuracy?
  2. Since the adversarial perturbation is computed from the full response, an earlier step's score can depend on what comes afterward. How does this affect its interpretation as uncertainty at that step?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Wdjz

We appreciate the comments and we will provide the clarifications and additional results.

1. Our method's difference to other perturbation methods. (W1, Q1)

We thank the reviewer for pointing out the relevant studies [a, b, c]. These works indeed share a similar high-level idea with ours: using perturbations to assess the stability of LLM predictions. However, our method differs from these studies in two important aspects.

  • (i) Different research objectives. The primary goal of these studies is to quantify uncertainty at the response level. In contrast, our work aims to identify the relatively more uncertain locations within a given reasoning trajectory. This requires fine-grained uncertainty signals at the token or step level, which are not provided by prior works.

  • (ii) Different perturbation strategies. For [a] and [b], the goal of perturbation is to ensure that the overall semantics of the input question change only slightly, without necessarily controlling the changes to each token's conditioning context. In contrast, our goal is to ensure that the conditioning context of each token's generation probability changes only slightly. Besides, our method perturbs not only the input question but also the preceding reasoning tokens.

To demonstrate the necessity of our perturbation strategy, we adapt the paraphrasing approach in [a] to our token-level uncertainty localization framework. Specifically, we use an auxiliary LLM to generate (n=10) paraphrases of the input question. We then pair each paraphrased question with the original response and recompute the generation probability of each response token. The variance of each token's probability across the 10 paraphrases is used as its uncertainty score. We follow the experimental settings of Table 1, with 300 cases per setting. The results are shown below.

MethodMath LlamaMath QwenMath MistralLogic LlamaLogic QwenLogic Mistral
10% 20%10% 20%10% 20%10% 20%10% 20%10% 20%
Paraphrase-max0.13 0.300.12 0.270.10 0.240.15 0.280.14 0.300.09 0.26
Paraphrase-avg0.13 0.270.10 0.290.11 0.280.14 0.310.12 0.270.11 0.25
Rand. Pert.0.40 0.540.33 0.500.31 0.500.33 0.500.38 0.550.28 0.46
Adv. Pert.0.45 0.590.35 0.520.34 0.500.42 0.580.40 0.570.35 0.50

The results show that question paraphrasing is substantially less effective for uncertainty localization than our method. We consider two possible reasons. First, although paraphrasing preserves the overall semantics of the input question, it may still introduce relatively large changes to the conditioning embeddings of individual token predictions, making it difficult to capture local prediction sensitivity. Second, paraphrasing only perturbs the input question, whereas our method also perturbs the intermediate reasoning trajectory. Therefore, paraphrasing may fail to capture uncertainty in later steps that depend more heavily on preceding reasoning instead of the original question.

[a] Spuq: Perturbation-based uncertainty quantification for large language models.

[b] Gradients with Respect to Semantics Preserving Embeddings Tell the Uncertainty of Large Language Models.

[c] Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking.

2. The influence of future steps on adversarial uncertainty scores. (Q2)

We agree with the reviewer that our adversarial perturbation score can be influenced by future tokens, as the perturbation direction is determined using the entire response. Essentially, the objective of our work is to identify the relatively more uncertain steps within an already generated reasoning trajectory. This is different from estimating uncertainty at each step online during generation. Therefore, we can interpret this score as a relative uncertainty localization signal for a complete reasoning trajectory.

Thank you for these detailed clarifications and additional experiments. The comparison and explanation fully address my concerns. I have no further questions.

Dear Reviewer,

Thank you for your positive feedback and support for our work.

Reviewer_4eMgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies uncertainty localization in intermediate LLM reasoning steps. It proposes two embedding perturbation-based methods: Random Perturbation, which measures token probability variance under Gaussian embedding noise, and Adversarial Perturbation, which uses sequence-level gradients to construct perturbations and evaluate token-level sensitivity. The resulting scores are used to identify potentially uncertain reasoning steps. Experiments on MATH, BBH, and GPQA demonstrate improvements over several existing uncertainty estimation baselines. The paper also explores the identification of fragile correct steps and targeted reasoning revision.

Critical strengths
  • Meaningful research problem. Localizing uncertainty within intermediate reasoning steps is an important problem beyond conventional response-level uncertainty estimation, potentially enabling more targeted verification and intervention.

  • Simple and efficient methodology. The proposed methods do not require additional model training or external process reward models. In particular, adversarial perturbation obtains token-level signals with only one additional backward pass and forward pass.

  • Promising experimental results. The proposed methods outperform several probability-based and Bayesian-based baselines in first-wrong-step localization. The investigation of fragile correct steps also provides an interesting extension beyond conventional error detection.

Critical weaknesses

W1. Limited methodological novelty. The proposed methods largely apply conventional Gaussian perturbation and gradient-based adversarial perturbation to token embeddings. The theoretical analysis mainly establishes their relationship with local gradient sensitivity, rather than introducing a substantially new uncertainty estimation principle.

W2. Insufficient justification of the core assumption. Embedding sensitivity does not necessarily imply reasoning uncertainty or incorrectness. Correct tokens may be sensitive due to multiple plausible alternatives, while confidently generated but incorrect conclusions may remain stable under perturbations. The paper should sufficiently discuss and establish the relationship between perturbation scores and actual reasoning errors or regeneration risks.

W3. Insufficient comparisons with related methods. The paper lacks direct quantitative comparisons with Forking Tokens, a closely related reasoning localization approach. Moreover, important baselines such as TokUR and Graph Uncertainty are missing from the GPQA experiments. More comprehensive comparisons are needed to establish the advantage of the proposed method.

W4. Limitations in experimental validation. The evaluation relies heavily on LLM-generated annotations and a limited amount of human examination. The sentence-based segmentation and maximum aggregation of token scores may also introduce biases related to step length. Furthermore, the correction experiments rely on an external powerful LLM, making it difficult to isolate the contribution of the proposed localization method.

Questions for authors
  1. The proposed methods apply Gaussian and gradient-based adversarial perturbations to token embeddings. Beyond applying these existing perturbation techniques, what is the key insight that distinguishes the proposed approach from conventional sensitivity or robustness analysis?
  2. The paper uses embedding sensitivity as a proxy for reasoning uncertainty. How do the authors distinguish sensitivity caused by multiple plausible continuations from genuine uncertainty in the reasoning process? How do they account for confidently generated but incorrect conclusions that may remain stable under perturbations?
  3. Why is there no direct quantitative comparison with Forking Tokens? In addition, TokUR and Graph Uncertainty are omitted from the GPQA experiments. Could the authors provide these comparisons, or clarify the practical obstacles?
  4. In the correction experiments, GPT-5.6 evaluates flagged steps and provides feedback for revision. How do the authors isolate the contribution of the proposed localization method from the external error-detection and revision capabilities of LLM?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 4eMg (Section 1)

We thank the reviewer for the comments. We will first explain why our method can identify reasoning uncertainty rather than being confounded by "multiple plausible continuations". This helps highlight the key contribution of our method. Then, we will address the reviewer's other concerns.

1. How does our method identify "reasoning uncertainty"? Can it distinguish "multiple plausible continuations"? (W2, Q2)

We provide a new theoretical analysis in Section 3.3 of our updated paper, to explain why our method can identify tokens that exhibit genuine reasoning uncertainty. We introduce its high-level idea here and kindly invite the reviewer to refer to Section 3.3 for details.

In our new theory, we consider that a generated token in the LLM response may have "conflicting alternatives" in the vocabulary. For example, the token "left" in our Figure 2 has a conflicting alternative "right". Similarly, for a simple reasoning step 1+1=2, the token "3" can be considered a conflicting alternative to "2". If the LLM generates such alternatives, it will lead to reasoning continuations that contradict the original generation.

For our metric, a high sensitivity score of the token's log probability captures a key phenomenon: when a small embedding perturbation reduces the model's logit support for the originally generated token xtx_t, the model may substantially increase its logit support for certain alternative tokens, thereby amplifying the change in the generation probability of xtx_t. Notably, we use logits here to ensure that the increase is not simply caused by softmax normalization (when calculating token probability from logits). Instead, we argue that such an increase is more likely to come from conflicting alternatives. This is because conflicting alternatives tend to "compete" with the original token xtx_t, so weakening the support for xtx_t may instead increase the support for one or some of its conflicting alternatives. In this way, our method can capture the model's tendency to favor conflicting alternatives, thereby reflecting reasoning uncertainty.

Besides, we agree with the reviewer that multiple benign continuations can appear. For example, in Figure 2 of the main text, the token "At" has plausible alternatives such as "After" and "Now", which also lead to correct continuations. Referring to our Section 3.3, these benign alternatives do not exhibit strong competition with the original token, so their logits remain relatively unchanged and even slightly decrease, when the original token's logit decreases. Thus, these positions typically have lower perturbation sensitivity scores. The reviewer also mentioned that LLMs can confidently make mistakes. In such cases, we believe that characterizing uncertainty alone is insufficient to identify unreliable reasoning. Addressing such errors may require further model training or calibration, which is beyond the scope of our work on uncertainty localization.

2. What are the key insight and novelty of our work? (W1, Q1)

The discussion above highlights the insight of our method: it may capture the model's tendency to favor conflicting alternatives. We agree with the reviewer that our methods use standard perturbation techniques, such as Gaussian and adversarial perturbations. However, our contribution lies in demonstrating their effectiveness for uncertainty localization in LLM reasoning, which has not been explored in prior work. Furthermore, to the best of our knowledge, our work is the first to systematically evaluate uncertainty localization accuracy. For example, although previous work such as TokUR also studies intermediate uncertainty, it does not systematically evaluate localization accuracy at the step or token level.

AuthorsOpenReview ↗

Response to Reviewer 4eMg (Section 2)

3. Comparisons to (a) Forking Tokens, (b) TokUR, (c) Graph Uncertainty. (W3, Q3)

  • (a) We thank the reviewer for mentioning forking tokens, which share a similar high-level goal with our work: identifying uncertain / unreliable parts of LLM reasoning. To our knowledge, Bigelow et al. [1] introduce the concept of forking tokens, but their method requires repeatedly regenerating complete responses from each token position for multiple times, resulting in extremely high computational cost. In our test, we try analyzing a single 260-token Llama response, and it took 4.65 hours on one NVIDIA GH200 GPU. Furthermore, other works, such as Wang et al. [2], refer high-entropy tokens as forking points. We have thoroughly discussed entropy-based metrics in our paper.

  • (b) TokUR: We exclude TokUR from our GPQA experiments due to implementation challenges. TokUR relies on model-specific hooks to perturb Q/K projection weights, and the author's original implementation only supports architectures such as Llama and Qwen2.5. Extending it to our GPQA models (gpt-oss-20b and Qwen3.5-27B) would require substantial reimplementation due to their different attention architectures. Moreover, the MoE structure of gpt-oss-20b introduces additional complexity about how perturbations should be aggregated across experts.

  • (c) Graph Uncertainty: We exclude Graph Uncertainty from our GPQA experiments due to its high computational cost, especially since GPQA responses are much longer than those in MATH and BBH (approximately 2,000 tokens per response). This method requires generating multiple complete responses and evaluating the consistency of each intermediate step against other responses, resulting in substantial computational overhead (813.40s on average for each GPQA response).

[1] Forking Paths in Neural Text Generation, Bigelow et al.

[2] Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning, Wang et al.

4. Is the max-aggregation of our result biased to step length? (W4)

We argue that sentence length is unlikely to explain the strong localization performance of our methods. To investigate this, the table below reports the average lengths of steps identified by different methods, as well as the ground-truth first wrong steps and all steps. First, several baselines, including NLL-max, Entropy-max, and TokUR-max, also use maximum aggregation, while other strong baselines, such as CCP and TokenSAR, use sum aggregation. As shown in the table, our methods do not select longer steps than these baselines. This suggests that our performance improvements are not simply due to maximum aggregation. Second, the steps identified by our methods are not substantially longer than the average steps. In fact, the steps selected by many methods, as well as the ground-truth first wrong steps, tend to be slightly longer than average. This suggests that longer steps may naturally be more likely to contain reasoning errors. For example, a short sentence like "Here is the detailed solution." is unlikely to contain reasoning uncertainty. Thus, selecting slightly longer steps does not necessarily indicate an undesirable bias.

MethodMath (Llama)Math (Qwen)
NLL-max24.43 ±\pm 2.8726.15 ±\pm 4.24
Entropy-max24.72 ±\pm 3.2025.45 ±\pm 3.89
TokUR-max25.17 ±\pm 3.4727.44 ±\pm 4.39
NLL-avg20.05 ±\pm 2.2721.98 ±\pm 3.26
Entropy-avg19.95 ±\pm 2.5121.60 ±\pm 3.05
TokUR-avg20.14 ±\pm 2.5522.94 ±\pm 3.35
CCP-sum27.38 ±\pm 3.4431.05 ±\pm 4.32
TokenSAR-sum27.57 ±\pm 3.9430.12 ±\pm 3.22
Graph Uncertain.21.18 ±\pm 3.4923.85 ±\pm 3.21
Rand. Pert.24.17 ±\pm 3.1425.82 ±\pm 4.63
Adv. Pert.23.99 ±\pm 3.2426.74 ±\pm 4.74
Ground-truth (GPT-annotated)25.12 ±\pm 4.5026.35 ±\pm 4.93
all steps22.40 ±\pm 5.0224.72 ±\pm 5.61
AuthorsOpenReview ↗

Response to Reviewer 4eMg (Section 3)

5. Reliability of External LLM Annotation. (W4, Q4)

Overall, we believe our LLM-based annotations are reliable. As detailed in Appendix B, we conducted a manual examination of 180 responses sampled across different subsets of our experimental datasets, achieving a 93.9% agreement rate between human judgments and LLM annotations. We believe this provides strong evidence for the reliability of our annotation pipeline.

To respond reviewer's Q4 to further isolate the contribution of our uncertainty localization methods in the correction experiment (section 4.3 and Table 4), we introduce an additional baseline called Uniform, which randomly samples 10% or 20% reasoning steps uniformly for examination by an external model. We keep the rest of the experimental pipeline unchanged. As shown in the result, our methods outperform this baseline, indicating the contribution of the localization itself.

MethodMath LlamaMath QwenMath LlamaMath QwenLogic LlamaLogic Qwen
10% 20%10% 20%10% 20%10% 20%10% 20%10% 20%
Uniform0.41 0.480.69 0.700.58 0.610.83 0.840.89 0.900.89 0.89
Rand. Pert.0.47 0.560.75 0.760.66 0.750.87 0.880.98 0.980.93 0.94
Adv. Pert.0.51 0.540.78 0.790.70 0.770.88 0.890.98 0.970.94 0.95

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response from Authors

We sincerely thank all reviewers and the AC for the time and effort during the rebuttal period. Throughout the rebuttal process, we tried our best to engage with the reviewers and address their concerns, particularly regarding the novelty and theoretical insight of our proposed method.

  • Regarding Novelty. We acknowledge that our proposed methods are based on standard random perturbation and adversarial perturbation strategies. Yet, we believe our contribution is to identify their particular effectiveness for a new problem: localizing the step where uncertainty arises in an LLM reasoning trajectory. To the best of our knowledge, our work is the first to systematically investigate this problem at the intermediate step level. Besides, in our rebuttal to Reviewer Wdjz, we compared our method with existing perturbation strategies (which measure uncertainty at the response level) to show their limitations in addressing this problem.
  • Regarding Theoretical Insight. During the rebuttal, we updated our theoretical analysis (Section 3.3). We show that embedding perturbations can help identify places where the model tends to choose a "conflicting alternative token" rather than the originally selected token. Such alternatives may lead to reasoning continuations that semantically conflict with the original generation, thereby reflecting the model's uncertainty in the reasoning process. Please see our response to Reviewer 4eMg for more details, including how our method can distinguish such uncertainty from "multiple plausible continuations".

Finally, we sincerely appreciate the reviewers’ constructive comments and the AC's help in encouraging further discussion. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.