acceptodds
Under review as a conference paper at ICLR 2027

On the Interplay between Backtracking and Latent Confidence Directions in Reasoning Models

Abstract

Reasoning models deliberate over multiple answer proposals, often resulting in excessively long traces due to backtracking, even when the correct answer has already been reached (On AIME 2026 with Olmo 3 32B Think, 87% of rollouts contain 1 unique answer yet still make 8 answer proposals on average). Given this potentially "decorative" chain of thought (CoT) reasoning, can we look to the model internals to determine if the model will backtrack or commit to an answer proposal? Operating under the hypothesis that models maintain an internal representation of their uncertainty, we seek to extract this representation and characterize its relationship with backtracking behavior at answer proposals. By querying the model's verbalized confidence following answer proposals throughout a response trace, we extract a linear direction for latent confidence that separates high and low verbalized confidence. This direction can steer verbalized confidence as expected, but it surprisingly can also steer the next token probability of backtracking, performing on par with a linear direction extracted directly from backtracking examples. % of backtracking vs. committing to answer. We then test the direction under progressively more realistic interventions. Steering continuously from an answer proposal modulates response length (-1k - +3k tokens) and the number of answer proposals (-4.52 - +3.26) symmetrically in both directions, with minimal cost to accuracy. Applying the confidence direction from the start of generation, where proposal positions are unknown, still shortens traces at comparable accuracy, and its effect surfaces in the linguistic confidence of proposals. Overall, our study reveals the intricate interplay between a model's internal confidence and its proclivity to backtrack or commit to an answer.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, reasoning models, backtracking, uncertainty, linear representations, steering

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 5 reviews, above 85% of the others.
Reviewer_UzVvOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies reasoning in 3 models and 5 datasets. The key issue is that they observe that models propose an answer that turns out to be their final answer, before spending considerable tokens on checking -- even though this doesn't change the final outcome. They are particularly interested in the link between the model's internal confidence and the initiation of backtracking behavior following an answer proposal. They use activation steering to show verbal confidence derived vectors, and commitment vectors (from late proposal marker vs early proposal marker) can reduce answer proposals, reduce backtracking token probabilities and cause early termination (shorten reasoning traces whilst preserving accuracy). Their conclusions are primarily in relation to the role of confidence in backtracking, answer proposals rather than regarding potential deployment benefits.

Critical strengths

S1: This is an important question -- whether confidence signals drive backtracking in LLMs -- often assumed but to my knowledge not clearly demonstrated before. S2: The methodology used -- the intermittent use of verbal confidence prompts to extract confidence vectors -- and activation steering are carefully applied. S3: The intervention findings i.e. the effects on answer proposal number, and early termination are convincing

Critical weaknesses

W1: Whilst the authors provide convincing evidence that intervention with confidence (and commitment vectors) can cause substantial alterations in answer proposals etc, I'm not convinced this shows that the model in it's natural state uses internal confidence signals to backtrack. Several analyses would help i) there is little demonstration apart from the mild correlation of verbal confidence with final vs early answer proposal, that confidence predicts these behaviors. This would provide much needed supportive evidence for their conclusions ii) Activation steering could be complemented by other techniques to show convincingly that internal confidence signals in its natural state are used by the model to trigger backtracking: a) activation patching b) activation swap experiments -- these try to eliminate limitations of activation patch experiments to isolate the contribution of the confidence-specific part of a representation. iii) the authors could take the confidence direction isolated by steering and look at the correlation between the projection of an activation state onto this direction and how frequently backtracking is triggered. W2: I'm not sure how to interpret the results of the commitment vector steering. This is because the final answer proposal marker representation likely contains lots of things that differentiate it from the early answer proposal marker. Can the authors perform additional analyses to narrow down what is driving the observed effects. W3: The evidence that these interventions like continued steering change backtracking behavior is indirect -- and it is not clear why proxies like the number of answer proposals and early termination were relied on. W4: (Minor) I found the related work section lacking a section on the link between internal confidence signals and behavior (e.g. in non reasoning LLMs -- e.g. Kumaran et al., Nature Machine Intelligence 2026; Ackerman 2026 https://arxiv.org/abs/2509.21545)

Questions for authors

Q1: Can the authors provide more evidence to back up their case that internal confidence signals in the natural operation of the model? Q2: What is driving the effects of the commitment vector Q3: Can more specific evidence be provided as to the link between the steering manipulations and backtracking behavior. Q4: Can the authors add relevant evidence on the link between confidence and behavior and also clarify if there paper is the first to show that a unique answer proposal is typically provided despite repeated answer proposals before?

1 comment

We thank the reviewer for the detailed feedback and address their points below. Changes in the revised manuscript are in red.

[W1, Q1] Naturalness of confidence

We note that while we motivate our work with the hypothesis that models use internal confidence to commit, our main contributions demonstrate a causal steering relationship with answer commitment behavior. We agree that this does not guarantee that the linear confidence direction is naturally being used by the model to the extent that steering applies it (Mishra et al., 2026 https://arxiv.org/abs/2604.09839). This reflects a broader open question on the “naturalness” vs “causality” of steering. We add this to our limitations, and we scaled down any claims that suggested vconfv_\text{conf} steering is synonymous with the natural mechanism.

Following the reviewer’s suggestion (iii), we compute Spearman correlations between the hidden state projection onto vconfv_\text{conf} or vcommitv_\text{commit} and (a) binary answer commitment on the test set (ρmax=0.87\rho_\text{max} = 0.87) or (b) pbtp_\text{bt}. We report results in Tables 28 and 29 and discuss in section 4.2.

Olmo 3 32B and Qwen 3 32B, projections onto vconfv_\text{conf} have ρ\rho=0.49,0.64 respectively, which exceed that of verbalized confidence on the test set (0.33,0.26). This disparity is largest on incorrect rollouts, where verbalized confidence is uncorrelated (ρ\rho=0.09,-0.03) while vconfv_\text{conf} projection remains at ρ\rho=0.36,0.58. Interestingly, on Distill 32B, the projection has no correlation with answer commitment (ρ\rho=-0.00), yet all three models have strong vconfv_\text{conf} projection correlations with pbtp_\text{bt} (ρ∈[−0.68,−0.46]\rho \in [-0.68,-0.46]). This suggests that Distill 32B tracks and steer local backtracking behavior which then influence answer commitment (consistent with results in sections 4 and 5), but cannot track answer commitment labels themselves. One hypothesis for this difference is the presence of RL post training. Olmo 3 32B and Qwen 3 32B undergo RL, while Distill 32B uses SFT. It is possible that RL training strengthens the link between latent confidence and answer commitment. Studying how post training strategies affect internal confidence would be an exciting future direction. We leave alternative methodologies for proving naturalness, like activation patching, to future work as well.

[W2, Q2] Interpreting the commitment direction

We add logit lens results for vcommitv_\text{commit} in Table 27 and discuss them in section 4.2 in the revised manuscript. Overall, the decoded top tokens suggest that vcommitv_\text{commit} operates at the syntactic level. The negative direction decodes backtracking tokens in English and Chinese, such as “However” and “But”. Olmo 3 32B’s negative direction also decodes affirmatory adverbs like “Therefore” and “Thus”, and its positive direction decodes “</”, the first token of </think>. This suggests that steering with Olmo 3 32B’s vcommitv_\text{commit} directly promotes the reasoning termination tag and suppresses adverbs leading to any additional reasoning, which is consistent with the construction of vcommitv_\text{commit} from final and mid-trace proposal markers. This contrasts vconfv_\text{conf}, which promotes concepts semantically related to confidence.

[W3, Q3] Linking metrics to backtracking behavior

Since backtracking behavior is a qualitative characterization of the CoT, any metric used, i.e. next token probability of a backtracking token, additional answer proposal counts, early termination, response length, even qualitative inspection of language itself, is arguably a noisy proxy with its own weaknesses. For example, next token probability of a backtracking token (section 4.1) doesn’t consider the later generated tokens, but early termination of thinking does (section 5.1). The number of answer proposals (section 5.2) measures how many times the model backtracks from an answer proposal, but it does not check the language used by the model. Qualitatively checking the prose (section 5.3) does. We argue that in the absence of a single, universally accepted metric to quantify backtracking, the consistency of all the different "proxy" metrics supports our conclusion that confidence steering modulates backtracking behavior.

[W4, Q4] Additional references and novelty of Table 1

Thanks for suggesting these references. We have added them to the manuscript. We also add a mention to Chen et al. (2025, https://arxiv.org/abs/2412.21187) at the end of section 3.1, who count the number of answer proposals, but not the number of unique solutions. To our knowledge, we are the first to explicitly count the number of unique solutions. However, due to the sheer volume of existing literature on reasoning redundancy, we provide this result primarily to frame the rest of our paper rather than as a substantially novel finding.

Reviewer_vzybOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the relationship between verbalized confidence, answer commitment, and backtracking in reasoning models. The authors construct a confidence steering direction using a Difference-in-Means approach over activations associated with high- and low-confidence answer proposals. They show that steering along this direction affects backtracking behavior, the number of answer proposals, and overall CoT length. The paper is generally well presented, and the experiments are carefully designed.

Critical strengths
  • Careful experimental design: The authors investigate several aspects of the proposed method, including the relationship between confidence and commitment and the effects of steering on backtracking and trace length.
  • Useful ablation studies. The Position Ablation in Appendix D.1 addresses whether the confidence direction is confounded by normalized trace position. The Probability DiM Ablation Experiment in Appendix D compares the confidence direction with a direction constructed directly from backtracking probability.
Critical weaknesses

As the authors acknowledge, since the steering direction is constructed from activations associated with verbalized confidence, it may also encode correlated factors such as problem difficulty, uncertainty about the solution, or normalized token position in the CoT. For example, easier problems may elicit higher confidence and shorter traces, while harder problems may elicit lower confidence and more backtracking. A direction learned from these examples could therefore capture difficulty-related features rather than confidence alone.

The authors make a careful effort to investigate this issue in Appendix D.1 (Position Ablation) and Appendix D (Probability DiM Ablation Experiment), which I appreciate. The results suggest that the observed effects cannot be explained entirely by coarse trace-position differences or by a direction trained directly on backtracking probability. However, they do not establish that the learned directions capture distinct, disentangled concepts. The substantial retention of steering effects after position matching is compatible with a confidence-related signal, but also with representations that encode several correlated aspects of the reasoning process.

The more interesting question is whether confidence itself determines how much additional reasoning a model performs, rather than merely correlating with reasoning progress or problem difficulty. The paper provides evidence that intervening on a confidence-derived direction can change CoT length, but this does not establish that confidence is the underlying mechanism.

A useful approach would be to examine cases that dissociate confidence from progress and task complexity:

  • A model is late in its trace but remains uncertain, repeatedly checking an answer.
  • A model is early in its trace but already confident, having found a plausible solution quickly.
  • A model is late in its trace because the problem requires many reasoning steps, rather than because it is indecisive.

These cases could help determine whether confidence predicts remaining reasoning independently of trace position and problem complexity. The paper's within-rollout confidence–commitment correlation (Spearman ρ=0.22\rho=0.22) provides some evidence of within-trace variation, but further analysis is needed to establish how substantial and prevalent those changes are, and whether they explain the learned steering direction. I think answering the central question of "whether confidence drives CoT length" could make the paper more interesting.

The paper is well designed, but its general approach, constructing activation steering directions from contrasting examples and using them to control CoT length, has precedents. Overclocking LLM Reasoning (arXiv:2506.07240), for example, steers representations associated with reasoning progress to control thinking length. Other approaches use short-verbose CoT pairs to construct steering directions for similar purposes(arxiv:2507.04742). This is of course not a reason to reject the paper.

Questions for authors

Do you think there are cases where:

  • A model is late in its trace but remains uncertain, repeatedly checking an answer.
  • A model is early in its trace but already confident, having found a plausible solution quickly.
  • A model is late in its trace because the problem requires many reasoning steps, rather than because it is indecisive.

I think the answer to this question can shed light whether confidence drives CoT length or not.

1 comment

We thank the reviewer for the detailed feedback and address their points below. Changes in the revised manuscript are in red.

[W1, Q1] Predicting remaining CoT length, disentangling confidence

I think answering the central question of "whether confidence drives CoT length" could make the paper more interesting.

We train a classifier to predict the remaining CoT until </think> from 4 manually-selected input features of the answer proposal. Our features include the verbalized confidence, number of tokens generated so far, the dataset (represented as a one-hot encoding), and the projection of the hidden state onto the latent direction vconfv_\text{conf} or vcommitv_\text{commit}. We divide the remaining CoT into five categories: 0-50 tokens left, 50-500, 500-2k, and 2k+. Our train set use the answer proposal markers used to compute each dataset’s vcommitv_\text{commit} (~1000 markers total). Our test set is the same 500 marker test set used in the paper, and we take the mean of the four continuations to assign our ground truth label. We use sklearn’s logistic regression with an L-BFGS solver trained with L2-regularized cross entropy.

The three cases you described definitely do exist, and they are cases in which confidence likely diverges from trace progress and task difficulty. If confidence were redundant with these features, it would carry no predictive information about remaining reasoning once they are accounted for. We therefore test whether vconfv_\text{conf} predicts remaining CoT length beyond a control classifier that includes only the other 3 features. Any gain over the control must come from markers where our internal confidence direction and progress/difficulty dissociate.

Rather than upper bounding the utility of the latent direction by learning a new direction specifically for this classification task, we use vcommitv_\text{commit} as the pseudo-upper bound since it separates mid-trace and final proposals by construction.

Below are the results on test set cross entropy loss, accuracy, and macro F1. The “prior only” column is the performance of the classifier if we only use the train set’s label frequencies. “v only” is the performance of a classifier that only uses the projection. "+v" is the full classifier with all 4 features.

modelv typeprior onlyv onlycontrol+vCE gain [95% CI]accuracy: control / +vF1: control / +v
Olmo 3 32Bv_conf1.3161.1341.1971.0890.108 [0.071, 0.146]0.50 / 0.540.30 / 0.35
v_commit1.3160.8581.1970.8040.393 [0.343, 0.443]0.50 / 0.650.30 / 0.50
Qwen 3 32Bv_conf1.2930.9811.2230.8530.370 [0.315, 0.423]0.48 / 0.610.27 / 0.50
v_commit1.2930.7971.2230.7730.450 [0.394, 0.506]0.48 / 0.640.27 / 0.55
Distill 32Bv_conf1.3671.3481.3521.3460.006 [-0.005, 0.016]0.36 / 0.360.17 / 0.18
v_commit1.3670.9721.3520.9780.374 [0.316, 0.432]0.36 / 0.550.17 / 0.48

Comparing against the control, the vconfv_\text{conf} projection feature improves performance substantially on Qwen 3 32B, dropping cross entropy loss by 0.270 and closely matching the vcommitv_\text{commit}'s accuracy and F1. On Olmo 3 32B, vconfv_\text{conf} improves cross entropy loss by 0.108 and raises accuracy (+0.4) and F1 (+0.5) above the control. Distill 32B’s vconfv_\text{conf} feature does not improve performance (This is consistent with the new correlation results in section 4.2, which we also discuss in our response to UzVv [W1, Q1]). Overall, these results suggest that the confidence direction does not solely predict CoT length, but rather is one of several factors that contribute to predicting CoT length. This is consistent with our steering results that show that latent confidence can influence CoT length.

However, they do not establish that the learned directions capture distinct, disentangled concepts.

We agree that the extracted latent confidence direction can carry correlated factors. However, we do not believe that complete disentanglement is a standard that any extracted direction can be guaranteed to meet. For example, orthogonalizing against a “task difficulty” direction would require that direction to be pure itself. Our aim instead is to test if a confidence-derived direction has causal influence beyond likely confounds, which we explore in Appendix D and the classifier analysis above.

The paper provides evidence that intervening on a confidence-derived direction can change CoT length, but this does not establish that confidence is the underlying mechanism.

We agree that the steering results do not establish the degree to which confidence is naturally used by the model. We add this to our limitations section. Reviewer UzVv makes a similar point, and we respond in more detail there ([W1, Q1]).

[W2] Additional references

Thanks for suggesting these references. We added them to the manuscript.

Reviewer_rxDYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper looks into the internal confidence of the model and connects it with backtracking. The authors provide a method to extract a linear confidence direction that could steer both verbalized confidence and the probability of backtracking. The experiments show that adding the extracted latent direction could generally increase verbalized confidence and backtracking, while subtracting it would result in the opposite. The authors further show that this confidence steering changes response length and the number of answer proposals. Steering from the beginning of generation can also shorten responses, with accuracy tradeoffs that vary across models and tasks. Overall, the study provides a way to extract latent confidence signals which can be used to steer answer commitment.

Critical strengths
  1. The paper offers an interesting view by connecting internal confidence with backtracking and answer commitment. The latent confidence direction is insightful.

  2. The main finding is supported by extensive experiments across three models and five benchmarks. Evaluations on generated continuations, together with prompt and position controls, strengthen the evidence for the claims.

Critical weaknesses

Although average token savings are promising, Table 4 includes noticeable accuracy reductions on individual benchmarks. Confidence intervals and more explicit reporting of the accuracy–length tradeoff would better support claims of comparable accuracy.

Questions for authors

Could the authors discuss more about the accuracy–efficiency tradeoff in Table 4? For example, steering from the beginning reduces OLMo’s AIME accuracy from 80.8% to 75.8%. I think it would be helpful to discuss when the confidence vector is "confidently" stopping useful reasoning and when it's reducing redundant backtracking.

1 comment

We thank the reviewer for the detailed feedback and address their point below. Changes in the revised manuscript are in red.

Accuracy Efficiency Tradeoff in Table 4

We add discussion on the accuracy-length tradeoff to section 5.3 in the revised manuscript. Reasoning models typically spend the initial reasoning steps exploring and formulating plans, which can strongly influence later reasoning steps. Increasing the model’s confidence too early can cause the model to be overconfident on pivotal steps where backtracking is useful, leading to a loss in accuracy. This observation lead us to also explore delaying steering after d=5000d=5000 tokens have already been generated, which helps close the accuracy drop from [-2.3, -0.4] to [-0.7, +0.4] but reduces the % gains in reasoning efficiency from [-20.0, -8.4] to [-10.4, -3.0].

We also record 95% confidence intervals in Table 25. With no delay d=0d=0, 2 out of the 15 model-benchmark pairs have Δ\Delta Accuracy CIs that do not include 0, and Olmo 3 32B’s overall Δ\Delta Accuracy CI does not include 0. With delay d=5000d=5000, zero model-benchmark pairs have Δ\Delta Accuracy CIs that do not include 0. All models have overall Δ\Delta % response length CIs that do not include 0 for both d=0d=0 and d=5000d=5000, with 8 out of 30 model-benchmark-dd tuples having Δ\Delta % response length CIs > 0.

Reviewer_95kuOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how internal confidence relates to a reasoning model's decision to continue checking or commit after proposing a candidate answer. The authors define sentences that explicitly state a candidate answer as answer proposals. They append a confidence prompt after each proposal and construct a confidence direction by taking the difference between the mean hidden states of examples with high and low self-reported scores. They then add this direction to hidden states during ordinary reasoning and examine changes in subsequent behavior.

For comparison, the authors construct a commitment direction from hidden states at final and earlier answer proposals. They compare the two directions' effects on self-reported confidence and the tendency to continue checking. The directions have similar effects on the probability of generating a token that signals backtracking, while the commitment direction has a weaker effect on confidence scores. The authors further report that continuous confidence steering changes reasoning length and the number of answer proposals, and that steering from the start of generation also shortens responses.

Critical strengths
  1. The paper finds that a direction extracted from self-reported confidence can influence whether a model continues checking during ordinary reasoning. Its effect on the probability of generating a backtracking token is comparable to that of a direction extracted directly from commitment behavior, yet the two directions have asymmetric effects on confidence scores. This observation offers useful empirical insight into when reasoning models accept their current answer.

  2. The experiments cover three 32B models and five datasets, progressing from the probability of a backtracking token to short continuations and complete generations. The authors also vary the confidence prompt and match proposal positions to examine whether the results depend on particular wording or positions in the trace. The experiments are clearly organized and provide several complementary checks on the main finding.

Critical weaknesses

1. The long continuation experiments lack a newly sampled unsteered control.

Table 24 compares newly generated steered continuations with stored original continuations. A proposal is labeled as mid-trace based on the original response: the model continues reasoning and makes a further answer proposal after that point. However, a new continuation from the same prefix may end immediately even without steering. Some of the observed reductions in length and proposal count may therefore come from resampling. The authors should repeatedly sample both unsteered and steered continuations from the same prefixes, then compare the mean effects and report their uncertainty. This is the experimental control I would most like the authors to add.

2. The test set is used to select the best direction.

Section 4.1 states that the authors use 500 test proposals to select the best direction among those extracted from different source datasets, then report results on the same data. Selecting and evaluating the best direction on the same data leads to an optimistic estimate of its performance. The direction, layer, and steering strength should be selected using validation data, followed by evaluation on independent test data.

3. The claim about the commitment direction's effect on confidence is too strong.

The paper argues that interventions along the commitment direction cannot effectively change the model's confidence score for its current answer. However, in Table 2, the score increase under positive commitment steering is about 31%–59% of that under positive confidence steering. The difference between the two directions is more pronounced under negative interventions. I suggest revisiting this conclusion in light of the effect sizes for both positive and negative interventions.

Questions for authors
  1. Could you repeatedly sample continuations with and without steering from the same proposal prefixes and re-estimate the changes in reasoning length and proposal count? In particular, how much do newly sampled unsteered continuations differ from the stored originals, and do the main effects remain after accounting for this difference?
  2. Do the steering effects hold on independent test questions once the direction, layer, and strength are fixed? Please clarify which data are used for direction selection and which are used for the final evaluation.
  3. For a fixed direction and layer, what can we say about the joint distribution of changes in confidence scores and the probability of a backtracking token across the same set of proposals? Is the asymmetry mainly driven by negative interventions, or concentrated in examples with particular initial confidence levels?
3 comments

We thank the reviewer for the detailed feedback and address each point below. Changes in the revised manuscript are in red.

[W1, Q1] Resampling continuations

We agree that resampling additional continuations and comparing against α\alpha=0 control samples is more robust. We find that with additional sampling, the overall findings remain the same. Positive (negative) steering decreases (increases) response length and answer proposals by approximately the same amount after performing resampling, it remains comparable to vcommitv_\text{commit}, and it has little cost to accuracy. Please see updated Table 24, Figure 2, and section 5.2 in the revised manuscript. Standard errors of the mean are in Table 24. We discuss results below as well.

Following our existing sampling parameters, we sample 4 continuations after each proposal marker within the non-degenerate alpha range for each model and vector. We also sample 4 continuations with α\alpha=0, and compare each steered continuation against the corresponding unsteered continuation. The original and new results for positive and negative steering with the confidence and commit directions are shown side-by-side in the table below. For visualization purposes, we average over Olmo 3 32B, Qwen 3 32B, and Distill 32B model. and vector’s best positive and negative α\alpha. Each entry has the form [original] →\rightarrow [new].

Vecα\alphaΔ\DeltaL allΔ\DeltaL mid-traceΔ\DeltaL finalΔ\Deltaprop allΔ\Deltaprop mid-traceΔ\Deltaprop finalΔ\Deltaacc
commitnegative1341 → 11601933 → 1777704 → 497+4.32 → +4.43+6.02 → +6.86+2.55 → +1.85−1.5 → −1.1
commitpositive−392 → −471−782 → −91327 → 4−2.39 → −2.39−4.70 → −4.40+0.07 → −0.23−0.3 → −0.4
confidencenegative1617 → 14782573 → 2511592 → 370+2.72 → +2.92+3.74 → +4.75+1.64 → +0.97−1.3 → −0.6
confidencepositive−650 → −736−1227 → −1323−30 → −107−2.27 → −2.26−4.52 → −4.18+0.16 → −0.19−0.9 → −0.4

[W3, Q3] vcommitv_\text{commit}'s seemingly strong positive confidence steering scores

Thank you for pointing this out. We stratify the verbalized confidence steering scores based on the initial verbalized confidences of the answer proposals, and we find that vcommitv_\text{commit}'s stronger positive steering scores are actually an artifact of the verbalized confidence having a max value of 9. Please refer to Figure 4 and Appendix E of the revised manuscript for additional discussion.

Across all initial verbalized confidences, positive steering with vcommitv_\text{commit} is consistently ΔVC<1\Delta VC < 1. On the other hand, at the smallest initial confidences, vconfv_\text{conf} positive steering has ΔVC∼3\Delta VC \sim 3. As the initial confidence increases, the ceiling for the effect of positive steering decreases, so vconfv_\text{conf}'s ΔVC\Delta VC also decreases to 0. For example, if an answer proposal marker has a high initial confidence of 8, positive vconfv_\text{conf} steering saturates with ΔVC=1\Delta VC = 1. If the proposal marker's initial confidence is 8.75, max ΔVC=0.25\Delta VC = 0.25. This ceiling deflates the overall scores of vconfv_\text{conf} but hardly affects vcommitv_\text{commit}, thus increasing vcommitv_\text{commit}'s relative score to vconfv_\text{conf}. Since this can be viewed a statistical artifact rather than an indication of asymmetric steering capability, this analysis supports our claim that vcommitv_\text{commit} does transfer to steering verbalized confidence.

[W2, Q2] Direction Selection

We clarify our direction selection process in section 4.1 in the revised manuscript. To clarify, we first obtain each dataset’s best direction using each dataset's respective validation set. We are not using the test set to select the best direction for single-token steering and then test single-token steering. Instead, we evaluate each dataset’s best direction on the test set, and select the best one for multi-token steering in section 5 based on performance steering pbtp_\text{bt}. Even though the markers we use in section 5.1 and 5.2 belong to the same test set, we believe this is acceptable since multi-token steering is a more complex task with distinct metrics (response length and number of answer proposals) from single token steering (pbtp_\text{bt}).

Furthermore, to demonstrate that our results are not an optimistic estimate, we create a 250 marker validation set by pooling the 50 markers from each of the five dataset’s validation set. These markers have no overlap with the test test. We evaluate single token steering with each dataset’s best direction on this validation set, and find that the best dataset direction is consistent with the best dataset direction when evaluated on the test test. Thus, using this validation set to select the best direction would have given us the same directions already used in the paper, demonstrating the generalizability of our selected directions. Below are results for steering pbtp_\text{bt}. Gap is the difference between −α-\alpha and +α+\alpha scores.

ModelVectorTrain setSiteVal GapVal −α-\alphaVal +α+\alphaTest GapTest −α-\alphaTest +α+\alpha
Olmo 3 32BCommitAIME 202635.mlp0.500+0.233-0.2670.499+0.209-0.290
GSM8K53.mlp0.423+0.153-0.2700.414+0.127-0.287
OMEGA-50039.attn0.316+0.074-0.2420.314+0.049-0.265
GPQA-D43.attn0.547+0.271-0.2760.556+0.258-0.298
MMLU-Pro43.attn0.489+0.220-0.2680.490+0.204-0.286
ConfidenceAIME 202623.attn0.602+0.398-0.2040.585+0.361-0.224
GSM8K25.attn0.484+0.292-0.1920.489+0.276-0.213
OMEGA-50023.attn0.416+0.257-0.1590.411+0.242-0.169
GPQA-D25.attn0.247+0.138-0.1100.237+0.124-0.113
MMLU-Pro23.attn0.324+0.205-0.1190.333+0.201-0.132
Qwen 3 32BCommitAIME 202651.attn0.235+0.008-0.2270.282+0.036-0.246
GSM8K52.attn0.163-0.063-0.2270.194-0.052-0.246
OMEGA-50052.mlp0.312+0.086-0.2260.348+0.103-0.245
GPQA-D51.mlp0.614+0.391-0.2230.641+0.399-0.242
MMLU-Pro51.mlp0.568+0.351-0.2170.595+0.359-0.236
ConfidenceAIME 202647.attn0.601+0.407-0.1940.645+0.423-0.222
GSM8K48.attn0.561+0.358-0.2030.607+0.385-0.222
OMEGA-50048.attn0.512+0.340-0.1720.565+0.354-0.211
GPQA-D50.attn0.429+0.326-0.1030.455+0.347-0.108
MMLU-Pro50.attn0.392+0.287-0.1050.429+0.306-0.123
R1 Qwen 32BCommitAIME 202647.mlp0.716+0.358-0.3590.730+0.356-0.374
GSM8K48.mlp0.351-0.019-0.3700.324-0.053-0.377
OMEGA-50047.mlp0.691+0.340-0.3520.703+0.333-0.370
GPQA-D46.mlp0.679+0.306-0.3730.673+0.292-0.381
MMLU-Pro46.attn0.819+0.447-0.3710.814+0.432-0.382
ConfidenceAIME 202646.attn0.634+0.325-0.3090.674+0.344-0.330
GSM8K46.attn0.710+0.371-0.3390.744+0.403-0.341
OMEGA-50046.attn0.588+0.312-0.2750.631+0.331-0.300
GPQA-D45.mlp0.544+0.307-0.2370.598+0.332-0.266
MMLU-Pro46.attn0.534+0.273-0.2610.583+0.295-0.288

Thank you for the additional experiments and clarifications. These have largely addressed my main concerns. I will maintain my rating of 3 (Weak acceptance).

Reviewer_aJ1WOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper investigate the question: whether reasoning models' tendency to backtrack at an answer proposal is driven by an internal sense of confidence. The authors use an LLM-judge pipeline, checked against human labels, to find "answer proposals" in reasoning traces. They show that 84–89% of rollouts contain only one unique answer, yet that answer is proposed 4–9 times. They test three models (Qwen3-32B, Olmo 3 32B Think, R1-Distill-Qwen-32B) on five datasets. v_conf steers the stated confidence, and it also steers the probability of a backtracking token after a proposal (p_bt). It does this about as well as v_commit, a direction fit on final vs. mid-trace proposals, while v_commit cannot steer stated confidence. The paper then moves to multi-token generation to compare steering against editing the proposal's wording and steering continuously after a proposal, which changes response length and the number of later proposals in both directions,

Critical strengths
  1. Clear question with a measured starting point. The redundancy statistics (Tables 1, 12) back up the motivation. The judge pipeline is checked against human annotations (precision 0.91 for math, 0.87 for multiple choice).

  2. Interesting one-way transfer. A direction fit at the confidence-prompt position steers backtracking at the proposal marker. The reverse fails, even though the two directions are nearly orthogonal. This suggests confidence sits upstream of the backtracking decision.

  3. Good controls for single-token steering. These include random-direction baselines, robustness to how the confidence prompt is worded (Spearman ≥ 0.93, CKA ≥ 0.88), a position ablation, a direction fit on p_bt itself (v_prob), and consistency across datasets (cosine ≥ 0.85).

  4. Nice breadth. Three models with different post-training (RL-trained and distilled), and five benchmarks covering math and multiple-choice knowledge questions.

  5. The authors go beyond the next token. They also study multi-token, bidirectional and from-scratch steering, plus a comparison with editing the text directly.

Critical weaknesses
  1. The paper never reports the correlation between stated confidence and p_bt under the same prefix (question + CoT up to the proposal), even though it has both numbers for every marker. The only correlation reported (§3.2, lines 173–176, Table 13) is between stated confidence and a final vs. mid-trace label. It is weak: ρ = 0.15 pooled and 0.22 within-rollout. On incorrect rollouts the pooled ρ is about 0.00–0.03. So the claimed "interplay" rests almost entirely on the steering results, while the observed relationship is close to absent where it matters most.

  2. Steering may suppress backtracking that is actually needed. 11–16% of rollouts propose more than one distinct answer. The paper never checks what happens there. Pushing toward acceptance on every question will inevitably cut off some useful backtracking. The overall accuracy number hides this: half the test markers are final proposals, and most rollouts never change their answer. So even frequent locking-in of wrong proposals would barely move the average. Table 4 hints at this cost. Accuracy drops concentrate on harder sets (Olmo AIME −5.0, OMEGA −2.7, GPQA −2.7; Distill OMEGA −4.8), while GSM8K is flat. Delaying steering by 5000 tokens recovers most of the loss. If models are less confident when a proposal is wrong, gating the steering on a confidence threshold could avoid the problem. The paper never tests whether stated confidence predicts proposal correctness in its own data.

  3. Baselines are under-specified, and a stronger baseline beats v_conf. v_commit gets one sentence in §4.1 (lines 252–254). Figure 1 already plots it and is cited in §3.2, before that definition. Section 3 should have a baseline subsection that defines it properly. v_prob (App. D), fit directly on high vs. low p_bt, beats both (steering gap 0.79–0.95 vs. 0.59–0.74 for v_conf; Table 16). So the abstract's "on par with a direction extracted directly from backtracking examples" holds only against v_commit. I understand that v_prob serves mostly as a sanity check to show that v_conf is not simply a dummy for pbtp_bt, but I believe it's worth to bring this ablation to the main paper. Otherwise this can be seen as a loophole in the story.

  4. Some presentation and claims are not totally accurate.

    • First abstract statistic. The opening number is wrong: for Olmo on AIME 2026, Table 12 gives 95% single-answer rollouts and about 10 proposals, not about 87% and about 8. Those figures are the average across all models and datasets.
    • In the end of the abstract, it's unclear what "progressively more realistic interventions" means. The sentence at lines 027–029 ("Steering continuously from…") is hard to parse; my reading is that steering can make the CoT shorter or longer.
    • "Symmetrically" doesn't fit −4.52 vs +3.26 or −1k vs +3k tokens.
    • The numbers are for mid-trace markers only, which the abstract doesn't say.
    • Line 072 ("when steering the full CoT…") describes the linguistics experiment wrongly. That experiment regenerates a single proposal sentence at α = ±0.5 for up to 128 tokens, with three qualitative examples.
Questions for authors

See my weakness points above.

3 comments

We thank the reviewer for the detailed feedback and address each point below. Changes in the revised manuscript are in red.

[W1] Correlation between verbalized confidence (VC) and probability of backtracking

To clarify, the pooled ρ\rho reported in Table 13 is confounded by variations between questions across the dataset. The same table also reports within-rollout correlations, which controls for this confound and is positive (\rho = 0.08-0.25). Since answer commitment is a binary label with unbalanced counts (more mid-trace markers than final markers), we provide the max attainable ρ\rho for context to Table 13, where \rho_\max = 0.51,0.62,0.66 across the models.

In section 3.2, we also state that we view VC as a surface-level approximation of the model’s underlying confidence. Thus, the majority of our contributions in section 3, 4, and 5 focus on steering, which is causal evidence on the “interplay” rather than correlational.

As requested by the reviewer, we correlate VC and pbtp_\text{bt} for each model and dataset. * means 95% bootstrap CI excludes 0. Across all models, pooled ρ=−0.26\rho = -0.26 and within-rollout ρ=−0.29\rho = -0.29. On incorrect rollouts, there is still an overall correlation when pooled or within-rollout, though, similar to answer commitment correlations, not all entries have CI that exclude 0.

ModelDatasetPooled AllPooled CorrectPooled IncorrectWithin AllWithin CorrectWithin Incorrect
Olmo 3 32BAIME 2026−0.39*−0.45*−0.06−0.40*−0.41*−0.27*
OMEGA−0.32*−0.38*−0.10*−0.30*−0.32*−0.27*
GSM8K−0.41*−0.43*−0.05−0.37*−0.39*−0.25*
GPQA-D−0.08*−0.12*+0.04−0.24*−0.32*−0.08*
MMLU-Pro−0.21*−0.22*−0.16*−0.23*−0.31*−0.06
All−0.27*−0.31*−0.08*−0.29*−0.34*−0.15*
Qwen 3 32BAIME 2026−0.34*−0.34*−0.14−0.32*−0.32*−0.30*
OMEGA−0.35*−0.36*−0.24*−0.30*−0.29*−0.33*
GSM8K−0.22*−0.19*−0.26*−0.19*−0.18*−0.26*
GPQA-D−0.12*−0.14*−0.10*−0.34*−0.36*−0.31*
MMLU-Pro−0.12*−0.16*−0.05−0.24*−0.26*−0.22*
All−0.19*−0.19*−0.12*−0.26*−0.25*−0.27*
Distill 32BAIME 2026−0.37*−0.40*−0.26*−0.35*−0.53*−0.06
OMEGA−0.33*−0.42*−0.19*−0.32*−0.35*−0.28*
GSM8K−0.47*−0.44*−0.48*−0.35*−0.36*−0.26*
GPQA-D−0.07*−0.13*+0.03−0.28*−0.39*−0.10*
MMLU-Pro−0.16*−0.21*−0.06−0.28*−0.43*−0.09*
All−0.31*−0.35*−0.15*−0.32*−0.38*−0.21*
All models−0.26*−0.29*−0.11*−0.29*−0.32*−0.19*

To provide more correlational evidence, we also measure how well the latent confidence direction tracks either answer commitment or pbtp_\text{bt} by projecting answer proposal hidden states onto vconfv_\text{conf}. Please also see our [W1, Q1] response to UzVv, and section 4.2, Tables 28 and 29 in the revised manuscript. To summarize, vconfv_\text{conf} shows stronger correlations than VC for pbtp_\text{bt} for all three models, and stronger correlations than VC on answer commitment for Olmo 3 32B and Qwen 3 32B.

[W2] Relationship between stated confidence and correctness

We agree that uniform steering can suppress useful backtracking, especially on harder questions. Additionally, initial reasoning steps can strongly influence later reasoning steps, so increasing the model's confidence too early can reduce accuracy. As mentioned by the reviewer, this lead us to delay steering after 5000 tokens to help recover accuracy. We note that our goal in section 5.3 is to demonstrate that the effects of confidence steering hold even when the answer proposal positions are unknown, rather than propose a practical efficiency method (i.e. we don’t optimize alpha or the delay per dataset). In the revised manuscript, we make the accuracy-efficiency tradeoff more explicit in section 5.3, and we leave precisely identifying which answer proposals would benefit most from confidence steering to future work.

[W4] Fixing Presentation

We thank the reviewer pointing these out, and make appropriate changes in the manuscript. The statistic in the abstract is indeed across the three models. The phrase “progressively more realistic interventions” means that we started with single token steering (section 4), progressed to multi-token steering after an answer proposal marker (section 5.2), and finally did steering at inference without knowing the marker positions in advance (section 5.3).

We will report a follow-up response to other points soon!

[W3] vprobv_\text{prob} ablation as a baseline

We agree that we can more clearly introduce vcommitv_\text{commit}, and we have cleaned up the prose in the revised manuscript.

With regards to baselines, we believe that this is not a loophole in our paper. In section 4, our goal is not to maximize performance on steering pbtp_\text{bt}, but rather showcase cross-task steering generalization. We select vcommitv_\text{commit} to be our baseline since we are studying answer proposals that lead to commitment vs backtracking. vcommitv_\text{commit} is directly constructed from these two classes, making it the most relevant baseline to compare with vconfv_\text{conf} in our experiments. We then show in section 4.1 that vconfv_\text{conf} is able to steer verbalized confidence as well as transfer to steering pbtp_\text{bt}, while vcommitv_\text{commit} can steer pbtp_\text{bt} but cannot transfer to steering verbalized confidence. Since vprobv_\text{prob} is constructed specifically to separate high and low pbtp_\text{bt} markers, we expect vprobv_\text{prob} to perform best on steering pbtp_\text{bt} (we add this clarification to Appendix D). However, vprobv_\text{prob} is the same as vcommitv_\text{commit} in the places that matter: (a) it is fit on the same position as vcommitv_\text{commit}, and (b) vprobv_\text{prob} cannot transfer to steering verbalized confidence (14.4% of vconfv_\text{conf}’s steering score, as shown in Table 17).

As requested by the reviewer, we generate steered continuations after answer proposal markers for each α\alpha. We generate 1 sample per continuation, and report results below, with the best row bolded.

Modelα\alphaΔ\DeltaL AllΔ\DeltaL Mid-traceΔ\DeltaL FinalΔ\Deltaacc
Olmo 3 32B−0.5+16546+19086 ± 483+13815 ± 596−29.0
−0.25+3087+4905 ± 228+1133 ± 133+0.6
−0.1+608+1029 ± 110+155 ± 39−0.2
+0.1−375−661 ± 95−67 ± 19−1.0
+0.25−804−1472 ± 120−85 ± 20−0.4
+0.5−1021−1869 ± 133−110 ± 20−0.6
+0.75−1100−2011 ± 143−122 ± 21−0.8
Qwen 3 32B−0.5+16942+16819 ± 763+17016 ± 596−41.6
−0.25+15300+16819 ± 763+14384 ± 820−40.8
−0.1+2737+4163 ± 285+1216 ± 160−2.0
+0.1−625−1085 ± 121−135 ± 17−1.8
+0.25−960−1619 ± 152−257 ± 19−3.0
+0.5−1177−1890 ± 163−417 ± 24−7.4
+0.75−1318−2038 ± 165−549 ± 26−15.6
Distill 32B−0.25+18294+21466 ± 706+14885 ± 838−44.0
−0.1+2398+3533 ± 551+1179 ± 244−3.2
+0.1−897−1446 ± 336−306 ± 138−1.6
+0.25−1179−1928 ± 330−375 ± 138−1.4
+0.5+18128+17895 ± 1062+18378 ± 842−46.6
+0.75+21185+21243 ± 1064+21122 ± 847−58.0

For comparison purposes, we display results for steering with vconfv_\text{conf} below, only with the best α\alpha values (full results in Table 24).

Modelα\alphaΔ\DeltaL allΔ\DeltaL mid-traceΔ\DeltaL finalΔ\Deltaacc
Olmo 3 32B−0.2521013472 ± 173627 ± 87+0.2
0.75−1057−1906 ± 121−144 ± 15−0.6
Qwen 3 32B−0.2512502183 ± 192254 ± 36−0.7
0.25−313−618 ± 7311 ± 8−0.6
Distill 32B−0.110821877 ± 245228 ± 46−1.2
0.5−839−1446 ± 194−188 ± 55−0.0

We aim to compare vprobv_\text{prob} with vconfv_\text{conf} steering at similar accuracy drops. On Olmo 3 32B, steering with vprobv_\text{prob} is comparable with vconfv_\text{conf} steering, and performs slightly better in the negative α\alpha setting. However, on Qwen 3 32B and Distill 32B, steering with vprobv_\text{prob} degrades accuracy with all α\alpha values, leading to accuracy drops by more than 1.2 points when compared to vconfv_\text{conf}.

[W2] Relationship between stated confidence and correctness (continued)

The paper never tests whether stated confidence predicts proposal correctness in its own data.

We measure the pooled and within-rollout correlations between VC and answer correctness. We report results on all rollouts and rollouts with 2+ answers in the table below (*: 95% bootstrap CI excludes 0).

Across rollouts and models, VC is higher for correct answers compared to incorrect answers (ρ=0.38\rho=0.38). However, within rollouts with 2+ answers, models are typically more confident on the answer it commits to, regardless of accuracy. Correct 2+ rollouts have high positive within-rollout correlations (ρ=0.32\rho=0.32), and incorrect 2+ rollouts have slightly negative within-rollout correlations (ρ=−0.05\rho=-0.05). Note that incorrect 2+ rollouts only contribute to the within-rollout correlation if they had also proposed a correct answer somewhere in the rollout.

ModelSubsetRolloutsMarkersWithin-rollout ρ\rho (All)Within-rollout ρ\rho (Correct)Within-rollout ρ\rho (Incorrect)Pooled ρ\rho (All)Pooled ρ\rho (Correct)Pooled ρ\rho (Incorrect)
Olmo 3 32BAll320232697+0.12*+0.19*−0.06*+0.39*+0.30*−0.03
2+ answers54610002+0.22*+0.37*−0.07+0.36*+0.37*+0.09
Qwen 3 32BAll264717255+0.08*+0.14*−0.05*+0.39*+0.24*+0.05
2+ answers2934176+0.14*+0.25*−0.07*+0.29*+0.23*+0.16
Distill 32BAll302517232+0.10*+0.18*−0.02+0.37*+0.22*−0.02
2+ answers3804759+0.16*+0.27*−0.01+0.27*+0.23*+0.04
All modelsAll887467184+0.11*+0.17*−0.05*+0.38*+0.26*−0.01
2+ answers121918937+0.18*+0.32*−0.05*+0.32*+0.30*+0.09

11–16% of rollouts propose more than one distinct answer. The paper never checks what happens there.

To understand how steering impacts rollouts where more than 2+ unique answers are proposed, we operationalize this through two scenarios:

  1. Among incorrect answer proposals that the model can demonstrably recover from (the current answer is incorrect but the model has proposed a correct answer to this prompt), how often does steering lead to the correct answer?
  2. Among correct answer proposals where the model can demonstrably arrive at the wrong answer (the current proposal is correct but the model has committed to an incorrect answer to this prompt), how often does steering lead to the incorrect answer?

We obtain relevant proposal markers from the test set by checking the proposals of all rollouts that share the same prompt as the relevant marker. We analyze the results using the steered continuations with the best positive and negative α\alpha per model, as well as the α=0\alpha=0 control (section 5.2).

Results for scenario 1 are below. # markers is the number of markers that fit the criteria, and each marker has four continuations. I → C counts continuations that change to the correct answer. “Same answer” means the final answer matches the proposed answer. Against the control, positive steering reduces the model's ability to recover the correct answer, confirming a small cost to suppressing backtracking. Conversely, negative steering slightly increases the model’s ability to recover the correct answer compared.

modelα\alphamarkerscontsI -> Csame answer
Olmo 3 32B06526042 (16%)207 (80%)
-0.256526048 (18%)191 (73%)
+0.756526028 (11%)226 (87%)
Qwen 3 32B05722836 (16%)176 (77%)
-0.255722840 (18%)166 (73%)
+0.255722834 (15%)173 (76%)
Distill 32B06425643 (17%)185 (72%)
-0.16425648 (19%)180 (70%)
+0.56425635 (14%)205 (80%)

Results for scenario 2 are below. C → I counts continuations that change to an incorrect answer. Positive steering is comparable to or slightly better than no steering at maintaining the correct answer. Combined with scenario 1, positive steering has a small net effect, consistent with near zero accuracy change in Table 24.

modelalphamarkerscontsC->Istays correct
OLMo-3-32B06827220 (7%)252 (93%)
-0.256827224 (9%)248 (91%)
+0.756827211 (4%)261 (96%)
Qwen3-32B04016019 (12%)141 (88%)
-0.254016021 (13%)139 (87%)
+0.254016020 (12%)140 (88%)
R1-Qwen-32B05522019 (9%)201 (91%)
-0.15522028 (13%)192 (87%)
+0.55522014 (6%)206 (94%)

We will add the steering analysis to Appendix G.2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.