acceptodds
Under review as a conference paper at ICLR 2027

Right Answer, Wrong Reasoning: Localizing Chain-of-Thought Restoration Errors in the Residual Stream

Abstract

Chain-of-thought traces are increasingly treated as explanations, yet that trust depends on whether the stated reasoning faithfully reflects how the model reached its answer. We study a failure that escapes surface-level checks: the model reaches the correct answer while producing a verifiably wrong intermediate step that is fluent and never corrected. We call this silent restoration. Answer-level verification misses it because the answer is correct. Across twelve model configurations and four domains, silent restoration appears universally and reaches 32% of answer-correct MedQA traces for Qwen2.5-7B. Across nine model configurations, linear probes identify wrong steps at 0.73–0.91 AUROC, consistently at interior layers. We also find that this signal is already present before the step is generated. Across layers, wrong steps exhibit a uniform upward shift in probe score rather than a change in profile shape. Exploiting this geometry with a probe-plus-distance detector yields 0.792 out-of-domain AUROC, outperforming all baselines and the strongest token-uncertainty signal at 0.655. The signal is also causal: steering along the probe direction induces verifiable errors in 46% of correct steps, and can also repair naturally wrong steps. The same intervention distinguishes load-bearing steps, on which the answer depends, from performative ones, on which it does not, suggesting that faithfulness is fundamentally a per-step rather than a per-trace property.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
interpretability, silent restoration, probe

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_M49gOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper looks at the phenomenon of trace errors – i.e., a model producing an incorrect reasoning step despite producing a correct final answer; dubbed “silent restoration” in the paper. While most prior work on this phenomenon tends to focus on domains like math, this work expands the analysis of trace errors to additional domains (math, finance, medicine, code) and identifies interesting inter-domain differences: for example, medicine tends to be the most prone to trace errors, which is particularly pernicious (since people are potentially more likely to use and trust reasoning traces in that domain). The authors then train probes to detect incorrect reasoning steps on answer-correct traces. They replicate several properties observed in correctness probes (i.e., peak accuracy in the middle layers, decodability before the wrong step is produced). Finally, the authors propose Probe-Targeted Activation Steering, a novel method for steering models along probe directions; they find that they can both steer for correct steps and for incorrect steps using this method. This work provides a meaningful and valuable extension of previous work on trace errors (particularly by expanding the domains), but some results and the overall novelty are overstated, and the presentation is heavily “AI-written” (to the work’s detriment; expanded upon below).

Critical strengths
  1. The authors investigate an important phenomenon (trace errors paired with correct final answers) and provide a real extension of prior work: they investigate trace errors in a broader range of domains beyond the typical choice of math (including medicine and finance) and uncover interesting inter-domain differences (like medicine being the most prone to trace errors).

  2. The authors devise a novel technique for probe-based steering that allows them to steer traces towards correct trace + correct answer and incorrect trace + correct answer, which is useful (either for creating evaluation data for monitors, or for understanding the mechanisms of trace errors).

Critical weaknesses

(Weakness 1 and 2 are the most important, but I felt that I needed to include the third, especially since it is fairly straightforward to fix.)

  1. Some of the results are a bit weak. For example, the authors’ findings that wrong step probe accuracy peaks in the middle layers as well as the pre-commitment finding are expected based on prior work (mid-layer probing: [1–4]; pre-commitment: [5–7]), so these are more like replications than novel findings. Furthermore, the translation vs. deformation result seems like it would be the case by construction (linear probes). Finally, their causal steering results need some additional baselines: a random direction baseline (since it’s possible that any perturbation of a given size will knock the model off its current trajectory), and a check for the steering judge (since the judge views the whole suffix, a successful corruption could mean that an error is induced at another step further downstream rather than the targeted step).

  2. The novelty is greatly overstated, and relevant prior work looking at the same phenomenon is largely not cited. The phenomenon the authors refer to as “silent restoration” – a correct answer preceded by an incorrect rationale in the CoT – is very well-documented in prior work under various labels (trace errors, process errors, etc.). The paper claims that “existing work is not positioned to catch this failure,” despite existing work addressing it [8–12]. The authors claim to be “the first to study this failure through the model’s internal representations at the level of individual steps,” but there is also prior work that does this [5, 13, 14]. The authors also claim to give “the first per-step definition of load-bearing versus performative steps,” but there is some existing precedent for this as well [15–17]. This weakness can be addressed if the authors add the missing citations and reframe their contributions.

  3. The presentation reads as heavily AI-written (e.g., “Our answers form a single progression from phenomenon to representation to intervention.” or “The wrong step is silently overwritten by a right answer.”). This sometimes adds unnecessary length or makes the point unclear. Editing the prose for clarity and succinctness would greatly help readers understand the work. The paper is also missing an AI Use Statement; it would be helpful to hear from the authors the extent to which they used AI in this work.

References

[1] Azaria & Mitchell. The internal state of an LLM knows when it’s lying. 2023.

[2] Marks & Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. 2024.

[3] Li et al. Inference-time intervention: Eliciting truthful answers from a language model. 2023.

[4] Orgad et al. LLMs know more than they show: On the intrinsic representation of LLM hallucinations. 2025.

[5] Zhang et al. Reasoning models know when they’re right: Probing hidden states for self-verification. 2025.

[6] Yuan et al. Hidden error awareness in chain-of-thought reasoning: The signal is diagnostic, not causal. 2026.

[7] Cox et al. Post-hoc reasoning in chain of thought: Decoding and steering pre-committed answers. 2026.

[8] Uesato et al. Solving math word problems with process- and outcome-based feedback. 2022.

[9] Zheng et al. ProcessBench: Identifying process errors in mathematical reasoning. 2025.

[10] Xia et al. Evaluating mathematical reasoning beyond accuracy. 2025.

[11] Wang et al. Examining false positives under inference scaling for mathematical reasoning. 2025.

[12] Yuan et al. Curing “miracle steps” in LLM mathematical reasoning with rubric rewards. 2026.

[13] Zhao et al. Verifying chain-of-thought reasoning via its computational graph. 2026.

[14] Ni et al. Efficient test-time scaling of multi-step reasoning by probing internal states of large language models. 2025.

[15] Bogdan et al. Thought anchors: Which LLM reasoning steps matter? 2025.

[16] Tutek et al. Measuring chain of thought faithfulness by unlearning reasoning steps. 2025.

[17] Lanham et al. Measuring faithfulness in chain-of-thought reasoning. 2023.

Questions for authors

Clarification requests:

  1. Can the authors situate their work against the prior work I cited in weaknesses (i.e. explain how their work is novel in the ways they claim, or update their claims?)
  2. Can the authors explain why the translation vs. deformation result is not expected by construction?
  3. Can the authors provide an AI Use Statement following the ICLR guidelines in their rebuttal? It would be helpful to know the extent to which AI was used in producing this work, especially if it was used for experimental design.

Suggestions for future work:

  1. Do the authors have an idea of why certain steps produce trace errors? Did they notice any regularities in the particular erroneous steps across models, or did models tend to make errors in unique locations?
2 comments

W1: Some of the results are a bit weak.

  • Response: We agree that several of our findings extend prior work, and we will present them as such. We also added two controls to the steering evaluation.
  • Mid-layer localization and pre-step predictability. Intermediate-layer correctness signals and prediction before generation have precedents in [1–7]. We will present our results as extensions of these findings to verifiably incorrect steps within answer-correct traces, not as new general phenomena. Predictive information about step correctness is available before the step is generated (Table 2), but this does not establish that the model has committed to the error, so we will replace "pre-commitment" with "pre-step predictability" throughout.
  • Translation versus deformation. We agree that higher scores for incorrect steps are the expected outcome of supervised probe training, although this is not guaranteed on held-out data. Our observation is narrower: in the per-layer mean held-out logit profiles, the incorrect-step curve lies at a roughly constant offset above the correct-step curve. Because each layer has its own probe, this is a statement about score profiles, not about a shared displacement in activation space. Layer-specific standardization and regularization may also influence the logit scales, so we do not interpret the approximately constant gap as establishing a geometric property of the underlying representations. The detector is evaluated separately in Table 4.
  • Random-direction control. On Qwen2.5-7B-Instruct with MATH, we evaluate 5 random directions per target step, matching each intervention to PTAS in layer, start position, token window, and actual edit norm. Random steering achieves mean flip-and-coherent rates of 31% for induction and 40% for repair, compared with 46% and 56% for PTAS under the original evaluation protocol.
  • Target-step judge. We re-evaluate correctness using only the original question, preceding context, and explicitly marked target step, excluding subsequent steps; continuation coherence is assessed separately. Under this evaluation, PTAS’s flip-and-coherent rate on MATH changes from 46% to 41% for induction and from 56% to 54% for repair.

W2: The novelty and relevant prior work.

  • Response: We agree that our framing overstated novelty. We will remove the "first" claims and the statement that existing work is not positioned to catch this failure, add the missing citations, and reposition our contributions relative to [5, 8–17].

  • Phenomenon [8–12]. We will acknowledge these works on process errors and on the limits of final-answer correctness as an indicator of reasoning quality. Our contribution is a systematic evaluation under an explicit four-condition criterion across twelve model configurations and four domains, together with an investigation of the internal signals associated with these errors.

  • Internal analysis [5, 13, 14]. We will explicitly acknowledge these works as precedents for internal error detection and intervention: [5] probes intermediate candidate-answer correctness, including before the answer is generated; [13] detects step errors and corrects reasoning through transcoder-feature interventions; and [14] uses internal-state probes for step verification and test-time scaling. Our analysis focuses on verifiably wrong steps within answer-correct traces, where final-answer correctness alone cannot distinguish correct from incorrect steps. In this setting we study layer-wise localization and pre-step predictability, and develop a probe-plus-distance detector that reaches 0.792 mean OOD AUROC (vs. 0.733 for the probe alone).

  • Load-bearing vs. performative steps [15–17]. We will acknowledge these precedents on analyzing individual steps' contributions to the answer. Our analysis measures answer sensitivity to step-level corruption induced by pre-step activation steering. We will describe it as sensitivity under the tested intervention, not as a validated faithfulness label, and revise the corresponding statements in §6.2.

W3: AI Use Statement.

  • Response: Thank you for the feedback. We will edit the manuscript for clarity and concision, removing rhetorical phrasing such as the “single progression” sentence and replacing “silently overwritten” with a precise description: an incorrect intermediate step remains uncorrected in the text despite a correct final answer.
  • An AI Use Disclosure is already included in Appendix F (p. 16, lines 857–863). It states that generative AI tools assisted with research brainstorming, refining research questions, feedback on experimental design, drafting portions of the manuscript, and language editing. All AI-assisted suggestions, experimental procedures, results, and manuscript content were reviewed and verified by the authors, who take full responsibility for the work. We will add a clearer pointer to this disclosure in the main text to make it easier to locate.

Questions For Authors:

Q1: Can the authors situate their work against the prior work I cited in weaknesses (i.e. explain how their work is novel in the ways they claim, or update their claims?)

  • Response: see response to 'W2'

Q2: Can the authors explain why the translation vs. deformation result is not expected by construction?

  • Response: Class separation is the expected outcome of supervised probe training, but this alone does not imply that the separation is similar across layers. Each layer has an independently trained probe and its own standardization, and nothing in the training objective requires the gap between the class-mean held-out scores to stay roughly constant across layers. That approximately constant offset is the empirical observation in Figure 3.
  • We agree that its interpretation should be narrower. Because normalization and regularization can influence logit scales, the observation concerns mean held-out score profiles, not evidence of a translation in activation space, and we will revise the "translation versus deformation" wording and avoid presenting it as a general geometric mechanism. Its practical use is as a motivation for the profile-level analysis in Table 3, where level features reach OOD AUROC 0.678–0.769 while spread, shape, and fluctuation features stay at or below 0.56. The probe-plus-distance detector is evaluated separately in Table 4.

Q3: Can the authors provide an AI Use Statement following the ICLR guidelines in their rebuttal? It would be helpful to know the extent to which AI was used in producing this work, especially if it was used for experimental design.

  • Response: Please see our response to 'W3', where we summarize the AI Use Disclosure already included in Appendix F. This disclosure explicitly covers assistance with research ideation and feedback on experimental design, as well as manuscript drafting and language editing. All AI-assisted content was reviewed and verified by the authors.

Q4: Do the authors have an idea of why certain steps produce trace errors? Did they notice any regularities in the particular erroneous steps across models, or did models tend to make errors in unique locations?

  • Response: We do not yet have a definitive explanation for why particular steps become erroneous. One hypothesis, which we have not tested, is that stating an intermediate step and producing the final answer can rely on partially different computations, so a model may emit an incorrect intermediate claim and still reach the correct answer through other information or later computation. Our pre-step probing results show that predictive information about an upcoming error is available before the step is generated, but they do not identify the mechanism that produces it.
  • What we can report are two descriptive observations. First, silent restoration is most frequent in medicine for 11 of 12 configurations (§3.2). Second, the illustrative cases in Appendix H, all from Qwen2.5-7B, show different error types across domains (a false arithmetic result, inverted code logic, a wrong clinical label). We do not claim these generalize across models. Regarding cross-model regularities, our results establish representation-level patterns, such as interior-layer detectability, not shared semantic error locations. We have not aligned reasoning steps across models, so we cannot say whether they fail at the same points or at model-specific ones. Step indices alone would not be a reliable comparison because models use different decompositions and solution paths. A useful follow-up would align intermediate claims or subcomputations on the same problems across models, then measure overlap in error locations and categorize the associated operations. We will state the distinction between observed regularities and these hypotheses explicitly in the discussion.
Reviewer_RdhROpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work explores silent error restorations (i.e. the verifiably wrong intermediate steps in reasoning traces ending up with the correct answer) by studying their structure, detectability and relationship to model's internal representations. They find out model internals provide significant gains on detectability of this behavior over surface-level textual cues. Also, proposing an optimization based steering method that can efficiently control the behavior and operationalize it to detect the steps models answer causally rely on.

Critical strengths
  1. The experiments are very comprehensive covering wide range of domains and models with careful considerations such as human validation of mainly used judge model and strong baselines to establish the gains from internal representations.

  2. The paper successfully addresses the failing steering approaches in previous work by optimizing steering vector itself based on the probes readout.

  3. Paper is generally readable and easy to ingest.

Critical weaknesses
  1. One of the biggest issues with the paper its incorrect contextualization with respect to previous work. There are many previous studies working faithfulness as step-level features [1-3] So, the claims such as "To our knowledge, we are the first to study this failure through the model’s internal representations at the level of individual steps" misrepresent this work as well as previous work. Also, silent error correction is framed as something identified by the authors, however it is also identified and studied before [4, 5]

  2. The proposed faithfulness evaluation setup resembles the Adding Mistakes setup from previous work [5] where step-wise mistakes added with a helper LLM. Some previous benchmarks [6,7] shows this is not a good way to measure faithfulness. Since the paper's main scope is on detectability of wrong steps from internal representations, the authors should mention this limitation and narrow their faithfulness claims or include one of these benchmarks to show that their "adding mistakes" approach is good and supports their claims on faithfulness measurement.

  3. This paper lacks a comparison with some very related papers [2, 3].

References

  1. Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps, Tutek et al., 2025

  2. Can Aha Moments be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought, Zhao et al., 2026

  3. Reasoning Models Know When They’re Right: Probing Hidden States for Self-Verification, Zhang et al., 2025

  4. Chain-of-Thought Reasoning in the Wild Is Not Always Faithful, Arcuschin et al., 2025

  5. Measuring Faithfulness in Chain-of-Thought Reasoning, Lanham et al., 2023

  6. A Causal Lens for Evaluating Faithfulness Metrics, Zaman & Srivastava, 2025

  7. Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth, Gur-Arieh et al., 2026

Questions for authors
  1. In L235, it's stated that they use greedy decoding for the experiments. Why is greedy decoding selected instead of each model's recommended sampling scheme? Does this sampling scheme affect the predictability or prevalence of silent restoration?

  2. It is not clear on which model the baseline detectors are trained on. The paper mentions that Appendix provides cross-model results, however it does not mention which model is used for the main results. From cross-model results, I assume all detector results are based on smaller models. I think it would be good to put a bigger judge model as a baseline.

  3. It's not clear what mean probe wrong score (OOF) means in Figure 3 and how to interpret it. Also, is wrong score calculated by fixing the probe or using the probe trained for corresponding layers?

  4. During identifying load-bearing and performative steps, do the authors steer one token and regenerate rest of that step or rest of all trace? If they regenerate rest of the whole trace, it is not a fair assesment due to compound effect of multiple steps.

3 comments

W1. Contextualization with respect to previous work.

  • Response: We thank the reviewer for the pointers. We agree that the "first" claim is too broad and will remove it, and we will revise the introduction, related work, and contribution statements to position our work relative to [1–5].
  • Our contribution. We operationalize silent restoration using four explicit conditions (correct answer, a verifiably wrong step, coherence, no self-correction) and measure it systematically across twelve model configurations and four domains. We then localize the corresponding signal in the residual stream, show that it is present before the step is generated, build a detector that transfers out of domain (0.792 mean OOD AUROC vs. 0.733 for the probe alone and 0.655 for the strongest token-uncertainty baseline), and test its causal role by intervening in both directions at the pre-step position.
  • Relation to prior work.
    • Arcuschin et al. [4] and Lanham et al. [5] study trace–answer inconsistency, including naturally occurring restoration-type errors [4] and answer sensitivity to inserted mistakes [5]. We will cite them as precedents for the phenomenon. They do not analyze naturally occurring, verifiably wrong steps in answer-correct traces through internal representations, which is what we study.
    • Tutek et al. [1] measure step-level faithfulness by unlearning individual steps and observing changes in the answer. We instead detect step-level correctness from activations and intervene on activations at inference time.
    • Zhao et al. [2] quantify step-level contributions and derive a steering direction that modulates whether a step is used. Our direction is derived from step-correctness labels and applied before the step is generated to induce or repair errors, so the targets differ (step use vs. step correctness).
    • Zhang et al. [3] probe the correctness of intermediate candidate answers. We probe the correctness of individual reasoning steps within answer-correct traces, where final-answer correctness alone cannot distinguish correct from incorrect steps. We also acknowledge their results on predicting correctness before an intermediate answer is fully generated.
  • We will add these comparisons to Related Work and revise the load-bearing/performative discussion in relation to [1, 2].

W2. The proposed faithfulness evaluation setup.

  • Response: We thank the reviewer for highlighting this distinction. We agree that sensitivity to an intervention is not, by itself, a validated measure of faithfulness, and we will narrow our claims accordingly and discuss the limitations identified in [6, 7].
  • Our intervention differs from Adding Mistakes [5] in implementation: we induce errors through pre-step activation steering rather than by having a helper LLM rewrite a step, and the judge only evaluates the resulting generations. This difference does not remove the underlying validity concern. An unchanged answer may reflect recovery or redundant computation rather than non-use of the step. Conversely, an answer change shows sensitivity to the intervention, but does not establish that the original verbalized step faithfully represented the model's computation. Activation edits may also affect downstream computation through pathways other than the generated step text.
  • We will therefore revise §6.2 to describe answer sensitivity under the tested intervention, rather than treating the load-bearing/performative distinction as ground-truth faithfulness labels. Table 6 will be interpreted as the rate of unchanged answers among successfully corrupted steps, a subset conditioned on steering success that may not be representative of all steps, and not as an estimated rate of unfaithful reasoning. We will also revise the statement in §6.2 about when silent restoration can occur.
  • Our main detection results concern verifiable step errors in naturally generated, answer-correct traces, and their evaluation does not rely on faithfulness labels derived from intervention outcomes. Likewise, the steering experiments test whether activation edits can induce or repair step errors, not whether they validate a general faithfulness metric. We will make these distinctions explicit and note that stronger faithfulness claims would require separate validation against benchmarks such as [6, 7].

W3. This paper lacks a comparison with some very related papers [2, 3].

  • Response: We agree that [2, 3] warrant more explicit comparison, and we will add a side-by-side methodological comparison covering prediction target, supervision, intervention, and evaluation protocol.
  • Zhao et al. [2] derive a steering direction from high- versus low-TTS steps to modulate step use. Our detector instead uses step-correctness labels, and our intervention targets the correctness of the subsequently generated step. The outcomes measured are therefore different, and the reported scores are not directly comparable.
  • Zhang et al. [3] train probes to predict the correctness of intermediate candidate answers. We instead predict whether individual reasoning steps contain verifiable errors within answer-correct traces. We will explicitly acknowledge their work as a precedent for hidden-state correctness probing, including prediction before an answer is generated.
  • Because the prediction targets and evaluation protocols differ, a direct comparison would require adapting their methods to a shared target. Our hidden-state MLP baseline (Table 4) already evaluates a hidden-state classifier trained on the same step-correctness labels.

Questions For Authors:

Q1: In L235, it's stated that they use greedy decoding for the experiments. Why is greedy decoding selected instead of each model's recommended sampling scheme? Does this sampling scheme affect the predictability or prevalence of silent restoration?

  • Response: We used greedy decoding for three reasons: (i) it gives one deterministic trace per problem, so each step's hidden state corresponds to exactly one label and probe training and evaluation are free of sampling noise; (ii) it allows all twelve configurations to be compared under identical decoding conditions, so differences in silent rate reflect the models rather than temperature settings; and (iii) it keeps judge cost per problem fixed, which let us label more problems (17,634 steps in the probing set). The steering evaluation also uses greedy decoding so that the ε=0 control isolates the effect of the edit.

  • We agree that greedy decoding is not the recommended setting for every model (e.g., thinking-mode models), and that the silent rate in Table 1 is specific to this decoding choice. Sampling could change the prevalence in either direction, and we do not claim that our absolute rates transfer to other decoding schemes. We will state this as a limitation and qualify the prevalence claims accordingly.

  • To assess decoding sensitivity, we re-ran Qwen2.5-7B-Instruct on MATH using its published sampling configuration (temperature=0.7, top-p=0.8, top-k=20, repetition penalty=1.05). The silent-restoration rate was 6.9%, compared with 6.9% under greedy decoding, and step-end probe AUROC was 0.883, compared with 0.872 under greedy decoding.

Q2: It is not clear on which model the baseline detectors are trained on. The paper mentions that Appendix provides cross-model results, however it does not mention which model is used for the main results. From cross-model results, I assume all detector results are based on smaller models. I think it would be good to put a bigger judge model as a baseline.

  • Response: The main detection results in Tables 3–4 use Qwen2.5-7B-Instruct as the trace-generating model and the source of hidden-state features. The probe, Mahalanobis distance, and MLP are fit on its MATH step-correctness labels and evaluated on held-out domains without retraining. The TF-IDF, BERT, and Mistral-7B text baselines use the same labeled steps, while token-uncertainty and prompted-judging baselines need no supervised training. Appendix D repeats the protocol for Mistral-7B and Gemma-3-4B, fitting each detector on its own model's MATH activations. We will state these model roles explicitly in the captions of Tables 3–4, in §5, and in Appendix C.
  • The probing signal is not limited to small models: Appendix J (Table 12) reports in-domain step-level AUROC of 0.871 and 0.892 for Qwen2.5-32B and Qwen2.5-72B, respectively. This addresses scale for the probing signal and is not an OOD detector comparison. Table 4 includes an external P(True) baseline scored by Qwen3.5-9B (0.655 mean OOD AUROC vs. 0.792 for our detector), and Table 9 includes a PRM baseline. Our step labels are produced by Claude Opus 4.8, so using that annotating judge as a baseline would not provide an independent assessment. Comparison with a separate, substantially larger judge is an additional evaluation beyond the reported baselines, and we will clarify this scope in the paper.

Q3: It's not clear what mean probe wrong score (OOF) means in Figure 3 and how to interpret it. Also, is wrong score calculated by fixing the probe or using the probe trained for corresponding layers?

  • Response: The "wrong score" in Figure 3 is the probe's pre-sigmoid logit, si,ℓ=wℓ⊤zℓ(hi(ℓ))+bℓs_{i,\ell}=w_\ell^\top z_\ell(h_i^{(\ell)})+b_\ell. Larger values mean stronger probe evidence that step ii is wrong; it is not a probability or an error rate.
  • "OOF" means out-of-fold: under grouped five-fold cross-validation, each step is scored by a probe trained on the other four folds, with all steps of a problem kept in the same fold. At each layer we pool these held-out scores and average them separately over correct and incorrect steps; these two class means are the curves in Figure 3.
  • We train a separate probe, with its own feature standardization, at each layer; no single probe is applied across layers. The "translation" therefore describes the profile of per-layer mean held-out logits (the incorrect-step curve lies roughly a constant amount above the correct-step curve), not a displacement along one shared direction in activation space. This curve-level observation motivates the two-view detector, whose effectiveness is evaluated independently in Table 4. We will clarify the score definition, the OOF procedure, and the layer-specific probes in §5.1 and the Figure 3 caption.

Q4: During identifying load-bearing and performative steps, do the authors steer one token and regenerate rest of that step or rest of all trace? If they regenerate rest of the whole trace, it is not a fair assesment due to compound effect of multiple steps.

  • Response: We compute the activation edit once, at the position immediately preceding the target step, and apply the fixed edit over the next kk generated tokens. We then regenerate the entire remaining continuation, including subsequent reasoning steps and the final answer, rather than only the target step. We will state this protocol explicitly.
  • We agree that this design does not isolate the causal contribution of the target step's text. Later steps can propagate, compensate for, or amplify the initial perturbation, and the edit may affect downstream computation beyond the verbalized step. Table 6 additionally conditions on successful corruption, a subset that may not be representative of all steps. We include an ϵ=0\epsilon=0 control that re-decodes from the same position under the same greedy-decoding protocol without an activation edit. This provides a no-edit reference, but does not isolate the target step's contribution from downstream effects.
  • Accordingly, we will interpret the results as final-answer sensitivity to a localized activation intervention followed by regenerated continuation, not as an isolated measure of step-level faithfulness. We will revise §6.2 and the Table 6 caption to state the intervention window, the continuation protocol, and this limitation, and will not treat an unchanged answer as sufficient evidence that the target step was unused.
Reviewer_LawqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies silent restoration: cases where models give correct final answers despite making errors in their reasoning steps that are never corrected. First, using an LLM as a judge, the authors show that such examples exist across four datasets: MATH-500, ConvFinQA, CruxEval, MedQA and in 10 different models. They then show that in some layers, a linear probe can distinguish between correct and wrong steps in two sites: 1) at the final token position of the step and, 2) right before the model is about to generate the step. Next, they show that they can induce or repair wrong steps right before they are about to happen using the probe direction. To do it, they optimize a vector that gets a high or low probe score (depends on the goal of the intervention). Finally, they show that corrupting some of the reasoning steps, especially at the beginning,  causes the model to switch from a correct final answer to an incorrect one.

Critical strengths

• The finding that linear probes can distinguish between correct and incorrect reasoning steps, even when the final answer is correct is interesting and could be useful for multiple use cases (monitoring, training, or human knowledge). • They test a broad range of settings: four datasets and 10 models.

Critical weaknesses

It is unclear to me why the authors need to optimize a steering vector to increase or decrease the probe score every step. A simpler and more common approach would be to steer directly along the probe direction. What does the optimization add compared with this more direct and common approach?

Questions for authors

N/A

1 comment

Weakness: It is unclear to me why the authors need to optimize a steering vector to increase or decrease the probe score every step. A simpler and more common approach would be to steer directly along the probe direction. What does the optimization add compared with this more direct and common approach?

  • Response: We agree that a direct probe-direction baseline is important. On MATH with Qwen2.5-7B, at matched k and t₀ and with ε tuned over the same grid for each method, direct steering along the normalized probe direction achieves 34%, compared with 51% for PTAS (flip-and-coherent rate, averaged over induction and repair).
  • We also clarify what our implementation does. Since the probe is linear, the gradient in Algorithm 1 is constant, g=c (w⊘s)g = c\,(w \oslash s), and the ℓ2\ell_2-constrained objective has a closed-form solution, so iteration is not mathematically required. Our implementation uses Adam (Appendix G), which under a constant gradient yields updates of η g/(∣g∣+ϵAdam)≈η sign(g)\eta\, g/(|g|+\epsilon_{\mathrm{Adam}}) \approx \eta\,\mathrm{sign}(g), except for coordinates with ∣g∣|g| comparable to ϵAdam\epsilon_{\mathrm{Adam}}. The resulting edit therefore has a different, per-dimension-equalized direction than normalized-gradient steering, and this difference is what the comparison above captures. We are also adding the closed-form sign(w⊘s)\mathrm{sign}(w \oslash s) control to separate the effect of direction from that of optimization, and will fix Algorithm 1 to match the implementation.
  • We will add these new results and explanation in our paper.

Comments on the paper

1 comment

Hi reviewers, thank you for your time and feedback! As the rebuttal period is coming to an end, we would appreciate it if you could take a look at our responses and consider updating your scores if your concerns have been addressed. Thank you!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.