- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper looks at the phenomenon of trace errors – i.e., a model producing an incorrect reasoning step despite producing a correct final answer; dubbed “silent restoration” in the paper. While most prior work on this phenomenon tends to focus on domains like math, this work expands the analysis of trace errors to additional domains (math, finance, medicine, code) and identifies interesting inter-domain differences: for example, medicine tends to be the most prone to trace errors, which is particularly pernicious (since people are potentially more likely to use and trust reasoning traces in that domain). The authors then train probes to detect incorrect reasoning steps on answer-correct traces. They replicate several properties observed in correctness probes (i.e., peak accuracy in the middle layers, decodability before the wrong step is produced). Finally, the authors propose Probe-Targeted Activation Steering, a novel method for steering models along probe directions; they find that they can both steer for correct steps and for incorrect steps using this method. This work provides a meaningful and valuable extension of previous work on trace errors (particularly by expanding the domains), but some results and the overall novelty are overstated, and the presentation is heavily “AI-written” (to the work’s detriment; expanded upon below).
Critical strengths
-
The authors investigate an important phenomenon (trace errors paired with correct final answers) and provide a real extension of prior work: they investigate trace errors in a broader range of domains beyond the typical choice of math (including medicine and finance) and uncover interesting inter-domain differences (like medicine being the most prone to trace errors).
-
The authors devise a novel technique for probe-based steering that allows them to steer traces towards correct trace + correct answer and incorrect trace + correct answer, which is useful (either for creating evaluation data for monitors, or for understanding the mechanisms of trace errors).
Critical weaknesses
(Weakness 1 and 2 are the most important, but I felt that I needed to include the third, especially since it is fairly straightforward to fix.)
-
Some of the results are a bit weak. For example, the authors’ findings that wrong step probe accuracy peaks in the middle layers as well as the pre-commitment finding are expected based on prior work (mid-layer probing: [1–4]; pre-commitment: [5–7]), so these are more like replications than novel findings. Furthermore, the translation vs. deformation result seems like it would be the case by construction (linear probes). Finally, their causal steering results need some additional baselines: a random direction baseline (since it’s possible that any perturbation of a given size will knock the model off its current trajectory), and a check for the steering judge (since the judge views the whole suffix, a successful corruption could mean that an error is induced at another step further downstream rather than the targeted step).
-
The novelty is greatly overstated, and relevant prior work looking at the same phenomenon is largely not cited. The phenomenon the authors refer to as “silent restoration” – a correct answer preceded by an incorrect rationale in the CoT – is very well-documented in prior work under various labels (trace errors, process errors, etc.). The paper claims that “existing work is not positioned to catch this failure,” despite existing work addressing it [8–12]. The authors claim to be “the first to study this failure through the model’s internal representations at the level of individual steps,” but there is also prior work that does this [5, 13, 14]. The authors also claim to give “the first per-step definition of load-bearing versus performative steps,” but there is some existing precedent for this as well [15–17]. This weakness can be addressed if the authors add the missing citations and reframe their contributions.
-
The presentation reads as heavily AI-written (e.g., “Our answers form a single progression from phenomenon to representation to intervention.” or “The wrong step is silently overwritten by a right answer.”). This sometimes adds unnecessary length or makes the point unclear. Editing the prose for clarity and succinctness would greatly help readers understand the work. The paper is also missing an AI Use Statement; it would be helpful to hear from the authors the extent to which they used AI in this work.
References
[1] Azaria & Mitchell. The internal state of an LLM knows when it’s lying. 2023.
[2] Marks & Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. 2024.
[3] Li et al. Inference-time intervention: Eliciting truthful answers from a language model. 2023.
[4] Orgad et al. LLMs know more than they show: On the intrinsic representation of LLM hallucinations. 2025.
[5] Zhang et al. Reasoning models know when they’re right: Probing hidden states for self-verification. 2025.
[6] Yuan et al. Hidden error awareness in chain-of-thought reasoning: The signal is diagnostic, not causal. 2026.
[7] Cox et al. Post-hoc reasoning in chain of thought: Decoding and steering pre-committed answers. 2026.
[8] Uesato et al. Solving math word problems with process- and outcome-based feedback. 2022.
[9] Zheng et al. ProcessBench: Identifying process errors in mathematical reasoning. 2025.
[10] Xia et al. Evaluating mathematical reasoning beyond accuracy. 2025.
[11] Wang et al. Examining false positives under inference scaling for mathematical reasoning. 2025.
[12] Yuan et al. Curing “miracle steps” in LLM mathematical reasoning with rubric rewards. 2026.
[13] Zhao et al. Verifying chain-of-thought reasoning via its computational graph. 2026.
[14] Ni et al. Efficient test-time scaling of multi-step reasoning by probing internal states of large language models. 2025.
[15] Bogdan et al. Thought anchors: Which LLM reasoning steps matter? 2025.
[16] Tutek et al. Measuring chain of thought faithfulness by unlearning reasoning steps. 2025.
[17] Lanham et al. Measuring faithfulness in chain-of-thought reasoning. 2023.
Questions for authors
Clarification requests:
- Can the authors situate their work against the prior work I cited in weaknesses (i.e. explain how their work is novel in the ways they claim, or update their claims?)
- Can the authors explain why the translation vs. deformation result is not expected by construction?
- Can the authors provide an AI Use Statement following the ICLR guidelines in their rebuttal? It would be helpful to know the extent to which AI was used in producing this work, especially if it was used for experimental design.
Suggestions for future work:
- Do the authors have an idea of why certain steps produce trace errors? Did they notice any regularities in the particular erroneous steps across models, or did models tend to make errors in unique locations?
2 comments
Questions For Authors:
Q1: Can the authors situate their work against the prior work I cited in weaknesses (i.e. explain how their work is novel in the ways they claim, or update their claims?)
- Response: see response to 'W2'
Q2: Can the authors explain why the translation vs. deformation result is not expected by construction?
- Response: Class separation is the expected outcome of supervised probe training, but this alone does not imply that the separation is similar across layers. Each layer has an independently trained probe and its own standardization, and nothing in the training objective requires the gap between the class-mean held-out scores to stay roughly constant across layers. That approximately constant offset is the empirical observation in Figure 3.
- We agree that its interpretation should be narrower. Because normalization and regularization can influence logit scales, the observation concerns mean held-out score profiles, not evidence of a translation in activation space, and we will revise the "translation versus deformation" wording and avoid presenting it as a general geometric mechanism. Its practical use is as a motivation for the profile-level analysis in Table 3, where level features reach OOD AUROC 0.678–0.769 while spread, shape, and fluctuation features stay at or below 0.56. The probe-plus-distance detector is evaluated separately in Table 4.
Q3: Can the authors provide an AI Use Statement following the ICLR guidelines in their rebuttal? It would be helpful to know the extent to which AI was used in producing this work, especially if it was used for experimental design.
- Response: Please see our response to 'W3', where we summarize the AI Use Disclosure already included in Appendix F. This disclosure explicitly covers assistance with research ideation and feedback on experimental design, as well as manuscript drafting and language editing. All AI-assisted content was reviewed and verified by the authors.
Q4: Do the authors have an idea of why certain steps produce trace errors? Did they notice any regularities in the particular erroneous steps across models, or did models tend to make errors in unique locations?
- Response: We do not yet have a definitive explanation for why particular steps become erroneous. One hypothesis, which we have not tested, is that stating an intermediate step and producing the final answer can rely on partially different computations, so a model may emit an incorrect intermediate claim and still reach the correct answer through other information or later computation. Our pre-step probing results show that predictive information about an upcoming error is available before the step is generated, but they do not identify the mechanism that produces it.
- What we can report are two descriptive observations. First, silent restoration is most frequent in medicine for 11 of 12 configurations (§3.2). Second, the illustrative cases in Appendix H, all from Qwen2.5-7B, show different error types across domains (a false arithmetic result, inverted code logic, a wrong clinical label). We do not claim these generalize across models. Regarding cross-model regularities, our results establish representation-level patterns, such as interior-layer detectability, not shared semantic error locations. We have not aligned reasoning steps across models, so we cannot say whether they fail at the same points or at model-specific ones. Step indices alone would not be a reliable comparison because models use different decompositions and solution paths. A useful follow-up would align intermediate claims or subcomputations on the same problems across models, then measure overlap in error locations and categorize the associated operations. We will state the distinction between observed regularities and these hypotheses explicitly in the discussion.
Response: We agree that our framing overstated novelty. We will remove the "first" claims and the statement that existing work is not positioned to catch this failure, add the missing citations, and reposition our contributions relative to [5, 8–17].
Phenomenon [8–12]. We will acknowledge these works on process errors and on the limits of final-answer correctness as an indicator of reasoning quality. Our contribution is a systematic evaluation under an explicit four-condition criterion across twelve model configurations and four domains, together with an investigation of the internal signals associated with these errors.
Internal analysis [5, 13, 14]. We will explicitly acknowledge these works as precedents for internal error detection and intervention: [5] probes intermediate candidate-answer correctness, including before the answer is generated; [13] detects step errors and corrects reasoning through transcoder-feature interventions; and [14] uses internal-state probes for step verification and test-time scaling. Our analysis focuses on verifiably wrong steps within answer-correct traces, where final-answer correctness alone cannot distinguish correct from incorrect steps. In this setting we study layer-wise localization and pre-step predictability, and develop a probe-plus-distance detector that reaches 0.792 mean OOD AUROC (vs. 0.733 for the probe alone).
Load-bearing vs. performative steps [15–17]. We will acknowledge these precedents on analyzing individual steps' contributions to the answer. Our analysis measures answer sensitivity to step-level corruption induced by pre-step activation steering. We will describe it as sensitivity under the tested intervention, not as a validated faithfulness label, and revise the corresponding statements in §6.2.