- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper investigates whether models contain internal states that represent the direction of correct reasoning, and whether causal interventions on these states can improve reasoning performance. The authors introduce a ChainSearch task and train a 3M-parameter model from scratch. They then use Linear Probe, CAA, and Direct Optimisation to extract correctness directions from different token positions, model layers, and components, and analyze the linear decodability and stability of these directions. The experimental results show that correctness information does exist inside the model and can be decoded using linear methods. Moreover, some correctness directions exhibit high stability across different reasoning positions.
Critical strengths
-
The research question is novel to some extent and has clear research value. The paper investigates the relationship between a model’s internal states and whether it ultimately produces correct reasoning, which helps improve our understanding of model-internal representations.
-
The paper designs a new controllable reasoning environment, ChainSearch, which makes the search, state-transition, and backtracking processes in arithmetic reasoning explicit and enables accurate annotation of the Reachability of intermediate states. This provides a relatively controllable and computationally efficient environment for studying reasoning correctness inside models.
-
The experimental design is relatively systematic, and the analysis is comprehensive. The paper uses Linear Probe, CAA, and Direct Optimisation to extract Correctness Directions, and analyzes their linear decodability, stability, and causal intervention effects across different Token Positions, Model Layers, and Model Components.
Critical weaknesses
-
The paper lacks an analysis of how the specific reasoning processes change before and after steering. It mainly reports how many problems transition from incorrect to correct and how many transition from correct to incorrect after intervening with the Correctness Direction, but it does not provide concrete examples of changes in the reasoning processes.
-
The paper lacks experimental analysis across problems of different difficulty levels. Simple problems may be solved in two steps, whereas complex problems may require multiple calculations and backtracking. Could the authors stratify the problems by difficulty and conduct a comparative analysis?
-
The paper lacks an analysis of the relationship between linear probe predictions and steering effectiveness. The paper conducts two main experiments: using a linear probe to determine whether the reasoning is correct, and injecting the Correctness Direction into the hidden states. Does better linear probe prediction performance correspond to more effective steering?
-
The paper lacks a comparison of Correctness Directions across different training stages. The current analysis examines whether Correctness Directions exist in fully trained models, but these directions may differ across training stages. Could the authors further compare the Correctness Directions learned during pretraining and SFT?
Questions for authors
please refer to weakness
3 comments
Checkin with reviewer WZDV
Dear Reviewer WZDV,
We appreciated your thoughtful engagement with the work, and use of detailed questions.
As we still have several days left in the discussion period, we wanted to followup, noting that we are very happy to have a bit more detailed discussion if you have any further thoughts, questions, or concerns.
Please do let us know at your convenience.
Best regards,
response
Thank you for your detailed response and the additional experiments. I appreciate the authors’ efforts to address my concerns. However, while I find the work interesting, I do not think its novelty meets the standard for a score of 4. I will therefore maintain my original score.
1st response to reviewer WZDV
Thank you for your response.
Regarding your third question, as we stated early on in the abstract and novel contributions (lines 85–87), strong linear decodability does not always correspond to effective steering. Sections 3 and 4 are specifically designed to examine this relationship (while going beyond linear probing to examine CAA and Direct Optimisation, for completeness).
For your fourth question, the answer is no, checking different training stages is not standard (see Dutta et al., 2026, Chen et al., 2025 and Venhoff et al., 2025). When performing linear steering, it is important to avoid using an undertrained model. Thus we always steer a model after training (this is the norm in the extensive steering literature we cite in our Introduction and Appendix A).
With regard to difficulty, next-state reachability looks 1 step ahead. This was indeed designed to address the counterargument that waiting till the final answer after several steps is difficult. Our experiments thoroughly cover next-state reachability (see Figures 6, 14, 16 and 18).
Regarding an analysis of steering effects, Figure 17 already reports changes in generation length, number of backtracks, and expression parseability. Example 1 in Section 2, Example 2 in Appendix B.3, and Examples 3–10 in Appendix J provide reference traces with different numbers of backtracks. These traces can become long, so we detail the steering effects using our various figures of results.
We also provided all the code so that the reader can reproduce paired examples beyond those we have space for in the paper. For example, for the test sample
IN: [ 3, 5, 6, 6, 8, 8 ] TGT: 293, we obtained the following trace endings (steering at the "before first operand" position for a final-answer correctness vector with steering strength β = 1):Correct answer without steering
Incorrect answer with CAA steering
The arithmetic goes wrong at
3 * 384 = 192, which should be1152. Also, despite printing293andSUCCESS, the steered expression evaluates to1143and uses an unavailable number (9). Both full traces have eight backtracks.Here is an alternative sample matching the cases we mentioned where steering helps. For the test sample
IN: [ 1, 2, 3, 3, 6, 50 ] TGT: 900, we obtained the following trace endings (steering for reachability at the semicolon ending a restored state afterBACK, with steering strength β = 2):Incorrect answer without steering
Correct answer with CAA steering
The unsteered expression evaluates to
466, missing the target. With CAA steering, the model produces a valid expression evaluating to900, and the full trace has six backtracks instead of eight.