acceptodds
Under review as a conference paper at ICLR 2027

How Large Reasoning Models Solve Problems: A Mechanistic Study on Tower of Hanoi

Abstract

Large reasoning models (LRMs) achieve strong performance that is widely attributed to deliberate thinking in long chains of thought (CoT). Whether the content of these traces is causally relevant to the final answer, however, remains actively debated. In this paper, we present a mechanistic study of CoT traces on a task adapted from the classic Tower of Hanoi puzzle to understand how open-source LRMs solve problems. We selected gpt-oss 20b and gpt-oss 120b, both of which achieve around 95% accuracy on the task under high reasoning effort. We identified 6 dominant strategies the models use to find solutions, including analogy to a known problem, exhaustive search, and means-ends analysis (MEA)–a strategy that has been identified in the study of human planning. Focusing on MEA episodes, we identified 4 dominant reasoning primitives, including subgoal proposal, action proposal, verification, and backtracking. We intervened on these primitives, strengthening or disrupting them by editing the reasoning trace, finding that these interventions had a strong impact on the accuracy and optimality of the resulting solutions. Using causal mediation analysis, we found that verification could be induced by a steering vector built from the outputs of the Mixture-of-Experts (MoE) layers with the strongest causal effects on the model's output. Together, these results provide causal evidence that the content of CoT traces shape the answers generated by LRMs, and show that these traces can be decomposed into meaningful primitives with reliable mechanistic signatures.

Primary area:
interpretability and explainable AI
Keywords:
Large Reasoning Models, Problem Solving, Tower of Hanoi, Casual Trace Intervention, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_U71iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how large reasoning models solve planning problems using a modified Tower of Hanoi task. The authors identify six reasoning strategies and further decompose means-ends analysis into four reasoning primitives: subgoal proposal, action proposal, verification, and backtracking. Through controlled interventions on reasoning traces, they demonstrate the causal influence of these primitives on solution accuracy and optimality. Additionally, causal mediation analysis identifies MoE experts associated with verification behavior, which can be manipulated through activation steering.

Critical strengths

S1. Interesting research question. The paper addresses an important question concerning the causal relevance of chain-of-thought reasoning, providing insights into how reasoning models solve structured planning problems.

S2. Systematic and well-designed analysis. The investigation progresses from high-level reasoning strategies to functional primitives and internal model mechanisms. This multi-level approach provides a coherent framework for understanding reasoning behaviors.

S3. Convincing causal interventions. The repair and corruption experiments provide meaningful evidence that specific reasoning primitives influence downstream performance. The additional activation-steering experiments further strengthen the mechanistic investigation.

Critical weaknesses

W1. Limited generalizability. The mechanistic analysis focuses exclusively on two models from the gpt-oss family and a relatively simple Tower of Hanoi task with three elements. It remains unclear whether the identified reasoning primitives and their causal effects generalize to more complex planning tasks or other reasoning model families.

W2. Limited validation of internal mechanisms. Although activation steering successfully induces verification-related behavior, the experiments primarily measure the occurrence of verification rather than its correctness or contribution to downstream performance. Therefore, the evidence connecting the identified MoE experts to functional verification capabilities remains incomplete.

W3. Potential confounding in trace interventions. The repair and corruption experiments involve selecting particular reasoning trajectories and iteratively modifying their content. While paraphrase controls are included, it remains unclear to what extent the observed improvements reflect the targeted reasoning primitives rather than broader changes in subsequent reasoning trajectories.

Questions for authors

Q1. Do the identified reasoning strategies and primitives generalize to other planning tasks or reasoning model families beyond gpt-oss?

Q2. Does activation steering improve the correctness of verification or downstream task performance, rather than merely increasing the frequency of verification-related expressions?

Q3. Can the authors further isolate the causal effects of individual reasoning primitives from the additional reasoning changes introduced by iterative trace interventions?

4 comments
AuthorsOpenReview ↗

Summary of New Experiments

Thanks for the reviewer’s suggestions and comments. We first summarize some major new results before replying to the specific concerns raised in the review:

  • We have now replicated some of our analyses in a more complex task, Mystery Blocksworld, from PlanBench[1]. We found that: 1) the main reasoning strategy used in this task is also means-ends analyses (MEA), and 2) MEA is implemented with the same set of four primitives (action proposal, subgoal proposal, verification, backtracking), and these primitives are employed with broadly similar frequency as in Tower of Hanoi (ToH). [Section 4 in the revision]
  • We have also replicated our analyses on ToH in a new model, Qwen3-235B. The Qwen model also mainly used MEA for solving problems (almost never performing exhaustive search, unlike the gpt-oss models), and implemented MEA using the same set of four primitives. [Section 4 in the revision]
  • We identified a more effective way to steer reasoning primitives using the embedding from MoE layers, and extended these steering experiments to all four primitives (rather than only targeting verification as in the original submission). [Section 3.2 in the revision]
  • We increased the sample size of our trace intervention experiments. Most of the interventions now yield a statistically significant effect in the expected direction (improvement after repair, decline after corruption) relative to the control condition. [Section 3.1 in the revision]

Please refer to the uploaded revision which highlight changes in blue.

[1] Valmeekam, K., et al. (2023). Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. Neurips 2023.

[2] Gerald J Sussman. A computational model of skill acquisition. 1973.

AuthorsOpenReview ↗

Response to W1/Q1

Response to W1/Q1

We thank the reviewer for this comment. As suggested, we have extended our analyses to a large reasoning model from a different model family, Qwen3-235B, and to a more challenging planning task, Mystery Blocksworld (Section 4 in the revision). The new results show that MEA and its four reasoning primitives generalize to both settings: MEA, the main focus of our analyses, remains the predominant strategy, and the models employ the same four primitives in MEA episodes at broadly similar frequencies (except that Qwen3-235B rarely proposes subgoals). In contrast, exhaustive search, which the gpt-oss models sometimes fall back on in 3-disk ToH, is rarely used by Qwen3-235B or in Mystery Blocksworld, suggesting that MEA is the more general strategy across models and tasks.

  • Experiments on More Challenging Planning Tasks.

We have extended our analyses to another more complex planning task, Mystery Blocksworld, from PlanBench [1]. Mystery Blocksworld from PlanBench [1] is an adaptation of the classic Blocksworld planning domain [2]. As in ToH, the goal is to transform an initial configuration of blocks into a goal configuration by moving blocks. However, Blocksworld involves four different actions, each with its own preconditions, and its problems involve 4 or 5 blocks, unlike the 3-disk ToH problems we mainly studied. Furthermore, the Mystery version replaces the names of the actions and of the block relations with unrelated words (e.g., "succumb X" means putting down block X). Please refer to Figures 24 and 25 in the revision for demonstrations of the task.

We evaluated the gpt-oss models on two versions of Mystery Blocksworld: plan generation (Task 1) and cost-optimal planning (Task 2), which also asks the model to reach the goal using the fewest actions. As expected, performance dropped substantially relative to 3-disk ToH, especially for gpt-oss-20b (Figure 14b in the revision).

TaskModelSamplesAccuracyOptimalThinking length (characters)
Plan generation (Task 1)gpt-oss-20b1000.14 (14/100)0.12 (12/100)106.0k
Plan generation (Task 1)gpt-oss-120b1000.78 (78/100)0.49 (49/100)99.7k
Cost-optimal planning (Task 2)gpt-oss-20b1000.20 (20/100)0.20 (20/100)138.7k
Cost-optimal planning (Task 2)gpt-oss-120b1000.67 (67/100)0.59 (59/100)145.6k
*Each task: 50 problems × 2 runs per model.

We parsed the reasoning traces in the same way we did for ToH tasks. The results showed that both gpt-oss models predominantly adopted MEA in Mystery Blocksworld tasks, rarely relying on exhaustive search or analogy (Figure 16 in the revision) since exhaustive search is not tractable in the complex state space of this task.

We further detected all four primitives in the MEA episodes of Task 1. Please see Figure 23 for an actual example of MEA episode. We then measured how often the model carries out the target primitive when it could. In the specific context, the model would decide to implement a specific primitive (P) or not (N). For example, after proposing a specific move, the model would choose to verify the moves against the rules (P for verification) or directly update the states with the move applied (N for verification). Different Primitives would have different types of context. Figure 20 in the revision illustrates, for each primitive, the context, and the P and N moments. Specifically, we computed the fraction of decision points at which the model carries out the primitive (P) among all decision points of the same context type, ρ=P/(P+N)\rho=P/(P+N). We measured this metric in both ToH and Mystery Blocksworld tasks. We found that they are employed at broadly similar frequencies and patterns in both tasks (Figure 6b in the revision), with backtracking the least frequent primitive.

  • Experiments on LRMs from another model family.

We extended our analyses onto Qwen3-235B-A22B-Thinking-2507 which achieved a high accuracy of 94.6% on the same 3-disk ToH task. We parsed its reasoning traces of Qwen3-235B into different reasoning strategies as we did for gpt-oss models. As shown in Figure 15 in the revision, Qwen3-235B, like the gpt-oss models, relies mainly on MEA, but unlike them it almost never resorts to exhaustive search.

We further successfully detected all four primitives in the MEA episodes. Please see Figure 22 in the revision for an example. We then measured how often the model carries out the target primitive when it could using the same metric which we used for gpt-oss models, ρ=P/(P+N)\rho=P/(P+N). Figure 6 in the revision shows that the three LRMs (gpt-oss 20b/120b and Qwen3-235B) have similar tendencies to perform verification, action proposal, and backtracking. However, Qwen3-235B proposes the subgoal of removing the blocker much less when a move is infeasible; instead it tends to propose an alternative move for the blocked number.

AuthorsOpenReview ↗

Response to W2/Q2

We appreciate the reviewer's comment. In our steering experiments, we mainly focused on extracting embeddings at the decision points where the model decides whether to implement certain primitives or not. Thus, the steering vectors were meant to control whether to do the primitive rather than how to do it, and it is hard to expect that such steering vectors would always induce correct verification.

Steering effects on the downstream performance. We agree that it is still meaningful to examine how such steering affects the correctness of the induced verifications and the downstream performance. Therefore, we designed another intervention loop with steering vectors in the revision. To gain stronger steering effects, we injected the steering vectors at the outputs of all MoE layers at once, with the best steering magnitude α\alpha selected on a small set of actual traces. In the intervention loop, we always started by detecting the moments when the model chose not to verify (i.e., an N moment) after proposing a move (i.e., the context), then injected the steering vectors at all layers. We let the model continue generation after injection and detected N moments again, repeating the loop until a path was found. Then we forced the model to output answers. We ran this loop on traces whose original answers were incorrect and that contained at least one unverified action proposal (N moment). As Figure 10 in the revision shows, after injection, the frequency of implementing verification dramatically increased compared to the control condition, in which we injected the steering vectors for backtracking with matched vector norm and magnitude (from ~11% to ~85%). However, the accuracy of the induced verifications did not differ between the two conditions. The downstream answer accuracy was also higher than in the control condition, but the difference did not pass the significance tests, so it is hard to confidently claim that steering with verification vectors would increase model accuracy. We looked at some failure cases after we injected the verification vector: the model did verify later but did not implement the verification correctly, thus causing invalid answers. This is consistent with our expectation above: the verification vector controls whether the model verifies, but not whether it verifies correctly.

ModelαEpisodesArmAccuracyVerification rateVerification accuracyChecks left out
gpt-oss-20B1035Steering16/35 (0.46)100/115 (0.87)86/93 (0.92)7
Control12/35 (0.34)23/198 (0.12)19/21 (0.90)2
p.45< .001***.67
gpt-oss-120B209Steering7/9 (0.78)35/41 (0.85)24/28 (0.86)7
Control3/9 (0.33)6/54 (0.11)6/6 (1.00)0
p.22< .001***1.00

*For measuring verification accuracy, we excluded verifications whose accuracy could not be reliably judged, e.g., when the board state could not be inferred from the text.

*Although the verification accuracy in the control condition is higher than that in the steering condition for gpt-oss-120B, the control induced far fewer verifications.

New steering experiments across all primitives. At the same time, we would argue that studying the internal embeddings responsible for deciding whether to do a primitive is also non-trivial and valuable. In the revision, we adopted a more effective and rigorous way to extract steering vectors and applied it to all four primitives, unlike just verification in the original submission [Section 3.2 in the revision]. The core idea of extracting steering vectors is taking the difference between the average embedding over all the decision points where the model chose to implement the primitive (P) and those where the model chose not to (N), in the same type of context. By extracting steering vectors from each MoE layer's output and injecting them on short synthetic prompts at the same layer, we managed to induce a higher frequency of starting to implement the primitives than both the control conditions, where we injected the steering vectors of another primitive onto the same prompts, and the unsteered baseline, at the most effective layer, across all primitives in both models (Figure 5). The significant steering effects mostly emerge when we inject at the later layers (L18-L22 in gpt-oss 20b, L22-L30 in gpt-oss 120b). Since the short synthetic prompts mainly mention the list configurations and certain moves and lack context about the ToH task (i.e., rules, task goal, etc.), such results demonstrate the generalization of these steering vectors to the synthetic prompts.

We also acknowledge that studying the internal mechanism behind the actual implementation of primitives, i.e., how the model performs a primitive correctly, would be an interesting open question worth further exploration, which we also mention in Section 7 of the revision.

AuthorsOpenReview ↗

Response to W3/Q3

To test whether the improvements come from the targeted primitive rather than from broader changes in the later reasoning, we checked whether the specific mistake we edited at each step still appears in the model's final answer.

Across all four primitives, the interventions largely remove the edited mistake from the final plan (0–11% carried over), while it persists under the paraphrase control (24–80%). These results measure the impacts of the specific steps we edited rather than overall accuracy, which would indicate that the interventions act through the targeted primitives, not through broad changes in the subsequent reasoning. We have added these results to the manuscript as a new figure (Figure 12).

The below are detailed results for your reference. We would also welcome the reviewer's view on other potential confounds, and would be glad to design additional controls to rule them out.


We first removed every trace in which either the repair intervention or the control run switched to exhaustive search after many edit rounds. This affected 10% of the edited runs (31 of 302 across the four repair experiments).

Verification. We identified checks in which the model wrongly accepted an illegal move. We edited each such check so that the move was rejected, and then asked whether the same move, from the same state, appears in the final answers.

ModelRepairControlp
gpt-oss-20B1/36 (3%)11/26 (42%)< .001
gpt-oss-120B1/28 (4%)8/33 (24%).031

*The repair and control could have different numbers of edited steps for the same original trace.

After the repair, the illegal move rarely reaches the final answer (3–4%). Under the control, it does in 24–42% of cases.

Action proposal. For the repair of non-optimal action proposals, we asked whether the model's original non-optimal move, from the same state, appears in a final answer.

ModelRepairControlp
gpt-oss-20B0/40 (0%)14/31 (45%)< .001
gpt-oss-120B0/49 (0%)22/66 (33%)< .001

None of the replaced moves appear in the final answers after the intervention, whereas 33–45% remain under the control.

Subgoal proposal and backtracking. The same pattern holds for the other two repairs:

  • Subgoal proposal: does the model still remove the worse of the two obstacles first in the final answer?
  • Backtracking: after the model leaves the optimal path, do the moves that we forced the model to abandon still appear in a final answer?

Subgoal proposal

ModelRepairControlp
gpt-oss-20B0/3 (0%)2/3 (67%).40
gpt-oss-120B0/7 (0%)8/10 (80%).002

*We counted only cases in which the blocked move has two obstacles, one on its source list and one on its target list. Such cases are rare.

Backtracking

ModelRepairControlp
gpt-oss-20B11/102 (11%)25/67 (37%)< .001
gpt-oss-120B1/23 (4%)10/27 (37%).006

p: Fisher's exact test.*

Reviewer_qBtVOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the reasoning of LRMs, specifically looking at how the model goes about multi-step tasks such as tower of Hanoi problems. The authors used another model (Gemini 3.1 Pro) to classify the reasoning steps that are taken by the model which they called: episodes. They focused on MEA (means-ends analysis) episodes during the reasoning. Their study revealed that LRMs overly rely on these episodes and they further studied how influencing or changing parts of these episodes influence the behaviour of the model.

Critical strengths
  • The analysis of the reasoning of LRMs is an interesting question that requires to be studied
  • The bridge with cognitive science is also quite interesting
Critical weaknesses
  • Although mentioned by the authors, only one task (tower of hanoi) is considered in this study, which limits the broadness of what the analysis demonstrates. We don't know if these strategies that were set for this analysis can be reused for another task
  • LRMs have been studied before in a similar setting in the paper 'The illusion of thinking[1]' where the authors also studied Tower of Hanoi problems although with a slightly different goal. This paper should be cited in the related works.
  • It would be interesting to know the percentage of CoT that is irrelevant to the final answer but still present. Depending on the task, an LRM could go on reasoning on something that is completely unrelated to what is produced later on as final answer. This is probably not prevalent in this constrained setting but could start appearing for a more generic task

[1] Shojaee, P., Mirzadeh, I., Alizadeh, K., Horton, M., Bengio, S., & Farajtabar, M. (2025). The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity. arXiv e-prints (arXiv:2506.06941).

Questions for authors
  • It would have made sense to study the reasoning of LRMs on multiple style of problems, did you try this analysis on another problem?
  • How do we know that the strategies identified for this case for tower of Hanoi problems would also be the same for other tasks done with LRMs?
  • Are exhaustive search and path simulation not included in means-end analysis? What distinguishes them from each other? I have a hard time understanding how they represent separate strategies
  • How could the results obtained in this paper help in better using LRMs or training ones?
3 comments
AuthorsOpenReview ↗

Response to Reviewer qBtV [Part 1]

Thanks for the reviewer’s suggestions and comments. We first summarize some major new results before replying to the specific concerns raised in the review:

  • We have now replicated some of our analyses in a more complex task, Mystery Blocksworld, from PlanBench[1] (reference in next comment). We found that: 1) the main reasoning strategy used in this task is also means-ends analyses (MEA), and 2) MEA is implemented with the same set of four primitives (action proposal, subgoal proposal, verification, backtracking), and these primitives are employed with broadly similar frequency as in Tower of Hanoi (ToH). [Section 4 in the revision]
  • We have also replicated our analyses on ToH in a new model, Qwen3-235B. The Qwen model also mainly used MEA for solving problems (almost never performing exhaustive search, unlike the gpt-oss models), and implemented MEA using the same set of four primitives. [Section 4 in the revision]
  • We identified a more effective way to steer reasoning primitives using the embedding from MoE layers, and extended these steering experiments to all four primitives (rather than only targeting verification as in the original submission). [Section 3.2 in the revision]
  • We increased the sample size of our trace intervention experiments. Most of the interventions now yield a statistically significant effect in the expected direction (improvement after repair, decline after corruption) relative to the control condition. [Section 3.1 in the revision]

Please refer to the uploaded revision which highlight changes in blue.

Response to W1/Q1/Q2

We have extended our analyses to another complex planning task, Mystery Blocksworld, from PlanBench [1]. By parsing traces into strategies and detecting reasoning primitives, we confirm that MEA which we were studying remains the predominant strategy for solving Mystery Blocksworld task, and four reasoning primitives occur in MEA episodes with broadly similar frequencies as in ToH. The convergence results show the generalization of MEA strategy across different tasks.

Mystery Blocksworld from PlanBench [1] is an adaptation of the classic Blocksworld planning domain [2]. As in ToH, the goal is to transform an initial configuration of blocks into a goal configuration by moving blocks. However, Blocksworld involves four different actions, each with its own preconditions, and its problems involve 4 or 5 blocks, unlike the 3-disk ToH problems we mainly studied. Furthermore, the Mystery version replaces the names of the actions and of the block relations with unrelated words (e.g., "succumb X" means putting down block X). Please refer to Figures 24 and 25 in the revision for demonstrations of the task.

We evaluated the gpt-oss models on two versions of Mystery Blocksworld: plan generation (Task 1) and cost-optimal planning (Task 2), which also asks the model to reach the goal using the fewest actions. As expected, performance dropped substantially relative to 3-disk ToH, especially for gpt-oss-20b (Figure 14b in the revision).

TaskModelAnswersAccuracyOptimalThinking length (characters)
Plan generation (Task 1)gpt-oss-20b1000.14 (14/100)0.12 (12/100)106.0k
Plan generation (Task 1)gpt-oss-120b1000.78 (78/100)0.49 (49/100)99.7k
Cost-optimal planning (Task 2)gpt-oss-20b1000.20 (20/100)0.20 (20/100)138.7k
Cost-optimal planning (Task 2)gpt-oss-120b1000.67 (67/100)0.59 (59/100)145.6k

We parsed the reasoning traces in the same way we did for ToH tasks. The results showed that both gpt-oss models predominantly adopted MEA in Mystery Blocksworld tasks, rarely relying on exhaustive search or analogy (Figure 16 in the revision) since exhaustive search is not tractable in the complex state space of this task.

We further detected all four primitives in the MEA episodes of Task 1. Please see Figure 23 for an actual example of MEA episode. We then measured how often the model carries out the target primitive when it could. In the specific context, the model would decide to implement a specific primitive (P) or not (N). For example, after proposing a specific move, the model would choose to verify the moves against the rules (P for verification) or directly update the states with the move applied (N for verification). Different Primitives would have different types of context. Figure 20 in the revision illustrates, for each primitive, the context and the P/N moments. Specifically, we computed the fraction of decision points at which the model carries out the primitive (P) among all decision points of the same context type, ρ=P/(P+N)\rho=P/(P+N). We measured this metric in both ToH and Mystery Blocksworld tasks. We found that they are employed at broadly similar frequencies and patterns in both tasks (Figure 6b in the revision), with backtracking the least frequent primitive.

AuthorsOpenReview ↗

Response to Reviewer qBtV [Part 2]

Response to W2

Thanks for pointing out the missing reference. We have added reference to the “Illusion of Thinking” paper in Section 5.1 of the revision.

Here are added sentences:

"In our paper, we chose two planning tasks, ToH and Blocksworld, which were also adopted by Shojaee et al. (2026) to understand the limitations of LRMs. Here, however, we use these tasks as testbeds to study the causal role of CoT in model performance."

Response to W3

We found almost none of the reasoning traces as irrelevant to the task. In our strategy parser, we left ‘Others’ category for the content which does not belong to 6 reasoning strategies used for solving the problems. Across both ToH and Mystery Blocksworld, only less than 0.3% of trace characters are labeled as the ‘Others’' category; nearly all of the remaining reasoning is spent understanding the problem and attempting it repeatedly with different strategies, mainly MEA (and, in ToH, also exhaustive search and analogy identification). Following the previous paper about measuring the faithfulness of CoT traces [3], we also truncated the original reasoning traces at different positions and forced the model to answer immediately. As the cutoff moves toward the end of the trace, accuracy keeps increasing, with no plateau at the later positions, in both ToH and Mystery Blocksworld (Figure 13 in the revision). These results suggest that, in the tasks we studied, the entire reasoning trace causally contributes to model performance.

Response to Q4

The goal of our paper is to understand how LRMs plan in their CoT traces. We analyzed this at three levels of granularity: we first parsed whole traces into reasoning strategies, then detected the prominent reasoning primitives within one human-like strategy, MEA, and finally examined the internal signal of each primitive by steering generation with the outputs of MoE layers. We further confirmed that MEA and the four primitives are prominent across different models and tasks. Together, our results provide a systematic and mechanistic account of the reasoning behaviors in CoT traces and demonstrate that these behaviors play a causal role in model performance. We see three ways in which these results can inform how LRMs are used and trained.

  • First, they suggest that identifying and monitoring these behaviors could help improve the reasoning abilities of LRMs. Prior work [4] showed that the presence of cognitive behaviors such as verification and backtracking in a base model is an important prerequisite for effective RLVR training. Detectors for the four primitives we identified could be used to measure such behaviors before and during training, or to reward them directly, and how to control and incentivize these primitives is a promising direction for future work.
  • Second, our steering experiments show that the primitives have internal signatures that can be used to induce them at inference time, e.g., steering with the verification vectors makes the model verify its proposed moves far more often. This offers a lightweight way to control and encourage specific reasoning behaviors at inference time.
  • Third, our results bear on the debate over the faithfulness of CoT traces, which matters for monitoring LLMs' reasoning for safety. A central question is whether CoT traces reflect the model's actual computation. We showed that the content of CoT traces, specifically the reasoning primitives in MEA episodes, is causally relevant to the final answer: repairing or corrupting a single primitive changes the model's accuracy and optimality.

[1] Valmeekam, K., et al. (2023). Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. Neurips 2023.

[2] Gerald J Sussman. A computational model of skill acquisition. 1973.

[3] Lanham, T., Chen, A., Radhakrishnan, A., Steiner, B., Denison, C., Hernandez, D., ... & Perez, E. (2023). Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702.

[4] Gandhi, K., Chakravarthy, A., Singh, A., Lile, N., & Goodman, N. D. (2025). Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01307.

AuthorsOpenReview ↗

Response to Reviewer qBtV [Part 3]

Response to Q3

We would like to clarify that exhaustive search and path simulation are not part of means-ends analysis (MEA).

In exhaustive search, the model applies a formal search algorithm such as BFS, exhaustively visiting all the states at each layer of the state graph to find a shortest path to the goal. In MEA, by contrast, the model searches for a solution informally, and the exploration is mostly goal-driven. Typically, the model compares the current state with the goal state and proposes a subgoal or an action that reduces the difference. If the action cannot be taken yet, it proposes a subgoal of removing the obstacles, i.e., satisfying the action's preconditions. It then grounds this subgoal in a specific action by choosing a destination list for the obstacle. The model always alternates among four primitives.

Path simulation is applied after a path has been found by MEA, exhaustive search or analogy identification. It is not used to find a solution, only to verify the correctness of a path that has already been found. Because it only verifies existing solutions, we did not include it among the three strategies used for finding a path.

Reviewer_SbdyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies LLMs' chain-of-thought on a text version of three-disk Tower of Hanoi. It compares 17 reasoning-model configurations on all 26 initial states other than the goal state. It then analyzes gpt-oss-20b and gpt-oss-120b with their reasoning effort set to high.

The paper identifies six strategies in the written reasoning. It describes four components of means-ends analysis (MEA), a strategy that reduces the difference between the current state and the goal. These components propose intermediate goals, propose moves, check moves against the rules, and return to earlier states after unsuccessful attempts. The paper relates these components to the General Problem Solver (GPS), an existing program that implements means-ends analysis.

The paper edits the text expressing these components and measure whether later solutions are correct and use the fewest possible moves. It also changes internal values from model submodules called experts to test their effect on text that checks moves.

Critical strengths
  1. The paper demonstrates that reasoning content can change later answers by controlled editing. The paraphrase controls make changed meaning a more credible explanation than mere rewriting or variation between generated continuations. They also address some effects of editing itself. Requiring legal destinations helps distinguish choosing a better plan from simply avoiding an immediate rule violation. These comparisons support a substantive causal claim about supplied reasoning content within the tested procedure.

  2. The human checks quantify which automated labels readers can trust, rather than requiring confidence in the judge alone. This strengthens the strategy descriptions and the identification of events targeted by edits.

Critical weaknesses
  1. Selected runs and forced early answers limit conclusions about normal reasoning. The authors retain runs with a complete segment in which means-ends analysis finds a path, then they force a final answer after the first such segment. Selection and forced early answers change the setting whose behavior the experiments measure. The accuracy drop before editing shows that removing later reasoning already changes outcomes substantially. Consequently, the measured editing effects may differ from normal reasoning, where the model might recover.

  2. Unclear run counts limit assessment of the reported effects. Figure 3 specifies 16 problems with eight runs each, and Section 3.1 states that it uses the same 128 runs. Appendix D and Figure 10 instead describe all 80 traces as those considered in Figure 3. The paper needs to clarify whether these are different collections or whether one is a subset of the other. Otherwise, readers cannot determine which automated labels and editing results concern the same evaluated runs. In the supplied manuscript, Figure 4 and its associated Section 3.1 do not state the sample count for each condition.

Questions for authors
  1. Do correcting reasoning and inserting mistakes have the same effects when the model can finish its normal written reasoning?
  2. Are the 80 traces described in Figure 10 a subset of the 128 runs specified for Figure 3? Or do these refer to different data collections?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Sbdy [Part 1]

Thanks for the reviewer’s suggestions and comments. We first summarize some major new results before replying to the specific concerns raised in the review:

  • We have now replicated some of our analyses in a more complex task, Mystery Blocksworld, from PlanBench[1]. We found that: 1) the main reasoning strategy used in this task is also means-ends analyses (MEA), and 2) MEA is implemented with the same set of four primitives (action proposal, subgoal proposal, verification, backtracking), and these primitives are employed with broadly similar frequency as in Tower of Hanoi (ToH). [Section 4 in the revision]
  • We have also replicated our analyses on ToH in a new model, Qwen3-235B. The Qwen model also mainly used MEA for solving problems (almost never performing exhaustive search, unlike the gpt-oss models), and implemented MEA using the same set of four primitives. [Section 4 in the revision]
  • We identified a more effective way to steer reasoning primitives using the embedding from MoE layers, and extended these steering experiments to all four primitives (rather than only targeting verification as in the original submission). [Section 3.2 in the revision]
  • We increased the sample size of our trace intervention experiments. Most of the interventions now yield a statistically significant effect in the expected direction relative to the control condition. [Section 3.1 in the revision]

Please refer to the uploaded revision which highlight changes in blue.

Responses to W1/Q1.

The reviewer notes that our intervention experiments diverge from the naturalistic reasoning setting by artificially forcing a premature response, preventing the model from engaging in multiple attempts to solve the problem using different strategies. This is true (and we have now confirmed that when the model is allowed to continue reasoning after our interventions it can recover by employing other reasoning strategies Figure 11 in the revision), but importantly this does not undermine the purpose of these experiments nor their conclusions. The purpose of these experiments was to assess the causal role of the reasoning trace, and specifically the causal role of the MEA episodes (see next paragraph for our rationale for focusing on these episodes in particular). It is possible for there to be two sufficient causes for a particular outcome, such that one can compensate in the absence of the other (as in the case of a backup generator that can provide power in the case of a power outage), a phenomenon known as ‘preemption’ in the causal inference literature [5]. In these cases, to assess the causal role of one factor independently of the other, it is necessary to construct a counterfactual scenario in which the other factor is removed [5]. This is precisely the approach that we employed, and our results allow us to establish the causal role of the MEA episodes. It is true that the models achieve higher accuracy when allowed to employ multiple strategies, but this does not contradict the causal contribution of the MEA episodes. Interestingly, we have found that model accuracy progressively increases as the reasoning trace is artificially truncated at various stages (Figure 13 in the revision), consistent with our claim that the entirety of the reasoning trace causally contributes to the final answer.

Given that the models can sometimes compensate for errors in the MEA episodes by performing exhaustive search, one might reasonably ask: why focus on MEA episodes, to the exclusion of exhaustive search? We have chosen to focus on MEA episodes for the following reasons:

  1. exhaustive search (e.g. BFS) is a fully understood algorithm, so it is not very interesting from an interpretability perspective to understand how the models implement a rote algorithm,

  2. exhaustive search is also not a tractable approach for more complex problems, as illustrated by the complete lack of exhaustive search in the models’ reasoning traces for our next experiments on the more complex Mystery Blocksworld task [Section 4 in the paper],

  3. MEA appears to be a more common mode of reasoning for LRMs, as illustrated by the lack of exhaustive search in the reasoning traces for the second model that we tested (Qwen3-235B) [Section 4 in the paper], presumably due again to the intractability of applying exhaustive search in more complex domains, and 4) experiments in cognitive science suggest that MEA is also a prominent strategy employed in human problem-solving [2,3,4].

We thank the reviewer for raising this point and for allowing us to refine our claims about the causal role of the different reasoning strategies employed by the model. We believe that these new results (Figures 11 and 13 in the revision) provide a more comprehensive picture of the respective contribution of different reasoning strategies, while our analyses on a new model and more complex new task suggest that MEA is a more commonly employed reasoning strategy for LRMs.

AuthorsOpenReview ↗

Response to Reviewer Sbdy [Part 2]

Responses to W2/Q2.

The 80 traces in Figure 10 (now Figure 9 in the revision) and appendix D are a subset of 128 traces in Figure 3. We selected 80 samples from 128 traces, 5 samples per problem. We have added the clarification in appendix D. We have updated Figure 4 with new results from a larger sample, and we have provided the sample count for each condition in Table 2 in the revision. We also listed the sample counts here.

PrimitiveInterventionModelNAccuracy: interventionAccuracy: controlp (acc)Optimality: interventionOptimality: controlp (opt)Samples switched to search
VerificationRepair20B2421/24 (88%)11/24 (46%).006**13/24 (54%)5/24 (21%).008**4
VerificationRepair120B2622/26 (85%)13/26 (50%).012*13/26 (50%)9/26 (35%).3882
VerificationCorrupt20B351/35 (3%)24/35 (69%)<.001***1/35 (3%)17/35 (49%)<.001***3
VerificationCorrupt120B427/42 (17%)30/42 (71%)<.001***7/42 (17%)22/42 (52%)<.001***7
Subgoal proposalRepair20B86/8 (75%)4/8 (50%).6253/8 (38%)0/8 (0%).2502
Subgoal proposalRepair120B106/10 (60%)5/10 (50%)1.0005/10 (50%)1/10 (10%).1253
Subgoal proposalCorrupt20B147/14 (50%)13/14 (93%).031*3/14 (21%)11/14 (79%).008**1
Subgoal proposalCorrupt120B145/14 (36%)9/14 (64%).2892/14 (14%)8/14 (57%).031*1
Action proposalRepair20B1311/13 (85%)11/13 (85%)1.00010/13 (77%)3/13 (23%).016*0
Action proposalRepair120B3024/30 (80%)25/30 (83%)1.00019/30 (63%)9/30 (30%).021*4
Action proposalCorrupt20B2410/24 (42%)18/24 (75%).039*7/24 (29%)18/24 (75%).003**6
Action proposalCorrupt120B3122/31 (71%)29/31 (94%).06513/31 (42%)27/31 (87%).001**3
BacktrackingRepair20B2014/20 (70%)14/20 (70%)1.00014/20 (70%)6/20 (30%).021*3
BacktrackingRepair120B2014/20 (70%)14/20 (70%)1.00013/20 (65%)6/20 (30%).039*8
BacktrackingCorrupt20B216/21 (29%)12/21 (57%).0705/21 (24%)11/21 (52%).1097
BacktrackingCorrupt120B2111/21 (52%)14/21 (67%).4536/21 (29%)13/21 (62%).016*6

Intervention = the repair or corrupt edit; control = the paraphrase control of the same trace. After the model found a path, we forced it to output answers and sampled 3 times. We use the majority of 3 forced answers for evaluation (accuracy = valid solution, optimality = shortest solution). p: exact McNemar test (two-sided); * p<.05, ** p<.01, *** p<.001. The last column counts samples in which the models, in either condition, switched to exhaustive search. Although it switched to search, we still keep the same loop of error detection -> repair -> sampling,

[1] Valmeekam, K., et al. (2023). Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. Neurips 2023.

[2] Herbert A Simon. The functional equivalence of problem solving skills. Cognitive psychology, 7(2):268–288, 1975.

[3] Anzai, Y., & Simon, H. A. (1979). The theory of learning by doing. Psychological review, 86(2), 124.

[4] Tate, A. (1975, September). Interacting Goals And Their Use. In IJCAI (Vol. 10, pp. 215-218).

[5] Halpern, J. Y., & Pearl, J. (2005). Causes and explanations: A structural-model approach. Part I: Causes. British Journal for the Philosophy of Science, 56(4), 843–887.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

We thank all reviewers for their constructive suggestions and comments. Below, we summarize the major new results and the core questions we addressed during the rebuttal period.

  • [Suggested by Reviewers qBtV and U71i] A more complex planning task. As suggested, we replicated our analyses on a more complex task, Mystery Blocksworld from PlanBench [1]. We found that (1) the main reasoning strategy in this task is also means-ends analysis (MEA), and (2) MEA is implemented with the same four primitives (action proposal, subgoal proposal, verification and backtracking), which are employed at broadly similar frequencies as in Tower of Hanoi (ToH). [Section 4 in the revision]
  • [Suggested by Reviewer U71i] Another model family. As suggested, we replicated our ToH analyses on a large reasoning model from another model family, Qwen3-235B. Qwen3-235B also mainly used MEA to solve the problems; in fact, unlike the gpt-oss models, it almost never performed exhaustive search. It also implemented MEA with the same four primitives, although it proposed subgoals less often. [Section 4 in the revision]
  • [In response to Reviewer U71i] Steering all four primitives. We developed a more effective way to steer reasoning primitives using the outputs of MoE layers, and extended the steering experiments to all four primitives, rather than only verification as in the original submission. [Section 3.2 in the revision]
  • [In response to Reviewer Sbdy] Larger trace-intervention experiments. We increased the sample size of our trace intervention experiments. Most of the interventions now show a statistically significant effect in the expected direction (improvement after repair, decline after corruption) relative to the control condition. [Section 3.1 in the revision]
  • [Suggested by Reviewer U71i] Effects of the targeted edits. As suggested, we measured whether the specific mistake we edited still appears in the model's final answer. Across all four primitives, the repairs largely remove the edited mistake from the final plan, whereas it persists under the paraphrase control. This indicates that the improvements come from repairing the targeted primitives rather than from broad changes in the later reasoning. [Appendix E in the revision]
  • [Clarification for Reviewer Sbdy] Recovery in later reasoning. We clarified that the model's ability to recover in later reasoning, through further attempts with other strategies, does not undermine the purpose or the conclusions of our causal interventions on early MEA episodes. Two causes can each be sufficient for an outcome, so that one can compensate for the absence of the other, a phenomenon known as preemption in causal inference. We therefore isolated the early MEA episodes to assess their causal role independently of later compensating strategies such as exhaustive search. [Appendix E in the revision]
  • [In response to Reviewers Sbdy and qBtV] The causal role of the whole trace. By truncating the original CoT traces at different positions and forcing the models to answer, we showed that accuracy keeps increasing as the truncation point approaches the end of the trace, in both ToH and Mystery Blocksworld. This suggests that, in the tasks we studied, the whole reasoning trace causally contributes to model performance. [Appendix E in the revision]
  • [Suggested by Reviewer U71i] Steering effects on downstream performance. As suggested, we tested the steering effects on downstream performance with an intervention loop that injects the verification vector whenever the model proposes a move without verifying it. Steering sharply increased the frequency of verification (from ~11% to ~85%), with no difference in the accuracy of the verifications. Downstream accuracy was higher than in the control condition, but the difference was not statistically significant and failure cases also show show the causes of verification errors rather than failure to do verification. Downstream accuracy was higher than in the control condition, but not significantly. In the failure cases we inspected, the model did verify the proposed moves but reached incorrect verdicts. This is expected, since our steering vectors are extracted at the decision points where the model chooses whether to perform a primitive, not how to perform it correctly. [Appendix F in the revision]

The changes in the uploaded revision are highlighted in blue.

We hope these new results address the reviewers' concerns. If our responses have addressed your concerns, we would be grateful if you would consider updating your evaluation. Thank you again for your time and thoughtful feedback.

[1] Valmeekam, K., et al. (2023). Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. Neurips 2023.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.