- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies how to improve the performance of transformers on planning. It proposes Chain of Computation (CoC), where a small transformer trained from scratch runs inside an iterative loop. At each step, the model reads a block of an external memory, the so-called Structured Context Window (SCW), selected by a pointer that the model itself outputs. It then writes new instruction blocks, emits a series of actions, and outputs the next pointer. The model is trained on [prompt; target] pairs derived from domain-dependent algorithms. With 500 training instances per problem size, the authors report 100% success on BlocksWorld and Pancake domains. On Tower of Hanoi, performance deteriorates as the amount of held-out data increases. The authors attribute these failures to arithmetic and reach 100% by delegating arithmetic to a symbolic solver. They also reformulate CoC as a pushdown automaton without explicit pointers, which solves Tower of Hanoi up to 20 disks and executes depth-first search in perfect mazes up to 31×31, where Qwen3.5-9B used as the controller completes no test case.
Critical strengths
The idea is interesting, and the overall framework is simple and easy to understand. The appendices also make the picture mostly clear.
The key strength, IMHO, is that the trained model is very small, and used little data.
I also liked the authors' ability to identify the arithmetic issue and the idea to delegate it to a symbolic solver.
The results are good & the connection to theory was an unexpected positive surprise.
Critical weaknesses
My main criticism is the framing of the paper with respect to the "planning and transformers" literature itself. The paper says (lots of times) that transformers struggle to reliably plan, and that there is a gap between theory and practice. This motivation is taken as an assumption to the entire paper, and it is never used as a testable hypothesis.
It turns out that this is outdated, and many papers already demonstrated similar conclusions to this one.
For example, Huang et al (ICAPS 2025) study several methods (fine-tuning with SFT, RL, etc.) and show that in-distribution results are close to ~100% for several planning domains. This is a similar result to yours, but using a vanilla post-training pipeline of a pretrained model. They also show some generalization to out-of-distribution problems when using RL fine-tuning. Unfortunately, the work by Huang et al. is classified as a "survey" (which it clearly is not) and dismissed from any comparison.
Similarly, Rossetti et al. (ICAPS 2024) introduce PlanGPT, a small model with ~83M parameters that is trained only on [initial state+goal; plan] pairs. Again, they show ~100% score in some domains for in-distribution problems. (They have a similar problem to you in out-of-distribution.) Note that their work is trained on plans only (no step-by-step) and it already achieves similar outcomes.
Last, but not least, Corrêa et al. (arXiv:2511.09378, 2025) show that frontier models are competitive to state-of-the-art planners, indicating that the key assumption (and motivation) for this paper is outdated even for pretrained models.
I still think the paper has lots of merits (e.g.., using much less data, using smaller models). But given the larger context of planning with transformers, I would need to see more comparisons (e.g., to simple SFT, to PlanGPT with the used data-scale) to evaluate any practical contribution. It is unclear to me whether the gains come from the entire method or simply from the supervised training.
On a minor side, I also dislike the fact that some key discussions were sent to the appendix. In particular, the abstract says that the methods can "generalize to unseen problems" but this is in-distribution generalization. The out-of-distribution is deferred to appendix E, and the discussion (while very valuable) received only a secondary treatment. In learning for planning, we are traditionally interested in the out-of-dist generalization. For example, the International Planning Competition (the main competition to evaluate planning algorithms) has a Learning track, where the planners have X hours to learn knowledge for a domain and then are evaluated in strictly larger problems.
Questions for authors
-
What is the practical value of your method given that previous work already showed that transformers can reliably plan?
-
How would you solve the out-of-dist problem? Could you do any sort of RL post-train?
-
What happens if you train your models only on the plans (i.e., like PlanGPT) or on the same step-by-step concatenated into a full context? This would show more clearly where the improvement comes from.
5 comments
Thank you for the response and the clarifications.
On the planning abilities of LMs: Yes, frontier LLMs fall short of 100% coverage on these benchmarks. But 100% is not the bar that classical planners meet either. In the IPC 2023 Learning track, the best planner solved only 604 of the 900 test tasks. My main point was much more specific: PlanGPT (Rossetti et al. ICAPS 2024) trains a GPT-2 model from scratch on plans only, and solves 99.5% held-out Blocksworld problems with 3 to 20 blocks. Huang et al. (ICAPS 2025) fine-tune a pretrained 7B model on plans and report 98.5% validity on in-distribution Blocksworld problems. I also agree that a pretrained model is a different setting, but PlanGPT, like your models, starts from scratch. So close to 100% in-distribution success does not, by itself, show the need for the SCW. What your results add is larger problems, much less training data and smaller models, but with step-by-step supervision. This comparison should be explicit in the paper (at a conceptual level, at least), but ideally with a PlanGPT-style baseline trained on your Blocksworld data (I obviously don't expect that for the discussion period).
On the SFT baselines: I think these results are worth reporting even if they are poor, since they are the most direct evidence for your central claim that the SCW mitigates the accumulation of irrelevant context. Could you add them, e.g., in the appendix?
On out-of-distribution generalization: I understand that unseen tokens are a real limitation of training from scratch, and that this is future work. My issue is the presentation. The abstract says that the models "generalize to unseen problems", which in the planning context indicates larger problems. This should be made clear in the text since the abstract.
Overall, I appreciate the clarifications. If the future revision includes the results and the clarified scope, I am open to raise my score to weak accept --- as aforementioned, I do not expect you to finish such baselines on time for the discussion, but I would like to agree on something that would make the submission stronger and more complete.
Follow-up on Reviewer ZsDz Comments
We appreciate your attention to our response and your willingness to increase your score. We have already started working on some of the experiments requested by you and the other reviewers, and we will provide the remaining results for the camera-ready version.
On the planning abilities of LMs: We agree that classical planners do not necessarily achieve 100% success either. Our goal in this paper is to show that even if large LMs achieve high accuracy in planning domains, their performance degrades as problem complexity increases. Our approach addresses this issue by combining CoC with SCW, which prevents the context window from growing with problem complexity. As a result, if the LM has learned the required instructions, it can scale to problems of arbitrary size. This issue becomes even more significant in TOH, where the plan length grows exponentially with the number of disks. Please also note that our experiments include BlocksWorld and Pancake Puzzle instances with up to (n=40), where our planner still achieves 100% accuracy. Furthermore, we conducted exhaustive evaluations over all configurations of BlocksWorld and Pancake Puzzle for (n=8), and our planner successfully solved every instance. These results provide further evidence that the model has learned the underlying planning algorithms.
Based on your initial review and the comments from other reviewers, we have already started running additional experiments to explicitly demonstrate the benefits of CoC and SCW. The table below summarizes our results so far. We aim to complete all the remaining experiments for the revised version of the paper. However, the current results already demonstrate the importance of SCW in improving the performance of our small LM trained from scratch. In the setting without SCW, we trained the model on full plans using a much larger context window to accommodate the entire plan similar to PlanGPT.
| Model | Context | BlocksWorld | Pancake | Extended BlocksWorld | Tower of Hanoi |
|---|---|---|---|---|---|
| Small LM trained from scratch | SCW | 100 | 100 | 99.89 | 92 |
| Small LM trained from scratch | Full Plan | 21.38 | 75.55 | 8.34 | 26.67 |
On out-of-distribution generalization: We plan to conduct additional experiments using the pretrained Qwen model with SCW on BlocksWorld, as well as evaluating our DFS planner on out-of-distribution maze patterns. We need more time to complete these experiments, but we expect to have the results ready in the final submission. We intend to include these results in the main paper.
On the SFT results: We agree that describing the performance as simply "poor" is too vague. We will include the quantitative SFT results in the appendix. These experiments also require additional time, and we will provide the complete results in the final submission.
Response to Reviewer ZsDz
Thank you for your constructive feedback and for taking the time to review our work.
Outdated assumptions regarding the planning abilities of transformers: The conclusions of this paper are not outdated. As noted in the literature cited throughout our work, LMs fail to maintain accuracy when the context window expands due to the accumulation of irrelevant tokens. The study referenced by the reviewer to argue that frontier LMs can plan (Corrêa et al., arXiv:2511.09378, 2025) does not show 100% accuracy for any of the tested frontier models on standard planning domains, with the exception of one domain: Spanner. Furthermore, that work includes no experiments evaluating whether planning performance degrades as problem scale increases, which is the primary focus of our study.
Comparison to simple SFT end-to-end on the plan: We analyzed how the context window grows for each problem in Figure 7. Without the SCW architecture, and given the model size used in our experiments, baseline performance was too poor to include as a meaningful comparison.
Out-of-distribution discussion: Our findings differ from those in Huang et al. (ICAPS 2025) because training a model from scratch relies entirely on the training distribution, whereas pretrained models draw from a much broader initial knowledge base. For instance, if a model is trained from scratch on blocks ranging from B1 to B40, it has no token representation for B41 in its vocabulary and will inevitably fail when presented with that new token during evaluation.
Questions:
1- What is the practical value of your method given that previous work already showed that transformers can reliably plan? As noted previously, the literature does not support the premise of this statement. Also note that this work is concurrent to ours, so it merits future study, but we could not have done this in this paper. In this work, we propose an architecture that enables LMs to maintain their planning accuracy as problem complexity increases. Our theoretical analysis and experimental results show that the accumulation of irrelevant tokens in the context window reduces LM planning accuracy. The CoC architecture with SCW addresses this issue by helping LMs maintain consistent planning accuracy across different levels of problem complexity.
2- How would you solve the out-of-dist problem? Could you do any sort of RL post-train? As mentioned above, the out-of-distribution problem is different for a model trained from scratch and a pretrained model. In this work, our goal was to demonstrate that a small Transformer can reliably learn planning algorithms. Addressing out-of-distribution generalization is left for future work, where we plan to investigate approaches using pretrained models.
3- What happens if you train your models only on the plans (i.e., like PlanGPT) or on the same step-by-step concatenated into a full context? This would show more clearly where the improvement comes from. We have tested these settings using the same model sizes. However, the results were very poor compared with our proposed approach, so we did not include them in the paper.