acceptodds
Under review as a conference paper at ICLR 2027

Planning with Transformers: Chain of Computation and Structured Context Windows

Abstract

Large Language Models (LLMs) have had a remarkable impact across many areas of machine learning. However, recent studies have shown that they struggle to reliably solve planning problems, particularly when compared with classical planners and planning frameworks such as PDDL. Even with recent advances in reasoning-oriented models, the extent to which LLMs can genuinely perform planning remains an open question. At the same time, theoretical results have shown that transformers, the core architecture underlying modern LLMs, are Turing-complete. In this work, we investigate this apparent gap between the theoretical computational power of LLMs and their empirical planning performance. We propose (), a computational architecture that places a transformer-based LM inside an iterative loop and leverages its strength as a pattern-matching system rather than requiring it to generate an entire plan in a single pass. uses a Structured Context Window (SCW) as external memory, allowing the model to access only the portion of the accumulated context required at each planning step and thereby maintain an approximately constant input size. Within this architecture, the LM learns the planning policy and world model and performs the arithmetic operations required during planning. We show that relatively small LMs trained from scratch can learn planning procedures from a small number of training instances and generalize to unseen problems, achieving success rates above 99% on BlocksWorld and the Pancake puzzle. We further show that can be formulated as a pushdown automaton (PDA), eliminating the need for explicit pointer operations. This formulation can solve Tower of Hanoi (TOH) instances with up to 20 disks and enables the planner to execute the depth-first search (DFS) algorithm in perfect maze environments. Theoretically, we characterize how the accumulation of irrelevant context increases planning failure probability under the standard full-context setup and how the SCW mitigates this effect.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Language Models, Planning, Chain of Computation, Structured Context Window, Pushdown Automata, Depth-First Search, Long-Context Reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ZsDzOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how to improve the performance of transformers on planning. It proposes Chain of Computation (CoC), where a small transformer trained from scratch runs inside an iterative loop. At each step, the model reads a block of an external memory, the so-called Structured Context Window (SCW), selected by a pointer that the model itself outputs. It then writes new instruction blocks, emits a series of actions, and outputs the next pointer. The model is trained on [prompt; target] pairs derived from domain-dependent algorithms. With 500 training instances per problem size, the authors report 100% success on BlocksWorld and Pancake domains. On Tower of Hanoi, performance deteriorates as the amount of held-out data increases. The authors attribute these failures to arithmetic and reach 100% by delegating arithmetic to a symbolic solver. They also reformulate CoC as a pushdown automaton without explicit pointers, which solves Tower of Hanoi up to 20 disks and executes depth-first search in perfect mazes up to 31×31, where Qwen3.5-9B used as the controller completes no test case.

Critical strengths

The idea is interesting, and the overall framework is simple and easy to understand. The appendices also make the picture mostly clear.

The key strength, IMHO, is that the trained model is very small, and used little data.

I also liked the authors' ability to identify the arithmetic issue and the idea to delegate it to a symbolic solver.

The results are good & the connection to theory was an unexpected positive surprise.

Critical weaknesses

My main criticism is the framing of the paper with respect to the "planning and transformers" literature itself. The paper says (lots of times) that transformers struggle to reliably plan, and that there is a gap between theory and practice. This motivation is taken as an assumption to the entire paper, and it is never used as a testable hypothesis.

It turns out that this is outdated, and many papers already demonstrated similar conclusions to this one.

For example, Huang et al (ICAPS 2025) study several methods (fine-tuning with SFT, RL, etc.) and show that in-distribution results are close to ~100% for several planning domains. This is a similar result to yours, but using a vanilla post-training pipeline of a pretrained model. They also show some generalization to out-of-distribution problems when using RL fine-tuning. Unfortunately, the work by Huang et al. is classified as a "survey" (which it clearly is not) and dismissed from any comparison.

Similarly, Rossetti et al. (ICAPS 2024) introduce PlanGPT, a small model with ~83M parameters that is trained only on [initial state+goal; plan] pairs. Again, they show ~100% score in some domains for in-distribution problems. (They have a similar problem to you in out-of-distribution.) Note that their work is trained on plans only (no step-by-step) and it already achieves similar outcomes.

Last, but not least, Corrêa et al. (arXiv:2511.09378, 2025) show that frontier models are competitive to state-of-the-art planners, indicating that the key assumption (and motivation) for this paper is outdated even for pretrained models.

I still think the paper has lots of merits (e.g.., using much less data, using smaller models). But given the larger context of planning with transformers, I would need to see more comparisons (e.g., to simple SFT, to PlanGPT with the used data-scale) to evaluate any practical contribution. It is unclear to me whether the gains come from the entire method or simply from the supervised training.

On a minor side, I also dislike the fact that some key discussions were sent to the appendix. In particular, the abstract says that the methods can "generalize to unseen problems" but this is in-distribution generalization. The out-of-distribution is deferred to appendix E, and the discussion (while very valuable) received only a secondary treatment. In learning for planning, we are traditionally interested in the out-of-dist generalization. For example, the International Planning Competition (the main competition to evaluate planning algorithms) has a Learning track, where the planners have X hours to learn knowledge for a domain and then are evaluated in strictly larger problems.

Questions for authors
  • What is the practical value of your method given that previous work already showed that transformers can reliably plan?

  • How would you solve the out-of-dist problem? Could you do any sort of RL post-train?

  • What happens if you train your models only on the plans (i.e., like PlanGPT) or on the same step-by-step concatenated into a full context? This would show more clearly where the improvement comes from.

5 comments
AuthorsOpenReview ↗

Response to Reviewer ZsDz

Thank you for your constructive feedback and for taking the time to review our work.

Outdated assumptions regarding the planning abilities of transformers: The conclusions of this paper are not outdated. As noted in the literature cited throughout our work, LMs fail to maintain accuracy when the context window expands due to the accumulation of irrelevant tokens. The study referenced by the reviewer to argue that frontier LMs can plan (Corrêa et al., arXiv:2511.09378, 2025) does not show 100% accuracy for any of the tested frontier models on standard planning domains, with the exception of one domain: Spanner. Furthermore, that work includes no experiments evaluating whether planning performance degrades as problem scale increases, which is the primary focus of our study.

Comparison to simple SFT end-to-end on the plan: We analyzed how the context window grows for each problem in Figure 7. Without the SCW architecture, and given the model size used in our experiments, baseline performance was too poor to include as a meaningful comparison.

Out-of-distribution discussion: Our findings differ from those in Huang et al. (ICAPS 2025) because training a model from scratch relies entirely on the training distribution, whereas pretrained models draw from a much broader initial knowledge base. For instance, if a model is trained from scratch on blocks ranging from B1 to B40, it has no token representation for B41 in its vocabulary and will inevitably fail when presented with that new token during evaluation.

Questions:

1- What is the practical value of your method given that previous work already showed that transformers can reliably plan? As noted previously, the literature does not support the premise of this statement. Also note that this work is concurrent to ours, so it merits future study, but we could not have done this in this paper. In this work, we propose an architecture that enables LMs to maintain their planning accuracy as problem complexity increases. Our theoretical analysis and experimental results show that the accumulation of irrelevant tokens in the context window reduces LM planning accuracy. The CoC architecture with SCW addresses this issue by helping LMs maintain consistent planning accuracy across different levels of problem complexity.

2- How would you solve the out-of-dist problem? Could you do any sort of RL post-train? As mentioned above, the out-of-distribution problem is different for a model trained from scratch and a pretrained model. In this work, our goal was to demonstrate that a small Transformer can reliably learn planning algorithms. Addressing out-of-distribution generalization is left for future work, where we plan to investigate approaches using pretrained models.

3- What happens if you train your models only on the plans (i.e., like PlanGPT) or on the same step-by-step concatenated into a full context? This would show more clearly where the improvement comes from. We have tested these settings using the same model sizes. However, the results were very poor compared with our proposed approach, so we did not include them in the paper.

Thank you for the response and the clarifications.

On the planning abilities of LMs: Yes, frontier LLMs fall short of 100% coverage on these benchmarks. But 100% is not the bar that classical planners meet either. In the IPC 2023 Learning track, the best planner solved only 604 of the 900 test tasks. My main point was much more specific: PlanGPT (Rossetti et al. ICAPS 2024) trains a GPT-2 model from scratch on plans only, and solves 99.5% held-out Blocksworld problems with 3 to 20 blocks. Huang et al. (ICAPS 2025) fine-tune a pretrained 7B model on plans and report 98.5% validity on in-distribution Blocksworld problems. I also agree that a pretrained model is a different setting, but PlanGPT, like your models, starts from scratch. So close to 100% in-distribution success does not, by itself, show the need for the SCW. What your results add is larger problems, much less training data and smaller models, but with step-by-step supervision. This comparison should be explicit in the paper (at a conceptual level, at least), but ideally with a PlanGPT-style baseline trained on your Blocksworld data (I obviously don't expect that for the discussion period).

On the SFT baselines: I think these results are worth reporting even if they are poor, since they are the most direct evidence for your central claim that the SCW mitigates the accumulation of irrelevant context. Could you add them, e.g., in the appendix?

On out-of-distribution generalization: I understand that unseen tokens are a real limitation of training from scratch, and that this is future work. My issue is the presentation. The abstract says that the models "generalize to unseen problems", which in the planning context indicates larger problems. This should be made clear in the text since the abstract.

Overall, I appreciate the clarifications. If the future revision includes the results and the clarified scope, I am open to raise my score to weak accept --- as aforementioned, I do not expect you to finish such baselines on time for the discussion, but I would like to agree on something that would make the submission stronger and more complete.

AuthorsOpenReview ↗

Follow-up on Reviewer ZsDz Comments

We appreciate your attention to our response and your willingness to increase your score. We have already started working on some of the experiments requested by you and the other reviewers, and we will provide the remaining results for the camera-ready version.

On the planning abilities of LMs: We agree that classical planners do not necessarily achieve 100% success either. Our goal in this paper is to show that even if large LMs achieve high accuracy in planning domains, their performance degrades as problem complexity increases. Our approach addresses this issue by combining CoC with SCW, which prevents the context window from growing with problem complexity. As a result, if the LM has learned the required instructions, it can scale to problems of arbitrary size. This issue becomes even more significant in TOH, where the plan length grows exponentially with the number of disks. Please also note that our experiments include BlocksWorld and Pancake Puzzle instances with up to (n=40), where our planner still achieves 100% accuracy. Furthermore, we conducted exhaustive evaluations over all configurations of BlocksWorld and Pancake Puzzle for (n=8), and our planner successfully solved every instance. These results provide further evidence that the model has learned the underlying planning algorithms.

Based on your initial review and the comments from other reviewers, we have already started running additional experiments to explicitly demonstrate the benefits of CoC and SCW. The table below summarizes our results so far. We aim to complete all the remaining experiments for the revised version of the paper. However, the current results already demonstrate the importance of SCW in improving the performance of our small LM trained from scratch. In the setting without SCW, we trained the model on full plans using a much larger context window to accommodate the entire plan similar to PlanGPT.

ModelContextBlocksWorldPancakeExtended BlocksWorldTower of Hanoi
Small LM trained from scratchSCW10010099.8992
Small LM trained from scratchFull Plan21.3875.558.3426.67

On out-of-distribution generalization: We plan to conduct additional experiments using the pretrained Qwen model with SCW on BlocksWorld, as well as evaluating our DFS planner on out-of-distribution maze patterns. We need more time to complete these experiments, but we expect to have the results ready in the final submission. We intend to include these results in the main paper.

On the SFT results: We agree that describing the performance as simply "poor" is too vague. We will include the quantitative SFT results in the appendix. These experiments also require additional time, and we will provide the complete results in the final submission.

Reviewer_7APVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Chain of Computation (COC), an architecture in which a Transformer language model iteratively executes instructions stored in a Structured Context Window (SCW). At each step, the model generates new instructions, actions, and a pointer identifying the next instruction, allowing it to operate on a limited portion of the accumulated computation history. The authors evaluate models trained from scratch on BlocksWorld, the Pancake Puzzle, and Tower of Hanoi, reporting high success rates within the evaluated distributions. They further investigate pointer arithmetic as a source of failure and introduce a stack-based formulation that solves Tower of Hanoi instances with up to 20 disks and executes DFS in perfect mazes. A theoretical analysis examines the relationship between irrelevant context and prediction errors.

Critical strengths

(1) A clear architecture for composing local predictions into long computations. COC provides an explicit interface between a learned controller and symbolic external memory. The model performs substantive tasks, including instruction generation, state updates, and, in the original formulation, pointer prediction. This makes the work a useful investigation of how local algorithmic supervision can support extended execution. The stack-based reformulation is also a practical simplification that removes a demonstrated source of errors. (2) Strong results within the evaluated distributions, accompanied by useful diagnostic experiments. The evaluation includes exhaustive testing over all 40,320 initial permutations at n=8, five random seeds for the Tower of Hanoi instruction-holdout experiments, and 450 DFS searches on unseen mazes. Separating exact arithmetic from instruction generation provides additional evidence about the causes of failure. These results support the narrower conclusion that small models can reliably execute the studied procedures under the proposed interfaces. The explicit discussion of out-of-distribution limitations in Appendix E is also appreciated.

Critical weaknesses

(1) The experiments do not adequately isolate the contribution of the proposed memory mechanism. For the small models trained from scratch, I could not find a controlled comparison between SCW and accumulated-history inputs with matched architecture, training instances, and target supervision. Section 5 does evaluate Qwen under both PDA-based prompting and an accumulated-context setting. However, Qwen fails in all settings and reportedly often fails at the first expansion step, so these results provide limited evidence about the effect of accumulating history. The comparison between Qwen and the trained models also changes task-specific supervision and input representations: Appendix L.1 describes explicit spatial and edit/copy-region embeddings for the proposed maze model, whereas Qwen receives character-grid prompts and textual instructions. These differences make it difficult to determine how much of the reported advantage is attributable specifically to SCW rather than to supervision or task-specific representations. A controlled ablation on the trained models would therefore materially strengthen the central architectural claim. (2) The scope and interpretation of the theoretical result need to be tightened. Theorem 1 states convergence of the full-context error rate to 1-1/V, with a Θ(t−1/2)\Theta(t^{-1/2}) gap, without explicitly stating the additional modeling assumptions used in Appendix D. These include a single attention layer, fixed-scaling approximations to normalization, a common ReLU activation region, Gaussian token contributions, and an additional symmetric Gaussian model for vocabulary logits. These assumptions are acknowledged in the appendix, but their necessity should also be reflected in the theorem statement. In particular, the random-guessing limit relies on the additional stochastic model. Furthermore, the analysis compares two lower bounds on failure probability; this does not by itself establish an ordering of the actual failure probabilities. The statement on page 4 that the pointer mechanism reduces planning failure therefore requires additional justification or narrower wording. This limits the theoretical support for the claimed architectural advantage.

Questions for authors

(1) Evidence for the SCW contribution: For the models trained from scratch, are results available comparing SCW with accumulated-history inputs while matching the architecture, training instances, and target supervision? A comparison at problem sizes where both setups are feasible would help isolate the contribution of the context mechanism. For the maze model, what is the effect of removing the additional spatial and edit/copy-region embeddings? Please distinguish any available evidence from experiments that have not yet been conducted. (2) Scope of the theoretical result: Can Theorem 1 be restated with the assumptions used in its derivation, and can the comparison of failure-probability lower bounds be distinguished from a claim about actual failure probabilities? Please also clarify how the next-token error analyzed in Appendix D connects to the transition-level error probability used in the failure recurrence, given that a COC call emits multiple tokens. (3) Action validity and evaluation criteria: Section 3 defines Pancake flips with 1<k≤n, but Appendix I, Step 3 on page 34 and Step 5 on page 35, includes a flip that leaves the state unchanged. With distinct pancakes, this cannot result from a flip of length at least two. Does the implementation permit k=1, omit such operations during execution, or use another convention? Please clarify whether success is determined by independently checking action legality and the final state, matching the teacher trace, or excluding redundant actions, and whether resolving this discrepancy changes the reported success rates.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 7APV

Thank you for your constructive feedback and for taking the time to review our work.

1. Ablation study on performance without SCW: You are correct that we do not provide explicit experiments in the paper regarding model performance without SCW. However, we theoretically show that without SCW, the failure probability grows at each step. Additionally, Figure 7 illustrates how the context window grows as irrelevant context accumulates. Performance without SCW is very poor, which is why we did not include it explicitly in the paper.

2. Assumptions made in the appendix: You are correct. We will move these assumptions into the main paper in the revised version.

Questions:

1. Additional spatial and edit/copy-region embeddings: Without these embeddings, performance on large maze sizes falls below a 100% success rate. We include them so the model can consistently achieve 100% accuracy across all sizes.

2. Next-token error vs. transition-level error probability: You are correct that the theory focuses on next-token error while model failures occur at the transition level. However, because a transition response must match the target token by token, the same theoretical results apply to transition-level errors, as any wrongly generated token causes the response to fail.

3. This is a typo that will be fixed in the revised version. The implementation allows a flip with k=1. Success is determined by matching all actions and intermediate states with those in the target.

AuthorsOpenReview ↗

Follow-up on Our Response to reviewer 7APV

Thank you for your attention to our response. We have completed the first experiment you specifically requested and have also revised the paper based on your second comment and third question. We will post the final version with all the modifications once the ongoing experiment is complete, before the end of the rebuttal period.

Comparison with full-plan generation. We compare our approach using SCW and CoC with a setting in which the same model is trained from scratch to generate the full plan, similar to PlanGPT. For full-plan generation, we use a substantially larger context window to ensure that the entire plan can fit within the context, even for the largest problem instances in each domain. The table below shows the results.

ModelContextBlocksWorldPancakeExtended BlocksWorldTower of Hanoi
Small LM trained from scratchSCW10010099.8992
Small LM trained from scratchFull Plan21.3875.558.3426.67

Without SCW, performance degrades substantially across all four domains. These results support our analysis that accumulating previous tokens in the context can significantly degrade planning performance which is the main focus of our paper.

Additional experiments in progress. We are currently conducting the remaining experiments, including applying SCW to a large pretrained LM and comparing its performance with full-plan generation. We hope to report these results before the end of the rebuttal period. We also plan to include the following experiments in the final version of the paper:

1- Pretrained LM with SCW for out-of-distribution generalization. We evaluate whether using a pretrained LM within the SCW framework can improve performance on out-of-distribution problem instances.

2- Pretrained and SFT models with SCW vs. full-plan generation. We compare pretrained and supervised fine-tuned (SFT) models under both SCW and full-plan settings. While our current results already demonstrate the degradation caused by removing SCW for the model trained from scratch, this experiment will provide a quantitative comparison for pretrained and fine-tuned LMs.

We appreciate your consideration of these additional results. If our response and new experiments address your main concerns, we would be grateful if you would consider updating your score. We also appreciate your understanding that some of the additional experiments require substantial computation and can’t be completed within the limited rebuttal period.

Thank you for adding the full-plan baseline. Comparing the same small model across all four tasks makes the empirical advantage of the CoC and SCW setup clearer. Your explanation that the Pancake implementation allows single-pancake flips and checks every action and intermediate state also answers my question about how success was measured. In light of these results and clarifications, I have raised my rating from 2 to 3 (weak acceptance).

AuthorsOpenReview ↗

Follow-up on Reviewer 7APV Comments

Dear Reviewer,

We thank you for considering raising your score in light of our explanations and the additional experiments we have provided. We fully agree that these experiments were needed to strengthen our paper and support our claims, and we appreciate you pointing out these gaps.

We have also corrected the typo in the paper and made the assumptions underlying our main theorem more explicit in the main paper rather than in the appendix, based on your suggestions. We will include all these modifications in the final version of the paper, along with the results of our ongoing experiments.

Reviewer_HwTGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes to address the current inability of language models to perform planning and multi-step reasoning by creating a specialized architecture in which a model is provided with an external memory that is then iteratively called. The proposed approach is evaluated on three domains. They broadly demonstrate high performance, and for the Tower of Hanoi, they additionally show that separating the arithmetic operation improves the performance of their approach.

Critical strengths

While the idea of using external memories and training models to directly operate on them isn’t novel, to the best of my knowledge, its use and focus on an explicit planning setting is unique. Additionally, the empirical evidence shows a lot of promise, particularly considering the model sizes and the dataset being considered. The theoretical contributions are also quite interesting.

Critical weaknesses

However, in its current form, I found the paper quite confused in its aims and approaches. The evaluation is quite shallow and missing important experiments.

Framing, motivation and language: To start with, I found the framing of the motivation a bit weak. The paper uses Turing completeness results to point at a gap between theory and practical performance. However, Turing completeness is, in general, a relatively easy bar to clear. Just because something is Turing complete, it doesn’t mean that it is amenable to efficient planning. Also, one should be careful to separate expressibility of transformers from learnability. Finally, I would recommend softening the language around Turing completeness of pure transformers. As I understand it, similar to your own method, more realistic constructions rely on external memory components.

Confusion about the contribution of the paper: I believe the authors are probably tying the contribution of their work most closely to the wrong set of previous work. Most of the introduction and empirical evaluation of the paper focuses on referring to works that have looked at large pre-trained foundational models and measuring their planning capabilities. Most of these works are interested in emergent planning capabilities, and of course they also consider how fine-tuning and other post-training methods may improve this capability. This is a very different question from whether you can train a transformer to solve sequential decision-making problems. A more reasonable point of comparison here would be works that focus exclusively on training transformers from scratch for sequential decision-making. Searchformer (which is cited here) would be one example. Another one would be a decision transformer [1], which, as far as I can see, is not referred to in the paper at all. While these two streams of work have overlap, they are fundamentally different methods.

Experiments: Finally, the empirical studies have fundamental limitations. Most importantly, the paper overlooks one of the most basic experiments that is required here, namely, how the transformer would work on the same dataset with and without the external memory.

[1] Chen, Lili, et al. "Decision transformer: Reinforcement learning via sequence modeling." Advances in neural information processing systems 34 (2021): 15084-15097.

Questions for authors

Please see my weakness section. Additionally, the paper presents two versions of the same system, one that uses an explicit pointer-based formulation and one that uses a stack. Is there a reason the authors keep both formulations and not just go with the more effective one?

4 comments
AuthorsOpenReview ↗

Response to Reviewer HwTG

Thank you for your constructive feedback and for taking the time to review our work.

External Memory Usage with LMs: You are correct that leveraging external memory alongside language models is an established concept. However, our approach does not rely on an external memory module for planning. Instead, the context window serves as the sole mechanism available to the model, acting as the self-contained memory space from which all responses are generated.

Turing Completeness and Efficient Planning: We agree that theoretical proofs of Turing completeness in Transformers often assume access to external memory. However, recent work [1] demonstrates Turing completeness for LMs without external memory, which directly motivates our formulation. Our primary goal was to establish that planning algorithms are learnable by Transformers. For scalable, practical planning, integrating more efficient algorithms (such as A*) remains a promising direction for future work.

Small Models vs. Large Pretrained Models: Evaluating large frontier models was cost-prohibitive for our computational budget. Our core objective was to show that the accumulation of irrelevant context causes planning failures, and that a smaller model equipped with a Structured Context Window (SCW) can overcome this issue to plan reliably. While these experiments could be easily extended to larger models, demonstrating success on smaller architectures sufficiently validates the underlying mechanism. We can replace the small model with a larger pretrained model without any training—using only a few-shot prompt setup—and still plan successfully. However, in that case, the computational cost would be significantly higher than using a smaller model trained from scratch.

Transformer Performance Without SCW: Figure 7 provides an empirical analysis of context window growth during iterative planning, and we show theoretically that failure rates compound without SCW. The model’s performance without SCW is severely degraded and this is the reason we did not mention it in the paper.

Two System Variants: The pointer mechanism caters to general planning algorithms that require random access within the SCW, whereas the stack-based variant is a streamlined implementation tailored to Depth-First Search (DFS). We included both variants to highlight generalizability beyond stack-only operations.

[1] Universal computation is intrinsic to language model decoding.

AuthorsOpenReview ↗

Follow-up on Our Response to reviewer HwTG

Thank you for your attention to our response. We have completed the first experiment that you specifically requested.

Comparison with full-plan generation. We compare our approach using SCW and CoC with a setting in which the same model is trained from scratch to generate the full plan, similar to PlanGPT. For full-plan generation, we use a substantially larger context window to ensure that the entire plan can fit within the context, even for the largest problem instances in each domain. The table below shows the results.

ModelContextBlocksWorldPancakeExtended BlocksWorldTower of Hanoi
Small LM trained from scratchSCW10010099.8992
Small LM trained from scratchFull Plan21.3875.558.3426.67

Without SCW, performance degrades substantially across all four domains. These results support our analysis that accumulating previous tokens in the context can significantly degrade planning performance which is the main focus of our paper.

Additional experiments in progress. We are currently conducting the remaining experiments, including applying SCW to a large pretrained LM and comparing its performance with full-plan generation. We hope to report these results before the end of the rebuttal period. We also plan to include the following experiments in the final version of the paper:

1- Pretrained LM with SCW for out-of-distribution generalization. We evaluate whether using a pretrained LM within the SCW framework can improve performance on out-of-distribution problem instances.

2- Pretrained and SFT models with SCW vs. full-plan generation. We compare pretrained and supervised fine-tuned (SFT) models under both SCW and full-plan settings. While our current results already demonstrate the degradation caused by removing SCW for the model trained from scratch, this experiment will provide a quantitative comparison for pretrained and fine-tuned LMs.

We appreciate your consideration of these additional results. If our response and new experiments address your main concerns, we would be grateful if you would consider updating your score. We also appreciate your understanding that some of the additional experiments require substantial computation and can’t be completed within the limited rebuttal period.

Reviewer_HwTGOpenReview ↗

Reply

Thank you for the additional results, I am happy to update my score as well.

AuthorsOpenReview ↗

Follow-up on Reviewer HwTG Comments

Dear Reviewer,

We appreciate your consideration of raising your score in light of the additional experiments we have provided to support our claims in the paper. We are also conducting further experiments that we believe are necessary to strengthen the presentation of our approach, based on your comments and those of the other reviewers. We will include the results of all experiments conducted during the rebuttal, along with those of our ongoing experiments, in the final version of the paper.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Out-of-Distribution Generalization Experiments

Dear Reviewers,

We have completed another experiment demonstrating how the CoC architecture with SCW can improve the out-of-distribution performance of large pretrained LMs. In this experiment, we used Qwen 3.5 4B and performed supervised fine-tuning (SFT) in two settings: one on full plans and the other on SCW instructions. The results for the BlocksWorld environment are presented in the table below. We performed SFT on problems with n=1n=1 to 2020 blocks, using the same amount of training data in both settings. We then evaluated performance on unseen problems with n=21n=21 to 2525 blocks, using 20 configurations for each value of nn.

SFT SettingOOD Success Rate
CoC with SCW100%
Full-plan42%

As shown in the table, CoC with SCW achieves perfect performance (100%) on more complex problems that were not seen during training, demonstrating its ability to generalize to larger problem sizes. This improvement can be explained by how the two settings use the context window.

In the full-plan SFT setting, the context window was designed to accommodate all plans in the training set. However, as the plan length increases during testing, the accumulation of previous tokens in the context window increases the probability of failure, since even a single incorrect token can cause the entire plan to fail. This explains the low success rate (42%) observed in the full-plan setting.

In contrast, with CoC and SCW, the model only needs to execute the current instructions and generate the next instruction, without accumulating the entire plan in its context window. By leveraging its pretrained knowledge, the model can learn these instructions from a limited number of examples during SFT and apply them to unseen tokens. This allows the model to maintain its accuracy even as the problem complexity increases.

We plan to repeat this experiment for the remaining environments and provide the complete results in the final version of the paper after the rebuttal period.

We appreciate the feedback from all reviewers and would be grateful if reviewers 7APV and HwTG could reconsider their assessments in light of these additional experiments, which we hope address their concerns.

AuthorsOpenReview ↗

Appreciation for Feedback from Reviewers

Dear Reviewers,

We sincerely appreciate your feedback and your consideration of raising your scores based on our explanations and experiments. We agree that the experiments suggested by the reviewers were important to further support our claims about the performance improvements achieved by combining CoC with the SCW. The results of these experiments have further strengthened the empirical evaluation of our approach.

We also plan to make the following updates and incorporate them into the final version of the paper:

  1. Clarify the motivation of our approach. Based on the reviewers' comments, we will better clarify the motivation and goal of our approach and update the related work to reflect recent improvements in the planning capabilities of frontier LMs.

  2. Extend the experiments provided during the rebuttal to the maze domain.

These updates will further clarify the motivation of our approach and provide additional validation of the findings already demonstrated in the paper and during the rebuttal.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.