acceptodds
Under review as a conference paper at ICLR 2027

Learning to Decompose and Act: Landmark-Based Manager–Worker Policies for Generalized Planning at Scale

Abstract

General policies learn from small instances to solve an entire instance class of a classical planning domain. Yet, as instances grow, so do the plans required to solve them, as well as the cost of evaluating alternatives at each decision. We introduce Landmarker, a manager–worker hierarchy that decomposes tasks into atomic subproblems: the manager generates one atom from a landmark vocabulary as the next subgoal, and the worker generates the primitive actions that solve it. The two policies are trained independently of each other with the help of polynomial width-based search: the manager via reinforcement learning, for which the search achieves its subgoals, and the worker from the plans that the search finds. At deployment, the learned worker replaces the search. We evaluate Landmarker on the IPC 2023 learning-track benchmark against state-of-the-art learned heuristics, learned policies, and LAMA. In the experiments, Landmarker solves 648 of the 900 test instances, more than every baseline except the strongest lookahead policy (668). Earlier landmark-based decompositions selected subgoals through search or heuristic rules. Landmarker makes landmarks viable as subgoals again by learning how to order and achieve them, solving more instances than LAMA, which uses landmarks only as a heuristic.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Graph Neural Networks, Sequential Decision-Making, Reinforcement Learning, Planning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_RSQ5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article presents Landmarker, a hierarchical Manager-Worker framework for Generalized Planning. The Manager selects subgoals, while the Worker executes corresponding low-level actions. Both parts are trained using Width-based Search and generate subgoals and actions through autoregressive R-GNN. Experiments were conducted in 10 planning domains of IPC 2023, demonstrating good generalization ability across task scales.

Critical strengths

The motivation of the paper is clear, utilizing Landmark for task decomposition, and separately learning sub-goal selection and execution strategies. The method design is reasonable and novel.

The experiment is complete and the results are quite remarkable.

Critical weaknesses

The Landmark extraction process is excluded from the testing time limit, but in some large tasks, this step itself can be quite time-consuming.

In Sokoban, switching to the learned worker results in a significant degradation. This seems to indicate that the manager's goals do not sufficiently take into account the capabilities of the worker.

Questions for authors

See Weaknesses

1 comment

Thank you for the positive assessment and questions. Since the submission we fixed a bug in the manager's training and retrained Landmarker (please see the general response). It now solves many more instances than every baseline.

Extraction time: Every Landmarker result now counts landmark extraction toward the hour. Coverage is not affected by the landmark extraction time: all 751 instances are solved within the hour including extraction. The extraction takes the longest in sokoban instances which our model could not solve regardless.

Sokoban: It is correct that the manager is trained with LIW(1) and never sees the learned worker, and hence the manager cannot learn jointly with the test-time worker. The paper lists this as a limitation. In sokoban, however, the results point to the worker learning as the issue:

  • The sokoban worker already struggles with single subgoals. It achieves 74% of them in validation, unlike the 93-100% for the workers of the other domains.
  • The manager's subgoals can be achieved, i.e., with LIW(1) as worker the same manager solves 39 instances, close to Stone Soup's 43, which is also the highest result on sokoban we have seen to date! With the learned worker it solves only 19.

In sokoban the worker therefore does not learn to execute its subgoals well enough. A manager adapted to this worker could avoid the subgoals it fails, which we have not tested, but it could not make the worker better at them.

Please note that not using LIW(1) as proxy worker during training carries two dangers. First, similar to a complete-search worker (see our reply to Reviewer Lwbq), the manager may come to rely on a worker that achieves more and more complex subgoals, and so overfit its decomposition to subgoals that do not generalize. Second, a worker that initially reaches no subgoal can prevent learning altogether. Both is avoided by the proxy LIW(1).

In the other domains, the step from search to the learned worker is what makes the large instances solvable: over all domains the learned worker solves 218 hard instances and LIW(1) 35. Childsnack and floortile are the clearest cases. Both have dead ends, so the manager has to order its subgoals to avoid them and the worker has to carry them out without taking an action back. Landmarker solves all 90 childsnack instances and 82 in floortile. No baseline solves a single medium or hard floortile instance.

Camera-ready. The revision reports the retrained model in its abstract, main results, ablations and subgoal analysis. Three analyses need new training runs that do not fit into the rebuttal period. The camera-ready version will contain them and every further update needed for the whole paper to reflect the retrained model.

Reviewer_LwbqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Landmarker, a two-level policy for generalized classical planning. A manager policy selects one landmark atom as subgoal and a worker policy generates the actions to achieve it. The policies are R-GNNs that generate their decisions one token at a time (a predicate or action schema

  • its arguments). The two policies are trained independently using width-based search: the manager uses RL, with LIW(1) (a variant of IW restricted to landmarks) executing its subgoals. The worker learns by imitating plans computed by IW(2). At deployment, the learned worker replaces LIW(1). On the learning trakc of the IPC 2023, the best learned pairings (i.e., best manager/worker pair) solve 648 of the 900 test instances, outperforming some of the used baselines.
Critical strengths
  • The idea of learning which landmarks to achieve next is interesting. It is also well motivated by the planning literature. Landmark-based algorithms are well-established in the literature as well.

  • The worker seems to generalize quite well for several of the tested domains.

  • The random-selector ablation is interesting. They show in which domains the learnt order matters and in which it only shortens the plans. The authors are also clear about some of the limitations (e.g., Sokoban and Spanner).

Critical weaknesses

The paper has severe methodological and experimental flaws. While I find the contributions conceptually interesting and worth pursuing, I cannot properly assess the practical relevance because of these experimental flaws. Therefore, I am negative about this paper.

Overall, I think your paper would benefit tremendously if you applied the advice listed in [1]. A few immediate issues worth pointing out:

  • The paper compares methods that were evaluated on vastly different hardware and limits. While I understand that this will happen at some level (e.g., you need a GPU for some methods), I think there must be a larger effort to run all the methods in similar limits. In your main results, we have methods running with 15m and methods running with 1h time limit.

  • The baselines used are not the best baselines for this test set from the competition itself. In fact, LAMA was only 4th in performance for coverage (the one you focused on) in the IPC learning track. If you want to keep a non-learning baseline, then please run Fast Downward Stone Soup or one of the winners of the last classical planning IPC.

One more relevant issue is regarding the properties and novelty of LIW. First, LIW is very similar "in spirit" to famous algorithms like BFWS(f5) or BFWS(R), see [2,3]. The core idea is to use "relevant" atoms to guide width-search algorithms in a more "goal-oriented" fashion. LIW does use it in a different way, of course, but using a set of relevant atoms to make width-search more "goal-oriented" is well known.

The more relevant issue is that LIW and IW are incomplete search algorithms! So saying that "it is polynomial" is misleading when stated without the incompleteness caveat. You do hint at that in the appendix, but this is crucial to the method. More importantly, other existing methods (like BFWS) make the search complete by ranking the nodes instead of pruning. This matters because LIW(1) failures are considered as subgoal failures in training, if I understand correctly (Appendix C4).

The landmark extraction part is also a bit fuzzy to me. From what is reported in section 5, the median time for hard Sokoban tasks to extract the landmarks was about half of total time limit. For some other domains (such as Rovers) the time is still non-negligible. Not taking into account this time into the total time is very arbitrary, and it may impact the results (although in Sokoban the performance is already 0 on hard tasks).

Last, but not least, I strongly recommend the authors to re-read the paper and try to smooth the text. In its current version, it has too many sharp turns and it assumes a deeper knowledge about symbolic planning than the average ICLR reader would have. For example, the introduction drops terms which are known only to planning folks (e.g., it mentions "childsnack" in the third paragraph as if this was a worldwide famous domain), and in the background it assumes that a reader knows what an action schema is, or what the difference is between a ground action and a schema.

[1] Michael Katz, Harsha Kokel, Christian Muise, Shirin Sohrabi, Sarath Sreedharan. "Make Planning Research Rigorous Again!". arXiv:2505.21674. 2025.

[2] Nir Lipovetzky and Hector Geffner. "Best-First Width Search: Exploration and Exploitation in Classical Planning." AAAI 2017.

[3] Guillem Francès, Miquel Ramírez, Nir Lipovetzky, and Hector Geffner. "Purely Declarative Action Descriptions are Overrated: Classical Planning with Simulators." IJCAI 2017.

Questions for authors
  • Could you please compare your method to Fast Downward Stone Soup using 1hr and 24 GiB per task as for your methods?

  • How does the performance change when we incorporate the landmark extraction time into the total time?

4 comments

Thank you for the concrete requests. We followed them, and they led to much stronger results for the revision. In summary, after the bug fix and retraining described in the general response, Landmarker solves many more instances than Fast Downward Stone Soup under the consistent limits you named.

Q1: Stone Soup solves 642 of the 900 instances, a true step-up from LAMA. Landmarker solves 751. We agree that this is the stronger classical baseline to refer to. We originally chose LAMA, since it still is a strong performing classical planner that is on top well known. In more detail, Stone Soup solves only one of the 30 hard childsnack instances, and neither it nor any other baseline solves a medium or hard floortile instance. More time does not change this: given more than 15 hours per instance instead of one, Stone Soup solves none of the 64 floortile instances it misses.

Q2: Every Landmarker result now counts landmark extraction toward the hour. Coverage does not change: all 751 instances are solved within the hour including extraction. Extraction is longest in sokoban, where it exceeds the hour on some hard instances, but these were never solved anyway. For clarity, the landmark extraction would not have led to any further timeouts in the submitted paper version. This extraction was originally done once and cached to avoid wasteful computations on the same data on our cluster during development, but we agree that this should not have carried into the final experiments.

Following the advice of [1], we reran Flat–AA, LAMA, WL-GOOSE and Distincter with the same one hour and 24 GB limits. AIW–AD keeps its published result, which uses one hour. Our Distincter reproduction (511) did not achieve the published result though (589, with 30 minutes and 8 GB), so the table shows both. Even after corresponding with its authors we could not reach this performance, and the checkpoints behind the published numbers are not available.

Incompleteness and BFWS. You raise a good point here. The revision now says that IW(k) and LIW(1) are incomplete as well. A failed LIW(1) search does count as a failed subgoal in training, as you understood. This incompleteness is desirable, and our method builds on it. LIW(1) is limited by its width, not by a time or memory limit that we would have to choose by hand. A width limit means the same in a small and in a large instance. The manager can therefore learn which subgoals the worker achieves and how to chain them, and crucially, this capability carries over to larger instances. In contrast, a complete worker would leave the manager little to decompose. It eventually reaches any subgoal, however far away, so the manager would learn to hand it subgoals that cover most of the task. Ranking the nodes instead of pruning them, as BFWS does to be complete, is thus what we want to avoid here. We agree that LIW(1) is close in spirit to BFWS and now cites it. Also note that LIW(1) is a tool of our method, not a claimed contribution.

The random-selector ablation that you also found interesting is rerun for the retrained model. With the same worker, random goal atoms solve 529 instances against the manager's 751. In childsnack and floortile they solve 10 and 0 instances against 90 and 82. In ferry and satellite they keep the coverage, with plans that are 15% and 31% longer.

Presentation. The revised text follows one running example from the transport domain and defines action schemas, ground actions and landmarks before it uses them. The introduction says what childsnack is where it names it, and a new table describes all ten domains. Further tables list the notation and every component of the method. We will make a larger effort to refine the writing for the camera-ready version, to ensure it is digestible by readers outside the planning community. Your further suggestions on this are most welcome.

Camera-ready. The revision reports the retrained model in its abstract, main results, ablations and subgoal analysis. Note that three analyses need new training runs that do not fit into the rebuttal period. The camera-ready version will contain them and every further update needed for the whole paper to reflect the retrained model.

Thank you for the detailed response and revision! A few questions remain:

  • The reported 751 is the best pairing per domain on the test set, and you chose the beam width from test-time ablations. Is this correct, or am I missing something? What is the coverage when you choose both on the validation or selection instances only?

  • LAMA solves 38 Blocksworld instances with one hour and 24 GiB, but 61 in the official IPC 2023 results with 30 minutes and 8 GB. Could you please double-check this run? The FDSS baseline matters more (and it seems correct), but this is a good sanity check of the new runs.

  • The paper now says that Landmarker runs "without search over states", but Table 9 shows that the closed set is essential in childsnack and floortile. Am I missing some detail here? Could you perhaps rephrase this claim?

Overall, I am happier with the revision than with the initial submission. Once you sort out these points, I will increase my score to weak accept.

Q1. Every checkpoint is selected on the validation and selection instances only, not on the test set. This is supposed to give the five best manager checkpoints and the five best worker checkpoints of each domain. Table 1 then reports the pairing of these selected checkpoints with the highest coverage, as the paper states. It is a best case, because we assess the extent to which these learned policies can generalize.

For the beam-width, the selection instances do not allow to distinguish widths 2 and 8. In nine of the ten domains, the best checkpoints solve the same or nearly the same selection instances at both widths, so selecting at width 2 would pick among equally good checkpoints; the tiebreak being the concrete primitive actions chosen by the worker on the subgoals, just as at width 8. We therefore kept the checkpoints selected at width 8 already and varied the width at test time. Widths 2 and 8 solve nearly the same easy and medium instances (533 and 531). But width 2 runs often more than twice as fast (on childsnack up to 22x faster!), and under the one-hour limit this decides the hard tier: where width 8 runs out of time, width 2 solves. We hope this answers the question you had on the checkpoint selection and its impact. If not, please let us know.

Q2. Thank you for catching this. Our LAMA and Stone Soup runs used a development version of Fast Downward from July 2024. On the medium blocksworld instances, its LAMA finds no plan within an hour where the LAMA of IPC 2023 needs only a few seconds. Release 23.06 of Fast Downward, from the summer of the competition, solves these instances within seconds. We have rerun both planners on all 900 instances with release 23.06.0 and updated both columns. LAMA now solves 61 blocksworld instances (30 easy, 27 medium, 4 hard), equal to the official result, totalling 582/900, 25 instances more than before. To ensure comparability we also reran the same Stone Soup portfolio on this FD build which loses 8 instances in aggregate and totals 634/900 now. Both runs used the same one hour and 24 GB per instance, on the same node as before.

Q3. By "no search" we mean no systematic state-space search, as in the table headings. LAMA, Stone Soup and Distincter maintain a frontier of alternative planning states and explore different continuations. Landmarker follows a single sequence of states. It keeps the states it visited to avoid re-entering them, but never returns to an earlier state to try an alternative continuation. The closed set stops cycles, but it cannot undo a choice or recover from a dead end, so the manager still has to choose subgoals in the right order. The beam search ranks partial subgoal or action decodings, rather than alternative planning trajectories. We have made that qualifier explicit in the paper: "Landmarker executes its policies without systematic state-space search: it follows one sequence of states, only excludes visited states, and never backtracks. Its subgoals must therefore be ordered to avoid dead ends and solve in time."

Thank you for the answers and the new runs. I consider all my questions as resolved.

I will raise my score, as previously promised.

Reviewer_89MGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Landmarker, a two-level policy for generalized planning in which a manager selects the next subgoal from a set of landmarks, facts that every solution must pass through, and a worker generates the primitive actions to achieve it. Experiments across the ten domains of the IPC 2023 learning track show that it outperforms most learned and classical baselines.

Critical strengths

The paper addresses an important problem: learning policies on small training instances that scale to much larger ones. Within the benchmark domains, the method delivers strong results, solving 648 of 900 test instances, including ones with over a thousand objects.

Critical weaknesses
  1. The method's success rests largely on structure that the symbolic planning setting provides for free. The subgoal vocabulary is structurally fixed, the candidate subgoals come from landmarks (facts or sets of facts that every solution must pass through) computed from the domain's action schemas rather than being learned, and verification is a trivial lookup of whether an atom holds. These are typically the hardest parts of hierarchical problem solving, which makes it unclear how much of the scaling comes from learning, and the paper does not discuss what would transfer to open-ended tasks such as coding or writing, where subgoals cannot be enumerated from a model and their achievement can only be partially or noisily verified.
  2. The proposed method involves many components and tuned settings: replay buffers, curriculum blocks, beam search, closed sets, exclusion rules (with a floortile-specific exception), singleton widening, retry budgets, fallbacks, and numerous hand-set constants. A more extensive ablation study is needed to show which of these are essential to the reported results and how sensitive performance is to each.
Questions for authors

See weakness

1 comment

Thank you for the review. Both of your points led to new experiments, and preparing them led us to the bug fix. Please see the general response. Our retrained method now shows significantly stronger results than the baselines.

How much comes from learning? The symbolic model is a key element of classical planning domains. The question therein is how to sequence actions to reach the goal of a given instance. Avoiding confounding errors, e.g. as from perception, allows this setting to be an excellent testbed for structural and scale generalization! Every instance shares the description language with other instances of its domain and merely differs in its initial state, goal, and objects. Note also that this model is available to all evaluated methods. The search methods LAMA and Stone Soup also use landmarks computed from this model, and they solve 557 and 642 instances. Which landmark to follow next, and how to achieve it, is what Landmarker adds by learning.

These choices are also decisive, as our ablation shows. When keeping the worker and the candidates and replacing only the manager by i) a random unsatisfied goal atom, ii) a random landmark, or iii) a fixed random order of the goal atoms, coverage falls from 751 to i) 529, ii) 147 and iii) 498. So the landmarks alone are far from enough: chosen at random they solve only 147. Childsnack, floortile, spanner, and sokoban have dead ends, so the order of the subgoals decides whether an instance can still be solved, as random goal atoms solve very few in these.

With LIW(1), the search the manager was trained with, as worker, only 35 hard instances are solved. With the learned worker, 218. With the published GABAR policy network as worker, only 318 instances are solved at all (second table). So the worker matters as well, and distilling width-based search into a learned policy is crucial to generalize to large instances.

DomainManager + workerRandom goal + workerRandom landmark + workerFixed order + workerManager + LIW(1)Random goal + LIW(1)Random landmark + LIW(1)Fixed order + LIW(1)
blocksworld90581295849042
miconic9090779090901290
satellite9090179059611755
transport90907905758858
ferry90904906362462
childsnack9010411519010
floortile8200060000
rovers7676297456561856
spanner3465629646
sokoban19193183937235
Easy (300)2882099519929321864207
Medium (300)245179341622341751173
Hard (300)218141181373535034
Total (900)75152914749856242865414

Columns pair a subgoal selector with our learned worker or LIW(1).

Which components matter? Each row of the second table changes one test-time setting. Of the budgets and checks, only the closed set of visited states matters, without which coverage falls to 567. Halving or doubling the worker's action budget, its retries or the cap on blocked subgoals, and switching the subgoal exclusions off, costs at most five instances. The manager's two decoding fallbacks never triggered in any decision in the 900 test runs. Beam width is what this ablation made us change: width 2 solves the most, and the wider beams lose almost only hard instances (218, 191, 180 and 159 solved at widths 2, 4, 8 and 16). The floortile exception you noted is gone now. The bug fix also removed the manager's separate distance head, so the method is simpler in structure.

Test-time setting (one change to the reported model)Solved of 900
Reported model751
Greedy decoding590
Beam width 3 / 4 / 8 / 16736 / 724 / 711 / 684
No closed set of visited states567
Worker action budget 50 / 200 (instead of 100)746 / 750
Worker retries 2 / 8 (instead of 4)748 / 750
Cap on blocked subgoals in a row 50 / 200 (instead of 100)748 / 750
Failed-subgoal exclusions off749
GABAR network as worker318

Open-ended tasks. The paper addresses generalized planning, where a symbolic model defines the task. Training the manager against a fixed worker, and generating each decision one argument at a time can be applied outside of this setting and the revision says it now in a scope paragraph. The remainder does need the symbolic model though.

Camera-ready. Singleton widening, replay buffers and the curriculum apply only in training. Ablations of these need training runs beyond the rebuttal period and will be in the camera-ready version.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Joint Response To Reviewers

We thank the reviewers for their careful reading. The requested experiments greatly improved our main result.

While preparing the ablations you asked for, we found a bug in the manager's training: the reward of some terminal states was wrong, which made its critic hard to learn. We fixed it and retrained Landmarker. The test-time ablations then showed that the retrained policies do best with a beam of width 2 instead of the submission's 8, so width 2 is now our reported model. We are grateful for both.

Landmarker now solves 751 of the 900 test instances, up from 648. This substantially improves the state of the art on this benchmark, where the best result so far was 668 (AIW–AD). Landmarker now solves more instances than every baseline, learned or classical. This includes Fast Downward Stone Soup 2023 (FDSS) with 642, which we added at Reviewer Lwbq's request and which has the highest coverage in the official IPC 2023 results for this benchmark. The lead is widest on the hard tier, the instances requiring strong size generalization the most: 218 of 300, versus 182 for AIW–AD and 97 for Stone Soup. The result does not hinge on the exact beam width: every width we tested, from 2 to 16, solves more instances than every baseline.

Each domain has 90 instances. All baselines we ran got one hour and 24 GB per instance, and Landmarker's time limit now includes landmark extraction.

DomainLandmarkerPreviousAIW–ADFDSSDistincter¹Distincter²Flat–AALAMAWL-GOOSE
blocksworld909090628890903877
miconic909090909067909090
satellite909088904864529058
transport909086685023797229
ferry909080668376867388
childsnack906764576472413529
floortile8243026220132
rovers765467734243367439
spanner343090679044903079
sokoban19413433230144239
Easy (300)288273253296–263248282265
Medium (300)245217233249–178186175165
Hard (300)21815818297–70144100100
Total (900)751648668642589511578557530

¹Published result with 30 minutes and 8 GB. ²Our reproduction. AIW–AD is its published result with one hour.

Childsnack and floortile stand out as results. Both domains have dead ends. In childsnack, a gluten-free ingredient spent on the wrong sandwich leaves an allergic child without a sandwich, a dead end. In floortile, a painted tile can never be visited again, so painting in the wrong order also leads to a dead end. Landmarker runs no search over states at test time, since its beam only ranks the partial choices of a single subgoal or action. It therefore has to pursue its subgoals in an order that avoids these dead ends. It solves all 90 childsnack instances and 82 of 90 in floortile, a result that no search baseline reaches here. On the 30 hard childsnack instances, Stone Soup and LAMA solve only one each and the best baseline 13 through clever pruning techniques. No baseline solves a single medium or hard floortile instance, where Landmarker solves all easy and medium, and 22 hard instances. Of the 8 remaining hard instances, 7 merely run out of time. The rerun manager ablation shows that this order is learned, because with the same worker, pursuing the same goal atoms in random order solves only 10 childsnack instances and none in floortile.

Also in the revision, counting landmark extraction toward the hour loses no solved instance. The bugfix also simplified the method: the manager's critic replaces the distance head, and both policies share one architecture. Floortile uses the same settings as every other domain, so its exception is gone. The manager ablation and the ablations of the test-time components are rerun for the retrained model (reply to Reviewer 89MG). The text defines the planning terms it uses and follows one running example. Sokoban and spanner remain the two domains where Landmarker falls short, and the paper says so.

Camera-ready: The revision states the retrained model in its abstract, main results, ablations and subgoal analysis. Three analyses need new training runs that will not finish for the rebuttal period (achiever-depth controls, spanner controls, and replay-buffer and curriculum ablations). The camera-ready version will contain these reruns and every further update needed for the whole paper to reflect the retrained model! Until then, we apologize that some appendix results (e.g., the pairing statistics) may be out of sync.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.