- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes SURVEYOR, a training-free framework for long-horizon planning with latent world models. The method introduces two mechanisms into existing hierarchical planning frameworks: an execution–plan consistency check that determines whether previously generated subgoals can be reused, and a reachability-based arbiter that decides whether hierarchical decomposition is necessary. Experiments across four simulated environments and five executors demonstrate the effectiveness of the proposed method.
Critical strengths
- The paper addresses a practically meaningful problem by targeting redundant subgoal generation and unnecessary task decomposition in hierarchical planning.
- The proposed method is simple, requires no additional training, and can be integrated with various existing planners, demonstrating practical applicability.
- The experimental evaluation is relatively comprehensive, covering different types of executors, multiple goal distances, and component ablations. The method achieves substantial improvements in success rates on certain long-horizon tasks.
Critical weaknesses
- The execution-discrepancy-based replanning mechanism is conceptually similar to existing event-triggered replanning methods, such as AdaReP. The reachability-based arbiter also relies primarily on a simple distance threshold in latent space. The authors should further clarify the technical contributions relative to prior work.
- Compared with flat planners, SURVEYOR substantially improves success rates and reduces execution time in certain long-horizon tasks. However, compared with the existing FF-JEPA hierarchical planning framework, the reduction in subgoal-generation calls does not translate into consistent end-to-end computational savings. Since the drafter accounts for only a small fraction of the total computation and the arbiter introduces additional planning overhead, the full method increases per-episode runtime in most settings.
- The arbiter uses a shared CEM planner to estimate goal reachability, although different executors may have substantially different capabilities, leading to performance degradation in some settings. Furthermore, the same threshold is used for execution consistency and goal reachability, which requires stronger justification. The authors should further analyze the accuracy of reachability estimation and the impact of incorrect decisions on task success rates.
Questions for authors
Please refer to the weaknesses.
7 comments
Response to Reviewer 4XqY (2/3): Accuracy of the Reachability Estimate
Q3. The reachability estimate
"The authors should further analyze the accuracy of reachability estimation and the impact of incorrect decisions on task success rates."
A3(1): Thank you for this valuable suggestion. Following it, we conducted a per-episode analysis (new Appendix H). For every episode, we log the decision of the arbiter and additionally run both the flat planner and every-step drafting on the same episode with the same seed. This allows us to measure the accuracy of each routing decision and to compare Surveyor with an oracle that selects, for each episode, whichever of the two planners solves it.
Table: Accuracy of the arbiter's routing (new Table 19). CEM, all values in %, eight repeated runs.
| Setting | Routed to the flat planner | Flat planner on routed episodes | Every-step on routed episodes | Surveyor | Per-episode oracle |
|---|---|---|---|---|---|
| PushT t=25 | 62.5 | 99.7 | 99.1 | 99.4 | 99.9 |
| PushT t=50 | 0.8 | 75.0 | 95.8 | 93.3 | 96.2 |
| Reacher t=25 | 91.2 | 99.5 | 66.0 | 98.2 | 97.7 |
| Reacher t=50 | 77.0 | 99.6 | 90.4 | 97.5 | 96.6 |
| Reacher t=100 | 49.7 | 99.0 | 99.0 | 95.0 | 96.1 |
| Reacher t=150 | 39.4 | 98.3 | 100.0 | 90.8 | 93.6 |
| Cube, mean over t | 48.3 | 72.8 | 62.3 | 45.9 | 49.2 |
| Two-Room, mean over t | 52.4 | 82.9 | 76.7 | 69.6 | 76.4 |
The results show that the estimate is accurate. Pooled over PushT and Reacher, the flat planner solves 99% of the episodes that the arbiter routes to it. On Cube and Two-Room, where all planners are weaker, the routed episodes are still the ones that the flat planner solves more often than every-step drafting (72.8% versus 62.3%, and 82.9% versus 76.7%). Correct routing brings a large gain: at Reacher t=25 the arbiter routes 91.2% of the episodes, of which the flat planner solves 99.5%, whereas every-step drafting would solve only 66.0%.
Regarding the impact of incorrect decisions, Surveyor stays within 2.9 points of the oracle at every PushT and Reacher distance, which bounds how much a perfect choice between the two planners could add. Note that Surveyor can slightly exceed this oracle, because it may also retire drafting in the middle of an episode.
"Furthermore, the same threshold is used for execution consistency and goal reachability, which requires stronger justification."
A3(2): Thank you for the question. The two tests share a tolerance because they ask the same question, namely whether two latents describe states that the task's success criterion treats as the same. The accept rule asks this of the achieved latent and the subgoal just pursued, while the arbiter asks it of the predicted final latent of a flat plan and the goal. The tolerance is the measured answer to this question, so a single calibration rule serves both tests and no second parameter needs to be tuned. We have added this explanation to Section 3.5. To be precise, the rule yields one value per latent space: on Two-Room, the accept rule reads DINOv2 embeddings (0.127) while the probe reads LeWM latents (1.323), and both values are measured by the same rule on the same pairs.
We also verified this choice empirically. Since the probe of every episode is logged, we can vary the routing threshold offline.
Table: Success (%) when routing at a multiple of the calibrated tolerance τ (Table 20). CEM, eight repeated runs. Episodes whose probe lies below the threshold go to the flat planner and the rest to every-step drafting, so the values isolate the routing decision and differ slightly from those of the full system above.
| Setting | 0.5τ | 0.75τ | τ (calibrated) | 1.5τ | 2τ | 3τ |
|---|---|---|---|---|---|---|
| PushT t=25 | 99.3 | 99.2 | 99.4 | 99.0 | 98.6 | 98.7 |
| Reacher t=25 | 64.5 | 79.3 | 94.9 | 96.9 | 96.9 | 96.9 |
| Reacher t=150 | 93.1 | 93.0 | 92.4 | 92.1 | 92.0 | 91.8 |
| Cube t=50 | 43.9 | 43.6 | 43.1 | 40.7 | 40.0 | 38.4 |
| Two-Room t=75 | 57.4 | 58.2 | 57.6 | 53.1 | 49.0 | 43.0 |
As shown in the table, a threshold that is too strict stops routing and loses the short-range recovery (64.5% at 0.5τ on Reacher t=25), while a threshold that is too loose routes episodes that the flat planner cannot finish (43.0% at 3τ on Two-Room t=75). The calibrated value avoids both cases without any tuning. It is within 2.0 points of the best multiple in all seventeen settings of the four environments, which are all listed in Table 20. In addition, moving the tolerance across its bootstrap interval changes no result by more than 1.8 points (Table 12).
Response to Reviewer 4XqY (3/3): The Shared Probe
Q3 (continued). The shared probe
"The arbiter uses a shared CEM planner to estimate goal reachability, although different executors may have substantially different capabilities, leading to performance degradation in some settings."
A3(3): Thank you for raising this concern. We use a shared probe by design, and our submission already evaluates the alternative that you describe, in which each executor serves as its own probe (Appendix J of the revised PDF). We summarize the comparison below.
Table: Shared CEM probe versus each executor as its own probe. Mean / worst success (%) of Surveyor over the nine PushT and Reacher settings, computed from the per-setting values in Appendix A. CEM is omitted because the two probes coincide for it.
| Executor | Shared CEM probe (used in the paper) | Executor as its own probe |
|---|---|---|
| iCEM | 94.0 / 89.7 | 95.6 / 92.8 |
| MPPI | 92.7 / 81.0 | 93.0 / 82.6 |
| Gradient | 96.3 / 93.3 | 96.3 / 94.1 |
| GC-IDM (amortized) | 95.5 / 93.5 | 91.3 / 64.3 |
For the search executors, sharing the probe has little effect: the two probes are within 1.6 points of each other in mean success for iCEM, MPPI and the gradient solver, which suggests that a CEM plan is a good estimate of what these executors can reach. For the amortized executor GC-IDM, which does not search the world model, the shared probe is necessary. With GC-IDM as its own probe, the worst setting drops from 93.5% to 64.3%.
Furthermore, the shared probe keeps the layer plug-and-play. Since it only reads the frozen world model, the same arbiter is applied unchanged to every executor, and it routes the same fraction of episodes for every search executor to within one point (Appendix G).
Since no probe is the best for every executor, we use one uniform setting instead of selecting a probe per executor on the evaluation results. With this setting, Surveyor improves on the vanilla version of all five executors in both the mean and the worst-case success over the seventeen settings. The settings in which the arbiter costs success are reported in the main text (Section 4.4, "What the arbiter costs"), and Appendix J reports the variants that recover them.
In summary, we made the following revisions in response to your review:
- Appendix H and Section 4.4 report the per-episode accuracy of the reachability estimate and the effect of incorrect decisions.
- Section 3.5 and Table 20, which covers all four environments, justify the single calibration rule.
- Section 4.4 now reports the distance-threshold control in the main text.
- Table 24 removes the time gap to FF-JEPA's every-step drafting.
- Section 2 and Table 29 position Surveyor against AdaReP and the other related planners.
We appreciate your detailed comments and hope we have addressed your concerns. Please let us know if you have any additional questions.
Sincerely, Authors
Response to Reviewer 4XqY (1/3): Technical Contribution and Cost
Dear Reviewer 4XqY,
We sincerely appreciate your careful review and constructive comments. We provide point-by-point responses below.
Q1. Technical contribution relative to prior work
A1(1): Thank you for pointing us to AdaReP. Section 2 of our submission already discusses AdaReP and event-triggered control and states how the accept rule differs, and we have expanded this discussion in the revision. While both mechanisms monitor a discrepancy during execution, they act on different objects. AdaReP caches the action plan of a single-level world-model MPC and re-solves it when the deviation between the predicted and observed states exceeds a tolerance that is adapted online. It does not involve subgoals. In contrast, our accept rule checks a drafted subgoal, i.e., whether the achieved latent agrees with the subgoal just pursued, so that the remaining subgoals can be reused. Moreover, its tolerance is not adapted online. It is measured offline from observation pairs that the task's success criterion treats as equivalent, so it requires no tuning.
More importantly, Surveyor makes a second decision that is absent from event-triggered replanning, namely whether the goal should be decomposed at all. This decision accounts for most of the gain of Surveyor over every-step drafting: over the search executors, the worst PushT or Reacher setting rises from 54.2%–66.8% to 81.0%–93.3%. We therefore view AdaReP as complementary to our work, since it could run inside the executor underneath Surveyor. The new Table 29 extends this comparison to FF-JEPA, SAGE and RDM.
A1(2): We would like to clarify that the arbiter does not threshold the distance between the current state and the goal. Instead, it rolls out a flat plan through the frozen world model and compares the predicted outcome of that plan with the goal. This distinction is essential. The appendix of our submission evaluates the simple distance threshold as a control, and we have moved this result into the main text (Section 4.4). We list it below.
Table: A distance threshold versus the arbiter. Success (%) at Reacher t=25.
As shown in the table, a distance threshold almost never routes an episode, while the arbiter's test recovers the short-range success. The contribution of the arbiter thus lies in the test itself, i.e., in using the predicted outcome of a plan together with a tolerance that is measured from the task.
Q2. End-to-end cost
A2: Thank you for this comment. The additional runtime comes from the arbiter's reachability probe, which is what recovers the short-range success lost by FF-JEPA's every-step drafting. To examine how much of this time is necessary, we ran Surveyor with a lighter probe (100 samples and 10 iterations instead of 300 and 30; the budget was fixed before the run). The results are reported below and in Table 24.
Table: Time and success with a lighter reachability probe. CEM on one A100. Seconds per episode are measured at t=150 and exclude model loading. Success is the mean / worst over the nine PushT and Reacher settings.
The results show that, with the lighter probe, Surveyor runs as fast as FF-JEPA's setting while keeping its higher success. The additional runtime is therefore determined by the probe budget. As for the accept rule, its saving is in subgoal-generation calls (0.58 to 0.73 of the calls of every-step drafting at PushT t=150). We have clarified in the introduction that this saving is most relevant when subgoals come from large generative models or from models served remotely, where every call has a computational or monetary price.