- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies candidate selection and computation allocation in world-model planning. The authors argue that selecting the best available candidate does not establish that it is good enough to execute or determine whether further search is worthwhile. StopTheDream combines candidate generation, outcome-supervised comparison, and bounded additional search. In the main HWM implementation, a controller first chooses between one full-budget search and two smaller searches. A scorer then estimates execution success from candidate actions and predicted trajectories. Low candidate scores trigger one additional search. The paper evaluates quality–computation trade-offs across planning, feedback scheduling, and active information acquisition.
Critical strengths
-
The paper highlights that selecting the best available candidate does not establish whether it is worth executing or whether further computation is justified. This distinction addresses a practical challenge in budget-constrained planning.
-
Supervising candidate evaluation with realized outcomes connects planning more directly to control performance. The integration of candidate selection and computation allocation produces favorable quality–computation trade-offs on HWM and CAST.
Critical weaknesses
-
In the HWM implementation, low scores trigger one additional search. However, the paper does not sufficiently explain when this identifies a recoverable search failure rather than a state where further search is unlikely to help. The threshold study shows the overall effect of changing the rule, but leaves this distinction unresolved.
-
HWM uses candidate scoring and additional search, while other applications involve feedback scheduling, observation reuse, or task-type routing. Key implementation choices are spread across task-specific appendices, making it difficult to distinguish the common algorithm from separate application designs. Figure 3 also leaves the additional-search condition implicit and labels supervision targets as “Allocator loss,” adding to the reading difficulty.
Questions for authors
-
What evidence supports using low candidate scores to trigger another search? Are the gains from additional search concentrated in these cases?
-
Disabling additional search still solves 116/120 HWM tasks, compared with 117/120 for the full method. Since this variant retains the initial allocator, how much of the improvement in success and computation comes from initial allocation, candidate scoring, and additional search, respectively?
-
Which components are shared across the evaluated tasks, and which are task-specific? Please make this distinction explicit in the method description.
7 comments
Response to Reviewer ftBs (2/2)
Question 2: Contribution of initial allocation, candidate scoring, and additional search
In our framework, additional search contributes one solved task (116 → 117) for 0.3% more computation, and allocation and scoring together contribute the rest of the success gain (116 vs. 105 for the original HWM planner). The computation saving comes from none of the three: it comes from replanning with half the usual samples after the first four actions, while the three components add almost no computation. Within the first decision, scoring improves the choice between the two plans: on 40 held-out states where both plans were executed, the scorer's choice succeeds in 38, against 35 for the planner's own cost. We measured each part on the same 120 HWM tasks and seeds, and ran two other planners at about the same computation (Tables 1 and 2, Section 5.2, Appendix K.2):
| Run | Tasks solved | Latent transitions |
|---|---|---|
| StopTheDream: allocation, scoring, additional search | 117 | 384.4 million |
| StopTheDream without additional search | 116 | 383.3 million |
| CoVO-MPC with half budget, without allocation and scoring | 106 | 376.8 million |
| Smooth MPPI with half budget, without allocation and scoring | 104 | 371.0 million |
| Original HWM planner with full budget, without allocation and scoring | 105 | 741.7 million |
Most of the success gain comes from the decision at the start of each task. Planners with the same half budget but without allocation and scoring solve more than ten fewer tasks. The additional search then adds one more task at almost no extra computation.
Scoring alone already chooses better than the planner's own cost. The planner's own cost is the predicted distance between the final state of a plan and the goal. On HWM, we executed both plans from 40 held-out states: the scorer's choice succeeds in 38, against 35 for the planner's own cost. On CAST, the scorer's choice among four saved proposals has an 11% lower trajectory error (Appendices K.2 and K.3). Figure 4a shows when the allocator keeps one full search.
Question 3 and Weakness 2: Shared and task-specific components, and Figure 3
Our framework shares one decision rule across all tasks: generate candidates, score them with a model trained on real execution outcomes, and either execute the best candidate or spend additional computation when every score is low. Each task then specifies three interfaces to this rule: what a candidate is, what the scorer reads and predicts, and what form the additional computation takes. These interfaces differ because the benchmarks differ in their actions and quality measures. HWM counts whether the agent reaches the goal, so the scorer predicts success. CAST measures trajectory error, so the scorer predicts the error. The LIBERO policy outputs a chunk of actions at a time, so a candidate is how much of the chunk to execute before calling the policy again. Section 4.4 and Table 4 (Appendix A.2, page 20) list the interfaces:
| Task | Candidate | Scorer input | Extra computation when every score is low |
|---|---|---|---|
| HWM maze navigation | plan from one full or two half-size searches | observation, goal, predicted trajectory, computation used | another world-model search |
| CAST visual navigation | proposed path | camera history, goal, predictions | more proposals |
| LIBERO-Long manipulation | 10 or 16 actions of an action chunk | images and actions | an earlier policy call |
| ROOM navigation from images | navigation action | visual history, goal, action summaries | comparing the actions again with cached image features |
| A-EQA question answering | answer now or after exploring | question and views | more exploration |
| SAT spatial reasoning | answer with the reasoning | images, question, choices, question category | more reasoning |
Table 4 also lists the two supplementary video benchmarks.
The Figure 3 caption now states that one extra search runs when both plans score below 0.5, and explains that "Allocator loss" is Eq. 7 applied to the labels of the allocator. The allocator learns whether one full search succeeds and whether either of the two half-size searches succeeds; the scorer learns whether each plan succeeds (Section 4.6).
Best,
The Authors
Dear Reviewer ftBs,
Thank you for your precise questions, which helped us sharpen the paper considerably. With the discussion deadline approaching on November 18, tomorrow, we wanted to kindly follow up on our responses.
In brief, we showed that plans chosen when both candidates score low fail three times as often in re-runs, attributed the gains to allocation, scoring, and additional search on the same tasks and seeds, and made the shared decision rule and the task-specific interfaces explicit in Section 4.4 and Table 4.
Please do not hesitate to let us know if any point needs further clarification; we will reply as quickly as we can.
Best regards,
The Authors
Response to Reviewer ftBs (1/2)
Dear Reviewer ftBs,
Thank you for your careful reading and precise questions, and for finding that supervising candidate evaluation with realized outcomes connects planning more directly to control performance.
In our framework, a low score for every plan indicates that the current plans are likely to fail, and two results support using it as the trigger. First, plans chosen when both scores are below 0.5 fail three times as often in re-runs (6 of 33 runs vs. 16 of 264). Second, searching again only on these 11 low-score tasks solves 117 tasks, against 116 with no extra search and 115 when searching again after all 99 two-plan tasks. The score indicates that a decision is at risk; whether the failure is recoverable depends on its cause. A failure is recoverable when the first two searches, each with half the samples, missed a good route by chance: the additional full search draws new samples and can find it (Figure 4b, c). A failure is not recoverable when the world model misjudges a whole region; the rule then costs only one extra search per task.
Another search can then help, because each of the first two searches used half the samples and can miss a good route by chance. The additional search is a full search with new samples, twice as many as each smaller search, so the additional search can cover more routes (Figure 4b, c shows one case). When the world model misjudges a whole region, another search cannot help, and the rule then costs only one search per task.
Notably, re-runs of the chosen plans support the design: plans with low scores fail three times as often. We took the 99 HWM test tasks where the allocator chose two half-size searches. From the same saved state, we executed the plan chosen without the extra search three times, with different seeds (Section 5.5, Appendix K.2):
Yes, the gains from additional search are concentrated in the low-score tasks (Table 2; threshold fixed on development data; one latent transition is one predicted step of one rollout):
Searching again after every task solves fewer tasks. When a plan with a high score already exists, another search rarely adds a working route. Each extra plan also gives the scorer one more chance to pick a plan that the scorer overrates. In the low-score tasks, the additional search is useful: the scorer preferred the new plan in 7 of the 11 tasks (Table 2).
CAST shows the same pattern: the less confident the scorer, the more often more proposals contain a better one. We grouped the 1,000 CAST trajectories by the confidence of the scorer after two proposals. The other two proposals contain a better one in 68% of the 200 least confident trajectories, against 57% of the 200 most confident (Appendix K.3).