acceptodds
Under review as a conference paper at ICLR 2027

Planning Before and After Feedback: Allocating Five Complete Planner Calls with Frozen Visual World Models

Abstract

Planning with a frozen visual world model poses a temporal allocation problem: given complete planner calls, how many calls should be spent on independent search before acting, and how many should be reserved to replan from fresh observations? We compare precommitted schedules while fixing the model, scorer, call count, and logical function evaluations within each panel. On Wall, a disjoint pilot selects ; relative to the all-initial-search schedule , the registered endpoint changes by (two-way 95% sensitivity interval ), with a descriptive success change of percentage points. PointMaze provides a boundary: both registered intervals for selected versus include zero. A matched Wall sham replaces fresh post-execution observations with cached imagined latents while holding later call slots and horizon work fixed; real minus sham is (two-way 95% sensitivity interval ), supporting fresh-observation conditioning relative to that control. The Wall direction recurs in targeted iCEM-component, DecentCEM-style, and one-checkpoint JEPA-WM adaptations. PushT repeats the fixed versus direction in a prospective train-split policy confirmation and a retrospective validation-split replication, although equal-call restart controls do not establish a task-general feedback mechanism. Planner-call timing is therefore a measurable, task-dependent controller choice under the tested protocols, not a universal replanning rule.

Primary area:
applications to robotics, autonomy, planning
Keywords:
visual world models, model predictive control, cross-entropy method, feedback replanning, planning budget allocation, robotic manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7kVaOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how a fixed budget of five complete CEM planner calls on a frozen visual world model should be split. The user selects a priori a number of planner calls of that budget, denoted r, that are allocated to planning from the initial observation, and the rest go to replanning from fresh observations after each execution. Call count and number of evaluated plans per call are matched across methods. Reserving calls helps a lot on Wall (r = 3 vs r = 5: ΔJ = −34.7, +31.6 pp success), has no significant effect on PointMaze, and gives a smaller gain on PushT (r = 4 vs r = 5). A matched control that replaces fresh observations with imagined latents attributes part of the Wall effect to feedback, and iCEM, DecentCEM-style and JEPA-WM variants on Wall go in the same direction.

Critical strengths
  • Methods are compared fairly under a clean protocol. There is no data leak since the model and scorer are frozen, r is chosen on a separate pilot set, and inference is done per scene.
  • The cached-latent ablation is a well-designed control for isolating the value of fresh observations.
  • Negative results (adaptive selectors, restart controls, repair attempts) are reported openly, and the claims are appropriately restrained.
Critical weaknesses
  • The motivation for this setup is unclear. It seems like the actual execution loop and the way to the budget is allocated can be done in many different ways and this one is not justified.
  • The main result comes down to open-loop execution failing on Wall. Every schedule that replans (r = 1–4) reaches J ≈ 9–12, and only r = 5 collapses (Table 4). It is well known that closed-loop control is far more robust.
  • The paper gives no actionable guidance. r shows no effect on PointMaze, and PushT is non-monotone (r = 3 is worse than r = 5 in some cases), so "task-dependent" says little about how to choose r.
  • The scope is narrow: only K = H = 5, three simple tasks and one main model, and the experiment code is not released.
  • The writing contains a lot of hedging, which hides a simple main finding among the words.
Questions for authors
  1. Why did you choose K=5? It seems arbitrary and unjustified. Do you have any results for other budgets or horizons?
  2. Why does r = 5 collapse on Wall specifically, and how do your r = 1 and r = 5 numbers compare with the published DINO-WM open-loop and MPC results?
  3. Can you please clarify what macro-actions are being performed in each environment and how they are implemented?
  4. What about taking multiple steps between intermediate planner calls? There is already extensive research on that, and this work seems to be a subset of this.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 7kVa (1/2)

Thank you for the review, and for noting the clean protocol, the cached-latent sham, and the openly reported negative results. We kept the protocol and the restraint of the claims, and rewrote the text to state the main finding directly (W5). Notation: a schedule (b, F) runs b searches from the initial observation, then one feedback call after each macro-action index in F; the r schedules are (r, {r, …, H−1}), and one terminal correction is (1, {H−1}). Differences are in task cost J (lower is better) with 95% CIs.

W2: "The main result comes down to open-loop execution failing on Wall"

We agree that closed loop beating open loop is not, by itself, a finding, and the revised introduction says so. The revision reports where the value of feedback sits and what cannot replace it (Wall, DINO-WM).

  • One late call carries it. With two calls, feedback before the last macro-action recovers 0.92, 0.98, and 0.91 of the per-step MPC gain at H = 5, 10, 15; after the first macro-action, the same call recovers 0.63, 0.20, and 0.04 (Fig. 1b).
  • It repairs failures and rarely causes them. Of 288 cells, 103 fail with a single search and succeed with one terminal correction, and 2 do the reverse; at H = 10, the counts are 60 and 0 of 128.
  • Search does not replace it. Best-of-5 open loop gains −11.83 over a single search, one terminal correction −38.02 [−45.35, −30.68]. The gap survives matched work and time (−39.27 to −41.34 against a single search with 5× samples), ten searches (J = 40.75 versus 18.12, one seed), and 3× CEM iterations (Sec. 6.4).
  • New observations carry 74% of the five-call gain in a matched sham split (Sec. 6.5).

No intermediate r is significantly better than per-step MPC (App. Table 7), and the revision no longer claims one.

W3: "The paper gives no actionable guidance"

Sec. 6.7 (p. 9) now gives three rules, each tied to evidence:

  1. Default at K = 5: r = 3. Over eight panels it has the smallest maximum regret in J: 2.66, against 3.19 for r = 4, 5.68 for r = 1, and 36.83 for best-of-5 open loop (App. Table 18).
  2. Tight compute: one terminal correction. Two calls recover 0.91 to 0.98 of the per-step MPC gain on Wall, with 13% to 40% of its work.
  3. Extra calls: measure τ on a pilot panel. The search-transfer ratio τ is the fraction of the predicted-cost improvement from extra search that appears in the realized cost. Where τ is well below 1 (Wall with DINO-WM, 0.18 and 0.36; PushT confirmation, 0.06), best-of-5 gains only 0.17 to 0.31 of what one terminal correction gains, so extra calls go to feedback. Where τ is near or above 1 (PointMaze, 0.79; JEPA-WM, 1.04; PushT validation, 1.36), search gains about as much as feedback or more (gain ratios 1.05 to 1.63). There, extra calls go to initial searches, and the terminal correction stays. τ is a post hoc diagnostic over six panels (Spearman 0.94).

The PushT non-monotonicity came from the 24-reset and 39-instance development panels (App. Tables 25 and 31). On the 128-unit confirmation bank, per-step MPC beats r = 5 by −7.21 [−10.34, −3.77]. On PointMaze, a single search fails in only 13 of 64 new scenes, consistent with low open-loop drift (0.041 against 0.098 on Wall).

Q2: "Why does r = 5 collapse on Wall specifically, and how do your r = 1 and r = 5 numbers compare with the published DINO-WM open-loop and MPC results?"

Why. The agent must pass through a door in the central wall, and open-loop latent drift rises from 0.024 after the first macro-action to about 0.10 from the third on. Best-of-5 changes the predicted cost of the selected plan by −0.092 but its realized cost by only −0.033 [−0.067, 0.003]: picking among more candidates is consistent with exploiting model error rather than correcting it. Failures concentrate at the end. On the original panel, r = 5 fails in 116 cells; r = 4, which replaces the fifth search with a terminal correction, repairs 95 of them and breaks 1 (App. Table 15).

Published numbers. We ran the released DINO-WM planning code and Wall configuration on 50 pairs from its official generator and seed. Official MPC reaches 0.58 cumulative success after its first round, which executes one open-loop CEM plan, and 0.94 after five rounds. Each round executes a full five-step plan, and the episode stops on success. Our evaluator on the same pairs executes exactly five macro-actions and scores the end: single open-loop search 0.56 (−0.02 [−0.11, 0.07] against official round 1), best-of-5 open loop (r = 5) 0.60, per-step MPC (r = 1) 0.81, and one terminal correction 0.83. The DINO-WM paper reports 0.74 for open-loop CEM and 0.96 for MPC. More CEM updates do not close this gap: with three times the updates, single-search success on the sham panel is 0.47 against 0.49 (one seed; App. Table 12). We report the gap as found (App. A.1, Tables 3 and 4).

(continued in 2/2)

AuthorsOpenReview ↗

Response to Reviewer 7kVa (2/2)

Q1 and W4: "Why did you choose K=5?" and scope

K = 5 was not tuned. It equals the official DINO-WM planning horizon (H = 5 macro-actions), and K = H is the smallest budget at which per-step MPC is possible. New Wall panels at H = 10 and 15, with K from 1 to H, give the two-call result above at all three horizons (details in our reply to smeF; Sec. 3, p. 4). The evidence covers three tasks and two model families (Sec. 7). The updated supplementary zip contains the anonymized end-to-end code (general response).

W1 and Q4: motivation, and "taking multiple steps between intermediate planner calls"

Each complete CEM call is expensive: 7.5 s on Wall and 65.5 s on PushT on one H100 with the official settings. A deployed controller therefore has a call budget, and when K < H it must choose when to spend the calls. The (b, F) notation separates the two choices such a schedule makes: how many searches before acting, and after which macro-actions to replan.

Multi-step execution between planner calls is a mature topic. The revision cites action chunking (ACT, Diffusion Policy), real-time chunking (RTC), Bidirectional Decoding, online replanning triggers such as AdaReP, regret-guided update scheduling (Musavi et al., 2026), and replanning budgets for LLM-based embodied agents (BRACE) (Sec. 2, p. 3). These works set chunk lengths, decide online when to replan, or schedule a limited number of updates. We fix the number of complete calls of a frozen world-model planner, compare search with feedback at matched calls and work, and separate new observations from re-optimization with the sham. Replanning every m steps is now an explicit baseline: at equal call counts, late placement and uniform spacing do not differ significantly in mean J (Holm p ≥ 0.19 in all five comparisons at H = 10 and 15). Late placement uses 33% to 44% less work at H = 15 (App. Table 9).

W5: hedging

Agreed. The abstract, introduction, and conclusion now state the finding and its strongest numbers first. Protocol terms (registered endpoint, two-way sensitivity interval) moved to the appendix, and disclaimers are collected in Sec. 7. The scope of the claims is unchanged.

Q3: macro-actions

All three released DINO-WM checkpoints use frameskip 5. For each macro-action, the planner proposes a 10-dimensional vector that the official code splits into five 2-D primitive actions, de-normalizes, and executes for five environment steps without reading observations (App. A.1, Table 2).

TaskPrimitive actionOne macro-action
Wall2-D displacement; the agent moves by twice the action, and the central wall can only be crossed through the door5 displacements (H = 5: 25 environment steps)
PointMaze2-D force on the two ball actuators, clipped to [−1, 1], held for 5 MuJoCo substeps5 actions, 25 substeps
PushT2-D target displacement relative to the agent, tracked by a PD controller for 0.1 s5 actions, 0.5 s

We hope these answers help in reassessing the paper.

Reviewer_3NWgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how to spend a fixed budget of five CEM planner calls when planning with a frozen visual world model. A schedule spends r calls up front from the initial observation and the remaining 5 − r on replanning from fresh observations. So r = 5 is best-of-five open loop, and r = 1 replans after every macro-action.

Critical strengths
  • The protocol seems solid. The full curves are reported. A resource ledger separates call counts from compute and time. Negative results are included which make the effects it reports are credible.
Critical weaknesses

Critical 1: On Wall, every schedule with any replanning beats open-loop r = 5 (J ≈ 9–12 vs. 45). No contrast among r ≤ 4 is tested, however. On PushT, replanning after every step is worse than r = 5 - the evidence shows only that open-loop execution fails, not where the calls should go.

Critical 2: Smaller r also uses less compute and time; Wall r = 3 uses 21,007 less horizon-weighted work and 8.86 s less time than r = 5. So r = 5 is dominated rather than traded off - no real allocation trade-off is characterized, and matched-compute comparisons are missing.

Secondary: the effect may be model-specific. With one JEPA-WM checkpoint, the Wall effect shrinks from −34.7 to −4.7 → this is consistent with DINO-WM's open-loop error rather than a general principle - it

Secondary: the evidence is narrow and weakly independent.

K = H = 5 on three simple tasks. The robustness checks all use one shared panel. PushT is confirmed on training-split episodes. No end-to-end code is released. → It is hard to know what transfers.

Secondary: the sham's principle is already published. Conditioning later planning on real rather than imagined observations, and replanning when observations diverge from expectations, is the published design of world-model look-ahead agents such as WebATLAS (Cheng et al., 2025), which the paper does not cite → the sham confirms a known principle in a new setting; the paper should cite and discuss ATLAS.

Questions for authors

[Clarification] Interior r vs. r = 1. Report selected-r vs. r = 1 contrasts, with intervals, on Wall and PushT → if no interior r beats r = 1, the paper reduces to closed-loop vs. open-loop.

[Clarification] Matched compute. Compare schedules at matched compute or wall-clock time, for example by giving r = 1 the work budget of r = 5 → an interior benefit that survives would substantially strengthen the paper.

[Clarification] The sham. Is the sham's real arm the r = 3 arm in Table 4? If so, the sham itself beats r = 5 by about 10.8 → roughly a third of the effect is not explained by fresh observations.

[Suggestion] . Beyond WebATLAS, some other papers which would be valuable for the authors to study:

  • Bidirectional Decoding (Liu et al., 2024), which analyzes the trade-off between committing to a chunk and reacting to new states;
  • Lambert et al. (2022) on compounding prediction error, which finds that system dynamics dominate error, a ready explanation for the task dependence;
  • MBPO (Janner et al., 2019), which uses short model rollouts branched from real data.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 3NWg (1/2)

Thank you for a precise review, and for noting the protocol, the full curves, the resource ledger, and the negative results. You stated what would make the paper informative (Q1, Q2), and we ran those tests; the answers below include the unfavorable ones. Notation: a schedule (b, F) runs b searches from the initial observation, then one feedback call after each macro-action index in F; the r schedules are (r, {r, …, H−1}), and one terminal correction is (1, {H−1}). Differences are in task cost J (lower is better) with 95% CIs.

C1 and Q1: "if no interior r beats r = 1, the paper reduces to closed-loop vs. open-loop"

No interior r beats r = 1. On the original Wall panel (96 scenes × 3 seeds), r = 2, 3, 4 minus r = 1 are −2.56 [−5.69, 0.26], −1.33 [−4.69, 1.86], and −2.43 [−5.74, 0.59] (Holm p ≥ 0.18). On PushT, r = 3 minus r = 1 is −0.29 [−3.48, 2.59] on the confirmation bank (64 episodes × 2 seeds) and −0.38 [−5.44, 4.76] on 60 new validation starts. App. Table 7 lists all 30 such contrasts on ten panels; none is significant. We removed every interior-advantage claim (Sec. 6.1, Fig. 2).

The revision therefore asks a narrower question than closed versus open loop: which call carries the value of feedback, and whether search can replace it.

  • Position. With two calls, feedback before the last macro-action recovers 0.92, 0.98, and 0.91 of the per-step MPC gain at H = 5, 10, 15 (new Wall panels for H = 10 and 15); after the first macro-action, the same call recovers 0.63, 0.20, and 0.04 (Fig. 1b). The late call is not significantly different from per-step MPC at H = 10 and 15, with 20% and 13% of its work. At H = 5 it trails by 3.47 [−0.37, 7.77] and fails its pre-specified non-inferiority test (margin 3).
  • Search. Extra search does not replace the late call, even at matched work (C2).
  • Sham. New observations carry 74% of the gain (Q3).

On PushT, per-step MPC beats r = 5 on the confirmation bank by −7.21 [−10.34, −3.77] (52 wins, 12 losses). The opposite impression came from the 24-reset development panel (19.59 versus 17.90; App. Table 25).

C2 and Q2: "Compare schedules at matched compute or wall-clock time"

You are right that under the submission's ledger r = 5 is dominated, not traded off. The revision gives every arm the work of best-of-5 open loop (W ≈ 75,000 sample-step evaluations) by changing the samples per CEM update: 1,500 for a single search, 1,250 for one terminal correction, 500 for r = 1, and 417 for r = 3. On the sham panel (96 scenes × 3 seeds), one terminal correction, r = 1, and r = 3 beat the single large search by −41.18 [−49.26, −33.33], −39.27 [−47.84, −30.64], and −41.34 [−49.55, −33.31] (Holm p = 0.0004 each); the two new seeds alone give −42.27 to −44.10. These arms take 36.9 to 41.0 s per episode and best-of-5 open loop 37.6 s, so the ranking holds at matched time (Fig. 3). Your example, r = 1 with the work of r = 5, beats best-of-5 open loop by −29.68 [−37.32, −22.06]. The single large search is worse than best-of-5 at the same work (+9.59 [3.32, 16.01]).

Your stronger test, an interior benefit at matched work, is negative: r = 3 minus r = 1 is −2.07 [−6.18, 1.51], and one terminal correction minus r = 1 is −1.92 [−6.23, 2.39]. What survives matched compute is the gap between feedback and search, and this is what the revision claims (Sec. 4, Eq. 6; Sec. 6.4; App. Table 10).

Q3: "Is the sham's real arm the r = 3 arm in Table 4?"

Partly. The real arm runs the r = 3 schedule, but on a separate 96-scene panel, disjoint from the original full panel (Table 4 of the submission), so the 10.8 subtracts numbers from two panels. We ran best-of-5 open loop on the sham panel itself. On the same scenes, seeds, calls, and work, J(r=3) − J(best-of-5) = −32.18 [−39.20, −25.06] splits into new observations, J(r=3) − J(sham) = −23.88 [−30.08, −18.26], and re-optimization without new information, J(sham) − J(best-of-5) = −8.30 [−14.67, −2.06]. Your conclusion holds in substance: about a quarter of the gain does not come from new observations. The revision reports both parts (Sec. 4; Sec. 6.5, Fig. 4a) and lists the remaining quarter as an open question (Sec. 7).

(continued in 2/2)

AuthorsOpenReview ↗

Response to Reviewer 3NWg (2/2)

S1: "the effect may be model-specific"

Agreed: the size depends on the model, and the revision measures this. On the 96 JEPA-panel scenes with the same seeds, one terminal correction gains −8.40 [−13.93, −3.88] with JEPA-WM (Holm p = 0.0002) and −45.10 [−55.86, −34.97] with DINO-WM; the difference is 36.70 [26.00, 47.60]. One quantity tracks the difference: the search-transfer ratio τ, the fraction of the predicted-cost improvement from extra search that appears in the realized latent cost. On these scenes τ = 0.18 for DINO-WM and 1.04 for JEPA-WM, and extra initial search helps JEPA-WM (five versus one: −10.03, Holm p = 0.0006) but not DINO-WM (−2.49, Holm p = 0.26). Over six panels, τ orders the value of search against feedback (Spearman 0.94, post hoc; Fig. 4c). Revision: Sec. 6.6 (p. 9), Table 1.

S3 and Suggestion: WebATLAS, Bidirectional Decoding, Lambert et al. (2022), MBPO

All four are now cited and discussed in Sec. 2 (p. 3).

  • WebATLAS (Cheng et al., 2025), an LLM web agent, simulates candidate actions with a learned internal model before acting and replans within a planner–simulator–critic loop. Conditioning later planning on real observations is a known design principle, and the revision says so. Our sham uses it as a measurement device: at matched calls and work, it splits the gain into new information (74%) and re-optimization without it (26%).
  • Bidirectional Decoding (Liu et al., ICLR 2025) finds that action chunking costs reactivity to unexpected states. This is consistent with our position curve: early feedback leaves most of the horizon to run on imagined states (Sec. 6.3).
  • Lambert et al. (2022) find that the underlying system dynamics are the strongest factor in the shape and magnitude of prediction error. This is consistent with our task dependence: with the same frozen encoder, open-loop latent drift after five macro-actions is 0.098 on Wall and 0.041 on PointMaze, where feedback adds little (Sec. 6.6; App. Table 16).
  • MBPO (Janner et al., 2019) limits compounding error with short model rollouts branched from real data. A late feedback call plays a similar role at planning time: it shortens the part of the episode that runs on imagined states.

S2: "the evidence is narrow and weakly independent"

  • K = H = 5. New Wall panels at H = 10 and 15 (above; details in our reply to smeF).
  • Shared panel. The original iCEM, DecentCEM-style, and JEPA-WM checks stay on the shared panel. The new evidence adds four panels with start states disjoint from all 986 submission starts (Wall at H = 10 and H = 15, 64 PointMaze scenes, 60 PushT validation starts) and 50 pairs from the official DINO-WM generator and seed. The feedback gain has the same sign on every Wall and PushT panel (Table 1, p. 9).
  • PushT training split. On 60 starts from the 21 validation episodes, one terminal correction beats a single search by −9.89 [−13.43, −6.81] (Holm p = 0.0003), and by −13.47 with 17 wins and 0 losses on the 17 frame-disjoint starts. The other 43 starts are at least 10 frames from earlier starts. The submission's r = 4 over r = 5 does not replicate (−2.29 [−4.84, 0.46], Holm p = 0.11), and we downgraded it in place (Sec. 6.2).
  • Tasks. Still three; we did not add a fourth environment (Sec. 7).
  • Code. The updated supplementary zip contains the anonymized evaluators, the official-pipeline wrapper, frozen protocols with SHA256, per-unit results, and analysis and figure scripts. Its README gives one CPU command that recomputes every sham-panel statistic (tested in a clean environment) and the GPU commands to re-run a panel.

We hope these answers help in reassessing the paper.

Reviewer_smeFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies how a controller built on a frozen visual world model should time its planning calls under a fixed budget. The question is how many calls to spend searching before acting, and how many to reserve for replanning once fresh observations arrive. With five complete CEM calls per episode, it formalizes and evaluates a family of pre- planned schedules. In each schedule, r calls search independently from the initial observation, the best plan’s first r macro-actions are executed, and each remaining call replans once from a new observation. The world model, scorer, call count, and logical function evaluations are held fixed across schedules.

Critical strengths

The motivation is clear and the paper is easy to understand.

Critical weaknesses

1: My main concern is the lack of justification for fixing the total planning budget at K=5K=5. The paper does not sufficiently explain whether a single-digit call budget is representative across different world models, task conditions, and execution horizons. Specifically, longer execution schedules may involve substantially replanning calls, making the practical relevance of the five-call setting unclear. The authors should justify this choice, explain how the call budget should be scaled with task duration and computational constraints, and evaluate whether the main findings persist under larger

budgets. Without this evidence, the applicability of the reported allocation effects remains limited.

2: The proposed method builds on DINO-WM’s closed-loop planning pipeline, but the additional gains from its call-allocation strategy appear modest. On Wall, changing from (r=1) to (r=3) improves the success rate by approximately one percentage point while increasing planning time by over 15%. The substantially larger headline improvement is measured against the open-loop (r=5) baseline.

3: The parameter r couples two distinct decisions: the number of initial planner calls and the number of macro-actions executed before feedback. These quantities should be discussed separately, and the author needs to conduct ablations with different values.

Questions for authors

Please justify the selection of total call budget, explain how the call budget should be scaled with task duration and computational constraints. Evaluate whether the main findings persist under larger budgets.

1 comment
AuthorsOpenReview ↗

Response to Reviewer smeF

Thank you for the review and for finding the motivation clear and the paper easy to follow. Notation: a schedule (b, F) runs b searches from the initial observation, then one feedback call after each macro-action index in F; the r schedules are (r, {r, …, H−1}), and one terminal correction is (1, {H−1}). Differences are in task cost J (lower is better) with 95% CIs.

W1 and Q: "the lack of justification for fixing the total planning budget at (K=5)"

K = 5 was not tuned. It equals the official DINO-WM planning horizon on Wall and PointMaze, H = 5 macro-actions (25 environment steps). K = H is the smallest budget at which per-step MPC is possible, so the r schedules span per-step MPC (r = 1) to best-of-5 open loop (r = 5). To test scaling, we added Wall panels at H = 10 (64 new scenes × 2 seeds) and H = 15 (64 new scenes × 1 seed), with K from 1 to H.

Task duration. At each horizon we tested, one search plus one feedback call before the last macro-action recovers 0.92 [0.82, 1.01], 0.98 [0.94, 1.00], and 0.91 [0.79, 1.03] of the per-step MPC gain at H = 5, 10, 15. What must scale with H is where the second call goes: after the first macro-action, the same call recovers 0.63, 0.20, and 0.04 (Fig. 1b, p. 2).

Compute. This two-call schedule uses 40%, 20%, and 13% of the work of per-step MPC at H = 5, 10, 15. On 50 official Wall pairs it plans in 10.0 s against 24.9 s (App. Table 4). Sec. 6.7 gives the resulting allocation rule. Under tight compute, use one terminal correction. Spend extra calls on feedback when the search-transfer ratio τ (realized over predicted gain from extra search) is well below 1, as with DINO-WM on Wall, and on initial searches when it is near 1 or above, as with JEPA-WM.

Larger budgets. Extra late calls add a little: at H = 15, two late feedback calls recover 0.94 and four recover 0.97 (App. Table 8). Larger calls do not change the ranking: at the work of best-of-5, with up to five times the samples per call, feedback schedules beat a single large search by −39.27 to −41.34. With three times the CEM updates, the terminal correction still gains −33.27 (one seed; Sec. 6.4).

Revision: Sec. 3, "Why K = H = 5, and what changes with H" (p. 4); Sec. 6.3 (p. 7). The longer horizons are on Wall only (Sec. 7).

W2: "changing from (r=1) to (r=3) improves the success rate by approximately one percentage point while increasing planning time by over 15%"

Your reading is correct, and the revision states it. No intermediate r is significantly better than r = 1: on the original Wall panel, r = 3 minus r = 1 is −1.33 [−4.69, 1.86] (Holm p = 0.37), and r = 3 plans 15% longer (28.9 versus 25.1 s). App. Table 7 lists all 30 intermediate-r versus r = 1 contrasts on ten panels; none is significant. The finding is therefore which call carries the value of feedback, not r = 3 beating r = 1. One terminal correction (two calls) is not significantly different from per-step MPC at H = 10 (+0.94 [−0.10, 2.93]) and H = 15 (+5.46 [−1.79, 13.12]), with 20% and 13% of its work. At H = 5 it trails by 3.47 [−0.37, 7.77] and fails its pre-specified non-inferiority test (margin 3). We keep r = 3 only as a minimax default at K = 5: over eight panels its maximum regret is 2.66, against 5.68 for r = 1 (App. Table 18). Revision: Sec. 6.1 (p. 6, Fig. 2), Sec. 6.7 (p. 9).

W3: "The parameter r couples two distinct decisions"

Agreed. The revision separates them: b is the number of initial searches, and the first feedback comes after min F macro-actions. On the sham panel (96 scenes × 3 seeds), a new grid of 45 schedules crosses b = 1 to 5 with the first feedback index t0 = 1 to 4, with feedback after every macro-action from t0 on. Single-feedback position curves cover three horizons.

  • Feedback position matters most. With one search and one feedback call, feedback before the last macro-action beats feedback after the first by −12.00 [−18.42, −5.62] (Holm p = 0.0006) at H = 5, and the gap widens with H (above).
  • Once a terminal correction is present, neither more initial searches (b = 5 versus 1: −3.00 [−6.75, 0.53], Holm p = 0.33) nor earlier first feedback (t0 = 1 versus 4: −1.75 [−5.25, 1.52], Holm p = 0.39) changes J significantly, and both add calls. Five searches plus a terminal correction beat best-of-5 open loop by −30.40 [−37.73, −23.45].
  • The value of b depends on the model. With JEPA-WM, five instead of one initial search gives −10.03 [−16.99, −4.19] (Holm p = 0.0006); with DINO-WM on the same scenes, −2.49 (Holm p = 0.26).

Revision: Sec. 3 (p. 4), Sec. 6.3 and 6.4 (p. 7), App. Table 11.

We hope these results address your concerns.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers and the AC

We thank the reviewers. They found the protocol solid and leak-free (3NWg, 7kVa), the full curves, resource ledger, and negative results credible (3NWg, 7kVa), the cached-latent sham well designed (7kVa), and the motivation clear (smeF). The shared concerns were the choice of K = 5 (all three), whether the result reduces to open loop failing (3NWg, 7kVa), matched compute (3NWg, smeF), and narrow evidence without code (3NWg, 7kVa). We ran new panels under frozen protocols with paired designs and scene-block bootstrap, without replacing any scene or seed, and rewrote the paper around what they show.

Notation. A schedule (b, F) runs b independent searches from the initial observation, then one feedback call from the new observation after each macro-action index in F. The submission's r schedules are (r, {r, …, H−1}): per-step MPC is r = 1, and best-of-5 open loop is r = 5. One terminal correction is (1, {H−1}): one search, then one feedback call before the last macro-action. J is task cost (lower is better); differences are paired, with 95% CIs.

New evidence (Wall, DINO-WM, unless stated)

  1. No intermediate r beats per-step MPC. On the original Wall panel, r = 2, 3, 4 minus r = 1 are −2.56, −1.33, −2.43 (Holm p ≥ 0.18). None of the 30 such contrasts on ten panels is significant (App. Table 7).
  2. The value of feedback sits in a late call. With two calls, feedback before the last macro-action recovers 0.92, 0.98, 0.91 of the per-step MPC gain at H = 5, 10, 15, with 40%, 20%, 13% of its work. After the first macro-action, the same call recovers 0.63, 0.20, 0.04 (Fig. 1b).
  3. Search does not replace it. Best-of-5 open loop improves on a single search by −11.83, one terminal correction by −38.02 [−45.35, −30.68]. When every arm gets the work of best-of-5, one terminal correction, r = 1, and r = 3 beat a single search with 5× samples by −41.18, −39.27, −41.34 (Holm p = 0.0004 each), also at matched time (Fig. 3). With 3× CEM iterations, the correction still gains −33.27 (one seed).
  4. New observations carry 74% of the gain. On the same 96 scenes, J(r=3) − J(best-of-5) = −32.18 splits into −23.88 from new observations and −8.30 from re-optimization without them (Fig. 4a).
  5. Model dependence is measured. Search lowers the predicted cost by 0.092 but the realized cost by only 0.033 (search-transfer ratio τ = 0.36). Over six panels, τ orders the value of search against feedback (Spearman 0.94, post hoc). With JEPA-WM, τ = 1.04, extra initial searches help (−10.03, Holm p = 0.0006), and the feedback gain is −8.40 against −45.10 for DINO-WM on the same scenes.
  6. Official pipeline. On 50 official Wall pairs, the released DINO-WM code reaches 0.58 success after one open-loop round and 0.94 after five (it stops on success). Our single search reaches 0.56, and one terminal correction 0.83 with two calls. The DINO-WM paper reports 0.74 and 0.96 (App. A.1).
  7. PushT. Feedback replicates on 60 new validation starts (one terminal correction minus single search −9.89, Holm p = 0.0003). The submission's r = 4 over r = 5 does not (−2.29 [−4.84, 0.46], Holm p = 0.11).

Revisions (new text in blue; pages of the revised PDF)

  • Title: "Allocating Five Complete Planner Calls" → "Allocating Planner Calls", because budgets now range from 1 to 15 calls. The question is unchanged. Abstract rewritten with fewer qualifiers (7kVa).
  • Sec. 3 (p. 4): (b, F) notation, why K = H = 5, macro-actions (App. Table 2).
  • Sec. 6.1 (p. 6, Fig. 2): r against r = 1. Sec. 6.2 (p. 7): PushT. Sec. 6.3 (p. 7, Fig. 1b): feedback position. Sec. 6.4 (p. 7, Fig. 3): matched work and time. Sec. 6.5 (p. 8, Fig. 4): sham split, predicted versus realized cost. Sec. 6.6 (p. 9, Table 1): JEPA-WM, PointMaze, official pipeline, τ. Sec. 6.7 (p. 9): an allocation rule.
  • Sec. 2 (p. 3): WebATLAS, Bidirectional Decoding, Lambert et al. (2022), MBPO, action chunking and replanning triggers, concurrent work.
  • Sec. 7 (p. 10): all limitations in one section. App. A (pp. 16–27): protocols with SHA256 and every new result.
  • Downgraded: PushT r = 4 over r = 5; any interior-r advantage over r = 1. Reported as found: one terminal correction fails its pre-specified non-inferiority test against per-step MPC at H = 5 (margin 3), and per-step MPC beats it on the official PointMaze pairs (+4.43 [0.33, 10.44]). Late placement and uniform spacing do not differ significantly.
  • Kept: all original results, including the negative ones.

Supplementary material (updated, anonymized): evaluators for DINO-WM and JEPA-WM, the official-pipeline wrapper, frozen protocols with SHA256, per-unit results of every panel, and analysis and figure scripts. The README gives a CPU command that recomputes every sham-panel statistic from the per-unit results (tested in a clean environment) and the GPU commands to re-run a panel.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.