acceptodds
Under review as a conference paper at ICLR 2027

Beyond Terminal Matching: Arrival-Aware Planning with World Models

Abstract

Planning with a learned world model involves deciding not only how to reach a goal, but also when to reach it. Matching the goal only at a fixed terminal step does not explicitly favor earlier arrival. We introduce an arrival-aware planning objective for learned world models that requires neither model fine-tuning nor training of auxiliary components. The objective treats each prediction step as a candidate arrival time and assigns it a cost combining elapsed time with a penalty for predicted goal mismatch. These costs are aggregated through a smooth minimum, allowing action optimization to favor earlier goal attainment without prescribing a fixed arrival step. We analyze how this objective adaptively weights gradient contributions from different candidate arrival times. Experiments on visual goal-reaching tasks show reductions in budgeted physical interaction steps under a shared planning configuration. Controlled comparisons separate flexible endpoints, which improve across four LeWM tasks, from explicit time preference, which provides further gains on TwoRoom and Reacher. These findings highlight arrival-aware objectives as a means of improving physical-interaction efficiency without additional training.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, training-free planning, model predictive control, trajectory optimization, arrival-time planning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Rgx8OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TIME, a training-free planning objective for goal-conditioned MPC through a frozen latent world model. Every predicted block h is treated as a candidate arrival time. Its cost is b_h = αh + ρ·softplus_κ(c_h − ε), where c_h is the latent goal distance normalized by the current-to-goal distance. The costs are aggregated with a smooth minimum, F = −τ log Σ_h exp(−b_h/τ).

  • α = 0 gives "Free" (endpoint flexibility only), and α = 1 gives TIME (adds a preference for early arrival).
  • The analysis covers the Gibbs-weight view, causal gradients, a monotone shift of the weighted endpoint index with α, a discounted-return identity, penalty/smoothing bounds and analytic counter-examples.

Experiments:

  • Frozen LeWM on TwoRoom, PushT, Reacher and Cube: 64 held-out test episodes × 3 seeds, a shared solver (96-start coarse-to-fine Adam + L-BFGS-B) selected on 16 development episodes, and paired bootstrap tests with Holm correction.
  • Free beats Terminal on all four tasks. TIME beats Free on TwoRoom (−9.3 steps) and Reacher (−4.7) but not on PushT or Cube.
  • TIME has the lowest mean steps against CEM/iCEM/MPPI*/GD/GRASP; 16 of 20 contrasts are significant, and all non-significant ones are on Cube.
  • There is a DINO-WM check on PushT and Wall. Initial-plan latency is 4.8 s vs 0.18–0.57 s for the sampling planners.
Critical strengths
  1. Exemplary experimental hygiene and honest reporting. The protocol separates development and test episodes, fixes the solver on the development set before testing (with a check that Free would select the same configuration), and uses paired episode-bootstrap intervals with Holm correction. Negative and unresolved results are disclosed rather than hidden, e.g. Cube favours Free over TIME, and Terminal beats both at H = 3 on Cube. The paper also states its caveats explicitly and gives analytic examples (App. A.5) showing that terminal matching under receding-horizon control can keep postponing arrival even with exact dynamics. This level of rigour is rare in planning papers and makes the reported numbers trustworthy.

  2. A clean conceptual decomposition. The Terminal → Free → TIME ladder separates "allow arrival at any step" from "prefer earlier arrival", so the contribution of endpoint flexibility and of explicit time preference can be measured separately. The permutation argument (Free is invariant to reordering endpoint costs, TIME is not) is a crisp way to state the distinction, and the analytic example in Fig. 5c makes it concrete.

Critical weaknesses
  1. Weak Terminal baseline and missing standard alternatives. Free vs Terminal accounts for nearly all of the improvement (Reacher 95.36 → 19.19 steps). The explicit time term — the part that is new — is significant on only 2 of 4 tasks, and Cube moves the wrong way (+3.70). The internal Terminal reference at H = 8 with the normalized softplus penalty is much worse than the external GD planner on the raw C_H: on Reacher, GD gets 32.87 steps / 93.2% vs Terminal's 95.36 / 7.3%. Even at H = 3, Terminal gets only 70.78 / 50% (Table 9). At H = 3 on Cube, Terminal (28.56) beats both Free and TIME. So "Terminal at H = 8" largely captures the receding-horizon postponement effect plus the loss transform, not a well-tuned terminal planner. The obvious objectives are absent: the cumulative running cost ΣhchΣ_h c_h (the default MPC stage cost, which already favours early arrival), a hard min over h (the "best-intermediate" distance the paper attributes to ProWorld), and a plain discounted sum. The paper itself notes that smooth temporal extrema and the discounted form are established (App. B, Eq. 7). Its novelty therefore rests on this particular composition, which cannot be judged without these baselines.
  2. Limited practical significance and scope. TIME's initial solve takes ~4.8 s, about 10–25× the sampling planners, and total wall-clock completion time is not reported. TIME is never plugged into CEM or iCEM, although it is solver-agnostic. Table 2 therefore still confounds objective and solver, and it remains unknown whether the benefit survives with fast samplers.
Questions for authors
  1. Please explain the Cube regression with α > 0: active-case success drops from 43.8% to 38.2%. Does the early-arrival preference produce more aggressive contact actions?
  2. Can you run Free and TIME as the cost inside iCEM/CEM with their development-selected budgets? Do the step reductions persist at sampling-level latency?
  3. Under the same shared solver and H = 8, how do (a) ∑hch\sum_h c_h, (b) min⁡hch\min_h c_h (hard), and (c) ∑hγhch\sum_h \gamma^h c_h compare with Free and TIME? If (a) matches TIME, what is the specific benefit of the smooth-min formulation?
  4. How sensitive are the results to α (e.g., 0.25, 0.5, 2), τ and ε? Is Cube's degradation monotone in α?
2 comments

Thank you for your careful review.

Q1. How does the time preference affect contact and success on Cube?

TIME made a larger first movement, and a positive time weight improved success on new episodes. We selected an H=8 setting on development episodes, then changed only α between TIME (0.092) and Free (0). Of 64 new test episodes, 49 required an action. TIME solved 24/49 (49.0%) vs 21/49 (42.9%) for Free. Across all 64, mean actions fell from 45.91 with Free to 41.27 with TIME.

On the paper’s 48 initially unsolved Cube episodes, first-action gripper travel averaged 24.29 mm with TIME vs 19.08 mm with Free. The cube moved 7.23 vs 6.02 mm closer to the goal on that action. Mean cube–gripper normal force over executed actions was 18.13 vs 15.45 N. On the 49 new active episodes, first-action travel was 21.92 vs 19.55 mm and force at the end of that action was 17.45 vs 13.64 N, including zero-contact episodes. The larger first move persisted when success improved. Whether force caused the original failures remains open.

Q2. Do the gains persist at sampling latency?

TIME averaged fewer actions than raw Terminal on all four tasks and under one second per first solve. Each sampler used its development-selected settings on 32 paired episodes. All three shared the first-solve budget. Free changed only α to zero. Raw Terminal used final unnormalized distance.

Task and samplerModel transitions in the first planning callMean actions (Terminal / Free / TIME)First solve, s (Free / TIME)Episode, s (Free / TIME)
Reacher iCEM, H=33,84038.53 / 18.78 / 14.560.685 / 0.6671.49 / 1.35
Cube iCEM, H=56,40053.78 / 62.97 / 51.130.765 / 0.7563.48 / 3.01
TwoRoom iCEM, H=545,00068.53 / 43.97 / 45.220.968 / 0.9344.34 / 4.37
PushT CEM, H=33,84070.94 / 67.25 / 68.500.638 / 0.6402.23 / 2.28

Against Free, TIME cut episode time on Reacher and Cube. Success rose from 96.9% to 100% and 37.5% to 50.0%, respectively. Raw Terminal took 1.80 s and 2.95 s per episode. First-solve means cover episodes requiring action (22 on Cube, 32 elsewhere). Episode times include preprocessing, planning and simulation after loading, through success or 100 actions.

Q3. What does TIME add over the three standard distance costs?

All five objectives used the H=8 gradient solver. TIME averaged fewer actions than all three distance costs on Cube, TwoRoom and PushT (normalized cₕ, discount γ=0.9). TwoRoom and PushT used the paper’s settings on the first 16 test episodes. For Reacher and Cube separately, we selected TIME’s cost parameters on 24 development episodes and tested all five objectives on the same 48 held-out episodes. Cube used α=0.0625 and τ=0.25. Free set α=0. Initial success counts as 0 actions and failure as 100.

TaskEpisodesTIMEFreeSum cₕMin cₕDiscounted sum
Reacher489.6010.029.3535.389.27
Cube4836.4835.0845.1040.5049.19
TwoRoom1614.2525.0029.7533.5028.75
PushT1664.0081.1968.0067.0665.69

TIME and both sums solved all 48 Reacher episodes. The sums averaged 0.25 and 0.33 fewer actions than TIME. By action 10, TIME reached 34/48 goals, each sum 33/48, Free 31/48 and the hard minimum 14/48. A running cost penalizes mismatch even after a trajectory reaches and leaves the goal. TIME can reward the earlier arrival and draw gradients from several endpoints.

To isolate smoothing on 64 TwoRoom episodes, we kept elapsed time and changed only how we combined endpoint costs. The hard minimum minₕ(αh+pₕ) averaged 13.56 actions, smooth TIME 13.75 and Free 22.38. The hard and smooth minima differed by 0.19 actions.

Does the penalty explain Terminal’s high action count on Reacher? With the same H=8 solver on 32 episodes, raw Terminal averaged 90.47 actions, penalized Terminal 92.38 and TIME 15.34. External GD used H=3 and AdamW (Table 6).

Q4. How do α, τ and ε affect performance?

On 16 Reacher test episodes at H=8 and τ=0.25, α=0, 0.25, 0.5, 1 and 2 gave 20.31, 14.00, 13.81, 15.19 and 16.38 mean actions. On 16 Cube episodes the same weights gave 38.94, 39.25, 39.19, 39.44 and 39.63 actions, a nonmonotone response. Each weight solved 4 of the 10 initially unsolved episodes. We also varied τ and ε one at a time on those same Cube episodes at H=8.

Cube parameterValues, in orderMean actions, Free (α=0)Mean actions, TIME (α=1)
τ at ε=0.050.125 / 0.25 / 0.5 / 134.25 / 38.94 / 44.56 / 39.5035.56 / 39.44 / 33.81 / 39.44
ε at τ=0.250.01 / 0.05 / 0.1045.63 / 38.94 / 42.3133.88 / 39.44 / 39.56

Q1’s 64-episode test used τ=0.068 and ε=0.01. Proposition 1 holds the predicted trajectory fixed. Here the planner reoptimizes its actions.

Thank you for the thorough rebuttal and for continuing to report the experimental outcomes transparently. The sensitivity sweeps over α\alpha, τ\tau, and ϵ\epsilon together with clarifying Proposition 1 under action re-optimization (Q4), as well as checking raw versus penalized Terminal at H = 8 on Reacher, address those specific diagnostic questions, and the iCEM/CEM experiments (Q2) confirm that the objective can run at sub-second first-solve latency.

However, my main concerns regarding the methodological novelty of TIME's specific formulation (W1, Q3), the unresolved negative result on Cube (Q1), and the consistency of practical gains (W2, Q2) remain, and are in several places reinforced by the new results:

  1. Regarding W1 and Q3 (and Q1), the new ablations narrow rather than establish the necessity of TIME's specific formulation. First, on 64 TwoRoom episodes, the hard minimum min_h(αh + p_h) (13.56) slightly beats smooth TIME (13.75), indicating that log-sum-exp smoothing and multi-endpoint Gibbs gradients add no empirical benefit once αh is included. Second, outside TwoRoom (tested on 16 episodes in Q3), untuned sum_h c_h (9.35) and sum_h γ^h c_h (9.27) match or slightly beat dev-retuned TIME (9.60) on Reacher, discounted sum is close on PushT (65.69 vs. 64.00), and Cube remains an unresolved negative result for α > 0 across Q3 (36.48 vs. 35.08 for Free) and Q4 (while Q1 leaves the failure cause open and relies on ε = 0.01, which degrades Free to 45.63). Finally, checking H = 8 still leaves unexplained why internal Terminal at H = 3 on Reacher (70.78) lags far behind external GD (32.87) and iCEM raw Terminal (38.53).
  2. Regarding W2 and Q2, under fast samplers TIME loses its advantage over Free on TwoRoom (45.22 vs. 43.97, despite TwoRoom being the only task where TIME clearly beat running sums under the gradient solver) and PushT (68.50 vs. 67.25), while on Cube its episode time (3.01 s) does not improve over raw Terminal (2.95 s). Moreover, total episode wall-clock time for the main 4.8 s gradient solver is still unreported, even though its initial solve alone exceeds the full-episode runtime of the samplers (1.35 to 4.37 s).

While I appreciate the rigorous and transparent empirical effort, these findings reinforce my concerns regarding novelty and practical scope, so I currently lean toward maintaining my score.

Reviewer_d7QCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TIME, Temporal Integration of Mismatch and Elapsed time, a planning objective that is arrival-aware and applicable to lerarned world models. The authors observes that conventional terminal state matching cannot actually distinguish trajectories that reach a goal early from those that reach it only at the end of the planning horizon. Therefore the authors address this issue by treating each predicted step as a candidate arrival time. Additionally, they assign a cost based on the goal mismatch and elapsed time, and aggregating these costs using a smooth minimum. Because of this design the method actually doesnt require addtional finetuning and additionaly learning components. The experiments are conducted across four LeWM tasks. It shows that flexible endpoint matching improves physical interaction efficiency. Additionally, an explicit time preference provides further improvements.

Critical strengths
  1. The problem is very well-motivated and clearly defined. The paper identifies a limitation of terminal-only planning objectives which they do not explicitly reward early goal attainment. This clarity between planning horizon and arrival time is both intuitive and useful.

  2. The method designed is very simple and practically useful. TIME is easily integrated into existing world-model planning pipelines without additional training.

  3. Also the authors show reductions in physical interaction steps, and it provides evidence of improved interaction efficiency, which is an important aspect in world model.

Critical weaknesses
  1. My main concern is that the planning objective does not necessarily guarantee minimum physical arrival time. The authors provided the theoretical analysis that operates on predicted latent mismatch and candidate arrival costs. But In the end many things could prevent the objective from producing truly minimum-time behavior, like imperfect latent distances, errors in the learned dynamics, and local action optimization etc.

  2. Most experiments uses LeWM, with a more limited extension to DINO-WM. It would be good if the authors could also show results on broader sets of world models to demonstrate the effectiveness on other models, architectures, longer horizon tasks or more complex environments, to increase the credibility and generalization ability of the conclusion.

  3. although the flexible endpoint matching improves performance across all four tasks, the additional time preference only show strength on TwoRoom and Reacher. It might be that it's the endpoint flexibility but not the time preference that leads to the improvements. it would be great if the authors could provide more analysis on that.

Questions for authors

It would be great if the authors could answer the questions brought in the weakness, and address each concern separately. I will consider raising the scores if those get properly addressed.

1 comment

Thank you for the thoughtful review.

1. What does TIME guarantee about physical arrival?

Our starting point is the task’s stopping rule. An episode ends at first success. Two plans can end at the same goal even though one reaches it much earlier and stays there. A terminal cost assigns them the same score. We designed TIME to distinguish these plans while retaining the full prediction horizon.

Proposition 2 characterizes the earliest predicted block at which an admissible action sequence can satisfy the goal-mismatch threshold. With continuous mismatch, compact actions, an attainable threshold, unit time weight and a sufficiently large penalty, the global minimum of the unsmoothed objective equals that earliest feasible block index. We also bound the effect of smoothing.

A physical guarantee requires two additional conditions. Predicted feasibility must agree with simulator success at every scored endpoint, and a fastest physical trajectory must remain successful until its next endpoint. With five actions per block, executing a global unsmoothed minimizer as planned would then reach the goal no more than four actions after the earliest attainable physical success. Latent-distance error, model error and finite optimization during replanning can break this link. We therefore evaluate our practical claim by checking first success in the native simulator after every executed action.

2. How far does the result transfer across models and longer goals?

To test transfer beyond LeWM, we expanded the DINO-WM evaluation to 32 cases per task and three planning seeds on both Wall and PushT. On Wall, TIME reduced mean actions from 30.23 to 19.56 at 100% success for both policies. TIME had the lower mean at every seed. On DINO-WM PushT, matched Free used fewer actions than TIME (80.65 vs 86.57). The Wall result establishes that the arrival benefit can carry over to a second frozen world model, while the PushT result shows that this transfer depends on the task.

To test goals farther along a demonstration, we kept each controller fixed and moved the goal image from +50 to +75 demonstration actions on 32 starts per LeWM task. On PushT, TIME reduced mean actions from 84.16 to 71.47 and raised success from 28.1% to 43.8%. On TwoRoom, mean actions fell from 36.78 to 29.28, with success changing from 90.6% to 87.5%. Free used fewer actions on Reacher and Cube at this offset. The offset measures distance along the demonstration in actions. The shortest physical path to each goal is unknown.

The submitted Table 9 separately tests a longer prediction horizon. At H=12, corresponding to 60 physical actions, Reacher TIME averaged 15.59 actions vs 22.50 for Free on 32 cases. Our evidence covers two frozen model families and later goals within the existing environments. Generalization to additional architectures and more complex environments remains open.

3. What does elapsed time add once every endpoint can already match?

Free already scores every endpoint in the rollout. Reordering the same endpoint mismatches leaves its score unchanged. TIME adds an elapsed-time cost to each endpoint, giving an equally good match more weight when it occurs earlier. To isolate this contribution, we compared TIME with the same controller with only the time weight α set to zero. Within each row, the policies share the world model, solver and starts. Mean actions count an initial success as 0 and a failure as 100.

LeWM task and settingMean actions, Free → TIMESuccess, Free → TIME
TwoRoom, H=8, 64 additional episodes22.38 → 13.75100% → 98.4%
PushT, H=8, 32 additional episodes56.84 → 45.4471.9% → 75.0%
Reacher, submitted H=8, 64 episodes, three seeds19.19 → 14.46100% → 100%
Cube, fixed H=6, three cohorts totaling 320 episodes40.61 → 38.8161.9% → 63.8%
Reacher, new H=8, 48 episodes10.02 → 9.60100% → 100%
Cube, new H=8, 48 episodes35.08 → 36.4868.8% → 66.7%

For the two new H=8 tests, we selected TIME’s parameters on 24 development episodes per task and locked them before evaluating the 48 new starts. The early-arrival measurements directly address your attribution question. Switching on α raised success by action 25 from 67.2% to 93.8% on TwoRoom and from 21.9% to 46.9% on LeWM PushT in the additional cohorts. On the new Reacher starts, every episode eventually succeeded under both policies, yet the fraction already finished by action 10 rose from 64.6% to 70.8%.

These paired experiments support our claim that an explicit time preference can advance physical arrival even when every predicted endpoint is already eligible to match the goal. On Cube, the H=6 setting improved mean actions in each of the three cohorts, while the new H=8 setting favored Free. The contribution of elapsed time is measurable, and its benefit depends on the planning setting.

Reviewer_mAxxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TIME, a training-free planning objective for gradient-based MPC using frozen latent world models. Instead of scoring only the latent distance between the final predicted state of the rollout and the goal, it treats every predicted step as a candidate arrival time. Each step’s cost combines an elapsed-time term αh\alpha h with a penalty on normalized latent goal mismatch, and the costs are aggregated by a smooth minimum. The analysis covers how α\alpha tilts endpoint weights, penalty/smoothing bounds relative to a discrete minimum-time problem, and perturbation bounds. Experiments use frozen LeWM on TwoRoom, PushT, Reacher and Cube (64 test episodes, 3 seeds), measuring budgeted physical steps (first-success time capped at 100). Under a shared solver at H=8H=8, TIME reduces budgeted steps relative to the terminal-only cost on all four tasks: 76.7 to 15.8 on TwoRoom, 95.4 to 14.5 on Reacher, 95.0 to 64.2 on PushT and 56.1 to 48.3 on Cube. The authors further ablate the time term (α=0\alpha=0, “Free”) and show that scoring all steps alone already improves over the terminal cost on every task, while the time term adds further gains on TwoRoom (−9.3 steps) and Reacher (−4.7), is unresolved on PushT, and worsens Cube (+3.7)

Critical strengths
  1. Identifies a practically relevant failure mode of terminal-cost planning with frozen latent world models. Under receding-horizon replanning, a terminal latent-distance cost (the default in LeWM/DINO-WM) ties arrival to the prediction horizon and can postpone it indefinitely. The paper illustrates it well (A.5) and shows large gains with their proposed method. TIME is a cheap, training-free change to how these latent world models could be used during inference.

  2. The objective needs no fine-tuning or auxiliary components, and applies to any differentiable latent predictor. This gives it practical value if the gains hold against stronger references.

Critical weaknesses
  1. The gains shown in the paper are not attributed against the natural baselines. The main reference is a terminal-only cost at H=8H=8 (40 physical actions), while successful arrivals take about 15 steps. In this horizon-misspecified regime, terminal MPC is known to postpone arrival, as the authors’ own Appendix A.5 shows. Table 9 confirms that Terminal improves sharply at shorter HH and beats both framework variants on Cube at H=3H=3. The literature comparison (Table 2) changes objective, horizon, solver and compute (4.8 s vs. 0.2–0.6 s per initial solve) at once, so it cannot show that TIME’s objective drives the gains.

  2. The novelty is limited and there is only weak support for the novel component. The paper’s distinctive ingredient is the time term; the rest of the objective is not new. The theoretical results (the exponential-tilting variance identity and the log-sum-exp/softplus bounds) are standard. Empirically, most of the improvement over Terminal is already achieved without the time term (α=0\alpha=0). Adding it helps on only 2 of 4 LeWM tasks, worsens Cube, and on DINO-WM is supported by a single-seed Wall check. It is also evaluated at a single value (α=1\alpha=1) with no sensitivity analysis over α\alpha or τ\tau, even though Proposition 1 itself notes that τ\tau trades endpoint concentration against suppressed gradients. The paper’s main new contribution is therefore its least supported one.

Questions for authors
  1. Under the shared solver at H=8H=8, how do Terminal, Free and TIME compare with (a) a cumulative running cost ∑hρ qκ(ch−ϵ)\sum_h \rho\, q_\kappa(c_h-\epsilon) or its discounted version, and (b) a hard minimum min⁡hph\min_h p_h? If TIME clearly beats both, my attribution concern would largely be resolved.

  2. How do CEM/iCEM perform when they optimize the Free and TIME objectives instead of the raw terminal CHC_H, at matched latency or matched model evaluations?

  3. How sensitive are the TIME vs. Free gains to α\alpha and τ\tau? Is there an α\alpha for which Cube does not regress?

  4. Table 4 suggests Terminal would select coarse Adam (mean ≈77.8 steps) rather than the shared coarse+polish configuration (≈78.7). Is this correct, and does Terminal’s result change under its own selection?

  5. Figure 1. show clear AI-generated artefacts, while AI Use statement do not make any mention of using AI to generate figures. Can the authors clarify this?

1 comment

Thank you for your careful review and for suggesting the cumulative penalty and hard minimum as baselines.

Q1. How does TIME compare with the two H=8 penalty objectives?

TIME took fewer physical actions on average than either control on all four test sets. All methods used the same frozen LeWM, gradient solver, 96 initial action plans and replanning interval of five actions. Cells give mean actions (success rate), counting 0 for initial success and 100 for failure.

TaskTest episodesTIMESum of penaltiesMinimum penalty
Reacher649.97 (100%)10.83 (100%)41.94 (78.1%)
Cube6441.27 (60.9%)50.30 (51.6%)45.61 (56.3%)
TwoRoom6413.50 (98.4%)22.70 (89.1%)22.36 (95.3%)
PushT3255.28 (62.5%)81.34 (28.1%)69.06 (50.0%)

The controls sum or minimize the paper’s per-step penalty pₕ over eight endpoints, with (ρ, ε, κ) = (30, 0.05, 0.01). On TwoRoom, TIME used that same penalty and the paper’s α=1 and τ=0.25. On the other tasks, we used separate development episodes to choose TIME’s time weight, temperature and penalty parameters. The selected values (α, τ, ρ, ε, κ) were (0.0609, 2, 6.9923, 0.01, 0.01) for Reacher, (0.092, 0.068, 36, 0.01, 0.0075) for Cube and (0.5, 0.12, 36, 0.025, 0.01) for PushT. These three comparisons include changes to the penalty. Q3 isolates α.

All methods shared episodes and seeds within each task. Reacher and Cube each used 64 new episodes, 32 per seed. The 32 PushT episodes were held out of tuning. On Reacher, TIME and the sum both reached all 64 goals, but TIME reached 44 by action 10, versus 39 for the sum and 11 for the minimum. Table 1 gives the Terminal comparison on the paper’s original episodes.

Q2. Does TIME also help sampling planners?

We chose TIME’s settings separately for each sampler on development episodes. Each row gives mean actions on 32 test episodes for raw terminal distance, Free and TIME. Free sets only α to zero. First-solve transition counts match within each row.

Task and samplerFirst-solve transitionsRaw TerminalFreeTIME
Reacher iCEM, H=33,84038.5318.7814.56
Cube iCEM, H=56,40053.7862.9751.13
TwoRoom iCEM, H=545,00068.5343.9745.22
PushT CEM, H=33,84070.9467.2568.50

TIME used fewer mean actions than raw Terminal on all four tasks and fewer than Free on Reacher and Cube. Relative to Free, mean episode time fell from 1.49 to 1.35 s on Reacher and from 3.48 to 3.01 s on Cube. These times include preprocessing, planning and simulation through success or the 100-action limit, after model loading.

Q3. Which time weights and temperatures help?

On the new Cube episodes in Q1, changing only α from 0 to 0.092 reduced mean actions from 45.91 to 41.27 and raised success from 56.3% to 60.9%. The corresponding Free and TIME means were 10.56 and 9.97 on Reacher, and 63.44 and 55.28 on PushT. On the 64 TwoRoom episodes, a second H=8 comparison at τ=0.5 gave 22.38 actions at α=0 and 13.75 at α=0.25.

We varied one parameter at a time on the first 16 original test episodes per task, using the paper’s H=8 settings. On Reacher at τ=0.25, α=0, 0.25, 0.5, 1 and 2 gave 20.31, 14.00, 13.81, 15.19 and 16.38 mean actions. On Cube, the same 16 episodes gave the following means as we varied τ.

Cube temperature τ0.1250.250.51
Free (α=0)34.2538.9444.5639.50
TIME (α=1)35.5639.4433.8139.44

The advantage of α=1 was largest at τ=0.5 in this test. On DINO-WM Wall, we also extended the single-seed comparison to three seeds on 32 episodes. Free and TIME both reached every goal, averaging 30.23 and 19.56 actions, respectively.

Q4. What changes when Terminal selects its own solver?

Table 4’s selection rule favors coarse Adam for Terminal (77.80 development actions vs 78.72 with refinement). In a paired rerun on the first 32 original Cube test episodes at H=8, Terminal took 42.94 actions with coarse Adam and 56.72 with the shared solver. Table 9 also showed that Terminal can win at H=3. To test that horizon on 64 new Cube episodes, both objectives used the shared coarse Adam solver with L-BFGS-B refinement. We chose TIME’s five cost parameters on 24 separate development episodes. TIME took 44.28 actions (57.8% success), versus 50.23 (51.6%) for Terminal.

Q5. How was AI used in Figure 1?

Thank you for raising this. We used generative AI to help plan the layout of Figure 1. The observation, goal, and environment thumbnails show PushT inputs. The intermediate-state thumbnails are schematic illustrations. Our submitted AI use statement described language editing and should also have mentioned this layout assistance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.