acceptodds
Under review as a conference paper at ICLR 2027

AIM: Action-Supervised Intent as a Search Prior for World Model Planning

Abstract

Latent world models let an agent plan goal-directed behavior from visual observations by evaluating candidate action sequences inside learned dynamics. Under a finite compute budget, however, effective sampling-based planning depends not only on how accurately candidates are evaluated but also on whether the search starts in the region of action space relevant to the current task. Planners such as CEM and MPPI typically start from a generic proposal distribution, so even when offline trajectories already record the actions that connect the current observation to a future goal, the planner must rediscover the coarse structure of goal-directed behavior. A substantial share of the sampling budget can therefore be spent locating a useful action region before the world model begins to refine candidate plans. To address this, we propose AIM (Action-Supervised Intent Module), which uses offline goal-conditioned action supervision to build an informed search prior for a pretrained, frozen world model. Given frozen current and goal latents, AIM predicts a compact temporal intent for the connecting action block—the leading coefficients of its orthonormal discrete cosine transform (DCT)—and, through the inverse transform, uses it only to initialize the planner's action-sequence mean, leaving the representation, dynamics, rollout, sampling rule, and planning objective unchanged. This frequency-domain parameterization gives a compact, coarse description of goal-directed behavior and splits the initialization error exactly into a prediction term and a truncation term, which ranks intent dimensions before any planning run. Across three frozen world-model hosts, AIM consistently improves finite-budget planning, raising the five-task average from to with a single 300-candidate CEM iteration in the largest-gain setting, while retaining its advantage across larger search budgets and under MPPI.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Models, Model-Based Planning, Action Supervision, Sampling-Based Planning, Action Priors

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_58cfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

AIM is a framework for world-model-based planning, providing the goal-conditioned initial point for efficient searching, rather than goal-agnostic start. It trains an intent predictor on frozen world model features, which predicts compact action blocks connecting the current and goal images. To summarize action blocks to intent, AIM uses discrete cosine transform (DCT), and also reweighting layers to focus on important components of features for planning. AIM shows improvement compared to default CEM, across three types of world models, various budget ranges, and two planners.

Critical strengths

None

Critical weaknesses
  • I am not convinced why DCT and reweighting layers should be used. DCT and reweighting do not show any clear gain in terms of the planning success rates, compared to Direct-H5 in Table 7 and compared to no reweighting in Table 8, respectively. I also think that there might be no cost advantage of DCT for planning. Also, without DCT and reweighting layers in the proposed method, it seems that it lacks novelty, since there are several similar approaches providing better initial points for planning (e.g., INTACT [1], PRISM [2]).

  • The proposed experiments are limited to somewhat simple environments and tasks. I am not sure that the proposed intent prediction module will also work on complex, long-horizon, and real-world scenarios.

[1] Sun et al., INTACT: Isomorphic intent-to-action learning for search-free world models.

[2] Wang et al., PRISM: PRior-guided imagination sampling in world Models.

Questions for authors
  • Can the authors clearly show why DCT and reweighting layers should be used? What advantages does it bring to world-model-based planning?

  • Can the authors clarify the novelty and practical gain of the proposed methods compared to similar approaches providing better initial points (e.g., INTACT [1], PRISM [2]) or policy-guided planning?

  • Can the authors show some evidence that the proposed method will also work in more complex, long-horizon, or real-world scenarios?

  • Also, considering 300 trajectories with planning only improves 3.67pp compared to prior-only, I think additional computational cost for planning is diffcult to justify. Can the authors show more convincing scenarios where prior-only is not sufficient?

4 comments
AuthorsOpenReview ↗

Why DCT and the reweighting layer are needed (1/2)

We thank you for the detailed comments and for pointing us to closely related work. We ran new controlled experiments for each concern; numbers are mean ± SD over three training seeds, 100 episodes per task and seed.

W1 & Q1: Why DCT and the reweighting layer are needed.

A1: We appreciate this comment, and we would like to clarify why DCT is needed: it is the choice an exact analysis prescribes, and its benefit is measurable.

Theory (Section 4.3). The DCT is orthonormal, so the initialization error splits exactly into the prediction error of the kept coefficients and the energy of the dropped ones: E(K+1)=E(K)−RK2eKE(K+1)=E(K)-R_K^2e_K, where eKe_K is the share of the action energy in coefficient KK and RK2R_K^2 its validation R2R^2 from the frozen latent. A coefficient is thus worth its share of the action energy times its predictability; one the latent predicts no better than zero only adds error. Direct-H5 predicts all five coefficients, including those. DCT with K=2 retains the first two low-frequency coefficients. Our analysis and empirical results support this compact representation as a robust default across the evaluated tasks.

The decomposition needs only offline data, and it predicts where the second coefficient pays off:

TaskEnergy in the two kept coefficientsPredicted error reduction, K=1→2Observed change in success, K=1→2 (points)
PushT60%25.5%+22.33
Cube63%19.8%+23.67
Cube-Double66%19.3%+17.00
TwoRoom43%0.1%+3.00
Reacher40%0.1%−1.67

On TwoRoom and Reacher the first temporal mode still holds about one fifth of the action energy, but the frozen latent predicts it at a validation R2R^2 of only 0.008 and 0.003, against 0.45–0.68 on the other three tasks, so predicting it brings almost nothing. Every further increment of K is ranked below the three large ones and changes success by at most 3 points. A target with more coefficients therefore adds outputs without improving control.

Why Table 7 does not show it. Table 7 compares the two targets after search, where an initial mean matters only while the search still depends on its starting point (Section 4.3); 300 candidates largely erase the difference between two good priors. Where search contributes less, DCT is clearly stronger:

  • Without search: 89.20 ± 0.60% for DCT, against 86.73 ± 0.83% for Direct-H5 and 81.93 ± 0.64% for a Gaussian prior (its PoG-fused mean). The ordering DCT > Direct-H5 repeats on the paper's original, independently trained heads (89.40% vs 85.33%). On Reacher, where the action blocks are temporally white and the frozen latent barely predicts them, DCT reaches 90.00% against 74.33% and 62.33%, as the analysis predicts: a direct head must output all five actions, so it has to fit high-frequency content that the latent cannot predict, and that content adds error to the executed block; DCT keeps only the two low-frequency coefficients, the smooth trend of the block.
  • Beyond the training range: at twice the goal offset DCT stays above Direct-H5 on average, both without search (56.60% vs 53.40%) and with search (65.47% vs 64.60%; W&A3).
  • Cost: because the DCT prior is the stronger controller on its own, more of the success is available without search, at about 10 ms per control call instead of 46 ms. With search it stays within 0.8 points of Direct-H5 at the same latency, with 60% fewer outputs.
  • Diagnosis: the same decomposition ranks candidate targets before any planning run; its three top-ranked increments of K are the three that gain 17–24 points.

The reweighting layer is an auxiliary design. Learned and uniform weighting both reach 92.93%, so we will present it as optional; no conclusion depends on it.

AuthorsOpenReview ↗

Novelty, longer horizons, and the role of planning (2/2)

W1 & Q2: Novelty and practical benefit over INTACT, PRISM, and policy-guided planning.

A2: AIM is not a variant of these methods.

  • INTACT [1] trains the world model jointly with the prior, so the model itself must be trained for it. AIM attaches to hosts that are already trained and frozen: three in the paper.
  • PRISM [2] also uses frozen features but fuses a predicted mean and variance into the planner's sampling distribution (MPPI in the original; our Gaussian-PoG control applies the same fusion to CEM). AIM writes only the mean.
  • The plain form of policy-guided planning, a network that predicts the action block and seeds the planner, is our Direct-H5 control.
  • Beyond the interface, AIM contributes the exact decomposition of W&A1, which states before any planning run what a prior should predict.

All three heads were newly trained on the same host with the same training pairs and update count, and evaluated on the same start/goal pairs:

Success (%)No searchCEM@300, offset 25CEM@300, offset 50Parameters
Direct-H586.73 ± 0.8393.67 ± 0.8164.60 ± 0.200.26–0.28M
Gaussian prior, PoG81.93 ± 0.6494.87 ± 0.1263.93 ± 1.220.51–0.59M
AIM (Ours)89.20 ± 0.6092.87 ± 0.6465.47 ± 1.220.25–0.26M

All columns average the five tasks. Offset 25 is the standard setting; offset 50 doubles the largest training offset (W&A3). AIM is thus stronger without search and at twice the goal offset, and within 2 points of a prior twice its size after search on the standard tasks, through an interface that touches only the mean and with no host retraining.

W2 & Q3: More complex, long-horizon, or real-world scenarios.

A3: Thank you for raising this concern. We answer it on two levels.

Scope already in the paper. The evaluation covers three frozen hosts (ours, SMWM, and a DINO-WM-style host on DINOv2 features), five tasks that include single- and two-object manipulation (Cube, Cube-Double), two planners (CEM and MPPI), and a thirtyfold range of search budgets. With one CEM iteration AIM raises mean success in 14 of the 15 task–host cells and ties the remaining one (Table 1).

New experiment on longer horizons. The heads are trained only on goals up to 25 environment steps away. With the heads frozen and evaluated on goals twice as distant (offset 50, 100 interaction steps, four planning calls) on all five tasks, without intermediate goals, AIM with search reaches 65.47%, above Direct-H5 (64.60%) and, in every seed, the Gaussian prior (63.93%). It is the best method on Cube (60.67% vs 57.67% for both others) and exceeds the Gaussian prior on Cube-Double (47.33% vs 43.00%). The advantage therefore carries over to goals beyond the training range.

Real-world scenarios. Our evaluation follows the benchmark protocol of the world-model baselines we build on and compare with (LeWM, SMWM and DINO-WM), all of which are evaluated in simulation, so that our results are directly comparable with theirs. Accordingly, this submission does not include real-robot experiments either. The rebuttal period is too short for real-robot experiments, so we will add these results in future work.

Q4: Scenarios where the prior alone is insufficient and planning is necessary.

A4: The prior fixes where sampling starts; only search can correct the cases where that start is wrong. The 3.67-point average understates this role, because the prior alone already succeeds on about 89% of episodes:

  • Standard tasks: search removes one third of the remaining failures (10.80% → 7.13%). Failures fall from 28.00% to 16.67% on PushT and from 6.67% to 1.67% on TwoRoom, and PushT success keeps rising with budget: 72.00 → 83.33 → 93.67% at 0/300/9,000 candidates.
  • Distant goals: at offset 50, search lifts AIM from 56.60% to 65.47% on the five-task average (+8.87 ± 0.61) and from 16.00% to 34.33% on PushT. On the four tasks other than Reacher it rescues 153 prior failures and breaks 14; on Reacher, discussed in W&A1, the prior alone already reaches 95.00% at this offset, above every search condition.
  • Against search alone: one AIM iteration (300 candidates) beats thirty default iterations (9,000) on all three hosts, 92.93% vs 75.07% on ours, in 5–12× less evaluation time.

These discussions and experiments will be integrated into the revised version.

[1] INTACT: Isomorphic Intent-to-Action Learning for Search-Free World Models

[2] PRISM: PRior-Guided Imagination Sampling in World Models

While I appreciate the authors' efforts in this rebuttal, it is still very hard for me to find practical benefit to DCT for planning. Also, without the search, this method is closer to goal-conditioned policy learning than to planning. In this context, predicting low-frequency DCT coefficients for action chunks is similar to ideas already used in FAST [1] and there is no comparison to a policy learning baseline.

[1] Pertsch et al., FAST: Efficient Action Tokenization for Vision-Language-Action Models

We thank you for the follow-up and for engaging with our rebuttal. Your three points are connected, and one controlled comparison below speaks to all of them: AIM with search against a goal-conditioned policy trained on the same frozen features and the same data.

W1: Practical benefit of DCT for planning.

A1: All conditions below use the same frozen host, the same training pairs and the same number of updates. The first column is Direct-H5 executed without search, i.e., goal-conditioned behavior cloning with action chunking.

Success (%)GC policy (Direct-H5)AIM prior aloneAIM + CEM@300 (Ours)
Standard tasks, five-task mean86.73 ± 0.8389.20 ± 0.6092.87 ± 0.64
Twice the training goal offset, five-task mean53.40 ± 0.8756.60 ± 1.0665.47 ± 1.22
Cube, p = 095.33 ± 1.5395.67 ± 2.0899.00 ± 1.00
Cube, p = 0.2581.00 ± 1.7386.00 ± 0.0087.33 ± 0.58
Cube, p = 0.551.67 ± 1.5357.00 ± 3.0068.67 ± 0.58
Cube, p = 0.7544.00 ± 2.6548.67 ± 2.0854.00 ± 1.73
Cube, p = 145.33 ± 4.0445.67 ± 2.0852.67 ± 2.08

Mean ± SD over three training seeds. The Cube rows are a new experiment: the heads are trained on 2,000 trajectories in which each step is a uniform random action with probability p, with the host, the update count and the evaluation fixed. All Cube rows use one frozen host and the standard Cube start/goal pairs, so their SD reflects head training only.

The practical benefit of DCT (first vs. second column) is a stronger starting point for the same search, at the same latency and with a more compact output. It shows wherever the planner's start is hard to learn from the data:

  • Non-expert data. Offline data for world models are often collected by exploratory or noisy policies rather than experts. When the data mix purposeful and random actions (p = 0.25–0.75), the DCT prior is 4.67–5.33 points above Direct-H5; with half or more random steps (p = 0.5 and 0.75), the advantage remains after the same search (+7.67 and +5.00 points).
  • Goals beyond the training range: 56.60% vs 53.40% at twice the training goal offset.
  • Hard-to-predict actions: on Reacher, 90.00% vs 74.33% without search.

On the standard expert datasets, 300 candidates largely erase the difference between two good starts, so the benefit is clearest where the search still depends on where it starts. In addition, the number of coefficients is chosen from offline data before running the planner.

W2: Without search, the method is closer to goal-conditioned policy learning; no policy-learning baseline.

A2: The policy-learning baseline is the first column of the table in W&A1, and planning with AIM (last column) exceeds it in every row:

  • Planning adds what the policy cannot: AIM with search is 6.13 ± 0.99 points above the policy on the standard tasks and 12.07 ± 1.92 points on distant goals.
  • The gap is largest where imitation is weakest: on distant goals (+12.07) and on Cube data with half the steps random (+17.00). There search on the world model corrects starts that imitation alone cannot get right.
  • Each column adds one component, and each raises success in every row: the DCT target improves the start, and search on the world model improves on it.

We agree that the prior executed alone is a goal-conditioned policy; we report it as a probe of the start the planner receives, as asked in Q4 of the original review. AIM itself is a planning method: in every main result the executed actions are selected by CEM rollouts of the frozen world model, and the head writes only the planner's initial mean.

W3: Similarity to FAST [1].

A3: Thank you for the pointer. Our paper already cites FAST for the energy compaction of the DCT (Section 1), and we will expand the comparison in Related Work. The transform is shared; its role and what is kept differ:

  • Role. FAST tokenizes an action chunk so that an autoregressive VLA can predict it and then execute it. In AIM the coefficients are not executed: they set where CEM starts, and the search over the full action sequence on the world model refines them.
  • What is kept. Because FAST's decoded chunk is executed, FAST keeps all coefficients up to quantization. AIM truncates to the leading K and ranks the choice of K with the exact decomposition of Section 4.3: a coefficient that the frozen latent predicts no better than zero cannot reduce the initialization error. FAST's rounding also zeroes small coefficients, but it selects them by magnitude, to preserve the reconstruction of the executed chunk; AIM selects them by how well the frozen latent predicts them (RK2eKR_K^2 e_K), which is what matters for a search prior.

AIM's contribution is therefore not the transform itself but a truncated intent target for the initial mean of a planner on a frozen world model, together with an offline analysis of how many coefficients are worth predicting.

[1] Pertsch et al., FAST: Efficient Action Tokenization for Vision-Language-Action Models

Reviewer_FM1UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AIM predicts low-frequency DCT coefficients of an action block from frozen current-state and goal latents, and uses the reconstructed block only to initialize the planner's action-sequence mean. The paper evaluates several frozen world-model hosts, search budgets, and planners, and analyzes prior-only execution, direct action-block prediction, temporal parameterization, and over-search.

Critical strengths

The method has a simple interface that preserves the frozen model and planning objective, with clear gains under limited search budgets. Multiple hosts, CEM/MPPI evaluations, and budget sweeps make the evidence broader than a single operating point.

Critical weaknesses
  1. The incremental contribution should be stated precisely. In the matched ablation cohort, search raises average success from 89.40% to 93.07%, with a 12-point improvement on PushT but a 4-point decrease on Reacher. This supports a strong prior with task-dependent refinement. In Table 7, Direct-H5 has a higher mean on three tasks, ties on one, and is lower on one; this does not establish statistical superiority, but neither does it establish a consistent success-rate advantage for DCT. The specific evidence for DCT is compactness with largely retained performance.
  2. The offline error decomposition is informative, but the prospective value of the dimension-selection threshold remains unvalidated. Section 6.4 and Appendix F state that τ is set on the same 15 increments it separates. This supports a descriptive analysis of those observations, rather than an independently validated rule for selecting dimensions on a new task without planning trials.
Questions for authors
  1. Please distinguish the contributions of learning a prior, compressing it with DCT, and refining it through search. What do the existing budget results suggest about when refinement is worth retaining?
  2. Is the dimension criterion intended as a descriptive analysis or as a selection tool for new tasks? If the latter, can the existing tasks support a held-out validation, including comparison with simply fixing K=2?
2 comments
AuthorsOpenReview ↗

Contributions of the learned prior, DCT, and search (1/2)

We thank you for the careful reading and for recognizing the strength of the learned prior and the value of our offline error decomposition.

W1 & Q1: Separate contributions of the learned prior, DCT, and search; when search is worth keeping.

A1: We appreciate this constructive suggestion. We ran a prior / parameterization / search factorial on newly trained heads, budget sweeps with measured latency, and distant-goal tests on all five tasks.

Five-task mean (%)No searchCEM@300
Default CEMn/a47.67 ± 1.15
Direct-H586.73 ± 0.8393.67 ± 0.81
DCT-K289.20 ± 0.6092.87 ± 0.64
Direct-H5, paper's original heads85.33 ± 0.6193.93 ± 0.70
DCT-K2, paper's original heads89.40 ± 1.1193.07 ± 1.01

Mean ± SD over three training seeds.

  • Learned prior: it carries most of the benefit. With zero rollouts, the new heads exceed Default CEM at 300 candidates by 39–42 points.
  • DCT: its advantage is more than compactness. The 60%-smaller target learns the stronger prior, as the exact decomposition of Section 4.3 predicts: a coefficient is worth its share of the action energy times its predictability, and one the latent predicts no better than zero only adds error. Direct-H5 predicts all five coefficients; DCT keeps the two low-frequency ones. Without search DCT exceeds Direct-H5 by 2.47 points on the newly trained heads and by 4.07 points on the paper's original heads (the two sets were trained independently). On goals 50 steps ahead instead of at most 25 in training, it reaches 56.60% against 53.40% on the five-task average. On the newly trained heads, both differences are positive in all three seeds. The advantage comes mainly from Reacher (90.00% vs 74.33%), whose action blocks are temporally white and barely predictable from the frozen latent. A direct head must output all five actions, so it has to fit high-frequency content that the latent cannot predict, and that content adds error to the executed block; DCT keeps only the two low-frequency coefficients, the smooth trend of the block, and leaves the high-frequency content out. After search, the two targets are within 0.8 points on the standard tasks, in line with your reading. Section 4.3 explains why: an initial mean matters only while the search still depends on its starting point, and we observe that 300 candidates largely erase the difference between two good priors.
  • Search: it removes a third of DCT's remaining failures (10.80% → 7.13%) and half of Direct-H5's (13.27% → 6.33%). Per 100 paired episodes, search rescues/breaks 16.0/4.7 prior outcomes on PushT, 5.0/0 on TwoRoom, 1.7/0 on Cube, 4.3/0.7 on Cube-Double, and 7.3/10.7 on Reacher.

When is search worth keeping? Search minimizes the model's predicted cost, so it helps only where that cost reflects the task.

DCT controllerPushT SR (%) / latency (ms)Reacher SR (%) / latency (ms)
No search72.00 ± 4.00 / 10.390.00 ± 1.73 / 10.0
300 candidates83.33 ± 2.52 / 47.686.67 ± 4.16 / 45.3
90087.33 ± 3.79 / 123.978.67 ± 3.21 / 101.7
9,00093.67 ± 0.58 / 1171.170.67 ± 1.53 / 913.4

Latency is the batch-one control-call median on an RTX 3090.

  • Search is worth keeping where the model can correct the prior's failures: on four of five tasks it rescues more prior outcomes than it breaks, and PushT keeps improving up to 93.67%.
  • On Reacher, more search lowers success for every initializer, including Default CEM (82.33% → 69.33% from 300 to 9,000 candidates, Table 2), while the predicted cost keeps falling (Section 6.2): the limit there is the host's planning objective, not the prior. The DCT prior alone is higher than DCT or Direct-H5 at any search budget.
  • Search matters most beyond the training range: on goals 50 steps ahead (all five tasks, frozen heads) it raises DCT from 56.60% to 65.47% (+8.87 ± 0.61), rescuing 153 failures and breaking 14 on the four tasks other than Reacher (Reacher being the case discussed above).
  • How to decide in practice: both modes run from the same checkpoint, so the choice needs no retraining. One validation run per task, with and without search on the same start/goal pairs, tells which mode to deploy. Larger budgets should be validated in the same way, because more search can lower success, as on Reacher.

We will state the contribution accordingly: a learned prior supplies most of the gain; the compact DCT target makes that prior stronger, as the decomposition predicts; and search corrects it where the model's objective is reliable.

AuthorsOpenReview ↗

Dimension criterion and held-out validation (2/2)

W2 & Q2: Whether the dimension criterion is a descriptive tool or a practical rule; held-out validation against fixed K=2.

A2: Thank you for this insightful suggestion. The decomposition is our tool for deciding how many coefficients to keep (Section 4.3): from offline data alone, before any planning run, it ranks every increment of K by the error reduction it predicts.

TaskPredicted error reduction, K=1→2Observed change in success, K=1→2 (points)
PushT25.5%+22.33
Cube19.8%+23.67
Cube-Double19.3%+17.00
TwoRoom0.1%+3.00
Reacher0.1%−1.67

The three increments it ranks highest, K=1→2 on PushT, Cube and Cube-Double, are exactly the three that gain 17–24 points; all other increments are ranked below them and change success by only −1.67 to +3.00 points. K=2 is the choice this ranking supports, and it is within 2.33 points of the best tested K on every task.

The ranking needs only a task's own offline data; choosing K from it also needs a threshold on how large a reduction is worth keeping, which the paper sets from planning results. Your held-out validation tests whether a threshold fitted on some tasks transfers to a new one:

  • Leave-one-task-out: the threshold fitted on the other four tasks selects K = 3, 1, 1, 2, 2 when PushT, TwoRoom, Reacher, Cube and Cube-Double, respectively, are held out, and reaches 92.93 ± 0.50%, on par with fixed K=2 (93.07 ± 1.01%): the criterion transfers when related tasks calibrate it. Adaptive selection thus matches the simple K=2 strategy rather than exceeding it, which is why K=2 remains our default.
  • Leave-one-family-out: with both Cube tasks held out, only PushT calibrates the threshold, which rises to 0.212 and narrowly excludes K=1→2 on Cube (0.198) and Cube-Double (0.193), costing 23.67 and 17.00 points. The ranking itself stays correct: K=1→2 remains by far the top increment on both tasks.

The criterion thus ranks reliably and transfers given calibration from related tasks; without it we recommend the default K=2, the value it selects wherever K matters. Section 6.4 and Appendix F will state this scope and report both analyses; all of the above will be integrated into the revised version.

Reviewer_mGKKOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies how offline action supervision can improve finite-budget planning with a frozen latent world model. AIM predicts a compact goal-conditioned action prior in the DCT domain and uses its reconstruction only to initialize the mean of CEM or MPPI. The host model, planning objective, and subsequent search remain unchanged. Across five simulated tasks and three hosts, this substantially improves low-budget planning. The paper also investigates how much performance comes from the prior itself, when additional search becomes counterproductive, and how action-spectrum predictability relates to useful prior dimensionality.

Critical strengths
  1. The paper cleanly separates proposing useful actions from evaluating them with a world model. Holding the host and solver fixed while changing only the initial mean gives convincing evidence for the benefit of informed initialization. Evaluation across hosts, search budgets, and a second planner makes this more informative than a single favorable operating point. The lightweight interface is also practically attractive because it does not require retraining the dynamics or modifying the search procedure.

  2. The diagnostic experiments provide substantive insight beyond the module itself. In the matched comparison, the prior alone reaches 89.40%, versus 93.07% with search; search helps substantially on PushT but hurts on Reacher. Additional search can also reduce environment success while improving predicted cost. These results distinguish action-prior quality, search efficiency, and the quality of the learned objective. I particularly appreciate that the paper reports where search hurts rather than attributing all of the gain to model-based optimization. These empirical lessons are the strongest basis for my positive assessment.

Critical weaknesses

The main reservation concerns the incremental contribution relative to existing learned action-prior methods. The controlled comparisons convincingly establish the benefit of informed initialization, but the published comparisons in Table 1 and the width-fusion study in Appendix G leave some uncertainty about the advantage over closely related priors under matched conditions. The direct-action prediction ablation also suggests that much of the benefit comes from informed initialization rather than the specific DCT parameterization. This does not diminish the value of the simple interface or the diagnostic experiments, but a clearer account of the trade-off in performance, computation, or simplicity would strengthen the paper's positioning.

Questions for authors

Under matched training data, frozen host, and evaluation conditions, what is the principal advantage over a closely related learned action prior: success rate, total computation, training cost, or interface simplicity? Can the authors identify controlled evidence that distinguishes the value of AIM's specific interface from the general benefit of supervised goal-conditioned initialization? This clarification would most directly affect my assessment of the incremental contribution.

2 comments
AuthorsOpenReview ↗

Matched comparison with closely related action priors (1/2)

We sincerely thank you for the thoughtful and encouraging review. We are glad that you found the controlled comparisons convincing evidence for informed initialization, and we appreciate your recognition of the simple interface and the diagnostic experiments. We are also grateful for your precise questions on matched conditions and on separating AIM's interface from generic supervised initialization: they led us to new matched experiments that make the positioning of the paper much clearer. We answer each point below.

W1: Advantage over closely related action-prior methods under matched conditions.

A1: We appreciate this comment. Under strictly matched conditions, AIM's advantage over closely related action priors is a stronger prior from a smaller module: it achieves the highest success both without search and on goals beyond the training range, with half the parameters of the Gaussian prior and the same control latency. The comparison uses three heads newly trained on the same frozen host: AIM, Direct-H5 (direct action prediction), and a conditional Gaussian action prior adapted from PRISM [1], whose Product-of-Gaussians (PoG) fusion we apply to CEM.

Success (%)No searchCEM@300, offset 25CEM@300, offset 50Parameters
Direct-H586.73 ± 0.8393.67 ± 0.8164.60 ± 0.200.26–0.28M
Gaussian prior, PoG81.93 ± 0.6494.87 ± 0.1263.93 ± 1.220.51–0.59M
AIM (Ours)89.20 ± 0.6092.87 ± 0.6465.47 ± 1.220.25–0.26M

Five-task averages, mean ± SD over three training seeds. "No search" executes, with zero rollouts, the mean each method hands to the planner (the PoG-fused mean for the Gaussian prior). Offset 25 is the standard setting; offset 50 is twice the largest goal offset seen in training. The heads are newly trained, so AIM's number differs slightly from Table 1 of the paper (92.87% vs 92.93%).

  • A stronger prior before any search. Executed with zero rollouts, AIM reaches 89.20%, 2.47 ± 1.42 points above Direct-H5 and 7.27 ± 0.42 above the Gaussian prior. This column compares what each method hands to the planner before search corrects it, so it shows the advantage most directly. The gain over Direct-H5 comes from the compact DCT target, which predicts only the low-frequency coefficients (W&A2). The stronger prior also makes AIM a usable controller on its own, at about a quarter of the latency with search (Q&A1).
  • The advantage persists beyond the training range. On goals twice as distant as any seen in training, with the same frozen heads, AIM again achieves the highest success (65.47%). It is the best method on Cube (60.67% vs 57.67% for both others) and exceeds the Gaussian prior on Cube-Double (47.33% vs 43.00%). The coarse intent therefore remains a reliable starting point for search when the goal lies outside the training distribution.
  • A smaller module at the same cost. AIM obtains these results with half the parameters of the Gaussian prior, with no variance head and no fusion step, and at the same control latency (Q&A1).

After search on the standard tasks, the three priors are within 2 points. In the no-search and offset-50 settings, AIM is at least as good as both baselines in every training seed.

[1] PRISM: PRior-Guided Imagination Sampling in World Models

AuthorsOpenReview ↗

DCT vs. informed initialization, core advantage, and controlled evidence(2/2)

W2: Whether the gain comes from informed initialization rather than the DCT parameterization.

A2: Thank you for raising this point. Informed initialization indeed provides the largest gain (Default CEM → Direct-H5: +46.00 points), and we will state this clearly. The DCT target adds a further benefit on top of it: a more compact target learns a stronger prior.

Evidence. With architecture, data and updates fixed, replacing the five-action target by two DCT coefficients (60% fewer outputs) raises no-search success from 86.73% to 89.20% (+4.07 points on the independently trained heads of the paper's ablations) and, at goal offset 50, from 53.40% to 56.60%, higher in every seed. The improvement is largest on Reacher: 90.00% for AIM vs 74.33% for Direct-H5 (+15.67 ± 2.52, in every seed). There AIM without search even exceeds Direct-H5 with search at any budget (at most 86.33%).

Why. This is the case the exact decomposition of Section 4.3 describes. Because the DCT is orthonormal, the initialization error splits exactly into the prediction error of the kept coefficients and the energy of the dropped ones, E(K+1)=E(K)−RK2eKE(K+1)=E(K)-R_K^2e_K: each predicted coefficient is worth its share of the action energy eKe_K times its predictability RK2R_K^2 from the frozen latent, and a coefficient the latent predicts no better than zero adds error. Reacher's action blocks are temporally white (every temporal mode holds about one fifth of the energy) and barely predictable from the frozen latent (validation R2≈0.003R^2 \approx 0.003 for the first temporal mode). A direct head must still output all five coefficients, so it has to fit high-frequency content that the latent cannot predict; AIM's two low-frequency coefficients leave it out. The compact target is thus what keeps the prior reliable when actions are hard to predict.

Q1: Core advantage: success rate, compute, training cost, or interface simplicity?

A1: The core advantage is that a more compact target learns a stronger prior, delivered through a simpler interface. On the four axes:

  • Success: higher without search and beyond the training range (W&A1).
  • Compute: rollouts per solve are identical and measured control latency is equal, 42.30 / 42.11 / 42.13 ms for AIM / Direct-H5 / Gaussian prior; at 0, 30 and 100 candidates the heads stay within 1 ms of each other on every task. Without search, AIM's control call takes about 10 ms, about a quarter of the latency with search.
  • Training cost: matched by design. The host stays frozen and every head receives the same number of updates per task (7k–20k); each head trains in 22–75 minutes. AIM uses 51–56% fewer parameters than the Gaussian prior, and doubling its width changes success by only 0.33 points, so the compact head suffices.
  • Interface simplicity: AIM writes only the planner's mean, with no variance head and no fusion step (Q&A2).

Q2: Controlled evidence separating AIM's interface from generic supervised initialization.

A2: Three controlled steps separate them:

  1. Supervised goal-conditioned initialization: Default CEM → Direct-H5 adds 46.00 points.
  2. DCT target: Direct-H5 → AIM gives a stronger prior with 60% fewer outputs (W&A2).
  3. Interface: AIM hands the planner one deterministic mean, whereas the Gaussian prior also predicts a variance and fuses both into the sampling distribution, in a module twice the size. The Gaussian prior is 2.00 ± 0.53 points higher at one CEM iteration on the standard tasks, but this lead does not carry over to goals beyond the training range (−1.53 ± 0.46) or to execution without search (−7.27 ± 0.42). AIM's interface is thus the smaller one and gives up nothing outside the standard setting.

These discussions and experiments will be integrated into the revised version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.