- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates learned priors for latent planning in JEPA-based world models. The authors systematically compare three types of priors—deterministic Behavior Cloning, Gaussian, and generative Flow Matching—across four world-model backbones (LeWorldModel, PLDM, DINO-WM, and its Straightened variant) and six control environments (Two-Room, Push-T, OGBench-Cube, Wall, PointMaze-UMaze/Medium). Experiments show that learned priors significantly improve success rate and reduce planning latency when trained on goal-directed demonstrations, with Flow Matching proving most robust under frequent replanning and long-horizon settings. The paper also analyzes how sample size, replanning cadence, and goal-distance weighting affect prior effectiveness.
Critical strengths
-
The trade-off between learned priors and uninformed search in latent planning is a central yet under-explored question. This work provides the systematic comparison of three priors with varying expressiveness (BC, Gaussian, Flow Matching) within a unified framework across multiple world models and planning settings.
-
By having all three priors share the same FiLM-conditioned residual MLP backbone and differ only in their output distributional form, the authors effectively isolate the variable of interest. The uniform sample budget and shared planning objective further ensure a fair comparison.
Critical weaknesses
-
Llimited experimental coverage undermines generalizability. All six environments are continuous-control tasks (2D/3D navigation and manipulation) with relatively low-dimensional action spaces. This raises two major concerns:
- Restricted environment scope: The paper completely omits discrete-action environments (e.g., Atari), high-dimensional visual tasks, or real-world robotic scenarios. The behavior of JEPA world models and the effectiveness of learned priors may differ drastically in discrete-action or sparse-reward settings, leaving the generality of the conclusions unclear.
- Narrow algorithmic comparison: The baselines are limited to CEM and gradient descent. Notably, the authors do not compare against—or even discuss—MCTS-based planners such as MuZero or UniZero, which also rely on learned priors (policy and value networks) to guide planning and have demonstrated strong performance on complex decision-making tasks. Given the authors' claim of being the first to thoroughly study the advantage of different priors, the absence of any comparison or discussion with these methods substantially weakens the contribution.
-
Limited discussion of prior acquisition sources. All priors in this work are trained from the same offline demonstrations via behavior cloning or generative modeling. However, prior acquisition is not limited to demonstration data. For instance, PriorZero and related works have explored using Large Language Models as external knowledge sources to generate task priors or subgoal decompositions, offering structured guidance without task-specific demonstrations. The authors do not discuss how such "non-data-driven priors" relate to, complement, or compete with the learned priors studied here. Given the rapid growth of LLM-for-robotics research, this omission makes the discussion of prior design feel incomplete.
Questions for authors
-
Have the proposed priors been validated in discrete-action environments (e.g., Atari)? If not, do the authors believe that continuous-space generative priors like Flow Matching can naturally generalize to discrete-action settings, or would fundamental architectural changes be required?
-
Compared with MCTS-based methods such as MuZero / UniZero, how do the sample efficiency and asymptotic performance of the proposed CEM-based informed planning compare? Under what circumstances should one prefer sampling-based priors over tree-search-guided priors?
-
Given that demonstration quality critically determines prior effectiveness, have the authors considered leveraging LLMs or other external knowledge sources to construct or augment priors, thereby reducing reliance on high-quality demonstrations?
-
Reasonableness of the long-horizon extending baseline. In the long-horizon experiment (Figure 2), short-horizon priors () are extended to by padding with Gaussian-sampled actions. This baseline seems artificially weak—why not extend the chunk autoregressively using the prior itself (e.g., feeding its own output back as the next conditioning input)? If a stronger extending strategy were used, would the relative advantage of retraining at remain as pronounced?
3 comments
Thank you for the additional experiments and clarifications, which address several of my concerns. Further experiments on non-data-driven priors would still help broaden the discussion of prior design, although I understand that they operate at a different level from the proposed low-level action priors. Likewise, although MuZero/UniZero cannot be applied directly to the current setting, preliminary results with an adapted MCTS baseline would further strengthen the paper’s empirical claims.
I still have a question regarding the paper’s core contribution. That a learned prior trained on goal-directed demonstrations substantially improves success rates is relatively intuitive. Although the general response already provides several useful practical guidelines, could the authors further clarify which findings go beyond this intuition, which lessons transfer robustly across environments and world models, and what their conditions and limitations are? This would help sharpen the paper’s reusable takeaways and distinct contribution.
We thank the reviewer for the follow-up and are glad that our reply addressed several concerns. As for the remaining question, several of our findings run counter to what intuition would suggest, four of which stand out. First, one would expect the most expressive prior to work best, yet the simplest, deterministic prior often matches or beats Flow. Flow’s advantage only emerges when the planner can choose among many candidates, with dense replanning, or at long horizons. Second, one would expect a prior to be useful only if it was trained on goal-directed data, yet priors trained on random-walk data did not mislead the search. Used to initialize CEM, they matched or improved on it. Third, one would expect the choice of prior to matter most, yet replanning cadence and cost weighting affect success far more. Fourth, one would expect the best planning objective to depend only on the task and world model, yet under frequent replanning our experiments suggest that it depends on how candidates are proposed: Flow tends to prefer weighting all steps equally, the unimodal priors favor early steps, and CEM favors later ones.
Among our findings, the following three proved most robust across world models. Seeding CEM with a Flow or Gaussian prior matches or improves on CEM, whereas the deterministic prior is less reliable. A prior alone reliably replaces search and cuts planning time several-fold when the demonstrations are goal-directed. Otherwise, it is best used to seed CEM. On Push-T, Flow is the most robust prior under dense replanning and at long horizons under all four world models, provided the sample budget is sufficient. With few candidates, the deterministic prior is preferable. These lessons are limited to simulated continuous-control tasks, and the priors are trained per world model and environment.
Our contribution therefore goes beyond showing that priors trained on goal-directed demonstrations help. It shows when priors help even without such demonstrations, which prior to choose, and how to configure the planner around it.
We thank the reviewer for their feedback and address the concerns below.
W1.1: Limited experimental coverage: No discrete-action, high-dimensional visual tasks, or real-world robotic scenarios.
Our study follows the experimental settings of current research on JEPA-based world models and latent planning (LeWorldModel, DINO-WM, temporal straightening): continuous control of differing complexity, from 2-D navigation and pushing to 5-D arm control in OGBench-Cube. Discrete action spaces such as Atari would require changing the priors' output parameterization and CEM's Gaussian refit (see Q1). They also motivate tree search rather than sampling-based planning (see W1.2). Regarding high-dimensional visual tasks, all six environments are visual control tasks. The world models plan from 224×224 RGB observations (about 150k input dimensions, about 5× the standard Atari input of 84×84×4 grayscale frames) in latent spaces of 192 (LeWM, PLDM) to 1,568 (DINO-WM) dimensions. The planners search over action chunks of 50–250 dimensions with only 300 samples. Real-world robotic scenarios remain a limitation, which we state in the manuscript.
W1.2, Q2: Absence of MCTS-based planners such as MuZero or UniZero.
MuZero and UniZero learn reward, value and policy from online interaction and use the policy as a search prior, typically over discrete actions. Our setting differs in each respect: The world models are trained on reward-free offline data, goals are given as images at test time (zero-shot), there is no value function, and actions are continuous chunks. Sampled MuZero extends MuZero to continuous actions and MuZero Unplugged to offline data, but both still require a reward signal to learn value and policy targets, which our reward-free, goal-image-conditioned setting does not provide. Without rewards, these methods do not apply out of the box, so sample efficiency and asymptotic performance are not directly comparable. Sampling-based priors are preferable for continuous, high-dimensional action chunks, reward-free goal-conditioned tasks, and frozen pre-trained world models. Tree-search-guided priors are preferable for discrete actions with a reward signal and online interaction. We will add a discussion of MCTS-based planners with learned priors, including MuZero, UniZero and these variants, to the related work.
W2: Limited discussion of prior acquisition sources.
We thank the reviewer and will add a discussion of non-data-driven priors. LLM priors such as PriorZero act on discrete, language-describable actions or subgoals, whereas our priors propose low-level continuous action chunks. The two are orthogonal: an LLM could propose subgoals that condition our action priors. Like previous work on JEPA-based world models, we limit the scope to image observations and continuous control. LLM priors are better suited to tasks whose actions or structure can be described in language, such as text games or high-level task decomposition.
Q1: Validation in discrete-action environments?
They have not been validated in discrete-action environments. The hindsight training and the conditioning on start and goal latents would carry over unchanged, but the output parameterization would have to change. The Gaussian and BC priors would need to predict categorical distributions over action chunks, and Flow Matching and CEM would need to be replaced by their discrete versions. Please see also W1.1.
Q3: LLMs or external knowledge to reduce reliance on demonstrations.
We deliberately used the world models' own training data, which comes at no extra cost and varies in quality. Even so, the priors reduce planning time and often raise success rates. Our data-efficiency study (App. Figure 9) further shows that training priors only on 25% of the demonstrations retains nearly the full benefit, which reduces the required size of demonstration data. Moreover, our new data-quality sweep on Two-Room (see our reply to Reviewer uP93, Q2) shows that priors trained on data that is only 25% goal-directed already reach near-ceiling success. We agree that LLMs or other external knowledge sources are a promising way to reduce this reliance further (see also W2).
Q4: Gaussian padding is an artificially weak extension baseline and extending the chunk autoregressively would be stronger.
We thank you for suggesting a stronger extension strategy. In short, autoregression improves on Gaussian padding in all 24 configurations and closes a substantial part of the gap, but retraining at H=10 stays ahead for Flow and for all priors combined with CEM, so the advantage of retraining persists, although it becomes smaller. The full results are in our general response.