acceptodds
Under review as a conference paper at ICLR 2027

How Far Can Latent World Models Imagine Ahead? Probing the Capabilities and Limits of Latent Planning

Abstract

World models offer a promising way to help robots understand how the physical world evolves and plan complex behaviours through imagination. Yet existing studies mainly demonstrate what these models can accomplish, leaving unclear when their predictions remain useful for planning and where they fail. We study this question using action-conditioned predictors built on five frozen self-supervised visual backbones: V-JEPA 2, V-JEPA 2.1, VideoMAEv2, VideoPrism, and DINOv2. We use frozen backbones to test representations intended to transfer across environments. We evaluate these models on diverse Meta-World manipulation tasks and real-robot interactions from BridgeData V2. We find that a world model guides action selection reliably only when the goal lies within, or slightly beyond, the trajectory it imagines during planning. With five-step rollouts, the length the predictor was trained on, the world model ranks actions reliably only for targets five to ten control steps ahead, whereas task goals lie 16 to 53 steps away. Neither an 81-fold larger predictor nor longer-rollout training extends this range; the encoder affects both range and closed-loop success, with V-JEPA 2.1 performing most consistently. More fundamentally, the limit persists under perfect prediction: using the real simulator, success falls from 92% to 41% as the target moves from five to twenty steps ahead of a five-step rollout (Figure 1). Planning therefore requires either longer imagined trajectories or closer subgoals. For distant goals, pure imagination succeeds in 23% of episodes, planning with feedback (MPC) raises success to 30%, imagining as far as the goal to 47%, and nearby expert subgoals to 76%. Used within its plannable range, a world model can also improve a vision-language-action (VLA) policy: choosing among eight actions the VLA proposes raises its success from 65% to 77% across 16 different tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World-models, Latent world models, video self-supervised learning, Model predictive control, JEPA, Planning, Vision-language-action models.

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_BxNaOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
3(+2)
Read review
Paper summary

This work analyses the latent forward-prediction performance on several pre-trained representations (mostly JEPA variants). The main analysis metric is comparing the distance between the latent of the expert demonstration, and an action-conditioned forward model trained on this representation. Importantly, only the action-conditioned model is trained on the task distribution, not the representation itself. Findings show that this autoregressive prediction fails to generate latent trajectories that are close to the latent demonstration trajectory.

Critical strengths

I really liked the motivation to understand the effect of pre-trained representations on decision-making, and the paper overall follows nice experiments. This paper would be quite interesting and strong if the evaluation process were different (see below). There are many analyses probing different axes of the representations. The paper is mostly well-organized, though some parts could benefit from further clarification, especially Section 3.

Critical weaknesses

I think the critical problem is drawing planning- or decision-making-related conclusions while the analyses measure the representation manifold. While one could expect the predicted trajectory to lie near the expert trajectory (in the latent space), for decision-making, this is not the primary concern. These self-supervised representations show most of their benefits when they are combined with a policy (e.g., an RL agent, or a behavior-cloned policy), which is also shown in Section 7 with the VLA experiments where the VLA chooses the actions to take. For instance, an RL agent trained on this representation space might take good actions even when the latent observation trajectories don't overlap. I don't see a sound reasoning on why we should treat the distances as a proxy for decision-making or planning performance. Maybe it makes sense in some form that I don't immediately see. If there's such a case, then it should be explicitly derived in the paper---for instance, a derivation on how this translates to the planning performance would be good. For instance, applying t-SNE (or even random numbers as new dimensions) would destroy the Euclidean distance interpretation, but you could still train an RL agent on it.

--- AFTER REBUTTAL ---

I understand that the paper's scope is planning with the learned representations. In that scenario, I agree with their experiments, even though, I still believe the usefulness of the learned representations could also be analyzed with a trained actor, or a mix of planning + RL.

Questions for authors

How does the latent distance to the expert trajectory formally connect to the model-free or model-based performance of an agent?

6 comments
AuthorsOpenReview ↗

Response to Reviewer BxNa (Weaknesses part 1)

Thank you for your review. We are grateful for your time and well-thought-out comments. We are also glad you "really liked the motivation to understand the effect of pre-trained representations on decision-making", and that you found the paper "follows nice experiments" and is "mostly well-organized". You wrote that the paper "would be quite interesting and strong if the evaluation process were different". Our evaluation process was designed carefully around how these world models plan. Below we explain why this evaluation fits the world models we study.


W1. Can latent distances support conclusions about planning?

Please note, there are two separate families of latent world models. Our study focuses on the emerging family, which plans with a latent distance cost, and the other is outside of this paper's scope. The RL family uses a policy or an RL agent, as in Dreamer [4] and TD-MPC2 [5]. Besides the world model, it learns a reward model and a policy or value function for each task or environment, so it needs task rewards or expert actions for every task. The policy chooses the actions, so the geometry of the latent space matters less. This is relevant to your t-SNE example, where a policy can still learn good actions on a distorted representation.

On the other hand, the family we study learns no policy, needs no reward and is gaining substantial traction (V-JEPA 2-AC [1], DINO-WM [9], NWM [2], PLDM [7], HWM [8]). These models learn only how actions change the world, in latent space, so they need no expert data. The world models in our paper were trained on random-action episodes only (App. B.1). At test time they imagine the outcomes of many candidate action sequences and execute the one whose imagined future is closest to a goal image (the latent distance, by design). This direction is promising because rewards and expert actions are hard to obtain outside simulated benchmarks, and because it can transfer to new settings. For example, V-JEPA 2-AC [1] was deployed "zero-shot on Franka arms in two different labs … without any task-specific training or reward", and DINO-WM [9] plans zero-shot "without expert demonstrations, reward modeling, or pre-learned inverse models". Our paper evaluates this family, and our evaluation follows from how it plans.

We note this in abstract, Sec. 2 and App. A.1. However, we will make this distinction between the world-model families and the scope of the paper clearer in Section 2.

(a) [Clarification] We measure whether the world model prefers the expert's actions to random ones, using the cost its authors designed it to plan with (latent distance). In the family we examine, the latent distance to the goal is the cost that chooses the actions. Thus, the latent distance cost was not 'our' choice. It is a design choice made by the authors of these models, and we follow it. For example, V-JEPA 2-AC uses an L1 distance, DINO-WM and PLDM use L2, HWM uses latent matching, and the latent distance we use (cosine distance on normalised features) equals half the squared L2 distance. We therefore compare the latent distance between the goal and the imagined outcome of the expert's actions with the same distance for random actions, which is the choice the planner makes at every step. Evaluating the latent distance cost is part of our paper's aims, and finding its strengths and weaknesses is central to the paper. The measurement shows how far ahead these world models can judge good actions with the planners they are normally used with. Using rewards or a policy would require extra data that this direction of world models does not use.

(b) The latent distance is a good cost, with some weaknesses. It needs no data, labels or rewards, so it works zero-shot on different tasks, which is what made the zero-shot results above possible. In our tests it works well when the imagined rollout reaches the target. For V-JEPA 2.1, the world model then picks the executed actions out of 16 candidates 98% of the time, 20 steps ahead (chance is 6%, Fig. 2(a), App. G), and the cost ranks the expert above all 100 random alternatives in every window (Table 4). The predictions therefore stay close to the demonstration in this setting, which differs from the summary of the review. The latent distance also transferred better than the learned costs we tried. A VIP-style head, trained on expert demonstrations to predict temporal distance, gave lower closed-loop success than the latent distance (0.188 vs 0.397 with subgoals 20 steps ahead), even on the tasks it was trained on. Its weakness shows when the planner imagines only a few steps at a time. One of the paper's findings is that the problem is not always long-horizon prediction error. It can instead be the latent distance cost that needs to improve, still without rewards or a policy. TEMPO [6], which appeared after our submission, is one of the first steps in this direction (Q1).

AuthorsOpenReview ↗

Response to Reviewer BxNa (Weaknesses part 2)

Weaknesses (continued)

(c) Rewards and a policy can help, but they need the extra data that the family we study avoids. A policy is strong where its training data exist. To show how much these data help, we trained a goal-conditioned behaviour-cloning policy (BC) on the same frozen V-JEPA 2.1 features, using expert actions from the 13 training tasks. On those tasks it beat every planner (0.92–0.96, against 0.54 for latent distance planning with a long rollout). Without such data, the RL family does not work or performs poorly. DINO-WM [9] trained TD-MPC2 on the same reward-free data, and it reached 0.00 success, which the authors attribute to "the lack of reward signal". On 10 new tasks, however, the BC policy failed to generalise and reached only 0.13–0.16, below latent distance planning (0.35). PLDM [7] also finds that "model-free RL benefits most from large amounts of high-quality data, whereas model-based planning generalizes better to unseen layouts". Rewards or a policy can therefore be a large advantage, but only because the model uses extra data about the task. This goes against the approach of the family of models we study in this paper, so a comparison with the RL family would not be fair and is outside the scope of the paper.

Section 7 shows that the two can still work together. An existing VLA [3] proposes candidate actions, and the world model, off the shelf, imagines the outcome of each and chooses among them with the same latent distance. Nothing is trained or tuned for this.

AuthorsOpenReview ↗

Response to Reviewer BxNa (Q1 & Remarks)

Q1. How does the latent distance formally connect to agent performance?

Thank you for this well-thought-out question. We will answer your question in several points so that it is easier to read.

[Clarification] The latent distance to the expert trajectory comes from the convention of the family of world models we study, but it is not the essence of the plannable range. We rely on the planning cost chosen by the authors of the models we have tested. These are policy-free, reward-free world models, such as V-JEPA 2-AC [1], DINO-WM [9], PLDM [7] and HWM [8], and their planning cost is the latent distance to a goal image (W1(a)). The idea of the plannable range is not to measure the latent distance to the expert per se, but to measure how well these world models tell good actions from random ones, based on their current planning convention. These models happen to use the latent distance, and therefore the latent distance to the expert becomes a proxy for whether the world model knows which actions are good to take.

These world models use the latent distance for a good reason. As we mentioned in W1(b), it has clear strengths, and these models have achieved good results with it, especially when the rollout reaches the target. For example, when the planner imagines far enough to reach the target, closed-loop success is 0.76. Its weakness shows when the planner imagines only a few steps at a time, because the cost is greedy (Sec. 5.5).

The main answer to your question depends on the type of agent. In a model-free RL agent, the policy chooses the actions directly, so the distance affects performance only if it is used to train the policy, for example as a reward. Model-based RL agents, such as Dreamer [4] and TD-MPC2 [5], also learn a world model, but their actions come from a policy, or from planning with a reward and value function learned for the task, so the latent geometry matters less. We make no claim about these two cases. The world models we study are also model-based, but they use no RL. Their planner executes the action sequence whose imagined outcome is closest to the goal in latent space. The philosophy of these models is to use no policy or rewards. The latent distance is therefore the cost of the planner, and it directly affects the performance of the agent. This is why the plannable range is defined with the latent distance.

Showing both the strengths and the weaknesses of the latent distance as a planning cost is one of the contributions of our paper. Improving this cost is a new direction of research, and we will make this clearer in the revised paper. However, the latent distance is not easy to replace, because a new planning cost must be learned without expert data or rewards, unlike the costs of RL world models. The learned costs we tried did not beat it on new tasks (W1(b)). TEMPO [6], which appeared after our submission, is one of the first steps in this direction. A policy can also help, but it needs extra labelled data (W1(c)).


We believe these answers address your main concern, including the soundness of our evaluation. We would appreciate it if you would re-evaluate the paper and consider raising your score. Its findings, and the weaknesses it reveals, should be of interest to the community, and especially to researchers working on world models that plan without a policy or reward.

Reviewer_c3q9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how far ahead a latent world model (frozen encoder + small action-conditioned predictor) can rank actions towards a goal. Using an "expert percentile" diagnostic on Meta-World and offline BridgeData, it finds that with a K=5K{=}5 rollout the usable range is 5–10 control steps while tasks need 16–53; predictor size and recursive training do not extend it, the encoder shifts it modestly, and only rolling out to the target does. A simulator oracle shows the same collapse, so the limit is attributed to target placement rather than prediction error. Closed-loop, expert subgoals five steps ahead raise MPC success from 30% to 76%, and world-model selection among eight π0 proposals raises success from 65% to 77%.

Critical strengths
  • This is an interesting problem.
  • The controls are well designed: the simulator used as a perfect predictor, an untrained-predictor control, matched-budget KK sweeps, and episode-paired McNemar tests.
  • Five encoders are compared under one shared predictor and planner, and the scope and limitations are stated honestly.
  • Table 7 shows that encoder rankings flip depending on the candidate distribution. This is a useful methodological caution for the community.
Critical weaknesses
  1. The main finding is expected and already documented. A greedy distance cost with horizon KK cannot rank actions toward goals far beyond KK. This is the classic myopic-MPC problem and the reason terminal value functions and hierarchical planners exist.
    • Recent JEPA work already shows that flat latent planners collapse at long horizons and that subgoals fix this (HWM, arXiv 2604.03208, the same frozen-feature planning setting).
    • Latent Euclidean distance is already known to misrepresent reachability (RC-aux, 2605.07278).
    • JEPA planning design choices have already been studied systematically (Terver et al., 2512.24497).
    • None of these is cited. The delta over them must be made explicit.
  2. P∗P^* is largely determined by the protocol.
    • Because JLJ_L takes a minimum over k≤Kk\le K, P^ is mechanically tied to KK (Table 4: P∈[K,2K]P^\in[K,2K]).
    • Clipped-Gaussian random negatives are trivially separable; the test saturates once K=LK{=}L.
    • The coarse grid L∈1,2,3,5,10,20L\in{1,2,3,5,10,20} makes the reported encoder differences ("5 vs 10") a single grid step.
    • The diagnostic uses only 4 articulated tasks, mostly one seed, and pooled tokens, whereas closed-loop control uses spatial token grids.
    • The claim that V-JEPA 2.1 is most consistent therefore rests on rankings that flip across protocols (DINOv2 goes from 1st to 4th) and is weakly supported.
  3. The positive results rely on near-oracle information.
    • Subgoals come from an expert rollout of the same reset in a deterministic simulator, where simply replaying the demo actions would likely succeed. The 76% and 65%→77% results therefore measure trajectory tracking, not planning.
    • The VLA also receives simulator object positions.
    • Needed: a demo-replay baseline, a demo-action-matching selector, non-oracle subgoals (generated or taken from another reset), and a comparison with world-model VLA steering such as DreamSteer (2607.02865) and GPC.
  4. The evidence is narrow for the claims made. Closed-loop results are Meta-World only, and success on new tasks is near zero. BridgeData V2 is evaluated offline only, with no real-robot control. Only the frozen-encoder family is tested, yet the title speaks of "latent world models" in general, and end-to-end models (TD-MPC2, Dreamer, LeWM, PLDM) are not evaluated.
Questions for authors
  1. What is the success rate of open-loop replay of the same-episode expert actions? What does a selector that simply picks the VLA candidate closest to the localized demo action achieve?
  2. Does P/KP^/K stay roughly constant across encoders as KK grows? If so, what does P^ measure beyond KK itself?
  3. Can you give the setup and numbers for the learned goal-scoring heads? Would a temporal-distance cost at K=5K{=}5, L=20L{=}20 close the 92%→41% gap seen with the simulator?
  4. Why are pooled tokens used for the diagnostic but spatial grids for control? Do the encoder rankings hold at full token resolution?
6 comments
AuthorsOpenReview ↗

Response to Reviewer c3q9: W1(a)–(b)

We thank the reviewer for the detailed review. We are glad the reviewer found the problem interesting and the controls "well designed" (the simulator as a perfect predictor, the untrained-predictor control, matched-budget sweeps and episode-paired McNemar tests). We are also glad the reviewer found the five-encoder comparison under one shared predictor and planner useful, and the scope and limitations "stated honestly". We also thank the reviewer for noting that Table 7 gives "a useful methodological caution for the community".

We ran the experiments the reviewer asked for in W3, Q1, Q3 and Q4, and each of them supports the paper's conclusions. The main results are listed here, and the details follow under each point.

  • A selector that picks the VLA candidate closest to the demonstration's actions reaches 0.725. The world model sees only images and reaches 0.772 (p = 0.011). It is at least as good as the action selector on all 16 tasks (Q1).
  • Without any subgoal, a longer rollout towards a goal image improves VLA selection (29 gains / 7 losses, p = 3.1e-4). This also holds when the goal image comes from another reset (24 gains / 10 losses, p = 0.024) (W3d).
  • With exact dynamics and the same 5-step rollout, changing only the cost to an oracle time-to-go raises success from 0.14 to 1.00. The limit therefore comes from the cost and not from prediction error (Q3).
  • At full token resolution the range stays between 5 and 10 control steps for all four encoders tested (Q4).

Using these experiments, we address each of the reviewer's concerns directly.


W1. "The main finding is expected and already documented"

(a) What the field expected. The usual explanation for long-horizon failure of world-model planners is compounding prediction error and a search space that grows with the horizon. HWM [13] states that "prediction errors compound and naive search is exponential in the planning horizon". Terver et al. [12] argue that compounding errors "grow exponentially with the horizon H". For the same reason, planners keep rollouts short (TD-MPC2 [4], DINO-WM [14], MBPO [5]). We test this explanation directly, and our results show that it is not the main limit.

  • With the real simulator (no prediction error), success still falls from 0.92 to 0.41 (165 paired gains, 0 losses).
  • An 81× larger predictor and 20× longer training rollouts do not extend the range. Ten times more search does not rescue three failed tasks (0/60 vs 0/60).
  • A longer rollout adds compounding error and a larger search space, yet it extends the range and improves control. A predictor trained on 5 steps still ranks the expert reliably 30 steps ahead. In closed loop, success rises from 0.30 to 0.47 at the same compute (57 gains / 5 losses).

These results go against the common design rule of short rollouts, so our finding was certainly not expected. It also matters for future work in this area, because it shows that the cost and the target placement, more than prediction error, decide how far these planners can reach. A later independent preprint (TEMPO [9], October 2026, after the ICLR submission deadline) reaches the same conclusion for end-to-end JEPA world models. It notes that "such failures are usually attributed to compounding rollout error and a search space that grows with the horizon".

(b) Myopic MPC and terminal values. This argument does not apply to our work. A terminal value function needs a reward. The models we study have no reward and no value function, only a distance to a goal image. Whether this distance gives a useful signal beyond the horizon is an empirical question. Along expert demonstrations, the latent distance to the goal falls almost monotonically (App. I, Fig. 13, with a median rank correlation of 0.98 for V-JEPA 2.1), so a greedy distance could have worked. We measure where it stops working (5–10 control steps when K = 5). This limit is not fixed by K. P∗P^* ranges from below K to about 2K across encoders and training (W2a), and once the rollout reaches the target the same cost ranks the expert first among 100 alternatives in every window (Table 4). The same limit appears with exact dynamics, so it is not caused by prediction error. We also show what moves the limit.

AuthorsOpenReview ↗

Response to Reviewer c3q9: W1(c), W2(a)–(c)

W1. (continued)

(c) The delta over HWM, RC-aux and Terver et al.

WorkWhat it showsWhat our paper adds
HWM [13] (a method)A learned hierarchy solves long tasks where a flat planner failsHow far apart the subgoals can be (0.76 at 5 steps, 0.40 at 20 steps). The same limit with exact dynamics. A longer rollout as an alternative to a hierarchy (0.30 to 0.47 at the same compute)
RC-aux [6] (a method)Retrains an end-to-end model (LeWM [8]) with multi-horizon prediction and reachability supervision, because "Euclidean distance may not reflect what is reachable"The failure also appears with the true state distance in the simulator, so it does not come only from the latent geometry. Multi-step training helps up to 5 steps (the range goes from 5 to 10) and then stops helping
Terver et al. [12] (a study, TMLR 2026)Which design choices give the most success when the goal can be reached within the planning horizonThe case they leave out, with goals beyond the horizon. Where the two studies overlap they agree (larger encoders and predictors do not help in simulation, and multi-step training gains stop early)

HWM and RC-aux propose methods, and our paper measures the limit that makes such methods necessary. Terver et al. state their scope in these words, "we sample an initial and goal state … such that the initial state attains the goal in H steps … Indeed, otherwise, the task … is not well solved with our planners." None of the three works tests the limit with exact dynamics, which is the control that separates the cost from prediction error. We also note that HWM is an arXiv preprint accepted at CoRL 2026. The conference is taking place now, and its proceedings are not yet available. We will cite and discuss all three in Section 2 and App. A.

We therefore strongly emphasize that it is not correct to regard our main finding as expected or already documented.

W2. "P∗P^* is largely determined by the protocol"

We are grateful to the reviewer for raising this point, but the concern does not hold. We clarify each bullet below, point by point.

(a) The definition does not fix P∗P^*. The abstract states that a world model guides action selection reliably "only when the goal lies within, or slightly beyond, the trajectory it imagines". Section 3 adds that the range "describes a particular model and planning configuration". A range that follows the rollout is therefore the result we report. The minimum over k ≤ K does not produce it by construction.

  • At K = 5, P∗P^* is 5 for three encoders and 10 for two. It is 5 for a predictor trained on one step and 10 for one trained on five (Fig. 3).
  • At the same K = 5, the per-task range goes from 5 to at least 20 control steps, and six tasks reach the largest lookahead we tested (Fig. 2b).
  • P∗P^* can be below K, with 0–5 on BridgeData and 0 for the released V-JEPA 2-AC (Sec. 4.2).
  • The minimum does not even guarantee better-than-chance ranking. On drawer-close at K = 5 and L = 20, the expert is at the 74.6th and 94.9th percentile (App. D.1).

Where P∗P^* falls depends on the encoder, the training, the task and the data. These are the properties we compare. The simulator shows the same pattern with exact prediction, and the pattern holds in closed loop.

(b) The random alternatives are the planner's own candidates. They come from the same clipped Gaussian that our CEM planner uses for its first candidates. The threshold of 10% equals the CEM elite fraction (10 of 100). So P∗P^* is the largest target distance at which the expert's action would be expected to survive the planner's first selection step. This is why the offline range predicts closed-loop results.

  • It predicted closed-loop success on 7 of 8 unseen tasks (App. J).
  • It predicted that a longer rollout would help, and closed loop confirmed this (App. E, from 0.169 to 0.369, 32 gains / 0 losses).

For harder negatives, App. F (Table 7) already uses perturbations of the expert's own actions at matched rollouts. The encoders separate again (13.0 to 35.4), and V-JEPA 2.1 still ranks the expert within the threshold 30 steps ahead (6.54 at σ = 0.25).

(c) The coarse grid. The main conclusion does not depend on the grid. Even the upper end of the interval (P∗P^{\ast} < 20 at K = 5) is far below the median task length (33 steps). We show the full curves (Figs. 2–3) and three thresholds (Table 2). The encoder claim does not rest on the one-step P∗P^{\ast} difference (see e).

AuthorsOpenReview ↗

Response to Reviewer c3q9: W2(d)–(e), W3(a)–(c)

W2. (continued)

(d) Four tasks, one seed, pooled tokens.

  • The per-task range is measured on all 16 tasks (Fig. 2b). We use the four articulated tasks only for the aggregate curves, because their demonstrations are long enough for lookaheads of 20–30 steps (App. P). The closed-loop results cover 16 + 8 tasks.
  • DINOv2's range is 10 on 3 of 3 seeds. The BridgeData curves average 3 seeds (Fig. 7). The learned-cost result is the same with a second seed (Q3). Closed-loop comparisons are paired over 320 episodes, with task-bootstrap intervals (Table 9).
  • The diagnostic and closed-loop control use the same predictor checkpoints. "Pooled tokens" means the encoder's patch tokens pooled to the 2×4×4 grid (2×7×7 for VideoMAEv2). This is the same spatial grid that control uses. Our sentence in App. B.1 was unclear, and we will fix it. Results with full-resolution tokens are in Q4.

(e) "V-JEPA 2.1 is the most consistent." This is a claim about consistency. It does not say that V-JEPA 2.1 is the best in every test. Rankings flip across protocols (as Table 7 shows), so we compare the encoders in three ways. V-JEPA 2.1 is the only encoder in the top two in all three. This narrow claim is our reading of the flip the reviewer points out.

W3. "The positive results rely on near-oracle information"

We thank the reviewer for raising this. We clarify below why it is incorrect to conclude that our positive results measure trajectory tracking rather than planning.

(a) The subgoals are a deliberate control, and the paper says so. Section 7 is not a method contribution. It gives initial results for a new way to use these world models that follows from our main finding. The world model is used off the shelf, with no retraining, as a critic within its plannable range, and an existing policy handles the rest. The subgoals are what place the target inside that range. The paper states this where the results appear. Section 6.3 says that obtaining the subgoals without an expert rollout "remains an open problem", and Section 7 says that the system "is not yet a controller that works without demonstrations". HWM [13] uses the same control. Its App. E gives manual subgoal images to the flat planner in order to separate planning from representation (W4b). We will also state this on the first page, by adding "with expert subgoals" to the abstract sentence on the VLA result.

(b) The gain does not come from tracking the demonstration's actions. The planner never receives the demonstration's actions, only images. It must find the actions with its learned dynamics.

  • Tracking the demonstration's actions does less well. A selector that picks the VLA candidate closest to the demonstration's actions reaches 0.725. The world model, which sees only images, reaches 0.772 on the same 320 episodes (23 vs 8 discordant episodes, p = 0.011), and it is at least as good as the action selector on all 16 tasks (Q1).
  • With the same subgoals, an untrained predictor succeeds in 0.8% of episodes (standalone planner) and 3.6% (VLA–WM, where the VLA alone reaches 21.4% on these 7 tasks). The subgoals are therefore worth little without the learned dynamics.
  • With the same demonstration but subgoals 20 steps ahead, success is 0.40 (learned) and 0.41 (simulator), against 0.76 and 0.92 at 5 steps. The information is the same and only the target distance changes. This distance effect is what the experiment measures.

(c) Object positions for the VLA. The VLA's state input includes object positions (App. B.2). This input is identical in the VLA baseline and in VLA–WM, because the same policy proposes the candidates. The world model sees only images. This input makes the baseline stronger, so it cannot explain the gain.

Reviewer_bNP4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
3(+2)
Read review
Paper summary

The authors study the usefulness of world models for planning as a function of the temporal distance of the goal state. Five different models are tested and compared on simulated and real world data and the ability to plan with the world model is tested in terms of whether it ranks expert action sequences above random ones. The authors then make an effort to study which factors and design choices have an impact on planning ability, including distance to target goal state, capacity, longer rollouts and intermediate goals.

Critical strengths
  1. This work proposes the evaluation of world models from the standpoint of how well they allow a planner to correctly rank action sequences which is a relevant research direction
  2. The authors make an effort to perform scientific testing of widely used algorithms, something that is often undervalued by the community in favour of chasing SOTA results in benchmarks
Critical weaknesses
  1. The choice of cosine similarity as the action sequence scoring function does not seem the most appropriate given that states that look perceptually similar may be far apart in terms of planning. This is especially the case in manipulation tasks such as the ones that are part of Meta World as discussed in [1]. My concerns regarding the used metric affect the soundness of both the results of Section 4 and Section 5, and the conclusions therein. This issue is actually discussed by the authors in Section 5.5 and Appendix I, further supporting the shortcomings of cosine similarity in the context of this study.
  2. The writing on the paper can be improved to better transmit and justify the ideas and design choices. For example, why do the authors choose the 3 questions outlined in section 6? Why is it relevant to test open-loop planning? I found It hard to follow many of the author’s choices. The level of rigour with which methods are described should also be improved. As an example, in section 4.3 the authors write “by asking [the world model] to identify the executed action sequence (…)” makes it seem that a world model is a classifier.
  3. Section 7 seems detached from the rest of the paper in terms of answering the research question of how far ahead world models are useful for planning. Why transition from studying the inner workings of world models to combining world models with VLAs? There is also no Related Work provided for this section despite it being the object of active research in the community. Could the authors justify the need for having this section in the context of the main research question?

References:

[1] Mendonca, R., Rybkin, O., Daniilidis, K., Hafner, D., & Pathak, D. (2021). Discovering and achieving goals via world models. Advances in neural information processing systems, 34, 24379-24391.

Questions for authors
  1. Would it make sense to study world models that also learn reward models, such as the Dreamer family and TD-MPC family? This could allow using the reward model of the expert instead of the cosine similarity, possibly avoiding the issues related with cosine similarity
7 comments
AuthorsOpenReview ↗

Response to Reviewer bNP4 (W1)

Thank you for your review. We are grateful for reviewer's time and well-thought out comments, we are also glad you found that evaluating world models "from the standpoint of how well they allow a planner to correctly rank action sequences" is "a relevant research direction", and that you valued our "effort to perform scientific testing of widely used algorithms".


W1. The choice of cosine similarity

Please note, the latent distance cost is not our choice. It is a design choice made by the authors of the world models we study. Conventional RL based world models, such as Dreamer [5] and TD-MPC2 [6], learn a reward model and a policy or value function for each task or environment. The emerging class of models we study learns no policy and needs no reward (V-JEPA 2-AC [1], DINO-WM [18], NWM [2], PLDM [14], HWM [17]). These models learn only how actions change the world, in latent space, and they plan by making the imagined future close to a goal image. This direction is promising because it needs no rewards, which could be hard to obtain outside simulated benchmarks, and because it can transfer to new environments. For example, V-JEPA 2-AC [1] was deployed "zero-shot on Franka arms in two different labs … without any task-specific training or reward", and DINO-WM [18] plans zero-shot "without expert demonstrations, reward modeling, or pre-learned inverse models". Our paper evaluates this family (we note this in abstract, Sec. 2 and App. A.1; see also Q1).

In this family, a latent distance to the goal image is the standard planning cost. V-JEPA 2-AC uses an L1 distance, DINO-WM and PLDM use L2, and HWM uses latent matching; Terver et al. [15] compare L1 and L2 costs across planners for this family. Cosine distance is the same kind of cost, since on normalised features it equals half the squared L2 distance. We therefore test the planner that these models actually use. We have also tested other planning costs. Over 421 matched goal settings, squared Euclidean distance gives the same plannable range as cosine for four of the five encoders and a shorter range for the fifth (Sec. 5.5, App. I). In App. H we note that cosine performed at least as well as learned goal-scoring heads trained on demonstrations in closed loop. Details in (a) below.

(a) Cosine is a good cost. We have tested multiple learned costs but they did not beat it. While emphasizing that cosine similarity is not 'our' choice, we acknowledge that it has weaknesses (which is central to our study in this paper), as also pointed out in [9]. However, it also has strengths that make it a widely adopted choice of the family of models we study. It needs no data, labels or rewards, so it works zero-shot on any task with a goal image. It works well when the imagined rollout reaches the target, and it transferred to new tasks better than the learned costs we tried.

For example, we trained a VIP-style head [8], which learns a goal-conditioned temporal distance, on expert demonstrations of the 13 training tasks. We then planned with it in place of cosine, with the same world model and planner, on the 16 main tasks (the 13 training tasks and 3 held-out tasks for 20 episodes per task). It gave lower success than cosine, 0.188 vs 0.397 with subgoals 20 steps ahead and 0.144 vs 0.250 with the image of the completed task. It was worse even on the tasks it was trained on, and with subgoals a second predictor seed gave the same result (0.206 vs 0.375).

In new experiments with the same world model, we also tested other learned temporal costs. Costs learned without simulator labels did not beat cosine. A cost learned with simulator labels helped on the trained tasks, but on 10 new tasks it reached 0.19, against 0.35 for cosine.

In addition, the reference you mentioned and later work agree with this. In LEXA [9], cosine beats the learned temporal distance in 2 of the 4 domains (Walker 73.1 vs 36.7; Quadruped 56.1 vs 31.4). LEXA also learns its temporal distance separately for each environment, from the agent's own imagined rollouts, and uses it as an RL reward rather than as a zero-shot planning cost. TEMPO [10] (October 2026, after ICLR's submission deadline) is one of the first works to try to improve on the latent distance in reward-free world models. It learns a temporal cost for each environment, and it still blends that cost with the latent distance, because "the latent term carries the fine positioning that manipulation needs". We will cite [9] and [10] in Section 5.5.

In summary, the latent distance cost is not our choice. It is the design choice of the authors of the reward-free, policy-free models we test. The planning cost is therefore a carefully considered and central component of our paper's systematic evaluation.

AuthorsOpenReview ↗

Response to Reviewer bNP4 (W2 & W3)

W2. Writing and design choices

(a) Why Section 6 asks three questions, and why open-loop planning matters. Section 6 asks how well current world models without a policy or reward can plan a complete task. Pure imagination (open-loop planning) is the baseline since the model sees only the first observation and then plans only once without any feedback or replanning. This is the most efficient setting, and a model that succeeds here can plan over long horizons efficiently. The other subsections test what improves this baseline, whether it is observation feedback, nearby subgoals or a longer rollout. Recent work has started to move in these directions, with visual subgoals [19] and longer planning horizons [17]. For example, ProWAM [19] (October 2026, after the ICLR submission deadline) predicts a sequence of visual subgoals together with its actions, and it clearly outperforms the strongest baseline in zero-shot real-world tests in new scenes. These directions are therefore an important part of our study. We will state this more clearly at the start of Section 6. We thank the reviewer for pointing this out, as it will make the paper easier to read.

(b) "Identify the executed action sequence" (Sec. 4.3). We used this wording because this specific experiment treats the world model as a predictor of good actions. The aim is to test how well the world model finds a good action when one exists among the candidates. We roll out the executed action sequence and 15 random sequences through the world model, and the test succeeds if the executed sequence gives the predicted future closest to the real future. We will rephrase the sentence to make this clear.

(c) Rigour. Section 3 defines the score (Eq. 1) and the percentile. App. C gives the estimator (Eq. 2), and App. B.3 with Table 1 gives every planner setting. We will add the following text to Section 3, after the definition of P∗P^*. "The random alternatives come from the same distribution from which the cross-entropy method draws its first candidates at the start of an episode, and the planner keeps the best 10 of its 100 candidates (App. B.3). The percentile therefore ranks the expert against the same kind of candidates that the planner compares, and the 10% threshold is the share of candidates that the planner keeps."


W3. Why Section 7?

(a) Section 7 shows a practical use of our main finding. Thank you for raising this important question. Our results show that a world model guides action selection reliably only within its plannable range. Section 7 asks whether world models, used within that range, can improve a policy that is already trained, by imagining the outcome of the policy's proposed actions and acting as a critic. We test this with a vision-language-action (VLA) model, since VLAs are now among the strongest approaches to robot manipulation [3, 7]. The VLA proposes a few candidate actions, the world model imagines and scores the outcome of each, and the robot executes the best one. Nothing is retrained for this.

To be clear, Section 7 is not claimed as a method contribution. It is part of the evaluation, and it provides interesting new results for the field on the emerging use and integration of these world models with policies, building directly upon our core findings in this paper. The world model can make a strong policy more reliable if it is used within its plannable range. We will make this link more explicit at the start of Section 7.

(b) Related work, and how we differ. The paper already discusses this related work. Sec. 2 (third paragraph) covers the methods that use predictive models to check VLA proposals before execution, and App. A (fourth paragraph) discusses them in more detail. We will also add V-GPS [11]. Using a world model to check a policy's proposals is a new and active direction. GPC [13] and FOREWARN [16] are both from 2025–2026, which supports including Section 7. These works add a model that scores the predicted outcomes for the task, such as a reward model (GPC) or a vision-language model (FOREWARN). Our critic needs no such extra scorer. It uses the same world model and the same goal-image distance as in Sections 4–6.

AuthorsOpenReview ↗

Response to Reviewer bNP4 (Q1 and remarks)

Q1. Would Dreamer or TD-MPC with reward models make sense?

Thank you for this query. It would indeed make sense to study RL world models in the same way as we study models without a reward or a policy. Such a study is, however, outside the scope of this paper, because these models follow a different approach from the family we study. They plan with a learned reward or value function instead of a distance to a goal image, so their evaluation would need its own criteria. We therefore keep the two studies separate.

RL world models are strong where they are trained. Dreamer and TD-MPC2 learn reward and value functions for a given task, and with rewards they perform very well in the environment they are trained on. They need reward labels for every task, however, and they are trained for each task or environment. For example, LEXA (the paper the reviewer cited) states that "we trained a new agent for every new environment". Without rewards, they do not work. DINO-WM trained DreamerV3 and TD-MPC2 on the same reward-free data, and TD-MPC2 reached 0.00 success, which the authors attribute to "the lack of reward signal".

Reward-free world models generalise better. PLDM [14] finds that "model-free RL benefits most from large amounts of high-quality data, whereas model-based planning generalizes better to unseen layouts". HWM notes that goal-conditioned RL methods "generalize poorly beyond training", and on a real Franka arm with subgoal images it reports 80% success for the reward-free V-JEPA 2-AC planner, against 20% or less for the Octo [12] policy given the same subgoals (HWM [17], App. E).

Our own results show the same trade-off. For the purpose of the rebuttal, we trained a goal-conditioned policy on expert actions (behaviour cloning) using the same frozen features. On the 13 training tasks, it outperformed every planner (0.92–0.96 vs. 0.54 for cosine planning with a 30-step rollout), but on 10 new tasks, it reached only 0.13–0.16, below cosine planning (0.35). Therefore, these experiments show that using such additional data (if available) can be highly advantageous for task- or environment-specific performance.

In conclusion, reward-based models may solve some of the weaknesses that appear with the cosine cost, but they also expose weaknesses of their own, because the improved performance comes at the cost of extra data (could be hard to obtain) that the family of world models we study in this paper does not use, and they become task-specific. Our paper, therefore, focuses on the reward-free, policy-free family, which aims to generalise beyond its training tasks. Adding RL-based world models is outside the scope of this paper. App. A.1 already names RL world models as future work.


Thank you again for your careful review and your time. We believe these answers address your main concerns, in particular about the soundness of our paper. We have run extensive studies and experiments on this family of world models, and the paper gives the community useful findings and clear weaknesses to work on, especially for researchers who build world models without a policy or reward. We would therefore be grateful if you would consider re-evaluating the paper and raising your score.

Comments on the paper

1 comment

Dear reviewers,

Thank you for your reviews, and thank you to the authors for their responses. Please read the authors’ rebuttal and discuss whether it addresses your main concerns. Please update your review and rating if warranted, and briefly explain your current assessment, including any concerns that remain unresolved. If your assessment is unchanged, please explain why.

Thank you for your continued engagement.

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.