acceptodds
Withdrawn from ICLR 2027

Reachability, Not Resemblance: Learning the Planning Geometry of Latent World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

12 views

Abstract

World models (WMs) let an agent act by imagining the consequences of candidate actions. Latent WMs roll candidate action sequences forward in latent space and keep the one whose predicted final latent lies nearest the goal, so the distance behind “nearest”, the planning geometry, is what planning optimizes. Current models, LeWorldModel (LeWM) among them, use a plain Euclidean norm in an encoder trained only to predict and to avoid collapse, a geometry with no reason to reflect temporal reachability, and whether the mismatch matters has never been measured. Here we show that the mismatch is benign for near goals and dominant for far ones, and that it can be repaired without retraining: Geometry for World Models (GeoWM), a head with of the parameters fitted to the frozen encoder so that consecutive states lie within one unit and random pairs far apart, approximates temporal distance and, read as the terminal cost, restores long-horizon planning at no added planning cost. On four benchmarks the head matches or exceeds the baseline at the standard goal offset and separates from it as the goal recedes, raising success from to on Push-T and from to on Two-Room at four times that offset. The gain is in the cost, not the representation, and the head transfers unchanged to a WM of another family. The planning geometry is thus the lever for long-horizon planning, the one component prior work has held fixed while improving the rollout and the search around it. More broadly, what a planner is asked to measure matters more than the machinery around it: plan by reachability, not resemblance.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, latent planning, model predictive control, quasimetric learning, temporal distance, goal-conditioned reinforcement learning, long-horizon planning, JEPA

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kkyXOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Since the modern latent world models often optimize their planning trajectories via Euclidean distance, the planning often has to rely on some unrealizable path that shows shorter Euclidean distance. The authors’ idea is to attach a lightweight head that measures “reachability”, which measures how many steps are required to go from a latent to another. It reflects the quasi-symmetric nature of the norm of reachability while satisfying triangular inequality, which seems to be a source of extrapolation capability that authors insist.

Critical strengths
  • The motivation is very sound and makes a good sense.
  • Essentially the way “reachability” is realized in this paper is to treat a distance between the two latents from an observed (s_t, a_t, s_{t+1}) triplet shorter than those not connected with an action in the train data, which sounds like an elegant and simple idea to me.
  • The results on longer-horizon tasks seem convincing.
Critical weaknesses
  • I do not necessarily think this method should bind to LeWM, as there are many other latent planning papers (e.g., [1, 2]) published recently. The training will not be heavy since it is primarily a framework attached on a pre-trained network and rely on simpler objective. Expanding this paper’s application and showing generalizability would have made this paper much stronger.
  • The reason for bringing “quasimetric” property of the measure isn’t really convincing; in the author’s claim in Sec 3.4, the asymmetric property of the distance does not seem to be an essential element. If so, please clarify.

[1] A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens (Kerssies et al., CVPR 2026)

[2] Planning in 8 Tokens: A Compact Discrete Tokenizer for Latent World Model (Kim et al., CVPR 2026)

Questions for authors

My question is primarily about its ability of extrapolation. The paper attributes the head’s extrapolation beyond the 3-block training window to the triangle inequality propagating the local constraint along trajectories, grounded in the QRL characterization of Wang et al. (2023). But:

  • Eq. (4) is a relaxation of that constrained program (fixed μ\mu, e−de^{−d} push), while the direction of the fixed-point deviation is unknown.
  • The matched-capacity unstructured head (Table A10) fails by collapsing to a trivial distance, not by violating the triangle inequality.
  • Such result supports “the MRN form excludes a degenerate solution” more directly than “the triangle inequality propagates local constraints”. Yet, Sec 3.3 claims the latter as its primary source of extrapolation.

Regarding above,

  1. Which of the two mechanisms are the authors claiming, and is the claim from Sec. 3.3 compatible with what Sec. M shows?
  2. Is there a control separating them, e.g., a head with the triangle inequality removed but d(x,x)=0d(x,x) = 0 and non-collapse enforced otherwise?
  3. On held-out trajectories, is the fitted dψd_\psi actually a lower bound on recorded path length at long range (the chain bound d(s0,sn)≤n)d(s_0, s_n) \leq n)?
Reviewer_rq7GOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argue that the euclidian norm commonly used by latent world model to do visual planning only reflect the similarity between the predicted state and the goal in latent space, not temporal reachability, which matters for longer horizon planning. The authors propose GeoWM, a lightweight quasimetric head trained on top of the frozen world model. This projection trains the planning geometry to approximate temporal distance by shaping consecutive state to be less than a unit of distance away with a hinge, while pushing away random pairs contrastively. This head is then used as a terminal cost directly, with the world model and the planner fixed. GeoWM, as well as the experiment settings, build on top of LeWM: they evaluate their proposed method on Push-T, Two-Room, Reacher and Cube while sweeping the goal offset to show the benefit in long horizon as the goal recedes. At the protocol offset of LeWM (25 steps), the quasimetric head performance is close to simply using euclidean norm, but when increasing the offset (100 steps), GeoWM reaches 38% SR against the 16% SR baseline on Push-T, and 94% SR against the 38% SR baseline on the Two-Room. The paper further supports this with direct measurements of the cost geometry and ablate the contribution of the cost to the planning performance rather than the representation or the search. They also show transferability of their proposed fix to PLDM on Push-T only.

Critical strengths
  1. A simple and lightweight method to tackle long-horizon planning with latent world models. With representation, dynamics and planner fixed, the author cleanly isolate the effect of replacing the terminal cost. This makes me confident that the long-horizon effect is real on the four benchmarks and attributable to the cost.
  2. Experiments and controls are detailed, conducted rigorously and delivered honestly, with in-depth statistical analysis of results.
Critical weaknesses
  1. The attribution of the gain is not clearly established against natural existing alternatives aiming to tackle the same issue (retaining long-horizon rollout capabilities). The paper concludes that the gain comes from injecting temporal reachability instead of the representation. But some controls are missing:
  • Evaluating very naive learned readouts such as a learned linear projection or MLP, trained with the same loss
  • The joint objective shapes the head's projection, not euclidean geometry on the encoder output latents, so recovery is surely expected. No comparison arm shapes the representation space directly with existing methods such as temporal straightening.
  • Tab. A4 shows LeWM’s cost correlates poorly with geodesic distance even where the wall is irrelevant (ρ = 0.66, vs. 1.00 for straight-line distance), while agent position is linearly decodable (Tab. A14). Much of the repair may therefore come from discarding task-irrelevant directions rather than from learning reachability. The asymmetry is also shown to matter in a single of four benchmarks.
  1. The evidence is not clearly supporting the generality the paper claims. All four benchmarks are low-dimensional simulated tasks in which long-horizons are artificially created along existing recorded trajectories. Transfer is shown on one additional backbone (PLDM) and one additional task only (Push T), with the harder image-based suite showing no gain. The paper therefore supports a consistent effect in this regime but not the claim that the cost alone is the lever for long-horizon planning.
Questions for authors
  1. Eq. (1) include a stop-gradient on the prediction target. LeWM's original motivation for introducing SIGReg was to find a collapse prevention mechanism which do not rely on the stopgrad, which they did remove. Did the author re-introduce the stopgrad in their experiments here? If so, why? and does it affect the joint arm?

  2. I am curious to better understand where the overhead of 40% in the joint-training setup comes from if the head is only 0.6% additional parameters with regard to the entire model, this seems counter-intuitive.

  3. Figure 1. misrepresents how the baseline fail: the euclidean geometry shown is only true if the distance was computed in pixel space, but the actual cost is euclidean in a latent space learned during joint training of the encoder. Figure 4. directly contradicts this, showing a patchy map with local minima. Can the author clarify figure 1?

  4. How does Euclidean distance after a learned projection (linear & MLP), trained with the same setup, compared on the offset sweep? If performance is close then the gain could mainly come from discarding task irrelevant directions rather than actually encoding reachability.

  5. If the proposed method uses negative pairs, why this objective over a standard InfoNCE head at matched capacity?

  6. Asymmetry only helps on Push-T, which is a mild asymmetric case. Can the authors test on an environment with a clear one way transition? Something like adding one-way traversal in Two Room for example.

Leaning towards improving my score if all questions are answered with well-supported evidence

Reviewer_nsVWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper argues that world models (WMs) are usually trained only to predict the next state, so Euclidean distance in the latent space measures resemblance between two states rather than reachability. When there's an obstacle between the start and goal states, relying on resemblance alone is wrong. To counter this, the paper proposed GeoWM, which adds an extra head on top of the encoder to output a quasi-metric distance. Across four environments, GeoWM aligns better with reachability and performs better overall than the standard WM.

Critical strengths
  1. The paper clearly explains the main issue in the current world models and the motivation behind adding a separate head, which involves reachability.
  2. Figures 1 and 4 are helpful to visualize the notion of reachability that GeoWM captures and where standard WMs fail.
  3. The ablation suite (frozen vs joint head, symmetric head, unstructured head, and reversed head) helps isolate the impact of components of GeoWM.
  4. The paper is very honest about all its limitations and describes the future work adequately.
Critical weaknesses
  1. The notion of reachability is not new. In fact, the proposed method is similar to the notion of contrastive learning, in which states that states that lie close to each other are pulled together, and a random state is pushed apart. In the GCRL literature, multiple algorithms, such as CRL [1], have been developed based on this idea. In fact, the Laplacian representation [2,3] inherently captures the notion of reachability and has proven useful for planning [4]. So, a more apt comparison for GeoWM should be with these. TRM [5] is also a concurrent work cited in the paper but not compared against.
  2. Isn't learning a quasimetric hard on its own? The paper's own results suggest the learned head only partially captures reachability outside the simplest environments. Quasimetric RL also struggled with the manipulation tasks on OGBench, and a similar behavior was observed in the cube task, where the rank correlation with steps-to-go only improved from 0.561 to 0.613. The same applies for reacher as well.
  3. All the results presented in the paper are with three seeds, which makes it difficult to assess robustness to training variance. Although Table 12 includes bootstrap intervals and McNemar counts, they only capture evaluation noise, not the training variance itself.
  4. None of the environments considered are actually long-horizon, which was the main motivation behind the paper. Adding results for the large and giant settings on OGBench, the cube-double/triple tasks will substantially improve the paper. Pixel-based variants can also be added for scalability.

References:

[1] Contrastive Learning as Goal-Conditioned Reinforcement Learning. Eysenbach et al. (NeurIPS 2022)

[2] Proto-Value Functions: Developmental Reinforcement Learning. Mahadevan S. (ICML 2005)

[3] Proper Laplacian Representation Learning. Gomez et al. (ICLR 2024)

[4] Laplacian Representations for Decision-Time Planning. Shehmar et al. (ICML 2026)

[5] World Model Control by Trajectory Reachability Metrics. Li et al. (2026)

Questions for authors
  1. While learning the quasimetric, why is just the next state pulled together, why not a geometric distribution over the future states in the trajectory?
  2. In QRL, the dual variable μ\mu, which is adapted, helps in making the constrained problem better. Why is this variable fixed while learning the quasimetric in GeoWM?
  3. Although the paper mentions that μ=5\mu=5 and β=1\beta=1 were the first values that were tried and never swept over, what was the thought process behind setting these values? Adding a sensitivity analysis on these parameters will show whether the method is robust or whether those values just happened to work.
  4. Figure 4 does show that GeoWm captures reachability distance better than LeWM, but this distance still needs improvement. In the left room, the top-right corners are greener, similar to the top-left corner of the right room, which suggests that reachability distance is still not fully preserved.

Minor Suggestions:

  1. Equation 4 underbraces are not being aligned properly.
  2. Section 2 and Appendix A mischaracterise TD-MPC/TD-MPC2, it does plan through a latent model with a learned terminal value.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.