acceptodds
Withdrawn from ICLR 2027

World Model Control by Trajectory Reachability Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Latent world models can learn representations that contain information needed for control, while the downstream controller may still rank candidate actions poorly when it relies on terminal latent distance alone. We study this failure in a fixed encoder and introduce a trajectory reachability model (TRM), a small temporal pairwise cost trained from logged trajectories and used to rank predicted endpoints of a candidate action sequence against a goal. In the TwoRoom evaluation on 100 episodes from a high distance range, the original controller with LeWorldModel reaches 7.0% mean success. Temporal TRM trained after excluding all evaluation episodes reaches 96.7%, shuffled label controls stay at 0.0%, and TRM also improves PLDM from 32.7% to 84.0%. TopoNav, a separate pixel navigation task, shows the same link between repaired ranking and closed loop control. The selection audit with shared candidates (SASC) and rowspace interventions localize the ranking failure to underweighted latent directions carrying the action decision. The XY rowspace contributes less than 1% of terminal latent MSE but carries most of the information needed for control. Coverage and boundary tests define the scope. Balanced doorway coverage restores a 0.0% coverage failure to 100.0% success, while an unseen wall orientation and contact rich PushT expose layout, dynamics, and recovery limits.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, trajectory reachability, latent representations, action selection, planning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_UFofOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the failure of Euclidean latent distance as a terminal cost for planning with pretrained world models and proposes a Trajectory Reachability Model (TRM) to improve action selection. TRM learns a pairwise temporal-distance predictor from logged trajectories and uses it to rank predicted terminal states without modifying the frozen encoder, dynamics model, or controller. The authors further introduce a selection audit with shared candidates (SASC) and subspace interventions to analyze why task-relevant information can be preserved in latent representations but underweighted during planning. Experiments on TwoRoom demonstrate substantial improvements in navigation success, with additional evaluations on PLDM, TopoNav, and PushT examining transferability and limitations. The results suggest that appropriately learned terminal costs can significantly improve latent-space planning, although their effectiveness depends on trajectory coverage, model characteristics, and task dynamics.

Critical strengths
  1. The paper provides useful evidence that the poor performance of Euclidean latent distance is caused by the underweighting of task-relevant latent directions rather than the absence of necessary information in the learned representation. In particular, Table 11 shows that the XY-related subspace accounts for less than 1% of the total latent squared distance, yet planning with this subspace alone achieves substantially higher success rates than the original latent distance. Together with the shared-candidate ranking audits, these results provide a convincing explanation of the terminal-cost failure and represent an informative contribution to understanding planning with latent world models.

  2. The paper demonstrates substantial improvements in the main TwoRoom evaluation while keeping the world model and controller fixed. In particular, Table 22 shows that TRM improves success from 7.0% to 96.7% even after excluding all evaluation-source episodes from training, whereas the matched shuffled-label control achieves 0.0% success. These results, together with the candidate-ranking analysis and temporal-horizon ablations, provide compelling evidence that the improvements are related to meaningful temporal supervision rather than merely introducing an additional learned prediction head.

Critical weaknesses
  1. Although the main TwoRoom results are impressive, the effectiveness of TRM varies substantially across checkpoints and evaluation settings. In particular, Table 40 shows that, among seven additional checkpoints, four improve and two remain unchanged, with the confidence intervals for these six comparisons including zero, while one checkpoint experiences a significant performance degradation from 90% to 60%. The underlying cause of this failure is not sufficiently explained. Furthermore, the evidence for applying the default temporal TRM beyond TwoRoom remains limited: TopoNav uses hand-designed features, exact dynamics, and shortest-route injection, while PushT relies on task-state distance supervision rather than the proposed temporal labels. Although these experiments are informative diagnostics, they do not fully establish the robustness or general applicability of the proposed temporal reachability approach to more challenging learned world models.

  2. The central idea of learning a goal-conditioned distance or reachability function for visual planning is closely related to existing work, including Model-Based Visual Planning with Self-Supervised Functional Distances (Tian et al., 2021) and Learning Temporal Distances through Contrastive Successor Features (Myers et al., 2024). While these approaches are discussed in the related work, the paper does not provide direct experimental comparisons that isolate the benefits of TRM's architecture, temporal supervision, and pair-sampling strategy. The current comparisons primarily involve raw latent distance, simpler metric transformations, and alternative supervision applied to the proposed pairwise head. A properly adapted baseline using the same frozen world model, offline trajectory data, and planning budget would be important for establishing the incremental technical contribution and novelty of TRM relative to existing learned-distance methods.

Questions for authors
  1. What causes the substantial performance degradation for one checkpoint in Table 40? The epoch-3 checkpoint experiences a decrease in success from 90% to 60% after applying TRM, while other checkpoints show improvements or unchanged performance. Could the authors investigate whether this degradation results from differences in latent representation quality, world model rollout accuracy, temporal-distance prediction errors, or candidate ranking? A more detailed explanation would help determine under which conditions TRM can be applied reliably and would materially affect my assessment of its robustness.

  2. Can the authors evaluate the default temporal TRM on PushT using the same experimental protocol? The current PushT experiments use task-state distance supervision, which differs from the temporal supervision proposed as the main contribution. Could the authors extend the existing offset-50 experiment by comparing raw latent distance, temporal-TRM hybrid, and shuffled-temporal hybrid under the same frozen world model, controller settings, and hybrid weight? Ideally, evaluation-source episodes should be excluded from temporal pair training, and success rates and final task distances should be reported across the existing evaluation seeds. This comparison would provide more direct evidence of whether the proposed temporal supervision generalizes beyond navigation tasks.

  3. How does TRM compare with existing learned functional or temporal distance methods under matched conditions? Could the authors include an appropriately adapted baseline based on the learned-distance component of Tian et al. (2021) or Myers et al. (2024), using the same frozen representation, offline training data, and CEM planning budget? Such a comparison would help clarify whether the observed improvements arise from the specific TRM formulation and full-horizon sampling strategy or from the more general benefit of replacing Euclidean distance with a learned reachability cost.

Reviewer_1EENOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The main idea of this paper is to improve planning in goal-conditioned world models using a trajectory reachability metrics (TRM). JEPA-like world models consist of two parts: learned representations and a planning algorithm, referred to as a controller in this paper. Default planning in this class of models has several weaknesses, including the “conditional mean” problem and reachability estimation. The authors focus on the latter.

The authors propose a reachability head intended to score endpoint reachability. The proposed method substantially improves performance, and the paper includes an analysis of the learned reachability head. However, the algorithmic novelty is limited, comparisons with related methods are missing, and parts of the paper are difficult to read.

Critical strengths
  1. The algorithm is simple and easy to reproduce, understand, and evaluate. It substantially improves baseline performance and provides a useful connection between reachability research and JEPA-based world modeling.
  2. The motivation is clear, and the paper addresses a real problem. The intervention is well controlled, and the analysis extends beyond success rates.
Critical weaknesses
  1. The algorithmic novelty is limited. In my view, the contribution can currently be characterized as the application of reachability methods to JEPA-based world modeling. This calls for a more thorough comparison with related work on value-based planning in world models, such as [1, 2, 3] (the real pool of paper wider).

  2. The paper lacks comparisons with other temporal-cost and reachability methods. In particular, comparisons with IQL and other established approaches from offline reachability research are missing.

  3. Some parts of the paper are difficult to read. For example, listing exact seed values makes the tables harder to read; these details could be moved to the appendix. The XY notation appears without an explanation. Although its meaning can be inferred from context, this makes the experiments harder to follow. Sections 3 and 4 could be combined, with clearer definitions and more polished presentation. For example, tit_i is not introduced in the text, although its meaning can be inferred.

[1] - Discovering and Achieving Goals via World Models

[2] - Value-guided action planning with JEPA world models

[3] - Model-Based Visual Planning with Self-Supervised Functional Distances

Questions for authors

My main questions concern the weaknesses listed above.

I would be open to raising my score if the authors provided a more thorough comparison with related work on reachability in world models and goal-conditioned RL, and improved the clarity of the presentation (W1, W2 and W3).

Reviewer_4pwGOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates temporal distance learning as an alternative cost function for latent-space planning in world models. Specifically, a separate network is trained on trajectory data to predict the number of time steps required to reach a given goal latent state. The predicted temporal distance is then used as the planning cost instead of the Euclidean distance in latent space. The paper demonstrates that replacing the cost function alone, without modifying the world model (i.e., the encoder and latent dynamics), substantially improves planning performance.

Critical strengths
  • The proposed cost function is intuitive and improves performance on challenging tasks. Furthermore, the proposed cost is shown to correlate with the oracle cost.
  • The paper provides an interesting investigation into the limitations of Euclidean distance in latent space. In particular, it demonstrates that although the latent representation encodes information about the oracle positions, Euclidean distance in the latent space does not adequately capture their relevance for control. The latent dimensions associated with position, which contain most of the information required for navigation, account for less than 1% of the overall latent Euclidean distance.
Critical weaknesses
  • Limited novelty: The proposed method is relatively simple, relying on regression to predict the number of time steps required to reach a goal latent state.
  • Limited baselines: The evaluation would benefit from comparisons with other temporal-distance estimation methods, particularly those used in goal-conditioned offline RL.
  • Poor presentation: The paper would benefit from substantial improvements in its presentation and organization.
    • Several terms (e.g., selection audit with shared candidates) are introduced in the abstract but only explained later in the paper, making the abstract difficult to understand.
    • The motivation and contributions of the paper are not sufficiently clear from the introduction alone. The introduction becomes meaningful only after reading the entire paper.
    • Some tables and figures provide limited information and could be improved to better communicate the results.
    • In Equation 6, using std may be misleading, as it is commonly understood to denote standard deviation, whereas the operation appears to be z-normalization.
Questions for authors

1. Regression and multimodal temporal distances: Since the Trajectory Reachability Model (TRM) uses regression to predict temporal distances, its predictions approximate the expectation of the distance between trajectory points. How does this affect performance when the temporal distance between two points (A and B) is multimodal in the dataset?

2. Comparison with Q-learning in goal-conditioned offline RL: How does TRM compare with Q-learning approaches commonly used in goal-conditioned offline RL, where a non-terminal reward of -1 is assigned at each time step? In principle, Q-learning can learn shortest-path distances through dynamic programming, even when the dataset contains trajectories with different lengths between the same states.

3. Feature design in Equation 3: What is the motivation for the feature map introduced in Equation 3, particularly given that the features appear to be dependent? Have the authors investigated a simpler feature map using only [zi,zj][z_i, z_j]? How does this compare with the proposed feature map in terms of performance?

4. Visualization of learned distances: Could the authors provide a visualization of the training trajectories alongside the corresponding learned temporal distances? Such a figure would help illustrate how the model captures temporal relationships between states and provide intuition for the learned distance function.

5. What does the statement "Layout and contact tests set its limits" in line 482 mean?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.