acceptodds
Withdrawn from ICLR 2027

How Long, Not How Close: A Learned Temporal Metric for Planning in Latent World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Latent world models plan by rolling a frozen predictor forward under candidate action sequences and ranking the candidates by the latent distance between their imagined end state and the goal. However, this ranking breaks down when the goal lies several plans away, because the latent distance measures how closely an end state resembles the goal rather than how far it remains from reaching it. To ad- dress this, we propose TEMPO, a temporal-distance planning objective that leaves the world model untouched, learns only from the demonstrations already used to train it, and adds negligible cost to the planner’s search. TEMPO learns a small map of the frozen latent in which the distance between two states of an episode reflects the number of environment steps between them, and blends this distance into the planner’s cost. It requires no rewards, policies or success labels and, being a cost rather than a model, applies to frozen world models with one latent vector per state that plan by a latent distance. We evaluate TEMPO on eleven simulated environments (e.g., maze navigation, tabletop pushing, robotic arm control and three-dimensional manipulation) with the LeWM and PLDM planners. With a 0.13M-parameter map that adds at most 0.3% to a plan’s arithmetic, TEMPO im- proves both planners at every goal distance, including the one-plan setting of their published evaluations, raises LeWM from 36% to 99% on TwoRoom three plans from the goal, and remains competitive on a broad range of 2D and 3D navigation, reaching and manipulation tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Latent World Models, goal-conditioned planning, temporal embedding, self-supervised representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_aFSjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TEMPO, a learned temporal metric for goal-conditioned planning with latent world models. The method trains a small projection network using temporal gaps between states from recorded trajectories, then combines the resulting embedding distance with the planner's original latent-distance cost. The goal is to improve long-horizon planning without retraining the world model. The evaluation covers navigation and manipulation tasks, with comparisons using matched world-model checkpoints and planning budgets. The results show substantial navigation improvements, particularly on TwoRoom, and the ablations support combining temporal and latent distances.

Critical strengths

The paper investigates an interesting mismatch between prediction and control. A representation that supports accurate short-term prediction may not provide a useful distance for evaluating progress toward a distant goal. And the proposed method's performance on navigation environments is strong. The ablation study is also informative.

Critical weaknesses

Overall, I see the practical contribution as a lightweight modification to the terminal cost of an existing frozen-model planner. The results demonstrate useful improvements in the tested settings. The scientific contribution would be clearer, however, if the paper better established what is gained from this particular temporal embedding and blending design, when it helps, and where its limitations lie. Although the evaluation covers several application families, its applicability beyond the closely related model and planning implementations remains less clear.

Evidence for the design choices. The ablation study supports combining the original latent distance with the learned temporal distance, and Appendices I–J provide useful diagnostics of the representation and candidate rankings. However, it remains unclear which benefits are specific to the proposed embedding geometry, rather than to adding a learned progress estimate more generally. Could the authors clarify the advantages and tradeoffs of their design relative to alternative learned costs? For example, would a scalar predictor taking a pair of state representations, trained on the same temporal-gap labels and used in the same blended planning cost, provide similar improvements?

Interpretation of temporal distance. Equation (4) uses the observed number of steps between two states as supervision. These gaps may reflect waiting or detours, rather than the minimum time needed to travel between the states. Could the authors clarify how the behavior that generated the trajectories affects the learned metric and planning performance? In particular, under what conditions should the observed time gap provide a reliable measure of progress toward the goal?

Presentation of the method. The method section would benefit from separating the general formulation from the experimental configuration. The roles of temporal normalization, the supervision cutoff, the blending weight, and the required model interface should be explained before introducing their chosen values, network dimensions, and training schedule.

Questions for authors

Please see weaknesses.

Reviewer_tt3wOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

TEMPO learns a distance function on top of a frozen world model to improve planning toward distant goals. A small network is trained to match temporal gaps between states in recorded trajectories, then updated with online experience. During planning, its distance is combined with the original latent distance to score candidate actions. The paper reports improvements on navigation and manipulation tasks with LeWM and PLDM.

Critical strengths
  1. The motivation is clear: a state that looks closer to the goal may still require a long detour. The navigation examples show why the original latent distance can mislead a short-horizon planner and why information from trajectories could help.

  2. The method changes the planning cost without retraining the world model or replacing the search procedure. The gains on both LeWM and PLDM, particularly on TwoRoom, show that this relatively small change can make a substantial difference.

Critical weaknesses
  1. Equation (4) learns elapsed time in recorded trajectories, not the time needed to reach a state. A detour or pause changes the target even if the endpoints stay the same. The symmetric embedding distance also cannot distinguish directions with different travel times. The distance may still be useful for planning, but calling it remaining time-to-go or a shortest-path-like distance goes beyond what this objective establishes.

  2. It is unclear what is new in the distance-learning method beyond applying it to frozen features. The paper discusses Hilbert representations and TD-JEPA, but the comparison mainly emphasizes that TEMPO leaves the world model unchanged. The differences in learning objectives, and why the proposed objective is preferable in this setting, need a more direct explanation.

Questions for authors
  1. How does the learned distance depend on the behavior in the training trajectories? In particular, what happens when the same states are connected by routes with different amounts of delay or detouring?

  2. Could existing temporal-distance objectives be used to train a mapping on the same frozen features? What distinguishes TEMPO from that approach?

  3. How much improvement remains with offline-only training? Please clarify the role of the earlier exploratory planner and report the additional online collection and training costs separately from the planning overhead.

  4. Appendix G says the mapping is trained in the training environment and kept fixed during evaluation, while the captions of Tables 6–8 describe online training in shifted environments. Which description is correct, and which results demonstrate generalization without adaptation?

Reviewer_R4dvOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose training a heuristic function that maps an encoding and goal encoding pair to the number of actions left to reach a state that produces the goal encoding. They train this heuristic function using expert demonstrations. This heuristic function is then used to guide search to solve problems instead of Euclidean distance.

The method is evaluated on domains that were not seen during training. Results how that this helps performance, even though they were unseen.

Critical strengths

The evaluation shows that this heuristic function performs better than Euclidean distance. Generalization to unseen domains is interesting.

Critical weaknesses

The paper builds on ideas from Hindsight Experience Replay (HER) [1] and learning heuristic functions from walks [2] (although it does not cite them); however, it makes no comparisons to these methods so one can understand the contribution of the proposed method. Crucially, this method assumes the existence of expert data. HER was able to learn a policy for solving problems similar to the ones used for evaluation in the paper, such as Reacher, when gathering data using an initially random policy. [2] was able to learn effective heuristic functions from random walks with no expert data.

In light of this previous work, it is not clear what the contribution of this paper is. One potentially interesting aspect is the generalization to new domains. However, the heuristic function is, inherently, domain specific. This means if the dynamics of the underlying Markov decision process were to change, the heuristic values may also need to change. However, the heuristic function only takes an encoding of the state and goal as an input, not the domain. Therefore, this generalization is inherently limited and somewhat serendipitous.

[1] Andrychowicz, Marcin, et al. "Hindsight experience replay." Advances in neural information processing systems 30 (2017). [2] Chervov, Alexander, et al. "A machine learning approach that beats Rubik's cubes." Advances in Neural Information Processing Systems 38 (2026): 172509-172531.

Questions for authors

How reliant is this method on expert data? If one were to instead train a heuristic function using HER or random walks, could this be used to guide search just as effectively?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.