acceptodds
Under review as a conference paper at ICLR 2027

Horizon-Aligned Scoring: A Longer View for Short-Horizon World Model Planning

Abstract

LeWorldModel (LeWM) learns visual latent dynamics from reward-free trajectories and uses the cross-entropy method (CEM) to plan toward image goals. It ranks candidate action sequences by their predicted endpoint-to-goal distances, so short rollouts can undervalue preparatory actions whose benefits emerge later. We introduce Horizon-Aligned Scoring (HAS), which uses an action-free latent predictor to extend each endpoint toward the goal's time scale before scoring. Trained on LeWM's existing trajectories, the predictor adapts its recursion depth to the remaining temporal gap, given the goal offset. HAS thereby extends candidate evaluation without lengthening the action-conditioned search. Across PushT, TwoRoom, and Reacher, HAS improves mean success when the goal image is sampled beyond LeWM's default planning span of 25 environment steps. For goal images sampled 50 environment steps after the initial observation, PushT success rises from 42.8% to 67.8%; at a 100-step goal offset in TwoRoom, HAS reaches 70.4%, compared with 55.4% for the matched hybrid variant of Trajectory Reachability Metrics (TRM-Hybrid) and 16.4% for LeWM. The gains persist across three independently trained checkpoints per task. HAS also outperforms a LeWM control with twice the action horizon at every tested longer goal offset under the same replanning interval. Together, the results show that HAS improves LeWM's long-range planning without additional training data or a longer action-conditioned planning horizon.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, latent planning, long-horizon planning, goal-conditioned control, model predictive control

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tGM9OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an extension to the action-candidate scoring in the CEM planning procedure of LeWM. An additional action-free next-state predictor is trained and used at test time to complete the predicted state sequence towards the goal beyond the reach of the fixed horizon action-conditioned prediction. This approach that builds on LeWM, named HAS, is compared to the standard LeWM on 2D visual tasks and outperforms it when the goal is not within the reach of the fixed-horizon action-conditioned planning horizon.

Critical strengths

None

Critical weaknesses

This paper does not pass the bar for an ICLR submission in my opinion.

The main reasons for my evaluation are detailed in the following:

Use of Privileged Information for Planning

The proposed candidate scoring method leverages the (already questionable) evaluation protocol which samples trajectory segments from the training data and selects initial-state and goal images within a known fixed temporal timestep distance. This known timestep distance is privileged information that is not available in standard goal-conditioned planning settings but the method relies on. Overall, the evaluation does not test the most basic form of generalization to in-distribution test trajectories and the validity of the state-only predictor relies strongly on this protocol.

Limited Scope of the Study

This paper proposes a modification to LeWM action candidate scoring mechanism. As stated above, it relies on privileged information and a very specific evaluation protocol. The method is only compared to LeWM on the full task suite. The environments are all 2D. In my opinion, the scope of this paper does not pass the bar for an ICLR contribution.

Lack of References to Prior Related Work

This paper lacks proper positioning and references to related work. The number of references (13) is exceptionally low and most focus on very recent papers from a small subset of authors.

Questions for authors

None

2 comments
AuthorsOpenReview ↗

Response to Reviewer tGM9

Thank you for the detailed assessment. Your concerns prompted us to separate HAS from its original timing rule, broaden the comparisons, and clarify its relationship to established planning methods. We have uploaded a revised manuscript and updated the abstract. All references below point to this revision.

1. Temporal information and evaluation protocol (W1).

Your concern led us to revisit our existing depth experiments. On PushT at the 75-step reference interval, the tested fixed positive depths achieve at most 4.0%, versus 40.4% for the decreasing schedule. On TwoRoom, fixed k=5 exceeds k=10 by 37.6 percentage points at the same interval. These results suggest that useful continuation depends on the task and planning stage, rather than simply extending predictions farther. Appendix G.2, Table 7, reports these experiments.

We consequently introduced HAS (adaptive), which advances candidates together and selects a shared depth from improvements in their minimum goal distance. Candidates are scored at the best visited depth, including k=0. The rule does not read the reference offset, elapsed time, or interaction budget, and uses the same stopping parameters across tasks. Section 4.3 describes the rule, and Appendix D provides pseudocode.

Adaptive HAS improves mean success over LeWM in 11 of 12 longer-offset conditions. For goals sampled 100 steps ahead along reference trajectories, it achieves 90.0% on TwoRoom, versus 20.8% for LeWM, 59.2% for TRM-Hybrid, and 11.2% for the best transferred value score. Section 5.3 analyzes depth selection, Figures 3-4 show performance and planning behavior, and Table 4 gives full results.

Section 4.2 clarifies that F learns local five- and ten-step predictions, rather than directly fitting the evaluated 25-100-step start-goal relations. The experiments retain LeWM's trajectory-based protocol and do not establish generalization to trajectories excluded from world-model training.

2. Breadth of baselines and tasks (W2).

Your concern motivated expanding both the comparison methods and task coverage. Beyond LeWM's latent-distance score, we added goal-conditioned value functions adapted from GCIVL, QRL, and CRL-V, and extended TRM to all four tasks. TRM is evaluated both as a replacement score and combined with latent distance. Together with LeWM and the two HAS depth rules, these form eight scoring variants under the same frozen world model and CEM settings.

These comparisons test two approaches to terminal evaluation: directly learning a goal-related score, or predicting continuation before applying latent goal distance. Section 5.1 motivates the baseline selection, and Appendix C specifies the terminal-score adaptations. We evaluate these methods as scoring components within the shared planner, rather than as reproductions of their original full systems.

On PushT with goals sampled 50 steps ahead along reference trajectories, HAS with the known offset achieves 68.0%, versus 36.8% for TRM-Hybrid and 24.8% for the best transferred value score. Adaptive HAS achieves the highest mean at every TwoRoom and Cube interval, while GCIVL and QRL perform strongly on Reacher. Section 5.2 and Figure 3 discuss these comparisons, and Table 4 gives full results. The longer-action-horizon controls additionally compare HAS with expanding action search itself (Section 5.4, Table 6).

To broaden task coverage, we added 3D single-object Cube manipulation. At the 100-step reference interval, adaptive HAS achieves 64.0%, compared with 60.8% for LeWM, 52.4% for TRM-Hybrid, and 48.8% for QRL. Appendix B specifies the OGBench environment and LeWM trajectory-goal protocol, which differs from official OGBench task-goal evaluation. Section 6 identifies multi-object scenes, independently specified goals, and other world-model backbones as remaining directions.

3. Related-work positioning (W3).

Your feedback prompted us to organize the related work around the design question rather than nearby world models alone. Section 2 now connects terminal values in POLO and TD-MPC, predictive representations, goal-conditioned value functions in UVFA and TDMs, and learned world-model scores to HAS. The bibliography has expanded from 13 to 32 references.

This discussion develops the distinction underlying HAS: learning local continuation on predicted endpoints, then composing that predictor before applying latent goal distance. Sections 4.1-4.2 explain the construction and training objective, and Section 5.1 connects this distinction to the selection of temporal and value-function baselines.

I appreciate your effort in addressing my concerns.

In my review I tried to make my opinion clear that the paper needs significantly more work to meet the bar for an ICLR submission.

It seems that you have made many changes and additions. As I see it, these are more than is acceptable in the scope of revisions during a rebuttal period. For me to assess them, I would have to essentially review the paper all over again.

I will thus maintain my rating.

Reviewer_q7A6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Horizon-Aligned Scoring (HAS) to improve candidate action evaluation in short-horizon visual goal-conditioned planning with LeWorldModel (LeWM). HAS first predicts the endpoint of each candidate action sequence using the action-conditioned world model, recursively extends that endpoint representation with a lightweight action-free latent predictor, and then measures its distance to the goal embedding. The recursion depth is determined by the supplied reference-trajectory goal offset and decreases as execution proceeds. The additional predictor is trained using local one-step and two-step recursive prediction losses on the existing offline trajectories, requiring neither additional data collection nor a longer action-conditioned search horizon. Evaluation on PushT, TwoRoom, and Reacher includes longer-action-horizon controls, a TRM comparison on TwoRoom, depth and offset sensitivity analyses, and validation across three independently trained checkpoints per task. The reported results show improved goal-reaching success at the evaluated longer goal offsets, with approximately 4% average replanning overhead under the reported latency benchmark.

Critical strengths

1.Consistent empirical gains. Under matched models and planning budgets, HAS improves performance across three tasks and independent training seeds, providing substantial evidence for the scoring method’s effectiveness. 2.Modest additional cost. HAS reuses existing data and retains the short action horizon, adding approximately 4% average replanning overhead in the reported benchmark, which strengthens its practical value.

Critical weaknesses

1.Reliance on additional temporal information limits applicability. HAS requires the goal’s reference-trajectory offset, which may be unavailable in image-goal tasks. The offset-perturbation experiments do not establish performance without this input, leaving its broader practical applicability unverified. 2.Comparisons with competing methods are limited. Evaluation primarily focuses on LeWM, with the TRM comparison restricted to TwoRoom. This supports an improvement to LeWM but provides limited evidence for HAS’s comparative contribution against other long-horizon planning and learned-scoring approaches.

Questions for authors

1.Beyond the TRM comparison on TwoRoom, have you conducted matched comparisons with other long-horizon planning or learned terminal-scoring methods, particularly on PushT and Reacher? If so, what are the main results? 2.Have you evaluated HAS with world-model backbones other than LeWM? If so, do the gains persist? If not, could you clarify the architectural or representation assumptions required to transfer HAS to another backbone?

1 comment
AuthorsOpenReview ↗

Response to Reviewer q7A6

Thank you for identifying the missing evidence on planning without the offset and comparisons beyond LeWM. These questions prompted us to revisit our depth experiments and expand the comparison of terminal scores. We have uploaded a revised manuscript and updated the abstract. All references below point to this revision.

1. Planning without the goal offset (W1).

Your observation that offset perturbation does not test the absence of supplied timing led us to revisit the existing fixed-depth experiments. On PushT at the 75-step reference interval, the tested fixed positive depths achieve at most 4.0%, versus 40.4% for the decreasing schedule. On TwoRoom at the same interval, fixed k=5 exceeds k=10 by 37.6 percentage points. These results suggest that useful continuation depends on the task and planning stage; reference timing is a proxy for that depth. Appendix G.2, Table 7, presents these experiments.

This motivated HAS (adaptive). Within each CEM iteration, candidates advance together until their minimum cost stops improving sufficiently. All candidates are then scored at the best visited layer, including k=0. The rule does not read the reference offset, elapsed time, or interaction budget, and uses the same stopping parameters across tasks. Section 4.3 explains the rule, and Appendix D provides pseudocode.

Adaptive HAS improves mean success over LeWM in 11 of 12 longer-offset conditions, with the highest mean among eight scores at every TwoRoom and Cube interval. At the 100-step reference interval, it achieves 90.0% versus 20.8% for LeWM on TwoRoom and 64.0% versus 60.8% on Cube. Section 5.3 reports that TwoRoom's mean first-call depth increases from 0.015 to 1.050 between the 25- and 100-step intervals without reading them, and analyzes the remaining PushT sensitivity. Table 4 gives full performance results. Goal sampling and the 2o environment budget remain unchanged.

2. Comparisons beyond LeWM (W2, Q1).

Your question helped organize comparisons around two ways to look beyond a short rollout. Longer-horizon controls test expanding action search; learned-score comparisons test replacing or supplementing latent distance. We extended TRM to all four tasks and added terminal-score adaptations of GCIVL, QRL, and CRL-V under the same frozen world model and CEM settings. Section 5.1 motivates these choices, and Appendix C specifies their implementation.

On PushT at the 50-step reference interval, HAS with the known offset achieves 68.0%, versus 36.8% for TRM-Hybrid and 24.8% for the best transferred value score. Adaptive HAS achieves the highest mean at every TwoRoom and Cube interval. Section 5.2 and Figure 3 discuss these comparisons and the strong GCIVL and QRL results on Reacher; Table 4 gives full results.

HAS with the known offset also exceeds both 50-step LeWM planners in all nine longer-offset conditions on the original three tasks (Section 5.4; Table 6). The H=50, R=25 planner takes 2.12 times LeWM's mean replanning time, while adaptive HAS adds 4.3% in its separate three-task benchmark. Appendix J, Tables 9 and 11, details these measurements.

3. Other world-model backbones (Q2).

Section 4.4 explains how HAS can transfer to other world-model planners. HAS operates between a predicted action endpoint and its comparison with the goal embedding, an interface also used by DINO-WM, PLDM, and V-JEPA 2-AC. A continuation module can be trained on a new backbone's predicted endpoints and future encoded observations using its existing trajectory data. Its architecture follows the representation: vector latents admit an MLP, while patch-token latents can use a token-to-token predictor without action conditioning.

The planner then applies its existing goal-distance function to the continued endpoint, leaving action search unchanged. Both depth rules carry over: known-offset scoring requires consistent temporal units, while adaptive scoring selects depth from successive goal-distance evaluations. These are the architectural requirements for transfer; the quality of the resulting score depends on the backbone's latent geometry and learned continuation.

Reviewer_1bsJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors present a horizon-aligned scoring. This mechanism fills the gap between the end of an action planned through LeWM and a target image farther in time. This scoring supports LeWM planning by connecting the end of the horizon to the goal without expanding the action search.

Critical strengths
  • This system is simple and clear, and the training is cheap.
  • Similarly, the benefits of the results are noteworthy, regardless of the simplicity of the idea.
Critical weaknesses
  • Examples tested with the proposed approach are very simple and only include 2D planning with simple or no background.
  • The novelty of the approach is somewhat unclear because of the parallelism you would see with value functions in reinforcement learning for the same task.
Questions for authors
  • When I read the paper and saw the first figure, the first thing I thought about was a value model and how the proposed approach resembles the way it is used in a reinforcement learning system. Can the authors expand on this thought?
  • Continuing on the idea above, I assume that the scoring somehow gives an average behaviour, since it is not conditioned on the actions. As such, what limitations do you think it exposes?
  • I think the authors should also include results on the OGBench benchmark to showcase their proposed system in more complex scenarios. If it doesn't work well, the authors can describe the limitations of their approach.
  • Finally, I would not put 3 contributions, as in my opinion, the contribution is the horizon-aligned scoring itself, and, especially, I wouldn't count the experimentation as part of the contribution.
1 comment
AuthorsOpenReview ↗

Response to Reviewer 1bsJ

Thank you for recognizing the simplicity and benefits of HAS. Your questions helped us clarify its relationship to value functions and extend the evaluation to learned goal scores and 3D manipulation. We have uploaded a revised manuscript and updated the abstract. All references below point to this revision.

1. Relation to value functions and novelty (W2, Q1).

Your connection to value functions helped sharpen the distinction between what HAS evaluates and what its learned component predicts. HAS serves as a terminal cost, while its continuation model F predicts future latents rather than directly fitting a scalar state-goal value. F learns one-step and two-step continuation on world-model endpoints; recursive application extends those endpoints before the planner applies its latent goal distance. Section 2 relates this construction to terminal values and TDMs; Sections 4.1-4.2 explain the scoring rule and supervision. F receives neither goals nor actions, but HAS uses the goal for scoring and adaptive stopping, and candidate actions determine F's input through the world model.

This distinction motivated comparisons with GCIVL, QRL, CRL-V, and TRM, which learn goal values or temporal separation. We adapt their scores to the same CEM planner, testing TRM both alone and combined with latent distance. On PushT with goals sampled 50 steps ahead along reference trajectories, HAS with the known offset achieves 68.0%, versus 36.8% for TRM-Hybrid and 24.8% for the best transferred value score. On TwoRoom at 100 steps, adaptive HAS achieves 90.0%, versus 59.2% and 11.2%, respectively.

Section 5.1 motivates these choices; Section 5.2 and Figure 3 compare performance, including the strong value-function results on Reacher. Appendix C documents the terminal-score adaptations, and Table 4 gives full results.

2. Averaged behavior without action conditioning (Q2).

Thank you for raising this issue. Because F predicts future latents without conditioning on future actions, squared-error training can average distinct continuations. HAS uses this prediction to score an action-conditioned endpoint; it does not execute the imagined continuation. Actions are selected through CEM, and subsequent planning incorporates new observations (Section 4.1 and Algorithm 1). Appendix I explains how averaging can nevertheless produce misleading goal distances.

Our existing depth experiments show why controlling continuation matters. On PushT at the 75-step reference interval, the tested fixed positive depths achieve at most 4.0%, versus 40.4% for the decreasing schedule (Appendix G.2, Table 7). These results do not isolate averaging as the cause, but show that extending prediction farther can substantially hurt planning.

This motivated HAS (adaptive): candidates advance together within each CEM iteration, and recursion stops when their minimum goal distance no longer improves sufficiently. All candidates are scored at the best visited depth, including k=0, allowing the planner to retain direct endpoint scoring when continuation offers no improvement. Section 4.3 describes this rule, and Appendix D provides pseudocode.

Adaptive HAS improves mean success over LeWM in 11 of 12 longer-offset conditions, including 90.0% versus 20.8% on TwoRoom at the 100-step reference interval (Section 5.3; Table 4). This supports selecting continuation depth from candidate costs rather than imposing a fixed positive depth. The rule does not eliminate averaging: predicted cost can improve without improving action quality, as the remaining PushT sensitivity illustrates. Section 5.3 analyzes this behavior, and Section 6 identifies it as a limitation.

3. More complex scenarios and OGBench (W1, Q3).

Your suggestion motivated extending the evaluation to 3D single-object Cube manipulation. We use the OGBench environment and LeWM's released trajectories, retaining the shared trajectory-goal protocol for comparison. This differs from OGBench's official task-goal evaluation.

Adaptive HAS achieves the highest mean among the eight scoring variants at every Cube interval. At the 100-step interval, it achieves 64.0%, compared with 60.8% for LeWM, 52.4% for TRM-Hybrid, and 48.8% for QRL. Sections 5.2-5.3 and Figure 3 discuss this extension; Appendix B specifies the setup, and Table 4 reports the results. Section 6 identifies multi-object interaction and richer visual scenes as remaining directions.

4. Contribution statement (Q4).

Following your suggestion, we rewrote the final paragraph of Section 1 around HAS itself: constructing a terminal cost through learned continuation while retaining a short action search and reusing the world model's training data. The experiments support this construction and its applicability, rather than being presented as separate contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.