acceptodds
Withdrawn from ICLR 2027

Beyond a Single Latent Space: A Dual-Latent World Model for Long-Horizon Planning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

18 views

Abstract

Latent world models often struggle with long-horizon planning, even when their short-term predictions are accurate. Prediction errors accumulate during recursive rollout, while distance concentration in high-dimensional latent spaces can weaken the distinction between states at different distances from a goal. We introduce the Dual-Latent World Model (Dual-WM), which separates local execution from long-range planning through two distinct state representations and dynamics models. The low-level model captures fine-grained action-conditioned transitions, while the high-level model uses learned macro-actions to plan over longer temporal spans. We further propose Long-Horizon Representation Learning with Weighted Rollout (LoRe), which supervises self-generated predictions at both levels. Motivated by an analysis of recursive error propagation, LoRe uses exponential horizon weights with separate decay rates to balance multi-step supervision at each temporal scale. During planning, the high-level model generates latent subgoals, and the low-level model refines them into actions for precise execution. We evaluate from-scratch Dual-WM on five goal-conditioned visual control tasks against the task-wise strongest baselines without actor-guided proposals. At goal offsets of 50 and 100 environment steps, mean success increases from 75.9% to 84.4% and from 61.4% to 69.5%, respectively. At offset 100, Dual-WM outperforms these baselines on all five tasks and improves mean success over LeWM by 30.8 percentage points. Ablations and supporting analyses provide evidence that the proposed design learns more informative state representations for goal evaluation and improves consistency under recursive prediction. These results highlight the value of separating temporal roles and training across multiple horizons for reliable latent planning. Our core implementation is available in an anonymous repository at https://anonymous.4open.science/r/Dual-WM-2411/.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
World Model, Latent Planning, JEPA, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_m8pEOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Dual-Latent World Model (Dual-WM), a hierarchical world model that separates short-horizon action execution and long-horizon goal planning into two distinct latent representations and dynamics models. The low-level model captures fine-grained action-conditioned transitions, while the high-level model uses learned macro-actions and a separate representation to support long-range planning. The two spaces are connected through a learned interface that enables high-level subgoals to guide low-level action refinement. The authors further introduce Long-Horizon Representation Learning with Weighted Rollout (LoRe), which supervises recursively generated predictions at both temporal scales using separately parameterized exponential horizon weights, together with Macro-Action Prior Shaping (MAPS) to improve the sampling properties of learned macro-actions. Experiments across five visual control tasks demonstrate improvements over several flat and hierarchical world-model baselines, particularly at longer goal offsets. Additional ablations and representation diagnostics support the benefits of the learned high-level representation, multi-step rollout supervision, and hierarchical planning.

Critical strengths
  1. The paper introduces a useful architectural distinction between local execution and long-range planning by learning separate state representations and dynamics models for the two temporal scales. Unlike conventional hierarchical approaches that reuse a shared latent representation, Dual-WM employs a learned interface to map low-level latent windows into a dedicated high-level space for subgoal evaluation and refinement. The representation ablations provide encouraging evidence for this design. In particular, the proposed model substantially outperforms shared-identity, endpoint-based, and direct window-concatenation alternatives while retaining the same low-level checkpoint and hierarchical planning procedure. These results suggest that learning a distinct high-level representation provides benefits beyond simply introducing macro-actions or extending the prediction horizon, making the architectural contribution meaningful.

  2. The paper evaluates Dual-WM across five visual control tasks spanning both continuous and discrete action spaces, with extensive comparisons against flat and hierarchical world models. The proposed method achieves consistent improvements at longer goal offsets, while the comparisons against the low-level-only variant using the same checkpoint provide evidence for the contribution of hierarchical planning. The component studies further investigate the effects of dual-scale rollout supervision, representation interfaces, and macro-action regularization. Importantly, the TwoRoom success–planning-time comparison shows that the full model can achieve higher planning success at comparable or lower computational latency than its low-level-only counterpart. Together with the representation and recursive prediction diagnostics, these results provide relatively strong support for the practical value of the proposed framework.

Critical weaknesses
  1. Although the paper reports results over three planning seeds, these seeds primarily capture variation in evaluation-task sampling and planning randomness rather than independent world-model training runs. Consequently, the reported uncertainty does not fully characterize the sensitivity of Dual-WM to model initialization and optimization. Furthermore, the most important representation-interface and LoRe weighting ablations are concentrated on TwoRoom, leaving it unclear whether their conclusions generalize consistently to contact-rich manipulation tasks such as PushT and Cube-Single. Since the framework introduces multiple interacting components, including separate latent representations, macro-action modeling, and multi-step rollout supervision, additional experiments with independently trained checkpoints would be important for establishing the reliability and generality of the proposed design.

  2. Dual-WM introduces additional high-level representation and dynamics modules, a macro-action encoder, and a multi-stage planning procedure, which potentially increase both training and inference costs compared with simpler world models. Although the paper provides a useful success–planning-time comparison on TwoRoom, it does not report comprehensive training time, total GPU-hours, or peak memory consumption for the full framework and its main baselines. Moreover, the planning-efficiency advantage has not been established on the more challenging manipulation tasks, where evaluating high-level subgoals and refining low-level action sequences may involve different computational trade-offs. A broader analysis of training and planning costs would strengthen the assessment of whether the reported performance gains justify the additional architectural complexity.

Questions for authors
  1. How robust are the benefits of the learned dual-latent interface across training seeds and manipulation tasks? The current representation-interface and LoRe ablations provide encouraging evidence on TwoRoom, but most reported uncertainty comes from planning seeds rather than independently trained models. Could the authors repeat the main representation-interface comparison on PushT or Cube-Single using multiple independent training seeds, while keeping the low-level checkpoint shared across the variants within each seed? Reporting the variation in planning success and long-range goal discrimination would help determine whether the benefits of the separate high-level representation are consistent beyond the navigation setting and would materially strengthen my confidence in the architectural contribution.

  2. What are the full computational trade-offs introduced by Dual-WM? Could the authors report training time, total GPU-hours, and peak GPU memory for the low-level and high-level components, together with comparable measurements for the main baselines? In addition, could they extend the existing success–planning-time analysis to at least one manipulation benchmark, such as PushT or Cube-Single, and report per-planning-call latency alongside the current amortized timing measurements? These results would help clarify whether the gains in long-horizon planning remain attractive when both the additional training cost and hierarchical inference overhead are considered.

Reviewer_uQamOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work proposes a 2-level hierarchical latent world model which learns separate latent spaces for each level. The high-level model is used to produce subgoals for the lower level. Both levels plan using the cross-entropy method where the high-level optimizes over a learned action space.

Critical strengths

Good motivation for the general approach taken by the method.

Critical weaknesses

The paper is not human-readable.

While the high-level idea seems sound, due to poor writing, I was not able to properly assess technical aspects of the approach and experiments. Good communication is a prerequisite for a solid contribution in my opinion, regardless of the method details and experimental evidence.

The notation is overly complex and hard to follow. The method description is somewhat cryptic about how planning is actually done and how the high-level subgoals communicate with low-level planning. The experiment section has many low-level details about evaluation but on the other hand does not discuss, e.g., basic details about baselines and the motivation for choosing them. New terms are presented in the experiment section and are not explained (e.g. “actor-guided”, “our Gemini 3.8 Flash closed-loop controller ”). The experiment section is not sufficiently self-contained, most of the experiment results and figures are deferred to the Appendix.

Questions for authors

How do the high-level subgoals guide the low-level model if they do not share a latent space?

Reviewer_PKvcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a solution to the problems of compounding errors and uninformative long-range latent distances during planning with latent world models. The paper proposes to learn two different latent world models operating at different time scales. The high-level model searches over sequences of macro-actions using CEM, picking the sequence whose predicted final state is closest to the actual goal. The intermediate predicted states serve as subgoals for the low-level model, which tries to reach them using CEM. Neither of the models is goal-conditioned. Instead, latent rollouts are evaluated based on a distance computed in the latent space. The dual latent model separates the coarse features required for planning from fine-grained features used for control, enabling learning world models that plan better. In addition, the paper proposes LoRe, which trains each world model on its own multi-step predictions rather than only one-step predictions. Simply put, in multi-step predictions, the errors are weighted with exponentially decaying weights, so that larger errors in the distant steps do not dominate the total loss. The proposed method is evaluated in 5 different goal-conditioned visual environments and is shown to outperform the baselines at longer goal offsets (50 and 100 steps).

Critical strengths

The problem is well motivated and explained. The paper clearly motivates the problem it is trying to solve in the introduction. Figure 1, panels A and B, helps establish the two problems with planning using latent world models.

Critical weaknesses

Overloaded Notations - The paper, specifically Section 3 where the method is explained, is overloaded with notations and variables, making it very difficult to follow. While the method itself is very simple, the paper introduces and uses more than 15 variables in Section 3. Most of these variables are used in a single equation and then dropped. Time indexing is inconsistent as well. The figures index high-level transitions with τ\tau, while the text uses tt for both levels and even refers to “high-level state t.” I strongly recommend rewriting Section 3 with fewer symbols and adding a notation table in the Appendix, making it easier to read. The Figures are also particularly hard to follow.

MAPS and LoRe - These two methods are proposed to assist with learning better latent world models. MAPS regularizes the macro-action posterior toward a standard Gaussian, so that sampled macro-actions during planning remain in-distribution for the high-level predictor. This method is not new and is used widely across the field, from VAEs [1] to other papers using world models in RL [2, 3]. LoRe is used to reweight multi-step predictions with exponentially decaying horizon weights, which is also already standard in prior work [4,5], and closely related to [6]. While the paper cites [5,6], it is not in the context of LoRe. The paper should position MAPS and LoRe relative to these works, and clarify how these are novel contributions.

The paper fails at positioning itself in the broader literature of reinforcement learning. Citing only papers from 2026 to support the claim about compounding errors with a learned model overlooks so much of what was done in the past [e.g., 7, 8, 9]. Similarly, a lot of existing work in the literature is ignored. Existing methods do decompose solutions into high-level and low-level controllers based on learned representations [e.g., 10, 11], and many others acknowledged this issue (even if in a different problem setting) [e.g., 12, 13].

I struggle to understand the point of Section 2. The manuscript barely interacts with its content, and it mostly seems like a dump of citations that are barely discussed in light of the paper’s claimed contributions. Similarly, I would expect Section 4 to discuss the consequences of the results, connecting them to the main claims of the paper. Instead, it basically restates the numbers in the tables/graphs.

[1] Kingma, Diederik P., and Max Welling. "Auto-encoding Variational Bayes." International Conference on Learning Representations (2014)

[2] Zhou, Wenxuan, Sujay Bajracharya, and David Held. "Plas: Latent Action Space for Offline Reinforcement Learning." Conference on Robot Learning (2021)

[3] Hafner, Danijar, et al. "Deep Hierarchical Planning from Pixels." Advances in Neural Information Processing Systems (2022)

[4] Hansen, Nicklas, Xiaolong Wang, and Hao Su. "Temporal Difference Learning for Model Predictive Control."International Conference on Machine Learning (2022)

[5] Hansen, Nick, Hao Su, and Xiaolong Wang. "Td-mpc2: Scalable, Robust World Models for Continuous Control." International Conference on Learning Representations (2024)

[6] Hafner, Danijar, et al. "Learning Latent Dynamics for Planning from Pixels." International Conference on Machine Learning (2019)

[7] Talvitie, E. Model regularization for stable sample rollouts. In Uncertainty in Artificial Intelligence (UAI), 2014.

[8] Talvitie, E. Self-correcting models for model-based reinforcement learning. In AAAI Conference on Artificial Intelligence (AAAI), 2017.

[9] Clavera, I., Rothfuss, J., Schulman, J., Fujita, Y., Asfour, T., and Abbeel, P. Model-based reinforcement learning via meta-policy optimization. In Conference on Robot Learning (CoRL), 2018.

[10] Koul, A., Sujit, S., Chen, S., Evans, B., Wu, L., Xu, B., Chari, R., Islam, R., Seraj, R., Efroni, Y., Molu, L., Dudik, M., Langford, J., and Lamb, A. PcLast: Discovering plannable continuous latent states. In International Conference on Machine Learning (ICML), 2024.

[11] Shehmar, D., Schlegel, M., Taylor, M., Machado, M. Laplacian Representations for Decision-Time Planning. In International Conference on Machine Learning (ICML), 2026.

[12] Seohong Park, Dibya Ghosh, Benjamin Eysenbach, and Sergey Levine. HIQL: Offline goal-conditioned RL with latent states as actions. In Neural Information Processing Systems (NeurIPS), 2023.

[13] Hongjoon Ahn, Heewoong Choi, Jisu Han, and Taesup Moon. Option-aware temporally abstracted value for offline goal-conditioned reinforcement learning. In Neural Information Processing Systems (NeurIPS), 2025.

Questions for authors
  1. Assumptions - How much of the work relies on deterministic dynamics?
  2. Experimental Setup - Are the reported performance results from a single trained model, and averaged over 3 different planning seeds? If so, please report results averaged from 3 different training seeds for the proposed method and the baselines (at least at offset 100) so that the actual gains can be separated from training variance (please refer to [14]).
  3. Reporting - In the Appendix, when reporting the standard deviation across seeded evaluations, what is being considered? Is the standard deviation of 3 numbers reported? Or is the standard deviation of 3 x 200 (the number of goal-reaching tasks) numbers computed?
  4. Baselines and Tasks - What criteria were used to select baselines? Why is Gemini, for example, a relevant baseline given the research question the paper asks? Similarly, there are many other challenging control tasks in the literature with longer horizons and more complex local control problems, and existing methods succeed in those [e.g., 10]. Why limit the evaluation in these four simple tasks?
  5. Hyper-parameters - How were the hyper-parameters selected for the proposed method and the baselines? The hyper-parameters vary by task for the proposed method (Table 3), while the baselines use the default configurations. Did the retrained baselines (HWM, DINO-WM) receive a comparable tuning budget on these tasks/datasets? Were the hyperparameters tuned for the ablated results?

[14] Patterson, Andrew, et al. "Empirical Design in Reinforcement Learning." Journal of Machine Learning Research (2024)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.