acceptodds
Withdrawn from ICLR 2027

A Low-Dimensional Deterministic Dynamics Core for Controllable Long-Horizon Latent World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Interactive world models must generate plausible observations while preserving the consequences of actions over long autoregressive rollouts for game agents, model-based planning, and embodied control. Yet commonly used one-step and teacher-forced metrics can make flat and factorized transition models appear similar under matched capacity and exposure-bias noise, even when their free-running behavior differs. We study this discrepancy in frozen pretrained latent spaces and introduce WorldBridge, a factorized transition model that predicts a compact dynamics code before rendering the remaining appearance coordinates deterministically. Both components retain access to the previous latent and the action, so the compact code guides prediction without imposing a strict information bottleneck. Under capacity-, training-budget-, and exposure-noise-matched comparisons, the factorized model has comparable short-range fidelity but produces more controllable and radially stable long-horizon rollouts than a flat predictor, with the clearest separation at the single-evaluation operating point. Comparisons with stronger memory-based predictors further show that the training objective and the factorization contribute distinct parts of the observed improvement, and that the advantage is not explained by generative stochasticity alone. A larger-scale validation retains the controllability advantage on temporally held-out trajectories and improves action identification. Finally, component-wise distillation yields a real-time student using only a few network evaluations per frame. These results support evaluating latent world models by their closed-loop behavior, not by short-range fidelity alone.

Primary area:
generative models
Keywords:
world models, latent dynamics, long-horizon rollout, flow matching, diffusion models, interactive generation, model-based reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_WdkrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper suggests a two-step process for latent dynamics modelling, in which instead of predicting the entire latent state conditioned on the previous latent state and action, the model first predicts a very low-dimensional latent, which is followed by a conditional prediction of a higher-dimensional latent, both of which are conditioned on the full latent in the previous time step and the action. The claim is that this provides a more stable rollout over long horizons compared to a one-shot approach and other baselines. This comparison of world models over long horizons also seems to be key theme of the paper.

Critical strengths

The experimental results seem interesting.

Critical weaknesses

Unfortunately, I found the paper very hard to digest because of heavy LLM-assisted writing, which has significantly undermined its readability. I believe the paper is not suitable for publication at ICLR due to its writing.

Questions for authors

N/A

Reviewer_5zqAOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes WORLD BRIDGE, a factorized latent world model that first predicts a low-dimensional dynamics core and subsequently generates high-dimensional appearance features conditioned on it. The authors investigate long-horizon controllability and stability under autoregressive generation, and introduce a distillation strategy for real-time inference.

Critical strengths
  • Important and well-motivated research problem. The paper addresses the discrepancy between short-horizon prediction accuracy and long-horizon rollout stability, highlighting an important limitation of conventional world-model evaluation.
  • Practical real-time distillation. The proposed distillation strategy enables real-time generation with only two network evaluations per frame while maintaining performance close to the multi-step teacher.
Critical weaknesses
  • Unclear sources of improvement. Although the paper includes several ablation studies, the contributions of low-dimensional factorization, learned representations, and multi-step training remain insufficiently disentangled. It is therefore unclear which component primarily drives the reported gains.
  • Limited comparisons with state-of-the-art world models. While the authors include Dreamer-style RSSM and attention-based baselines, they do not directly compare against modern large-scale world models such as Dreamer 4. This limits the assessment of the proposed method's competitiveness.
  • Lack of evidence for general applicability. The proposed factorization is evaluated primarily within the authors' own model architecture. No experiments demonstrate whether integrating this design into established world models, such as replacing the latent dynamics predictor in Dreamer 4, yields consistent improvements.
  • Limited visual quality and model scale. The qualitative generations appear relatively blurry, raising concerns about perceptual fidelity. Furthermore, the largest evaluated model contains only approximately 21M parameters, substantially smaller than contemporary large-scale generative world models. This leaves its scalability and effectiveness in high-fidelity generation unclear.
  • Limited methodological novelty. The core idea of separating low-dimensional dynamics from high-dimensional appearance modeling overlaps with existing work on factorized latent dynamics and motion–appearance decoupling. The paper's contribution appears primarily empirical, with limited conceptual novelty beyond prior approaches.
  • Counterfactual controllability does not necessarily imply accurate dynamics. The primary controllability metric measures the magnitude of changes in predicted latents under different action sequences, rather than their agreement with ground-truth action-conditioned transitions. Although the authors provide additional action-identification evaluations, higher controllability scores may reflect exaggerated action sensitivity rather than more accurate dynamics. This raises concerns about whether the reported improvements translate into better world modeling or downstream planning performance.
Questions for authors

See strengths and weaknesses

Reviewer_2XzVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a factorized model for world model prediction, leading to more stable long-term predictions compared to a flat predictor. It builds on two recent world models: OASIS (Minecraft) and SD-VAE (Atari). The factorized model first predicts a compact code and then generates the remaining latent representations.

Critical strengths

S1: The paper introduces a factorized world model objective and shows that it leads to more stable long-term predictions.

Critical weaknesses

W1: The paper trains factorized and "flat" world models on top of frozen representations, compare the prediction of the two alternatives. However, it does not compare with the frozen world models, which are already able to make stable long-term predictions. Figures 1, 13, and 18 show that while the factorized model leads to more stable predictions, it is unable to maintain the prediction quality of OASIS.

Questions for authors

Could you compare the method with OASIS's pretrained world model?

Reviewer_D4RfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WorldBridge, a factorized latent transition model designed to improve long-horizon autoregressive rollouts. The model factorizes the transition into a low-dimensional dynamics core and a high-dimensional appearance renderer, where the dynamics core is predicted first and then used to condition the appearance renderer. The authors evaluate the method on frozen pretrained latent spaces from Minecraft and Atari environments, showing that the proposed factorization improves long-horizon controllability and stability over flat transition models while maintaining comparable short-horizon prediction quality. The paper further investigates the effects of generative stochasticity, compares against stronger memory-based baselines, and explores few-step distillation for real-time inference.

Critical strengths
  • The paper highlights an important limitation of evaluating world models solely through short-horizon prediction metrics. The distinction between short-range fidelity and long-horizon behavior is relevant to interactive world modeling.
Critical weaknesses
  • The contributions are not sufficiently clear, as while the paper attributes long-horizon improvements to the low-dimensional dynamics core, experimental results suggest that learned representations and multi-step training are also important. The effects of the architecture and training procedure should be separated more clearly, and the multi-step training objective should be described in greater detail.
  • The dynamics core is initially constructed using principal components, which do not necessarily correspond to action-relevant dynamics. Moreover, the renderer receives both the previous latent and the action, allowing information to bypass the core, so whether the proposed representation captures intended information is unclear.
Questions for authors
  • How do the models compare under matched total network evaluations or inference latency, rather than matched integration steps per component?
  • How is the multi-step training objective defined, and how much of the improvement can be attributed independently to factorization, learned core representations, and multi-step training?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.