- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Summary
This paper introduces the Representation World Model (RWM), a world-modeling approach that jointly learns state representations and actionable transition structure in latent space, without an explicit forward dynamics model. Given start and goal observations, RWM constructs a straight-line path between their latent representations and uses an inverse dynamics model to recover actions along the path. The same construction is used during training and inference, enabling planning without autoregressive rollouts or online action-space search.
The authors evaluate RWM on four continuous-control benchmarks and ten LIBERO-Goal manipulation tasks. The results are strong, with ablations supporting the importance of inverse-dynamics supervision along the interpolated latent path. Additional probes suggest that both encoded and interpolated representations preserve relevant state and action information.
Assessment
I find this paper interesting and technically appealing. RWM offers a simple alternative to conventional latent world models by learning representations in which plans can be constructed directly and translated into actions, avoiding explicit forward-dynamics prediction and online search. The reported results are strong, and the ablations provide useful evidence for the proposed training objective.
My main concerns concern the limitations of straight-line latent planning and dependence on closed-loop execution. These issues limit how broadly the results can be interpreted, but I do not consider them sufficient to negate the contribution demonstrated on the current benchmarks.
Overall, I lean towards acceptance, subject to clarification of the planning horizon, observation modalities, and evaluation assumptions.
Critical strengths
Interesting and elegant formulation. Constructing executable plans directly in representation space is an appealing alternative to conventional world-model planning through forward rollouts and search. Using the same construction during training and inference is particularly nice.
Strong empirical results. RWM achieves a reported macro success rate of 98.25% across the four continuous-control benchmarks and 93.0% on LIBERO-Goal, outperforming the reported latent-world-model baselines.
Useful ablations and representation analysis. The experiments investigate the contribution of each training objective and show that interpolated representations preserve substantial state and action information. The closed-loop interval ablation also provides insight into the importance of feedback during execution.
Critical weaknesses
1. Limitations of straight-line latent planning. RWM constructs plans by linearly interpolating between the start and goal representations. While this works well on the evaluated tasks, it is unclear how well the approach handles environments where multiple qualitatively different trajectories connect the same endpoints or where a valid plan requires navigating around obstacles in a more complex state space. The authors acknowledge the limitation regarding multiple possible plans, but a more thorough discussion of the approach's applicability and failure modes would strengthen the paper.
2. Planning horizon and execution assumptions. The method's dependence on the planning and execution horizon could be clarified. The closed-loop interval ablation shows that performance degrades substantially when more actions are executed between replanning steps. This highlights the importance of feedback, but the paper could better explain the effective planning horizon, the choice of trajectory segment lengths, and how these settings affect performance. It would also be useful to distinguish the length of a constructed latent path from the number of actions executed before replanning.
Questions for authors
-
What observation modalities are used in each benchmark? In particular, does RWM use proprioceptive information in addition to images, and how does this affect the comparison with baselines?
-
How are the planning horizon, trajectory segment length, and closed-loop replanning interval chosen? How sensitive are the results to these choices?
-
Are gradients propagated through both sides of the interpolation-consistency loss, or is one representation detached? If so, what motivates this choice?
-
How does RWM behave when multiple distinct plans connect the same start and goal, or when the straight-line latent path does not correspond to a feasible physical trajectory?