- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
LeWAM jointly learns JEPA-based latent dynamics and a flow-matching action policy, supporting both direct control and visual planning, achieving a significant increase in the success rate.
Critical strengths
- Joint modeling of the world and actions based on the JEPA framework is clearly an emerging trend.
- This paper is very clearly written; apart from the experiments, which are not convincing, everything else is quite good.
Critical weaknesses
- The core innovation of LEWM lies in its low computational requirements; however, it does not represent the highest-performing variant within the JEPA architecture. Consequently, using LEWM as the primary benchmark fails to demonstrate whether the proposed method offers any performance improvement, as the research objectives are entirely different.
- Numerous existing implementations model the world and action jointly within the JEPA framework, such as ACT-JEPA and WA-JEPA, yet the paper provides no comparison against them.
Questions for authors
- How does the JEPA architecture achieve out-of-distribution (OOD) generalization?
- How can fairness across different environments be ensured?
- Given that tasks vary by environment, does this imply that a separate JEPA model must be trained for each individual environment?
5 comments
Q2: How can fairness across different environments be ensured?
A4: Whenever possible, we use public checkpoints for benchmark scores. Otherwise, we use the configs specified in the respective papers or repositories if available. Lastly, all models in all environments see the same number of episodes (50), drawn from the same eval seeds (42, 0, 1), and thus the exact same starting and goal frames. All models receive the same execution budget: 2x the horizon length. Beyond this, it would be great if the reviewer could clarify what “fairness” means in this question so we may answer it concretely.
Q3: Given that tasks vary by environment, must a separate JEPA model be trained for each environment?
A5: Yes, we discuss this in the paper (Conclusion and Limitations, “Single-Task and Single-Embodiment Learning”). We identify a version of the model that supports multiple tasks and environments as a direction for future work, since the current work has all the environment training share the same model architecture and hyperparameter setting.
References
[1] Ye, Seonghyeon, et al. "World action models are zero-shot policies." arXiv preprint arXiv:2602.15922 (2026).
[2] Wang, Xinlin, et al. "WA-JEPA: Rethinking the video JEPA paradigm for world-action modeling in autonomous driving." arXiv preprint arXiv:2608.20974 (2026).
[3] Vujinovic, Aleksandar, and Aleksandar Kovacevic. "ACT-JEPA: Novel joint-embedding predictive architecture for efficient policy representation learning." arXiv preprint arXiv:2501.14622 (2025).
[4] Maes, Lucas, et al. "LeWorldModel: Stable end-to-end joint-embedding predictive architecture from pixels." arXiv preprint arXiv:2603.19312 (2026).
The author did not resolve my issue.
Thank you for your response, but it has not addressed my concerns. Comparing only against LeWAM is insufficient, and the issue of OOD (out-of-distribution) generalization has not been considered at all. It is difficult for me to determine whether this approach possesses general applicability or holds any significant research value.
Response to Reviewer
Thank you for recognizing joint world-and-action modeling within the JEPA framework as an emerging direction and for finding the paper clearly written. Please see our point-by-point responses below.
A0: General Response:
Regarding the research objective, we want to respectfully clarify that:
LeWM [4] and LeWAM are in the same research direction. Both aim to (1) learn a strong action-conditioned world model that captures the environment dynamics, (2) plan and reason at test time using the world model, and (3) explore end-to-end training without additional techniques.
LeWAM is different from ACT-JEPA/WA-JEPA/JEPA-WAM. ACT-JEPA [3] and WA-JEPA [2] are JEPA-enhanced policy models; LeWAM is a world model with a coupled policy. Their world-model components provide only auxiliary training supervision, and their latent spaces support policy learning while retaining minimal information about the environment's dynamics. Instead, LeWAM's model-based planning and control rely on the learned dynamics, while the stochastic policy only provides a prior distribution.
LeWAM benefits from two design choices. Joint prediction of latent states and actions shapes the representation and dynamics. The learned policy distribution provides a prior that improves sampling efficiency and narrows the planning search space.
W1: LeWM is an efficiency-oriented JEPA rather than the highest-performing variant, so using it as the primary benchmark does not demonstrate a performance improvement, and the research objectives differ.
A1: While we agree that LeWM is not the highest-performing JEPA variant, we also do not attempt to create the highest-performing JEPA architecture. Our research goal is to study joint world and action modeling in a JEPA that is simple by design (requiring no extensive hyperparameter tuning), is trained end-to-end with no pretrained components, and works across a variety of tasks. In other words, we want to push toward a model that, like LeWM, is small and efficient yet supports both direct control and planning. Note that LeWM has approximately 16M parameters and LeWAM approximately 17M. Both models are trained end-to-end with SIGReg. Therefore, we respectfully disagree that the research objectives are entirely different and believe they are actually quite aligned.
W2: Existing methods that model the world and actions jointly within the JEPA framework, such as ACT-JEPA and WA-JEPA, are not compared against.
A2: While WA-JEPA [2] and ACT-JEPA [3] are indeed WAMs that predict in latent space, they do not model and leverage forward dynamics. They mirror works like DreamZero [1] in jointly predicting the next state and action. Modeling forward dynamics is a crucial part of our work, as it enables test-time planning beyond direct policy execution. We have added both methods to the Related Work section (Section 2).
Q1: How does the JEPA architecture achieve out-of-distribution (OOD) generalization?
A3: Thanks for the insightful question. As a simple OOD test, we analyze the robustness of LeWAM and LeWM to visual changes in the environment. Specifically, for each of the three evaluation seeds, we sample 50 random episode starts and set the goal 25 environment steps ahead. In every episode, we independently and uniformly redraw the colors of the agent, the block, and/or the background across RGB. The draws come from a fixed per-episode seed, so every method sees exactly the same colors on exactly the same episodes. The results have been added to Appendix C.9 (Table 14), and we summarize them below.