- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper "audits" world models (LeWM, DINO-WM, V-JEPA2-AC) on MPC using collections of "frozen" trajectories (what the authors call "banks" in their abstract, but which are better called probes). They study various error metrics such as next-state/embedding prediction error, and error in prediction future reward/success.
Critical strengths
Generally I think the genre of research is good -- too many papers champion some "novel" tweak to a method, and too few pick apart existing methods and try to understand their painpoints.
Critical weaknesses
The paper reads as an eclectic mix of tests that the authors ran on various JEPA models and various robot control benchmarks. I'm all for benchmark papers, but why only JEPA-style models? Why these benchmarks? There's no stochastic environments, no validation on a real robot, no Dreamer-style or Diamond-style or inverse-dynamics-style world models, and more significantly, no clear-cut indictment or vindication of the JEPA paradigm.
Clarity is also poor; prose appears AI-written and inherits the expected issues, such as obscure ad-hoc terminology ("banks" is but one of many examples).
Questions for authors
What should we really learn from your work? Do you think your findings would transfer to a real robot? To stochastic environments? To long-horizon tasks? To wildly-unobserved decision-making problems? To not just manipulation but also locomotion?