- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper considers the setting of JEPA-like world models and under which conditions they experience failure modes. The paper claims two main contributions: a theoretical distinction between representation collapse due to the encoder discarding state information and prediction drift due to JEPA only supervising next-state prediction and not full rollouts and analysis of these failure modes on a frozen V-JEPA 2 and a potential solution to address this coined MEND which instead fits the parameters of a damped oscillation model in latent space. The proposed method is experimentally verified on video observations of a mechanical pendulum.
Critical strengths
- The problem setting is relevant to the community. Also, the idea of adding physical priors to learned dynamics is an interesting research direction
Critical weaknesses
- The primary issue with the paper is that it is badly written, not well presented which makes it very hard for the reader to follow the paper - in terms of theory, methodology and results, etc. Indeed, most of the content/text seems to be heavily AI-generated. Therefore, the paper in its current form does not provide value to the reader or the broader community as the insight into the theory, the analysis of the frozen V-JEPA2 or the proposed method is non-existent/extremely limited.
- The validation of the proposed method seems to be limited as it has only been done on the experimental pendulum which will (naturally) agree well with the mechanical prior on the latent dynamics of a conformal-symplectic DMD.
Questions for authors
- What other systems beyond the mechnical pendulum do you envision this proposed conformal-symplectic DMD latent dynamics to be beneficial for? Would the latent dynamics have to be changed/adjusted?