- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes k-WM, which lets JEPA world models to predict the k-step future in a single step to remove autoregressive generation and to deal with compounding error, improving gradient based method for goal conditioned planning. Applied on DINO-WM, finetuned DINO and LeWM, k-WM presents improvements of CEM and GD on open-loop planning performance and efficiency. They also present representational analysis and rollout visualizations with less compounding errors.
Critical strengths
- Clear motivation and methodology. JEPA latent world models and planning suffered from representation collapse and compounding error during rollout making it hard for gradient based optimization to work properly. This work proposes minimal changes to the world model to support multiple horizon predictions and variable length inference, significantly improving gradient based planning performance in challenging scenarios.
Critical weaknesses
- Distinction relative to Fast-LeWM. The core mechanism, prediction of all future latents from an action sequence over multiple horizon, is already in Fast-LeWM, claiming similar long-horizon improvements, faster planning, and representational gains. The main distinction is how they implemented such design and their additional analysis. More matched comparison on the design choices seems to be required to explicitly compare the two methods.
Questions for authors
- Encoder-matched comparison. Can you also present results using the prior baseline methods (LeWM and Fast LeWM) with a DINO or ViT-S encoder to show a comparable architecture with k-WM with DINO-WM/DINO-FT? While planning performance is one of k-WM’s clear distinctions, it seems Fast-LeWM already matches performance of k-WM LeWM concat with a comparable backbone scale.
Thank you for the detailed response and additional experiments. While these additions address several of my concerns, I still do not believe that the paper's novelty and contributions meet the bar for ICLR, so I will maintain my original rating.