- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Flow-JEPA. Instead of using the one-forward-step autoregressive next-latent prediction as in LeWM, this paper introduces flow-matching over a full trajectory with multiple timesteps. Experimental results report the mean success from 86% to 92% under clean observations and from 67% to 86% under noisy conditions.
Critical strengths
- The presentation is sound and clear. Notions are easy to follow.
- The design choice is well motivated: stable representations do not imply robust dynamics and autoregressive latent prediction is vulnerable.
- The improvements, especially on the noisy-condition benchmark, are promising and consistent across various environments.
Critical weaknesses
- The proposed components are not well ablated: Although the authors admit it in the limitations, I would like to point out here that all the modifications, including joint trajectory prediction, predictor architecture, stochastic training, etc. are not ablated individually, which weakens the contributions of this paper. We do not know which components contribute the most and make the method work.
Questions for authors
- Table 1 includes DINO-WM, but the noisy-observation comparison in Table 2 includes only LeWM. Could the authors evaluate DINO-WM under the same perturbation protocol? This would help establish whether the reported robustness gains extend to a baseline using frozen pretrained representations. If this evaluation is infeasible, please explain the practical limitation.
2 comments
Additional experiment addressing W1
W1: The proposed components are not well ablated… joint trajectory prediction, predictor architecture, stochastic training, etc. are not ablated individually.
We now added an architecture-controlled multi-horizon MSE variant in Two-Room.
The predictor receives empty tokens as inputs and predicts all future latents in a single forward pass, supervised by the same MSE loss used in Fast-LeWM. Because flow time is absent, flow-time-conditioned AdaLN is replaced with RMSNorm. Apart from these changes, no other model component or training setting is modified.
| Method | Clean | Gaussian | Blur | Brightness | Saturation |
|---|---|---|---|---|---|
| MSE variant | 94 | 64 | 32 | 86 | 90 |
| Flow-JEPA | 100 | 96 | 82 | 98 | 98 |
Fast-LeWM controls for joint multi-horizon prediction, while the additional MSE variant further reduces the architectural confound. Together, these comparisons provide stronger evidence that the observed robustness is associated with the flow-matching formulation rather than either of these factors alone. The architecture-controlled results are reported in Table 11 of Appendix B.3.
Response to Reviewer VD1M
We thank the reviewer for recognizing that the motivation is well grounded, the presentation is clear, and the noisy-condition improvements are consistent across environments.
We added Fast-LeWM [1] as an additional baseline. Fast-LeWM predicts multiple future latent states in parallel across the planning horizon, avoiding recursive autoregressive rollout, while using deterministic MSE supervision.
Under localized Gaussian noise:
Fast-LeWM also performs substantially below Flow-JEPA under blur, brightness, and saturation perturbations. The Fast-LeWM comparison is reported in Table 2 for localized Gaussian noise and in Tables 7–9 for the additional perturbation types.
These results support our proposed flow-matching formulation beyond joint multi-horizon prediction alone.
We have now evaluated DINO-WM [2] under all clean and perturbed conditions, including localized Gaussian noise, blur, brightness, saturation, and the corresponding perturbation-severity sweeps.
Under localized Gaussian noise, DINO-WM achieves 48% mean success, compared with 67% for LeWM, 46% for Fast-LeWM, and 86% for Flow-JEPA.
DINO-WM is strongest in some individual settings under the additional perturbations, particularly on Two-Room under blur and under some stronger appearance shifts. Because DINO-WM uses a frozen, externally pretrained DINOv2 ViT-S encoder, whereas Flow-JEPA, LeWM, and Fast-LeWM use ViT-Tiny encoders trained only on environment-specific data, we treat DINO-WM as a complementary pretrained-feature reference and distinguish this comparison from the controlled comparison among the three ViT-Tiny methods. For these evaluations, we use the officially released LeWM and Fast-LeWM checkpoints. The DINO-WM checkpoints are taken from Oberweger et al [3].
References
[1] Gao, Y. and Xu, X. Fast LeWorldModel. 2026.
[2] Zhou, G., et al. DINO-WM: World Models on Pre-trained Visual Features Enable Zero-shot Planning. 2024.
[3] Oberweger, F. F., et al. Does Latent Planning Survive Point Clouds? Action-Conditioned JEPA World Models for Geometric Observations and Goals. 2026.