- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
WAVE-0 augments a pretrained policy (Physical Intelligence et al., 2025) with a latent world model that predicts future visual representations and uses them to condition action generation. In Stage 1, the world model and action/state encoders are trained on robot demonstrations in a frozen SigLIP-based representation space (Zhai et al., 2023), using future prediction, residual regression, and a counterfactual action-ranking objective. In Stage 2, the world model is frozen, the VLM learns latent action plans aligned with the action encoder's representations, and predicted future features are compressed into conditioning tokens for the action expert (Sec. 3.2, Fig. 2). The paper evaluates the approach on LIBERO, RoboTwin 2.0, several LIBERO generalization benchmarks, and three real-world tasks, reporting improvements over with modest additional inference latency (Tables 1–3 and 11).
References
-
Physical Intelligence et al. (2025). : A Vision-Language-Action Model with Open-World Generalization. arXiv preprint, arXiv:2504.16054.
-
Zhai, X., Mustafa, B., Kolesnikov, A., & Beyer, L. (2023). Sigmoid Loss for Language Image Pre-Training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 11975–11986).
Critical strengths
-
An action-ranking objective with evidence of downstream value. The counterfactual ranking loss is the most interesting methodological component to me. It targets a concrete failure mode: a latent predictor may rely heavily on the current observation while making insufficient use of the action input. By keeping the observation fixed and contrasting demonstrated and mismatched actions, the objective explicitly encourages action sensitivity (Sec. 3.2, Eq. (7)). The reported ablation provides evidence at both the prediction and policy levels: at 20k training steps, the action-sensitivity gap increases from 0.0020 to 0.1648, and adding the ranking loss improves success by 2.6 and 3.0 percentage points on LIBERO-Plus and LIBERO-Pro, respectively (Sec. 4.7.2, Fig. 4(a–c)). The downstream results make this component consequential to my assessment: they connect the proposed training intervention to improved robot performance and motivate further investigation of its role in learning action-dependent predictions.
-
Practical improvements to a pretrained VLA with modest additional inference cost. The method provides a concrete way to incorporate predicted visual consequences into an existing policy (Physical Intelligence et al., 2025), while retaining its action-generation framework and avoiding pixel reconstruction or additional large-scale video pretraining for the world model (Sec. 3.2). The reported gains extend across standard manipulation and distribution-shift evaluations: RoboTwin success improves from 80.38% to 91.47%, and LIBERO-Pro success improves from 53.0% to 61.4% (Tables 1(b) and 2). In the real-world inference configuration, mean server latency increases from 287.1 ms to 305.9 ms, approximately 6.5% (Appendix A.2, Table 11). These results support the practical relevance of the design: the reported performance benefits include challenging generalization settings, while the added inference cost relative to the base policy is limited.
References
- Physical Intelligence et al. (2025). : A Vision-Language-Action Model with Open-World Generalization. arXiv preprint, arXiv:2504.16054.
Critical weaknesses
-
The contributions of the coupled design choices are insufficiently isolated. The framework combines latent action queries, action-representation alignment, multiple auxiliary losses, future-conditioning tokens, and several conditioning inputs. The paper includes useful integration and loss ablations, but these do not establish which parts of this combination are essential. In particular, the auxiliary-only world-model variant already improves RoboTwin success from 80.38% to 89.51%, while the full method reaches 91.47% (Sec. 4.7.1, Table 1(b)). The additional 1.96-point gain supports the full design, but does not isolate the value of the predicted future content from the other changes in conditioning and training. Controls using current visual features or latent action tokens in place of predicted future tokens are missing, as is an ablation of the separate proprioceptive input to the world model. Consequently, I remain uncertain whether the improvements substantiate the proposed mechanism and justify the framework's additional complexity.
-
The distinctive methodological contribution over prior latent world-action models is not sufficiently established. The paper itself identifies LaWAM (Chen et al., 2026) as already conditioning action generation on predicted latent visual subgoals (Sec. 2.3). Direct action supervision and the ranking objective could provide a meaningful distinction, but the latter is underspecified and its dynamics interpretation is only partially validated. Equations (6)-(7) introduce mismatched actions without explaining their construction or how distinct actions with similar visual consequences are handled. Moreover, increasing the distance between the negative prediction and the demonstrated future does not establish that the negative prediction accurately represents the alternative action's outcome. The diagnostics in Fig. 4(a–b) measure the separation encouraged by the ranking loss, and Fig. 3(b) visualizes policy attention rather than probing the world model's representations. Despite the reported performance gains, the current evidence leaves the main new methodological insight insufficiently characterized.
References
- Chen, J., Wang, K., Chen, K., Chen, S., Gao, F., et al. (2026). LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies. arXiv preprint, arXiv:2606.15768.
Questions for authors
-
Clarification: definition and validity of mismatched actions. How exactly is the mismatched sequence in Eqs. (6)-(7) constructed: from another trajectory or task, by temporal permutation, or through action perturbations? How is the mask computed? How are distinct action sequences that produce equivalent or nearly identical visual outcomes handled? This is important because action inequality alone does not guarantee that the observed future is an inappropriate target for the alternative action.
-
Evidence request: attribution of the future-conditioning benefit. Have you compared predicted future tokens against current-observation tokens or directly projected latent action tokens, while keeping the remaining architecture, auxiliary supervision, token budget, and training budget as comparable as possible? Such controls would help determine whether accurate future prediction is necessary for the gain over the auxiliary-only world-model baseline in Sec. 4.7.1.
-
Evidence request: necessity of the separate state-conditioning path. Section 3.2 and Eq. (4) condition the world model on , while the action encoder already receives the same proprioceptive state through . Have you removed the separate input while retaining in the action encoder? How does this affect future prediction, action sensitivity, and downstream success? This would clarify whether the additional state pathway provides information that the action representation does not already supply.
-
Evidence request: dependence on the prediction space. Have you compared the frozen SigLIP-based prediction space (Zhai et al., 2023) described in Sec. 4.1 with an alternative such as DINO (Caron et al., 2021), using appropriate dimensional adapters and comparable training settings? In particular, how much do future prediction and policy performance depend on reusing 's visual representation space? This would clarify how specific the reported benefit is to the selected visual representation.
-
Evidence request: potential distribution shift in world-model conditioning. In Stage 1, the world model is trained on action-encoder outputs, including encodings of demonstrated and mismatched action sequences. In both Stage 2 training and deployment, the frozen world model instead receives planner-generated latent actions (Sec. 3.2, Eqs. (4), (6), and (9)). What evidence shows that the alignment and future-prediction objectives in Eqs. (10)–(12) adequately address this potential mismatch? Specifically, on held-out trajectories, how do future-prediction errors compare under encoded demonstrated actions versus planner-generated latents, with current observations and robot states held fixed? Does this gap increase on generalization benchmarks, and is it associated with degraded policy performance?
-
Clarification of evidence and scope: Stage 1 scaling. Are there controlled studies varying the amount or diversity of Stage 1 training data, with Stage 2 training held fixed, and measuring both prediction quality and downstream success? Section 4.6 compares training budgets, and Fig. 4(a–b) varies optimization steps; neither establishes scaling with dataset size. Please clarify whether such results are available. I would treat a new, extensive study on larger datasets as future work; the immediate question is what evidence currently supports the data-efficiency and generality of the learned world model.
References
-
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021). Emerging Properties in Self-Supervised Vision Transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 9650–9660).
-
Zhai, X., Mustafa, B., Kolesnikov, A., & Beyer, L. (2023). Sigmoid Loss for Language Image Pre-Training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 11975–11986).
11 comments
For Weakness 1: "The contributions of the coupled design choices are insufficiently isolated. The framework combines latent action queries, action-representation alignment, multiple auxiliary losses, future-conditioning tokens, and several conditioning inputs. The paper includes useful integration and loss ablations, but these do not establish which parts of this combination are essential. In particular, the auxiliary-only world-model variant already improves RoboTwin success from 80.38% to 89.51%, while the full method reaches 91.47% (Sec. 4.7.1, Table 1(b)). The additional 1.96-point gain supports the full design, but does not isolate the value of the predicted future content from the other changes in conditioning and training. Controls using current visual features or latent action tokens in place of predicted future tokens are missing, as is an ablation of the separate proprioceptive input to the world model. Consequently, I remain uncertain whether the improvements substantiate the proposed mechanism and justify the framework's additional complexity."
For Question 2: "Evidence request: attribution of the future-conditioning benefit. Have you compared predicted future tokens against current-observation tokens or directly projected latent action tokens, while keeping the remaining architecture, auxiliary supervision, token budget, and training budget as comparable as possible? Such controls would help determine whether accurate future prediction is necessary for the gain over the auxiliary-only world-model baseline in Sec. 4.7.1."
First, π0.5 + WM and WAVE-0 in Table 1(b) are two parallel ways of integrating our proposed Stage-1 world model into π0.5 (Fig. 1(3) vs. Fig. 1(4)) rather than an incremental stack. We designed π0.5 + WM to place the world model after the action expert for training loss supervision only, mirroring the auxiliary-supervision paradigm of VLA-JEPA and JEPA-WAM, so that we can directly compare that paradigm against WAVE-0's pre-expert future conditioning on the exact same backbone and world model. Beyond the +1.96 lead on RoboTwin 2.0, our new evaluation of π0.5 + WM on LIBERO variants conducted during the rebuttal shows clear gains from pre-expert conditioning under distribution shifts, improving LIBERO-Plus by +1.6 points from 86.4% to 88.0% and LIBERO-Pro by +2.6 points from 58.8% to 61.4%.
Second, to isolate the value of predicted future content from extra token capacity, latent action queries, and Stage-2 auxiliary losses, we conducted matched controls on the same π0.5 backbone, data, and training budget:
- (a) π0.5 + WM + Non-predictive Tokens. Adds non-predictive conditioning tokens (8 tokens on LIBERO and 12 on RoboTwin, matching WAVE-0's conditioning-token budget) to π0.5 + WM.
- (b) Direct Action Queries w/o WM. Projects VLM action-query states directly into matched conditioning tokens for the action expert without the world model.
- (c) Current-Feature Conditioning. Conditions the action expert on resampled current visual features instead of predicted future features while retaining the exact same latent-action queries, resampler, and Stage-2 world-prediction and alignment losses.
| Method | LIBERO | LIBERO-Plus | LIBERO-Pro | RoboTwin Clean | RoboTwin Random | RoboTwin Avg. |
|---|---|---|---|---|---|---|
| π0.5 | 96.9 | 84.5 | 53.2 | 81.12 | 79.64 | 80.38 |
| π0.5 + WM | 98.4 | 86.4 | 58.8 | 89.48 | 89.54 | 89.51 |
| (a) π0.5 + WM + Non-predictive Tokens | 98.4 | 86.5 | 58.4 | 89.78 | 89.28 | 89.53 |
| (b) Direct Action Queries w/o WM | 98.2 | 86.9 | 58.1 | 90.84 | 90.50 | 90.67 |
| (c) Current-Feature Conditioning | 98.4 | 87.4 | 60.4 | 90.82 | 90.06 | 90.44 |
| WAVE-0 (Ours) | 98.7 | 88.0 | 61.4 | 91.46 | 91.48 | 91.47 |
Adding non-predictive tokens in (a) brings virtually no gain over π0.5 + WM (86.5% vs. 86.4% on LIBERO-Plus and 89.53% vs. 89.51% on RoboTwin Avg.) and trails WAVE-0 by 1.5 points on LIBERO-Plus, 3.0 points on LIBERO-Pro, and 1.94 points on RoboTwin Avg. While VLM action-query routing in (b) already improves over π0.5, bypassing the world model still drops performance across every benchmark relative to WAVE-0, including −1.1 points on LIBERO-Plus, −3.3 points on LIBERO-Pro, and −0.80 points on RoboTwin Avg. Most directly, (c) Current-Feature Conditioning keeps all latent-action queries, alignment and world losses, and resampler parameters identical to WAVE-0, yet replacing predicted future features with current visual features consistently degrades performance across all benchmarks, including −1.03 points on RoboTwin Avg. (−1.42 points under randomization) and −1.0 point on LIBERO-Pro, isolating a consistent additional benefit from conditioning on predicted future content beyond query routing and auxiliary supervision alone.
For Weakness 1 (State-conditioning input) and Question 3: "Evidence request: necessity of the separate state-conditioning path. Section 3.2 and Eq. (4) condition the world model on , while the action encoder already receives the same proprioceptive state through . Have you removed the separate input while retaining in the action encoder? How does this affect future prediction, action sensitivity, and downstream success? This would clarify whether the additional state pathway provides information that the action representation does not already supply."
Under identical training settings on LIBERO, we remove the separate state token from the world model while retaining the proprioceptive state in the action encoder. Across sampled demonstration segments from all 40 LIBERO tasks, Stage-1 future prediction error and temporal-roll counterfactual action sensitivity compare as follows:
| Method | Future Cosine Error ↓ | Mismatched-Action Error | Action Sensitivity |
|---|---|---|---|
| WAVE-0 w/o | 0.0641 | 0.0633 | |
| WAVE-0 (Ours) | 0.0574 | 0.0566 |
Downstream policy success rates (%) across the three LIBERO benchmarks are:
| Method | LIBERO | LIBERO-Plus | LIBERO-Pro |
|---|---|---|---|
| WAVE-0 w/o | 98.5 | 87.2 | 59.2 |
| WAVE-0 (Ours) | 98.7 | 88.0 | 61.4 |
Architecturally, the two pathways play distinct roles. In Stage 1, because the action encoder receives the proprioceptive state to compute relative action displacements and directly feeds the world model, removing changes offline metrics only slightly, modestly increasing future prediction error from to while routing all state-action conditioning through (0.0566 vs. 0.0633 sensitivity). More importantly, in Stage 2 the action encoder only provides alignment targets (Eq. 10) rather than feeding the world model (Eq. 9), and at deployment it is not used at all. Instead, the frozen world model receives VLM-generated latent action plans, while π0.5's VLM only observes coarsely discretized text tokens of proprioception rather than continuous joint states. Consequently, serves as the only direct continuous proprioceptive input to the world model during Stage-2 future prediction and online deployment. Consistent with this design, removing drops LIBERO by 0.2 points and causes larger drops under distribution shift on LIBERO-Plus (−0.8 points) and LIBERO-Pro (−2.2 points), where perturbed initial robot states, camera viewpoints, and object layouts require accurate continuous proprioceptive grounding.
Dear Reviewer tbC5,
Thank you for your thorough review and constructive feedback. We appreciate your recognition of our counterfactual action-ranking objective as a consequential methodological contribution that mitigates observation-only shortcuts and improves both Stage-1 action sensitivity and downstream OOD success, as well as WAVE-0's practical gains on a pretrained VLA with modest (~6.5%) inference overhead without pixel reconstruction or large-scale video pretraining. Below, we (1) isolate the value of predicted future tokens via matched non-predictive, direct-query, and current-feature controls, (2) ablate the separate proprioceptive state pathway in the world model, (3) establish our distinction from prior latent WAMs with a matched Video-IDM comparison and specify the construction of mismatched actions and the validity mask in Eqs. (6)–(7), (4) compare π0.5's built-in SigLIP encoder against a frozen DINOv3 visual encoder, (5) diagnose Stage 1 to Stage 2 conditioning alignment, held-out OOD prediction errors, and rollout failures, and (6) present controlled Stage-1 scaling experiments on demonstration count and task coverage.