- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Dream-Lift-VLA, a progressive framework that first uses future-video supervision to learn scene dynamics and subsequently introduces future-depth supervision to strengthen spatial representations. The learned world embeddings support action generation, while both generative branches are removed during inference. Experiments evaluate the framework across simulation benchmarks and real-world manipulation tasks.
Critical strengths
1.The evaluation covers multiple simulation benchmarks and real-world tasks under both in-distribution and out-of-distribution conditions, demonstrating competitive performance and consistent overall improvements over the compared baselines. 2.The ablation studies are extensive, examining training order, training duration, supervision targets, and generative backbones.
Critical weaknesses
- Although the curriculum ablations support the effectiveness of the proposed training schedule, the rationale for learning future 2D dynamics before introducing depth supervision remains insufficiently explained. In particular, the paper provides limited analysis of how this progression improves representations relevant to action prediction, beyond the observed performance gains.
- The contribution relative to DreamVLA also warrants clearer characterization, as the comparison does not fully control for differences in the backbone and pretrained generative components. Consequently, the extent to which the improvements over DreamVLA stem from the proposed curriculum remains difficult to assess.
Questions for authors
1.Why should future video prediction precede depth prediction for VLA learning? Can the authors provide further analysis showing how this ordering improves representations relevant to action prediction? 2How sensitive is the advantage over joint training to the relative weights of the video and depth objectives? Does progressive training remain advantageous when these weights are carefully tuned? 3.Do the findings from the training-order ablations on MetaWorld also hold on LIBERO and real-world tasks? Such results would help establish the broader applicability of the proposed curriculum. 4. Despite the empirical gains, I find the methodological contribution somewhat incremental, with the novelty appearing to lie primarily in the integration and training schedule of existing components. I currently lean toward rejection, but remain open to revising my assessment if the rebuttal convincingly addresses the concerns above and better substantiates the methodological novelty.
4 comments
Response to the Reviewer 2Dwb (2/3)
[W1, Q1] Further Representation Analysis (continued)
Analysis:
(1) Under the same 60k-step training budget, joint SVD+PPD training with loss weights of (0.25, 0.25) achieves a success rate of 67.3% (see our response to Q2), close to the 68.0% achieved by PPD-only training. This suggests that the model may not fully benefit from video supervision when both objectives are introduced from the outset. One possible explanation is that introducing depth supervision from the outset in joint training may encourage the model to prioritize overall scene geometry for action prediction without sufficiently focusing on manipulated objects, targets, and their changes, making it difficult to develop a robust understanding of manipulation semantics. In contrast, the Dream stage first develops action-relevant semantic understanding, providing semantic and dynamic context for future-depth prediction. The Lift stage uses the dynamics learned during Dream to understand the resulting spatial relationships, further refining its understanding of manipulation semantics from a spatial perspective. Moreover, reversing the order introduces geometric supervision before Dream’s semantic and dynamic context is established, potentially weakening the grounding of learned manipulation semantics in task-relevant spatial relationships.
The attention maps support this interpretation: attention under joint training is relatively diffuse, whereas the Dream stage focuses more clearly on manipulated objects and target locations, and the Lift stage maintains and further strengthens the focus on manipulated objects. Reverse-order training shows some signs of semantic understanding, with attention to objects in certain cases, but remains limited in precise localization.
(2) Moreover, joint training can fit actions in scenes seen during training (as discussed in our response to Reviewer g5bK’s comment W2), but it does not fully develop stable representations of manipulation semantics, which limits generalization. Our curriculum learning approach builds on a more stable understanding of manipulation semantics and incorporates spatial relations relevant to manipulation, reducing reliance on scene-specific cues. In reverse-order training, the semantics learned during Dream may remain insufficiently grounded in spatial relationships, leading to less precise localization and higher probe errors.
The probe results further support this interpretation: compared with joint training and reverse-order training, our curriculum achieves lower errors in both the future-dynamics and action probes on unseen trajectories.
[1] AdaPolicy: Action-Driven Anticipatory Robot Policies via Next-Chunk Prediction, arXiv 2026.
[2] Vision-Language Models Unlock Task-Centric Latent Actions, arXiv 2026.
[3] Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature, Geosci. Model Dev. 2014.
[W2] Additional Controlled Experiments
A: Thank you for the comment. To distinguish the contributions of the training curriculum from those of the generative components, we conduct two groups of controlled experiments based on DreamVLA on MetaWorld.
Settings. The first retains DreamVLA’s original components but replaces joint supervision with a two-stage curriculum: learning future dynamics and semantics first, followed by depth geometry, with action supervision retained in both stages. The second keeps the same policy backbone and curriculum learning but replaces the auxiliary prediction components with SVD and PPD. The total training budget remains 40 epochs, divided into 20 epochs per stage. Each stage uses an initial learning rate of and cosine decay, following DreamVLA’s official configuration.
Result. Without changing the model components, the curriculum learning yields absolute success-rate gains of 4.7% on MetaWorld. Introducing SVD and PPD provides further gains of 3.3%. These results further support the effectiveness of the proposed curriculum and highlight the benefits of leveraging pretrained generative components.
| Method | Total Epochs | MetaWorld SR (%) |
|---|---|---|
| DreamVLA | 40 | 66.0 |
| DreamVLA + Two-Stage Curriculum | 20+20 | 70.7 |
| DreamVLA + Two-Stage SVD–PPD | 20+20 | 74.0 |
Response to the Reviewer 2Dwb (3/3)
[Q2] Sensitivity to Loss Weights
A: We conduct additional loss-weight sensitivity experiments for joint training, varying both the total auxiliary-loss weight and the relative weights of video and depth prediction. With a total weight of 1.0, (0.5, 0.5), (0.2, 0.8), and (0.8, 0.2) achieve success rates of 65.3%, 65.3%, and 66.0%, respectively. With a total weight of 0.5, (0.25, 0.25), (0.1, 0.4), and (0.4, 0.1) achieve 67.3%, 66.0%, and 66.7%. Although reweighting improves joint training from 65.3% to 67.3%, it remains below progressive training’s 82.0%. Thus, joint training exhibits some sensitivity to loss weighting, but progressive training retains a substantial advantage across the tested configurations.
| Total Weight | Avg. SR (%) | |
|---|---|---|
| Action-only baseline | 0 | 68.0 |
| (original) | 1.0 | 65.3 |
| 1.0 | 65.3 | |
| 1.0 | 66.0 | |
| 0.5 | 67.3 | |
| 0.5 | 66.0 | |
| 0.5 | 66.7 |
[Q3] Curriculum Generalizability
A: Thank you for the suggestion. We extend the training-order ablations to LIBERO and real-world demonstrations, evaluating on LIBERO, LIBERO-Plus, and four real-world tasks. The table reports average success rates (%). Dream→Lift achieves absolute gains of 2.7%, 9.7%, and 13.7% over joint training across these three settings, respectively, and consistently outperforms Lift→Dream. These results reproduce the trend observed on MetaWorld and support the broader applicability of our training curriculum.
| Training Strategy | LIBERO SR (%) | LIBERO-Plus SR (%) | Real-World ID SR (%) |
|---|---|---|---|
| Dream→Lift (ours) | 99.0 | 83.7 | 85.0 |
| Lift→Dream | 97.0 | 77.7 | 73.8 |
| Joint Dream+Lift | 96.3 | 74.0 | 71.3 |
[Q4] Method Contributions
A: Thanks for your valuable suggestions, which have helped us further clarify our contributions. Our method introduces a novel concept and architecture, as reviewer u75x also noted: “The idea to ‘2D-to-3D imagine’ is interesting and novel to the best of my knowledge.” We further clarify our contributions from three perspectives, namely conceptual novelty, architectural design, and extensibility.
(1) Concept. In world modeling for VLAs, no prior work has explored our progressive world-modeling paradigm. Learning future semantics first and then developing geometry foresight has been shown to be important for action generation, yet all previous methods have overlooked this point.
(2) Architecture. We introduce a novel two-stage generative VLA architecture that deliberately leverages pretrained generative priors through tailored conditioning and adaptation, connecting world modeling to action learning rather than simply assembling existing components.
- Native-pathway conditioning: We carefully design an interface in which PPD’s native VFM features attend to world embeddings, integrating Dream’s future dynamics while preserving its native geometric context (Table 9).
- Generative priors for policy learning: We deliberately adapt pretrained video and depth generators jointly with the VLA, harnessing their native generative capabilities to shape action-relevant dynamic and spatial representations (Table 10).
- From depth estimation to spatial foresight: We explicitly repurpose PPD from current-depth estimation to future-depth prediction, grounding VLA representations in future geometry and encouraging the policy to anticipate action-relevant spatial changes rather than only encode the current scene (Table 7).
(3) Extensibility. Our design supports extensions across generative backbones (Table 11), supervision paradigms (Appendix B.3, Table 16), and VLA baselines (our reply to W2). These results support progressive world modeling as a reusable learning principle, offering new methodological insight and a strong, extensible baseline for the VLA community.
We will incorporate these clarifications into the revised manuscript to help readers better understand Dream-Lift-VLA’s design rationale and novelty.
We thank the reviewer again for the constructive suggestions, which help us clarify Dream-Lift-VLA’s contributions and strengthen the experimental evidence, and we will include these results and our discussion in the latest version.
With Appreciation for Your Feedback and Consideration #20374.
Dear Reviewer 2Dwb,
Thank you again for your thoughtful and constructive feedback. In our rebuttal, we have analyzed the learned representations through linear probes and attention visualizations, added controlled DreamVLA experiments to distinguish the benefits of the curriculum from those of the generative components, evaluated loss-weight sensitivity, extended training-order ablations to LIBERO, LIBERO-Plus, and real-world tasks, and clarified our conceptual and architectural contributions.
We hope that our response helps address your concerns, and we would sincerely appreciate any further feedback.
Best regards,
The Authors
Response to the Reviewer 2Dwb (1/3)
We thank the reviewer for the constructive feedback. Below, we analyze the learned representations, present controlled experiments with DreamVLA, evaluate sensitivity to loss weights and the generalizability of the curriculum, and clarify our methodological contributions.
[W1, Q1] Further Representation Analysis
A: We thank the reviewer for highlighting the need to explain why future-video learning should precede future depth supervision and how this order affects action-relevant representations. To assess the benefits of the proposed curriculum, we follow prior works [1, 2] and use linear probes to examine the future-dynamics and action-relevant information encoded in the learned representations. Furthermore, we visualize attention maps to show which image regions the action tokens attend to. Finally, we discuss possible mechanisms behind the observed differences.
Both probe experiments compare Joint, Dream, Dream→Lift, and Lift→Dream, using the same 600 observations for fitting and all 96 observations from 12 unseen expert trajectories across six tasks for testing. The VLM remains frozen, and neither the original action head nor proprioceptive states are used. We evaluate prediction errors using standard root mean square error (RMSE) and mean absolute error (MAE), with lower values indicating better performance [3].
(1) Future-dynamics probe. We mean-pool the final-layer features of the six world tokens and train a linear probe to predict the 3D displacement of the primary manipulated object or tool from t to t+4. RMSE is computed in centimeters against simulator-recorded ground-truth displacements from the expert trajectories.
(2) Action probe. We separately mean-pool the final-layer world-token and action-token features, concatenate them, and train a linear probe to predict the current and next expert actions. We report translational-action RMSE and gripper-action MAE.
Compared with joint training and reverse-order training, the Dream stage yields lower future-displacement and translational-action prediction errors, although gripper prediction does not improve. The subsequent Lift stage further reduces all three errors, with gripper MAE decreasing from 0.0660 to 0.0403. These results suggest that our curriculum learns representations that better capture future dynamics, improve geometric understanding, and support action prediction.
(3) Attention Visualization. We visualize attention from action tokens to visual tokens using the same observations and instructions. In the presented examples (https://anonymous.4open.science/r/dream-lift-vla-20374/figures/attention-visualization.png), Joint training yields relatively diffuse attention, whereas the Dream stage encourages clearer attention to manipulated objects and a stronger response to the target placement location in Shelf Place (red box). The subsequent Lift stage further strengthens the focus on manipulated objects.