- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper aims to train VLAs to predict future information for better action generation. The authors focus on transitions between current and future observations, which include motion and spatial relationships during manipulation. In particular, they introduce MoAct, a VLA that learns visual-transition dynamics constructed in compact representations using the MoT architecture. They first train a visual-transition representation using transformer blocks by predicting future DINO features from current DINO features with transition tokens and VLM features. They then predict actions by conditioning these transition representations as conditioning input. They adopt two-stage training that connects visual-transition dynamics learning with task-specific action generation using heterogeneous robot and egocentric human videos.
Critical strengths
- The proposed approach shows relatively strong performance only with a 1B-parameter model.
- The proposed approach is efficient during inference because the DINO encoder and predictor are not required during inference.
Critical weaknesses
-
This paper lacks conceptual novelty. Recent work such as LAPA [1] already learns compact representations of visual transitions from action-free videos and leverages them for downstream action learning. Moreover, DUST [2] adopts a dual-stream, MoT-style architecture that jointly models compact future visual representations and future actions. While MoAct differs in how the transition representations are constructed and trained, it remains unclear what fundamentally new capability or insight this formulation provides beyond these closely related approaches.
-
For the methodogically, this paper lacks an ablation study of core components of the proposed method. Specifically, it does not evaluate the following ablations:
- Sensitivity analysis of transition tokens by varying the number of them
- Effect of spatial gating by comparison to a simpler fusion mechanism, and thus it is unclear whether explicitly modeling where the scene changes is useful
- Effect of layer-wise modulation by removing the FiLM-style modulation, and thus it is difficult to claim the benefit of modeling how the scene changes.
- Effect of MoT architecture by comparison to a single-stream architecture.
- Effect of stop-gradient.
- Effect of two-stage training by comparing with training the full model jointly from scratch.
- It is unclear whether the performance of the 1B model comes from the proposed method, because it uses Qwen3.5 as the VLM backbone, but baselines use relatively weaker VLM backbones (e.g., Wog [3] uses Qwen3 as the VLM backbone).
Because the method is relatively complex, the contribution of each component should be carefully identified. Without such ablations, it is difficult to know which parts of the design choices are actually necessary, particularly given the limited conceptual novelty from prior works.
[1] Ye et al., Latent Action Pretraining from Videos, ICLR 2025
[2] Won et al., Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model, ICML 2026
[3] Su et al., World guidance: World modeling in condition space for action generation, ICML 2026
Questions for authors
Please answer the weakness part.
10 comments
Response to Reviewer fTMC (Part 2/8)
Response 1 (1/2)
Reviewer's Comment: This paper lacks conceptual novelty. Recent work such as LAPA [1] already learns compact representations of visual transitions from action-free videos and leverages them for downstream action learning. Moreover, DUST [2] adopts a dual-stream, MoT-style architecture that jointly models compact future visual representations and future actions. While MoAct differs in how the transition representations are constructed and trained, it remains unclear what fundamentally new capability or insight this formulation provides beyond these closely related approaches.
Response: We respectfully disagree. The core idea of MoAct is that imagining only the part of the future that changes is sufficient to improve action generation. This makes MoAct highly efficient, and because learning such changes requires no action labels, it allows low-cost egocentric human videos to improve downstream manipulation (Section 4.2). We believe this offers a promising route to exploiting egocentric human data and to alleviating the cost of teleoperated robot data. As their own descriptions show, LAPA and DUST differ from MoAct in what the compact representation encodes, how it is learned, and whether the deployed policy uses it.
Relative to LAPA [1]: MoAct's transition tokens are a continuous, learned condition that the deployed policy reads at every step, rather than discrete pseudo-labels that are discarded.
- LAPA's latent actions are discrete codes produced by a separate quantization model. LAPA first trains a VQ-VAE whose encoder "takes the current frame and the future frame ... and outputs the latent action " (LAPA, Section 3.1), and the difference of the two frame embeddings is discretized "by finding the closest embedding from the codebook" (LAPA, Appendix A). This frozen encoder then serves "as an inverse dynamics model to label all frames" for pretraining (LAPA, Section 3.2). In MoAct, there is no codebook and no separate labeling model: the continuous transition tokens are produced by the VLA itself from the current observation and instruction, and are learned end to end through the successor-feature prediction loss.
- LAPA's latent actions are removed before deployment. LAPA states that "VLAs that are pretrained to predict latent actions are not directly executable on real-world robots", and therefore "we discard the latent action head ... and replace it with a new action head to generate ground truth actions" (LAPA, Section 3.3). The latent actions are thus a pretraining target only. In MoAct, the transition tokens are recomputed at every control step and condition the action expert through layer-wise keys and values, and the deployed policy depends on them: freezing them at their first-step values reduces success from 71.7% to 20.6% on Google Robot and from 65.6% to 41.7% on WidowX, and removing them reduces success to 26.4% and 15.6% (Appendix D.2).
References
[1] S. Ye, J. Jang, B. Jeon, S. Joo, J. Yang, B. Peng, et al. Latent Action Pretraining from Videos. In ICLR, 2025. [2] J. Won, K. Lee, H. Jang, D. Kim, and J. Shin. Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model. In ICML, 2026.
Response to Reviewer fTMC (Part 3/8)
Response 1 (2/2)
Relative to DUST [2]: MoAct conditions actions on the predicted change, while DUST generates the future state as a second diffusion modality.
- DUST's target is the future state itself. DUST defines its world-modeling target as "the representation of the future observation produced by the vision encoder", the observation "obtained after executing the action chunk" (DUST, Section 3), namely "the future image embedding derived from the SIGLIP-2 representations" (DUST, Section 5). In MoAct, the predictor receives the preceding grid as an explicit reference, so each token only needs to explain the change to rather than re-encode the future scene.
- DUST generates future embeddings at inference; MoAct generates none. In DUST, the diffusion model "takes the triplet as input, ... composed of the robot state, noised actions, and noised future observation embeddings" (DUST, Section 4.1), and "during inference, we jointly sample actions and vision in parallel" (DUST, Section 4.3). In MoAct, the dynamics stream is not a diffusion modality: it is computed once per control step without noise and reused across all flow-integration steps, and the DINOv3 encoder and predictor are discarded at inference (Appendices A.3–A.4).
- The two dual-stream designs exchange information differently. In DUST, the action and vision streams "are concatenated ... during the shared cross-modal attention layer", so that "generated actions constrain plausible future states, and predicted states guide action generation" (DUST, Sections 4.1 and 4.3). In MoAct, information flows one way: the action queries attend to the dynamics keys and values under stop-gradient, so the action loss does not propagate back into the dynamics stream (Section 3.1).
The tokens carry change-specific information, as the formulation intends. On 1,024 samples each from Fractal and Bridge, the tokens reduce the successor-prediction error on changing regions by 39% and 37% relative to copying the preceding features, while matching the copy baseline on static regions (Appendix D.1). The compact representation thus carries the information about where and how the scene changes, and static content is propagated from the reference.
References
[2] J. Won, K. Lee, H. Jang, D. Kim, and J. Shin. Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model. In ICML, 2026.
Response to Reviewer fTMC (Part 1/8)
Thank you for recognizing the strong performance of MoAct with only a 1B-parameter model and its efficiency at inference, where neither the DINOv3 encoder nor the predictor is required. We appreciate the time you spent on our paper.
Before the point-by-point responses, we would like to correct three statements in the review that are central to its assessment:
Beyond these corrections, we have added component-level analyses of the designs you list (Appendix D of the revision):
The component-level evidence is summarized in Response 9.
Anonymous resources:
We address your comments point by point below.
References
[3] Y. Su, S. Chen, H. Shi, M. Liu, Z. Zhang, N. Huang, et al. World Guidance: World Modeling in Condition Space for Action Generation. In ICML, 2026. [4] D. Driess, J. T. Springenberg, B. Ichter, L. Yu, A. Li-Bell, K. Pertsch, et al. Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better. In NeurIPS, 2025.