- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CtrlWAM, a unified world action model designed to improve the physical consistency between predicted actions (intent) and generated visual futures (foresight). The central observation is that conventional joint video-action diffusion independently perturbs recorded actions and videos, potentially producing intermediate denoising states in which the perturbed actions are inconsistent with the visual observations.To address this issue, the authors introduce three components:physically aligned noising,warped video-action noise schedules and structured multi-agent conditioning.The method is evaluated on autonomous driving and robotic manipulation. On the driving benchmark, CtrlWAM-M reduces Self ADE from 0.538 m to 0.513 m, Rollout ADE from 2.920 m to 2.804 m, and Cmd ADE from 1.023 m to 0.981 m compared with the SFT baseline. On RoboTwin/WorldArena, the proposed method improves EWMScore from 58.70 to 61.75 over Cosmos 3 SFT.
Critical strengths
1.This paper identifies an important issue in joint video-action generation: independently noised actions and visual observations may not correspond to the same physical future. This issue is particularly relevant for embodied applications in which action trajectories and visual consequences must remain consistent. The motivation is intuitive and clearly illustrated in Figures 1-3.
2.Physically aligned noising provides a concrete way to incorporate the visual consequences of action perturbations into model training. Rather than introducing only an additional consistency loss, the method modifies the training inputs to expose the model to counterfactual action-observation pairs. This is a meaningful training-design contribution.
3.The experiments cover autonomous driving and bimanual robotic manipulation, including action prediction, video-action consistency, command following, and video generation quality. The paper also provides comparisons across rendering sources and noise schedules, as well as matched-budget robotics controls.
Critical weaknesses
-
Table 2 reports a VLM compliance improvement from 10.07% for CtrlWAM-S to 18.94% for CtrlWAM-M. However, Appendix E.2 explicitly states that only CtrlWAM-M receives the edited surrounding-agent command, whereas the other baselines do not receive this information.Therefore, the comparison primarily demonstrates the advantage of providing an explicit agent-control interface over models without such an interface. It does not fully isolate the effectiveness of the proposed representation or conditioning mechanism.
-
An absolute compliance score of 18.94% remains relatively low, raising questions about the practical reliability of counterfactual agent control.A stronger comparison would include an alternative agent-conditioned baseline receiving the same command information, along with quantitative trajectory-following metrics for controlled agents.
-
The ablation results in Table 4 are informative but reveal a nuanced relationship between the proposed components. The GT-Src shared-schedule configuration achieves a lower Rollout ADE (2.968 m) than the uncleaned Rendering warped configuration (3.019 m), while the Cleaned warped configuration achieves the best Rollout ADE (2.881 m).This suggests that rendering quality and the proposed noise schedule may interact substantially. Furthermore, the warped schedule decreases PSNR for all three rendering sources in Table 4, indicating a trade-off between motion-related metrics and visual reconstruction quality.
-
The method requires simulator execution and rendering of perturbed trajectories. The driving training set contains approximately 269,000 aligned items, and the experiments rely on a large pretrained backbone and substantial GPU resources.This paper does not sufficiently quantify the additional simulation and rendering costs, preprocessing time, or training-efficiency trade-offs relative to standard fine-tuning.
Questions for authors
Please refer to the weakness