- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes UniWAM, a unified World-Action Model integrating physical reasoning, world generation, and action prediction. The main contributions include: (1) A Mixture-of-Transformers architecture that enables information exchange among vision-language reasoning, future video generation, and robotic action prediction through joint multimodal attention. (2) A multi-source pretraining strategy with physical-language supervision, leveraging VQA, human egocentric videos, and robot demonstrations. (3) Future-Frame Noise Augmentation and History-Conditioned Flow Matching to improve action prediction robustness and inference efficiency. Experiments on LIBERO, LIBERO-Plus, RoboTwin 2.0, and real-world robotic tasks demonstrate competitive performance in manipulation success, out-of-distribution generalization, instruction following, and long-horizon execution.
Critical strengths
- UniWAM integrates semantic reasoning, world generation, and action prediction into a unified framework with strong practical relevance.
- The model achieves competitive results on LIBERO, LIBERO-Plus, and RoboTwin 2.0, particularly demonstrating strong out-of-distribution generalization.
- Real-world robotic experiments and multiple ablations support the effectiveness of multi-source pretraining, physical-language supervision, and history-conditioned action generation.
Critical weaknesses
- UniWAM shares substantial similarities with Motus in its three-expert MoT architecture and unified multimodal modeling. Although the authors emphasize differences in training recipes, data composition, and model capabilities, the distinct contributions of these improvements over Motus are not sufficiently clarified.
- The method relies on a relatively large unified Transformer architecture. Although some action-generation inference times are reported, a systematic analysis of overall computational cost, memory requirements, and scalability under constrained hardware settings is still lacking.
Questions for authors
- What are the key differences between UniWAM and Motus in terms of training recipes, data composition, and model capabilities? Can the authors further clarify the independent contributions of the proposed designs?
- Can the authors provide additional analysis of computational cost and inference efficiency, including GPU memory consumption, inference latency, and deployment feasibility under limited GPU resources?
- It would be interesting to discuss how UniWAM could be extended to more complex 3D environments involving geometric structures, occlusions, and multi-object interactions, and what architectural or training modifications might be required.
4 comments
Response to Reviewer 1v8m(2/2)
[Q3] It would be interesting to discuss how UniWAM could be extended to more complex 3D environments involving geometric structures, occlusions, and multi-object interactions, and what architectural or training modifications might be required.
Response: Thank you for this suggestion. We consider three complementary directions for extending UniWAM to more complex 3D environments.
1. Geometric structures. 1) Implicit alignment, as in Spatial Forcing [1], could be incorporated by aligning UniWAM's internal representations with geometry-aware features during training. 2) Explicit 3D inputs, as in SpatialVLA [2], could be incorporated by adding depth-derived 3D positional encodings to the visual tokens of our VLM-based physical reasoner. We have conducted preliminary experiments incorporating Spatial Forcing during post-training by aligning intermediate-layer features of the VLM-based physical reasoner with geometry-aware features from VGGT. The results are summarized below (C2C: clean-to-clean; C2R: clean-to-random).
| Configuration | RoboTwin 2.0 C2C (%) | RoboTwin 2.0 C2R (%) | Average (%) |
|---|---|---|---|
| UniWAM | 75.14 | 68.32 | 71.73 |
| UniWAM + Spatial Forcing | 78.72 | 70.41 | 74.57 |
As shown in the table above, incorporating implicit spatial representation alignment inspired by Spatial Forcing improves both C2C and C2R success, increasing average success from 71.73% to 74.57%. These preliminary results support geometric representation alignment as a promising extension to UniWAM. We leave the evaluation of explicit 3D inputs to future work.
2. Occlusions. SaPaVe's active-perception strategy [3] offers a feasible direction for extending UniWAM: the VLM-based physical reasoner could guide viewpoint selection, while a separate camera-pose prediction head could generate camera movements alongside the existing manipulation actions. This would require training the added head on image–language–camera movement data, followed by joint training on camera-control and manipulation trajectories with changing viewpoints. With a movable camera, the predicted movements could reveal occluded objects and provide updated observations for subsequent action prediction.
3. Multi-object interactions. We could extend UniWAM in two ways: by incorporating more multi-object interaction data during training, and by introducing an agent as a high-level planner to specify the order of object interactions and the execution plan. We have conducted preliminary experiments integrating GPT-6 Astra as the high-level planner, which passes its execution plans to UniWAM as text instructions for low-level action execution. We evaluate this extension on selected multi-object interaction tasks from RoboTwin 2.0, with results summarized below.
| Configuration | stack_blocks_three (C2C/C2R, %) | put_object_cabinet (C2C/C2R, %) | blocks_ranking_size (C2C/C2R, %) |
|---|---|---|---|
| UniWAM | 74.00 / 52.00 | 40.00 / 28.00 | 36.00 / 38.00 |
| UniWAM + GPT-6 Astra high-level planner | 76.00 / 59.00 | 47.00 / 39.00 | 48.00 / 46.00 |
As shown in the table above, adding the high-level planner improves success on all three tasks under both C2C and C2R, with C2R gains of 7, 11, and 8 percentage points, respectively. These results suggest that high-level planning can complement UniWAM's action prediction for multi-object interactions.
We will expand the discussion of these extensions and future research directions in the camera-ready version.
References
[1] Li, F., et al. Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Model. ICLR, 2026. https://openreview.net/forum?id=euMVC1DO4k
[2] Qu, D., et al. SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model. RSS, 2025. https://arxiv.org/abs/2501.15830
[3] Liu, M., et al. SaPaVe: Towards Active Perception and Manipulation in Vision-Language Action Models for Robotics. CVPR, 2026. https://openaccess.thecvf.com/content/CVPR2026/papers/Liu_SaPaVe_Towards_Active_Perception_and_Manipulation_in_Vision-Language_Action_Models_CVPR_2026_paper.pdf
Dear Reviewer 1v8m,
I hope this message finds you well. As the discussion period is nearing its end with less than five days remaining, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and were eager to address any remaining issues to improve our work.
Thank you for your time and effort in reviewing our paper.
Best regards, Authors.
Thank you for the detailed rebuttal and clarifications. I appreciate the authors' efforts in addressing my concerns. The paper presents a unified framework with promising experimental results and practical relevance. I will maintain my original score of 3.
Response to Reviewer 1v8m(1/2)
[W1] UniWAM shares substantial similarities with Motus in its three-expert MoT architecture and unified multimodal modeling. Although the authors emphasize differences in training recipes, data composition, and model capabilities, the distinct contributions of these improvements over Motus are not sufficiently clarified.
Response: Thank you for this comment. We acknowledge that UniWAM builds on the three-expert Mixture-of-Transformers (MoT) architecture of Motus. In the following, we will clarify our improvements over Motus, which constitute our distinct contributions.
1. Training recipe. Whereas Motus uses a frozen VLM as a multimodal encoder, we train the VLM-based physical reasoner during both pretraining and post-training, using physical language supervision to learn a shared mapping between semantics and actions while preserving pretrained language capabilities. As shown in Figure 5(b), unfreezing the VLM and adding physical language supervision progressively improve downstream performance. During post-training, history-conditioned flow matching uses recent action history as an initialization prior, improving action prediction performance and reducing the required denoising steps. Future-frame noise augmentation encourages the action predictor to tolerate imperfect future visual predictions, providing additional performance gains (see Figure 5(c)).
2. Data composition. We combine human egocentric data, robot teleoperation data, and visual question answering (VQA) data. In contrast, Motus includes neither VQA data nor action labels for its human data. We apply physical language supervision to both human and robot actions, while VQA data provides complementary visual–semantic and embodied reasoning supervision. Figure 5(a) shows that the three-source mixture achieves the highest average success rate among the evaluated data combinations. Human–robot co-training also yields increasing benefits at larger data scales (see Figure 4(a)).
3. Model capabilities. These designs support instruction following and generalization across environments. As shown in Figure 3(a–b), UniWAM achieves strong real-world Instruction-Following Rate and Task Success Rate compared with Motus and other baselines. Its generalization is further evaluated on RoboTwin 2.0 clean-to-random (C2R) and LIBERO-Plus (see Tables 1 and 2). Together with the component ablations, these results support the benefits of our training recipe and data composition.
We will make these distinctions and their supporting evidence more explicit in the camera-ready version.
[W2] The method relies on a relatively large unified Transformer architecture. Although some action-generation inference times are reported, a systematic analysis of overall computational cost, memory requirements, and scalability under constrained hardware settings is still lacking.
Response: Please see our response to Q2.
[Q1] What are the key differences between UniWAM and Motus in terms of training recipes, data composition, and model capabilities? Can the authors further clarify the independent contributions of the proposed designs?
Response: Please see our response to W1.
[Q2] Can the authors provide additional analysis of computational cost and inference efficiency, including GPU memory consumption, inference latency, and deployment feasibility under limited GPU resources?
Response: We have evaluated UniWAM's inference cost on NVIDIA H800, RTX 4090, and Jetson Thor T5000. All evaluations use four denoising steps and generate 16 actions per chunk, with an estimated computational cost of approximately 17,200 GFLOPs per chunk.
Chunk latency measures the end-to-end time from receiving image observations to outputting the complete action chunk.
For Jetson Thor, GPU memory usage is drawn from unified system memory (128 GB).
As shown in the table above, UniWAM runs on all three platforms with peak GPU memory usage below 20 GB. The RTX 4090 and Jetson Thor T5000 achieve action-generation throughputs of 10.83 and 7.94 actions/s, respectively, demonstrating deployment feasibility on consumer and embedded hardware. We will explore asynchronous inference to overlap action generation with execution and reduce deployment idle time in future work.