- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces UNITAS, a 3D-native world action model for embodied manipulation that represents visual observations, robot and human-hand motion, and scene dynamics within a shared metric coordinate frame. It represents agent motion as action flow and environmental changes as action-conditioned scene flow. World-aligned positional embeddings ground visual features with or without observed depth, while a physical-time trajectory tokenizer encodes each point trajectory into a spatially anchored token. Coupled diffusion transformers predict point flows and executable robot commands, supporting both action-conditioned scene prediction and policy-only inference without future-flow generation. The 1.7B-parameter model is evaluated on LIBERO, LIBERO-Plus, RoboTwin, VLABench, and two real-robot tasks. Reported results include 99.8% success on LIBERO, 89.1% on LIBERO-Plus, and approximately 35% lower moving-point trajectory error than PointWorld on RoboTwin.
Critical strengths
-
The paper presents a coherent 3D representation that places visual observations, robot and human-hand motion, and scene changes in a shared metric coordinate frame within each interaction window, rather than treating depth merely as an additional input. Action flow, scene flow, and spatially anchored trajectory tokens form a unified framework grounded in manipulation geometry. This is a valuable conceptual contribution.
-
The evaluation is broad, covering single-arm and bimanual manipulation, robustness to distribution shifts, real-robot execution, and action-conditioned scene prediction. This breadth provides evidence for the framework’s effectiveness across multiple settings.
-
The model demonstrates strong quantitative results in 3D scene prediction. Compared with PointWorld, which also directly predicts 3D trajectories, it reduces Moving ADE from 0.656 cm to 0.427 cm, an approximately 35% reduction, while improving Moving Acc@1cm from 88.22% to 92.31%.
-
The attention structure and key–value caching enable efficient policy-only inference, allowing the model to generate executable commands without predicting future point flows. The reported latency of 55.77 ms per action chunk on an RTX 5090 D highlights the practical value of this design.
Critical weaknesses
-
Line 106 reports a success rate of 65% for Pack the Ball, whereas Table 2 and Section 4.3 report 80% (16/20).
-
According to Eq. (3) and Section 3.5, action flow (A) and executable commands (u) are generated separately given a shared context, using independent noise, while scene prediction (S) is conditioned on (A) as its sole action input. This design does not explicitly ensure consistency between the generated action flow and the motion resulting from executing the generated commands, particularly when multiple valid actions exist for the same task. Appendix F.2 provides indirect evidence by comparing predicted and observed scene changes, but does not directly quantify the discrepancy between the two action representations. I suggest comparing the generated action flow with the actual gripper-point trajectories obtained by executing the generated commands, and reporting both trajectory and endpoint errors. This would more directly assess whether the predicted scene consequences correspond to the actions actually executed, strengthening the case for unified world–action modeling.
-
Table 3 shows that removing action-flow supervision or history tokens reduces overall success by 6.85 and 9.82 percentage points, respectively, whereas adding scene-flow supervision to the reference model containing both components yields only a 0.73% improvement, from 85.74% to 86.47%. These ablations therefore provide stronger evidence for the value of geometric action supervision and motion history in policy learning than for the additional benefit of scene-dynamics supervision. Since scene flow is not generated during policy-only inference, I encourage the authors to distinguish clearly between the value of scene prediction as an auxiliary training objective and the necessity of generating scene predictions at inference time.
Questions for authors
Please refer to the "Critical Weaknesses" for specific issues. If these can be resolved, I will raise the score.
5 comments
Response to Reviewer J9Sa (Part II)
2. Inference-time action selection
To evaluate the additional value of consulting predicted scene dynamics, we independently sample command chunks and predict scene flow under each candidate’s associated action-flow condition. We fit rigid object poses to the predicted points and render the available simulator meshes at those poses. Qwen3-VL-4B-Instruct [2] evaluates these rendered outcomes against the current observation and instruction to select a command chunk. After execution, the procedure repeats with the updated observation.
With 100 trials per task and method, the full scene-guided selection procedure improves success on all three tasks, increasing mean success from 71.67% to 79.67% (+8 percentage points).
| RoboTwin task | Direct policy (%) | Scene-guided best-of-4 (%) | Gain (pp) |
|---|---|---|---|
hanging_mug | 61 | 74 | +13 |
put_object_cabinet | 77 | 82 | +5 |
place_can_basket | 77 | 83 | +6 |
| Mean over all 3 tasks | 71.67 | 79.67 | +8.00 |
These results demonstrate an additional use of generated scene dynamics for action selection, while scene generation remains optional for policy execution.
Manuscript updates
The revised manuscript includes the following additions.
- W1 — Corrected success rate in Section 1.
- W2 — Direct action–execution agreement in Appendix H.1, Table 15.
- W3 — Point subsampling in Section 3.4 and supervision ablation in Appendix H.2, Table 16. Action selection in Section 4.5, Figure 6 and Table 5, and Appendix H.3.
References
[1] T. Yuan et al. Fast-WAM: Do World Action Models Need Test-Time Future Imagination? 2026, Section 4.3.2. arXiv:2603.16666
[2] S. Bai et al. Qwen3-VL Technical Report. 2025.
Kindly Reminder
Dear Reviewer J9Sa,
Based on your valuable feedback, we have supplemented our experiments to analyze the consistency between action flow and action command, and the role of scene flow prediction during training-time and test-time stages. We have also updated the revision as you suggested. As we approach the end of the discussion period, we would be grateful if you could review our response and consider our revised manuscript. Your constructive comments have helped us to significantly improve our work, and we believe that we have thoroughly addressed your concerns.
Thank you very much for your time and detailed review. We welcome any further additional questions or requests for clarification.
Yours sincerely,
The Authors
Response to Reviewer J9Sa (Part I)
We appreciate your recognition of our unified 3D representation, broad evaluation, and efficient policy-only inference. Your suggestions helped us clarify the role of scene prediction and provide direct evidence of action–execution consistency.
W1. Pack the Ball success rate
Thank you for catching this typo. We have corrected the introduction to 80% (16/20), consistent with Table 2 and Section 4.3.
W2. Direct agreement between action flow and executed motion
We appreciate the reviewer’s suggestion to directly evaluate action–execution consistency. Although the loss does not explicitly enforce kinematic agreement, the two branches share context and are trained against corresponding demonstration targets. We added the requested execution-based comparison.
We execute the generated commands and reconstruct gripper trajectories from the actual joint states using URDF forward kinematics. We compare these with predicted action flow in the same coordinate frame, matching point identities and executed steps. Correlations use displacements relative to their common initial positions.
ADE measures mean trajectory error, and FDE measures error at the last executed step of each prediction window. Pearson measures displacement correlation, while CCC additionally accounts for differences in mean and scale. Across both benchmarks, Pearson correlations exceed 0.95 and ADE remains below 1 cm, indicating close agreement between predicted action flow and executed motion under the evaluated protocol.
W3. Training-time and inference-time contributions of scene prediction
Thank you for highlighting this distinction. Our original policy-only results evaluate the training-time contribution of scene supervision; future action flow and scene flow are not generated during policy-only inference. We now provide additional supervision ablations and evaluate an optional scene-guided action-selection procedure.
1. Training-time scene supervision
To accelerate training and reduce GPU memory usage, our default setting drops 99% of scene points after voxelization and capping, retaining only 1% for supervision. This sparse supervision may partly explain the modest LIBERO-Plus gain from 85.74% to 86.47% (Section 4.4, Table 3, L417–431). Fast-WAM [1] similarly finds that future-video co-training benefits action learning.
We added a RoboTwin ablation comparing no scene supervision with drop rates of 0.99, 0.90, and 0.75, retaining 1%, 10%, and 25% of scene points, respectively. All configurations are evaluated at their 60k-iteration checkpoints. The main RoboTwin results in Table 1 use 200k-iteration checkpoints; extending all ablation runs to 200k iterations was not feasible within the available computational budget and rebuttal period. Trajectory and position errors are in mm, rotation errors in degrees, and success rates in percent.
Scene supervision reduces all reported prediction errors and improves closed-loop success by 3.7–4.6 percentage points across the tested drop rates. Retaining 10% or 25% of scene points provides additional success gains of 0.9 and 0.6 percentage points over the default 1% setting.