acceptodds
Under review as a conference paper at ICLR 2027

WSA: a 3D-Centric World-Spatial-Action Model for Generalizable Robot Control

Abstract

World-Action Models (WAMs) have emerged as a promising paradigm for robot control by using predicted world evolution to guide action generation. However, existing WAMs largely follow a one-way world-to-action paradigm. Real-world interaction, in contrast, is inherently driven by bidirectional temporal causality with implicit physics: the evolving world informs what actions should be taken, while the executed actions, in turn, determine how the world evolves. Without explicitly modeling this action-to-world direction, WAMs cannot assess whether the consequences of their generated actions align with the world evolution they predict, preventing WAMs from capturing consistent interaction dynamics from video demonstrations and thereby constraining their generalization capabilities in real-world scenarios. In this paper, we introduce WSA, a new modeling paradigm for embodied foundation models. WSA develops a World-Spatial-Action understanding by jointly modeling three interconnected processes: how actions transform the 3D world (world level), how spatial states evolve toward desired outcomes (spatial level), and how target 3D world guide appropriate robot behaviors (action level). By integrating these processes, WSA enables a more coherent and tightly coupled understanding of embodied interactions. Compared with prior WAMs trained on tens of thousands of hours of real-robot data, WSA leverages 6k hours of video demonstrations (only 1k from real robot), demonstrating strong data efficiency. Moreover, even our 3B-parameter WSA surpasses larger WAMs with 6B–8B parameters, highlighting its parameter efficiency. Experiments show that WSA achieves a 93.3% success rate on RoboTwin2.0, delivering an average 20% improvement over state-of-the-art models on real-world OOD robot control tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
embodied foundation model, world-action model, robot policy

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_DW69OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WSA1 presents a well-motivated approach that integrates forward and inverse dynamics modeling (i.e., action-to-world and world-to-action prediction) into a unified world-action framework, distinguishing it from many existing approaches that primarily use predicted world states to guide action generation. The experimental results on both simulation benchmarks and real-world robotic tasks demonstrate promising performance and generalization capabilities. The ablation studies further provide convincing evidence of the effectiveness of the proposed design. I also appreciate the authors' efforts to conduct real-world experiments with representative baseline comparisons. Overall, I find this work promising and have only a few relatively minor concerns. I would be willing to increase my score if the authors adequately address the questions raised below.

Critical strengths
  1. Well-motivated and intuitive methodological design. The paper proposes jointly learning forward dynamics (action-to-3D-world prediction) and inverse dynamics (3D-world-to-action prediction) within a unified world model. Although similar bidirectional learning ideas have been explored in flow-matching-based dynamics modeling, I still find their application to world-action modeling intuitive and interesting. The authors further leverage a Mixture-of-Transformers (MoT) architecture to establish dependencies between the forward and inverse prediction branches within a shared latent space, potentially improving the robustness of the learning framework.

  2. Comprehensive experimental comparisons. Table 2 provides extensive comparisons with representative baselines across simulation benchmarks, where WSA1 achieves competitive and, in several cases, superior performance.

  3. Convincing ablation studies and generalization performance. The ablation studies clearly demonstrate the benefits of introducing bidirectional world-action modeling, yielding substantial improvements in average task success rates. I am also convinced by the scene generalization results in Table 6, where WSA1 demonstrates stronger robustness to scene randomization than the compared methods.

  4. Promising real-world experimental results. The real-world experiments further demonstrate the effectiveness of WSA1, showing clear improvements over the evaluated baselines. I particularly appreciate the inclusion of real-world robotic experiments and direct baseline comparisons, which strengthen the practical significance of the work.

Critical weaknesses
  1. Presentation and clarity. Although the paper is generally well written and logically structured, the connections between the main text and figures could be improved. For instance, I could not find an explicit reference to Figure 1 in the main text. Similarly, the methodological descriptions do not sufficiently guide readers through Figure 2, making the proposed architecture more difficult to follow. I encourage the authors to integrate these figures more explicitly into the narrative.

  2. Insufficient theoretical justification for bidirectional modeling. While I can intuitively understand the benefits of jointly learning forward and inverse dynamics, I would appreciate a more rigorous theoretical explanation or discussion of why this design improves causal reasoning and world-action consistency. In particular, it would be helpful to clarify how bidirectional modeling provides advantages beyond learning statistical dependencies between actions and world states.

  3. Unclear handling of coupled forward and inverse dynamics. The paper does not sufficiently explain how the proposed architecture handles the temporal dependencies and potential asynchrony between the forward (action-to-world) and inverse (world-to-action) processes. These processes are inherently coupled, as actions influence future world states, which in turn inform subsequent actions. I would appreciate a clearer explanation of how the MoT architecture coordinates these interdependent predictions and maintains their consistency.

  4. Additional experimental concerns.

  • LIBERO performance: WSA1 does not outperform LingBot-VA on LIBERO, although the authors attribute the limited improvements to benchmark saturation. I consider this a relatively minor concern, particularly given the promising real-world results, but a more detailed discussion would be helpful.

  • OOD ablations: I am curious how the bidirectional modeling ablations would affect the out-of-distribution generalization results reported in Tables 6 and 7. Such experiments could further substantiate the relationship between bidirectional modeling and generalization.

  • World prediction quality: The current evaluations primarily focus on robotic task success rates. I would also be interested in understanding how bidirectional modeling improves the quality of predicted future world states, particularly in terms of geometric accuracy, dynamics prediction, and physical consistency. Quantitative evaluations of these aspects would provide stronger evidence for the claimed benefits of the world model.

Questions for authors
  1. Could the authors clarify how the proposed MoT architecture handles the coupled temporal dependencies between forward and inverse dynamics? Specifically, actions induce changes in world states, while evolving world states simultaneously influence subsequent action predictions. I suspect that the bidirectional attention mechanism plays an important role in coordinating these processes, but a more explicit explanation of how temporal consistency is maintained would be valuable.

  2. For the remaining questions, please refer to the concerns raised in the Weaknesses section, particularly regarding the theoretical justification for causal modeling, the influence of bidirectional modeling on OOD generalization, and the quantitative evaluation of future world-state prediction quality.

2 comments

I sincerely thank the authors for their detailed responses and substantial additional experiments, which have effectively addressed most of my concerns regarding experimental validation, particularly the benefits of bidirectional modeling for OOD generalization and world prediction. I appreciate the authors' efforts, and I find the new experimental evidence convincing. My only remaining concern is the physical intuition underlying the proposed bidirectional mechanism. While the authors have clarified how forward and inverse predictions interact through shared attention, I still feel that the explanation is primarily learning-driven, with limited discussion of the underlying forward–inverse dynamical consistency from a physical perspective. I would encourage the authors to explore this aspect more explicitly in future work, as it could further strengthen the physical interpretation of the proposed framework. Given this remaining concern, I have decided to maintain my original rating. Nevertheless, I believe this work makes a valuable and well-supported contribution, and I support its acceptance at ICLR.

Thank you for your positive feedback and support!

Reviewer_n3ofOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes WSA$_1$, a 3D-centric robot policy that jointly learns future visual and 3D feature prediction with action generation. Its Mixture-of-Transformers architecture couples two spatial experts and an action expert through bidirectional world–action interaction, using frozen Depth Anything features for 3D supervision. WSA$_1$-L reports 93.3% average success on RoboTwin2.0 and 80.5% across the real-robot tasks,. Ablations examine the contributions of bidirectional interaction and the prediction objectives.

Critical strengths
  • The joint-learning design coherently connects world prediction and action generation. Bidirectional interaction improves success from 73.5% to 80.2%, and visual and Depth Anything supervision during training both enhance WSA's performance. These ablations support the proposed coupling architecture design.
  • Real-robot improvements are substantial: WSA-B/L achieve 77.3%/80.5% mean success versus 55.5% for π0.5π_{0.5} across 7 tasks. The strong performance in RoboTwin2.0 and LIBERO benchmarks strengthen the contribution.
Critical weaknesses
  • Table 4 shows that Depth Anything supervision improves performance, but does not establish how much of this gain is specific to geometric information rather than additional feature supervision or teacher distillation. A comparison replacing Depth Anything targets with general-purpose visual features, while preserving the architecture and information paths, would better support the paper’s central 3D-centric claim.
Questions for authors

Beyond the 3D-loss ablation, is there further experimental evidence that the gains specifically come from 3D geometry rather than additional supervision?

Reviewer_3VM1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes WSA1, a 3D-centric world-action model that jointly learns future visual states, action-conditioned 3D world representations, and robot actions. The key idea is to connect world prediction and action generation through bidirectional attention. The model is pretrained on 6,000 hours of demonstrations from simulation, human videos, and real robots. Experiments on RoboTwin2.0, LIBERO, and seven real-world tasks show strong performance, including improvements in scene and cross-embodiment generalization.

Critical strengths
  1. The bidirectional world-action modeling is an interesting idea. Rather than using future world predictions only to guide actions, the model also learns how actions affect the world. The ablations in Tables 3 and 4 support the usefulness of jointly modeling these relationships.

  2. The experimental results are strong. The paper evaluates a wide range of simulation and real-world tasks. In particular, WSA1 achieves 80.5% average success across seven real-world tasks, compared with 55.5% for the strongest baseline. The scene generalization results are also encouraging.

Critical weaknesses
  1. The claim about learning physical causality needs stronger evidence. The ablations show that bidirectional attention improves task success, but it is unclear whether the model actually learns how different actions change the physical world. The 3D predictions are mainly supervised using features from a pretrained model. A quantitative evaluation of action-conditioned 3D prediction would better support the central claim.

  2. The cross-embodiment generalization results are somewhat difficult to interpret. Appendix B.3 shows that using normalization statistics computed from target-robot demonstrations improves success from around 17% to 47%. This is a substantial improvement that does not come from the learned policy alone. It would be helpful to clarify how this affects the zero-shot claim and whether the baselines use comparable normalization.

Questions for authors
  1. Can the authors quantitatively evaluate whether the 3D world model predicts different and physically consistent outcomes under different candidate actions?

  2. For cross-embodiment evaluation, why are normalization statistics from target-robot demonstrations needed? Are the same normalization procedures applied to all baselines?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Area Chairs and Reviewers

We sincerely thank the Area Chair and Reviewers (3VM1, n3of, DW69) for their constructive feedback and for recognizing our bidirectional World–Spatial–Action formulation, OOD generalization, and real-robot results. We have revised the manuscript accordingly, with all updates highlighted in blue in the updated PDF document, and summarize the main revisions below:

  • Quantitative evaluation of bidirectional world–action interaction (3VM1, DW69, n3of; Section 4.3, Appendix B.6): Added symmetric dual-future evaluations on LIBERO and RoboTwin, showing that action-conditioned predictions (At→GtA_t \rightarrow G_t) consistently match their own executed futures over alternative futures in both RGB and decoded 3D depth (Tables 8, 23–25; Figures 4, 6–9), while perturbing intermediate 3D tokens (Gt→AtG_t \rightarrow A_t) monotonically degrades decoded geometry and joint control (Table 9; Figures 10–11).
  • 3D geometric vs. 2D visual semantics (n3of; Section 4.1, Table 4, Appendix B.4): Replaced the frozen DA3-Large with backbone-matched DINOv2-Large under identical architecture and training; using DINOv2 reaches 76.4% average success (+0.6 over no 3D loss) versus 80.2% with DA3 (+3.8 over DINOv2), providing evidence for the benefit of cross-view geometry-aware supervision beyond generic 2D feature distillation.
  • Directional world–action ablations under OOD scene shift (DW69; Appendix B.4, Table 20): Evaluated directional attention ablations under Clean-to-Randomized scene shift (OOD-C2R); removing either interaction direction reduces OOD performance from 77.8% to 74.7% / 74.0%, indicating that coupled bidirectional modeling strengthens OOD robustness.
  • Cross-embodiment calibration and baseline fairness (3VM1; Section 4.1, Appendix B.3, Table 18): We clarify that zero-shot transfer in Table 7 involves zero target-robot policy training while applying target-robot joint statistics at inference for kinematic alignment, and verify that under the exact same ARX-X5 statistics, WSA$_1$-B (47.4%) outperforms all joint-space baselines (17.8%–26.3%) as well as calibration-free EEF baselines.
  • Exposition, LIBERO analysis, and claim precision (3VM1, DW69; Sections 1, 3.1–3.2, 4.1, Figures 1–2, Appendix B.1): We now explicitly guide readers through Figures 1–2 in the text, clarify forward–inverse coordination across subgoal alignment, denoising synchronization, and receding-horizon execution, expand the LIBERO saturation discussion (Table 16), and refine phrasing throughout to frame claims precisely as action-conditioned 3D prediction and geometry-guided control.

Detailed point-by-point responses and experimental breakdowns are provided in our individual replies to each reviewer. We thank the Area Chair and all three reviewers again for their dedicated time and constructive efforts in helping improve our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.