- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SimVLA, a sim-to-real framework for mobile manipulation that addresses the high cost of collecting expert demonstrations. It trains a π0.5-style VLA purely on three simulation-derived data sources. SimAction contains ~200K trajectories over 35 tasks, generated in procedurally built kitchens with skill APIs and motion planning. SimVQA contains ~1M VQA pairs derived from privileged simulator state. SimDeploy contains policy rollouts collected in diverse simulated environments. The framework also uses hybrid system identification and domain randomization. Extensive real-world experiments validate the effectiveness of the proposed method: SimVLA transfers zero-shot to a kitchen, a pantry and a real home, and outperforms π0.5 fine-tuned on 50 real demonstrations.
Critical strengths
- The proposed sim-to-real framework is a key approach to the difficulty of collecting mobile manipulation data, and the data synthesis pipeline is comprehensive.
- The paper is clearly written, and the experimental results are convincing.
Critical weaknesses
- Missing real-to-sim component. In my view, the key to sim-to-real is real-to-sim rather than data synthesis: building visually and physically faithful 3D assets of real scenes, then applying simple domain randomization. The paper skips this step. All scenes are procedurally generated from Objaverse assets, and the only real-to-sim step is system identification of the robot's own controller gains. Environment physics, such as the friction and PD gains of drawers and handles, is only randomized over hand-set ranges (Table 5). As a result, transfer depends on whether hand-crafted generators happen to cover the target domain. This limits generality beyond kitchen-like settings.
- Insufficient study of the key sim-to-real components. All ablations (Sec. 4.4–4.6) target VLA training ingredients rather than the sim-to-real gap. There is no analysis of how the visual gap between simulator and real world is reduced. For example, the author can explore whether 3D point-cloud input has a smaller sim-to-real gap than 2D images.
Questions for authors
- Could SimVLA be combined with or compared against a real-to-sim pipeline, e.g., reconstructed scenes with measured physical parameters? How much effort would it take to extend it beyond kitchens?
- Can you ablate visual vs. physical domain randomization and analyze the visual sim-to-real gap?
- Have you tried RGB-D or point-cloud inputs, and do they reduce the sim-to-real gap compared with RGB-only inputs?
3 comments
[Q1] Could SimVLA be combined with or compared against a real-to-sim pipeline, e.g., reconstructed scenes with measured physical parameters? How much effort would it take to extend it beyond kitchens?
We thank the reviewer for the constructive feedback. A real-to-sim pipeline can be integrated with SimVLA as a complementary extension to enable deployment in specific target scenes. Specifically, a single image of a target environment can be processed using ACDC [5] to segment fixtures and estimate metric depth, followed by scene reconstruction using SimVLA's procedural assets and demonstration generation. Now, our code supports this integration; however, we have not conducted fine-tuning or experiments using this real-to-sim integration in the current work. We emphasize that our core contribution remains zero-shot sim-to-real transfer for mobile manipulation without environment-specific information.
A procedurally generatable environment is the main requirement for extending SimVLA beyond kitchens, making such an extension straightforward once available. Furthermore, a trained policy can be deployed outside kitchens as long as the task remains the same. For example, we trained a "Push Chair" policy using data collected from kitchen table chairs and evaluated it zero-shot in a study room generated with SceneSmith [6], where it successfully pushed a study desk chair.
We added the real-to-sim integration in Appendix K (Real-to-Sim). The supplementary material includes updated code and corresponding generated data.
References
[1] Deng et al., GraspVLA: A Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data, CoRL, 2025.
[2] Deshpande et al., MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation, CoRL, 2026.
[3] Black et al., : A Vision-Language-Action Model with Open-World Generalization, CoRL, 2025.
[4] Bjorck et al., GR00T N1.5: An Improved Open Foundation Model for Generalist Humanoid Robots, NVIDIA, 2025.
[5] Dai et al., Automated Creation of Digital Cousins for Robust Policy Learning, CoRL, 2024.
[6] Pfaff et al., SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes, ICML, 2026.
If you have any further questions or if there are remaining points you would like us to clarify, we would be happy to provide additional clarification before the discussion period ends.
Thank you again for your time and thoughtful feedback.
We thank the reviewer for the detailed and constructive feedback.
[W1] Missing real-to-sim component. … building visually and physically faithful 3D assets of real scenes, then applying simple domain randomization.
We agree that real-to-sim—building visually and physically faithful 3D assets of real scenes—is an important factor in improving sim-to-real performance. However, our core contribution is zero-shot sim-to-real transfer for mobile manipulation without environment-specific information. Following prior sim-to-real VLA works [1], we leverage domain randomization to promote generalization across diverse environments without environment-specific fine-tuning. Concurrently, MolmoB0T [2] demonstrates the effectiveness of large-scale simulation training with diverse environments and domain randomization. Accordingly, SimVLA generates SimAction data across 100 diverse kitchen scenes, incorporating multiple object instances per category (e.g., 80 different mugs from Objaverse) and 41 types of domain randomization spanning camera positions, kitchen environments, object assets, robot physical configurations, material properties, and lighting conditions.
While deployment environments beyond kitchen-like settings may introduce additional visual and physical variations, these do not fundamentally limit our approach, as SimVLA does not rely on hand-crafted generators. Therefore, we respectfully suggest that the absence of real-to-sim components is not a weakness, but is a deliberate difference in research focus.
[W2-1, Q2] All ablations (Sec. 4.4–4.6) target VLA training ingredients rather than the sim-to-real gap. … no analysis of how the visual gap between simulator and real world is reduced. … Can you ablate visual vs. physical domain randomization and analyze the visual sim-to-real gap?
The key component that enables sim-to-real transfer is leveraging three complementary forms of simulation supervision—SimAction, SimVQA, and SimDeploy—within a two-stage VLA training. SimVLA suggests that simulation can serve not only as a source of action trajectories (SimAction), but also as a source of visual-language supervision (SimVQA) and deployment-oriented rollout data (SimDeploy). By these, SimVLA demonstrates robust performance across five long-horizon mobile manipulation tasks in three real-world environments. In a real home, SimVLA achieves an average task progress of 58.5% and a binary success rate of 45% (Table 15), without real-world training data or environment-specific information.
We acknowledge in Section 6 (Limitations & Future Work) that our work does not measure how the sim-to-real visual gap is reduced or compare against physical domain randomization. Instead, our work places greater emphasis on providing the ingredients and instructions that enable zero-shot sim-to-real transfer in mobile manipulation.
We clarified our experimental design in Section 4 (Experiments) and made this point clearer in Section 6 (Limitations & Future Work) of the revised paper.
[W2-2, Q3] Have you tried RGB-D or point-cloud inputs, and do they reduce the sim-to-real gap compared with RGB-only inputs?
Thank you for suggesting an interesting idea! Now, our simulation pipeline supports generating RGB, depth, and point-cloud data, although our VLA training currently uses only RGB inputs. Our goal is to develop an RGB-camera-based VLA, which is widely adopted design choice of recent robot foundation models [1, 2, 3, 4]. We agree that these additional modalities may help reduce the sim-to-real gap. However, we respectfully claim that comparing RGB-D and point-cloud inputs is beyond the scope of our work.
We added RGB-D/Point-cloud data generation to Appendix D (SimAction) and Figure 15 of the revised paper. The supplementary materials include the updated code.