- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Asset2Sim, a pipeline that uses vision-language models (VLMs) to improve the simulation readiness of articulated URDF assets. Given an asset, the method uses multi-view renderings and mesh geometry to estimate physical properties such as mass, inertia, friction, damping, and scale. It then checks the predicted parameters for physical validity and uses simulator feedback to refine the initial joint configuration when self-penetration is detected, while attempting to preserve the intended object state. The paper evaluates the approach in terms of initialization stability, perceived physical realism, robot policy performance, and sim-to-real correlation. Comparisons with VLM-based and human-annotated baselines are conducted on articulated manipulation tasks, including evaluations with VLA policies and state-based reinforcement learning. The authors also plan to release a collection of refined articulated assets based on PartNet-Mobility.
Critical strengths
1.I think the paper addresses an important problem for robot manipulation. Existing articulated object assets often have usable geometry and joint structures, but their physical parameters are not necessarily suitable for simulation. Improving these assets directly could save considerable manual effort when setting up manipulation environments. I also find the focus on refining existing URDFs more practical than building new assets from scratch.
2.One strength of the paper is that it evaluates the assets beyond whether they are physically valid at initialization. The experiments also examine interaction realism and downstream policy performance, including VLA-based manipulation and state-based RL training. These evaluations provide a more useful picture of simulation readiness than static checks alone. The planned release of Asset2Sim-X, with 1,892 assets across 44 categories, could also be valuable to the robotics community if the assets prove reliable in downstream use.
Critical weaknesses
1.I am not entirely sure how simulator feedback is used to refine the physical parameters. The paper describes how the VLM estimates properties such as mass, inertia, friction, and damping, but Section 2.4 seems to focus on optimizing the initial joint configurations to reduce self-penetration. It is therefore unclear whether the physical parameters themselves are updated iteratively based on simulation results, or whether they are estimated first and then kept fixed.
2.I am not fully convinced that the reported improvements come from more accurate physical parameters. The method also adjusts the initial joint configurations, but I could not find an ablation separating the effects of these two components. A better initial configuration alone might improve stability and task success without necessarily making the dynamics more realistic.
Questions for authors
1.The source of the reported improvements in simulation stability and manipulation performance is unclear. The current experiments do not seem to isolate the effects of physical property estimation from those of initial joint configuration refinement. As a result, it is difficult to determine whether the gains are primarily attributable to more accurate physical parameters, better initial configurations, or both.
2.The reported greater-than-2.12x improvement in mean sim-to-real correlation is confusing to me. Using a simple arithmetic mean of the three VLA SRCC values in Table 2 gives approximately 0.513 for Asset2Sim and 0.267 for the strongest baseline, Human (GenSim2), corresponding to about 1.93x rather than 2.12x. It is unclear how the reported 2.12x figure is obtained.
2 comments
Dear Reviewer QpWd,
Thank you for your time and effort in providing your thoughtful review. As the discussion period is ending soon, we would like to gently follow up on our rebuttal response. We are happy to provide any further clarifications if any of our points remain unclear, or if you have any further questions.
Thank you!
Rebuttal (1/1)
Thank you for the feedback, and for recognizing the utility of our approach for improving robotics simulation design.
Simulator-in-the-Loop Refinement
We update the positions of the focus joints (defined in lines 194-196), which are configurations in the feasible joint space (defined in lines 188-190). Thus, we only updating the kinematic properties of the joints during refinement for the subset of joints in Jfocus(t). The other properties (mass, inertia, friction, damping, etc.) are generated in a single pass, as shown in Figure 1.
We respectfully disagree that we do not isolate the effects of physical property estimation from joint configuration refinement: our Direct VLM baseline is equivalent to running the Asset2Sim pipeline without any iterative simulator-guided refinement. The Direct VLM baseline uses the same physical properties as Asset2Sim alongside the VLM’s one-shot estimated joint positions. The performance difference between Asset2Sim and Direct VLM can hence be reasonably attributed to the usage of iterative refinement for optimizing joint configurations. Additionally, our experiments on the GenSim2 assets use human-assigned object size and joint configurations alongside default simulator values for the remaining properties. The difference between GenSim2 and Asset2Sim results quantifies the impact of correct physical property estimation, as joint configurations are optimized for low interpenetration in both cases.
SRCC Score Calculation
Our original estimate reported the improvement in mean SRCC across all environments between GenSim2 and Asset2Sim for π0.5, which is the median result in terms of performance increase between the 3 evaluated VLAs. We have incorporated your feedback to update this estimate to reflect the mean of improvement in mean SRCCs across all 3 VLAs, and updated our paper to use an updated 1.92x figure.