ASSET2SIM: Automatically Improving Simulation Physics of Articulated Objects

Approximate physics simulators have emerged as a key resource for training and evaluating robotic policies in simulation, especially for locomotion and navigation capabilities. A key bottleneck to realizing their utility for manipulation tasks is the dearth of assets (objects and environments) compared to the real world. Today's largest repositories of object assets focus on rigid objects offering limited affordances for manipulation beyond global repositioning. On the other hand, articulated objects which afford more nuanced manipulation (e.g., staplers, trashcans, cabinets, microwaves, folding knives) remain largely absent from simulators. Widely available curated libraries of articulated assets are limited: (1) they are either geometry-only, lacking annotations for physics properties like link inertias and joint frictions, or (2) they cover a limited number of assets designed with extensive manual effort and still have noisy physics annotations, as we show. We present Asset2Sim, an autonomous vision-language model-guided procedure to improve the physics properties of articulated object assets for sim-readiness. Asset2Sim utilizes multiple render views, geometric data, semantic annotations, and volumetric properties to iteratively refine estimates of an articulated asset's physics properties and initial states. We also develop an evaluation protocol to thoroughly validate the physics fidelity, RL training-readiness, and initialization stability of Asset2Sim assets in robot manipulation tasks, establishing over $2.12\times$ higher mean sim-to-real correlation and over $2.14\times$ higher average policy training success rate over the next-best baselines. We will release the Asset2Sim-X library of 1892 sim-ready articulated object assets spanning 44 categories, as well as our code to enable scaling such repositories further in the future.