From Parts to Sim: Bottom-Up Autoregressive Generation of Sim-Ready 3D Assets
Abstract
Recent advances in 3D generation have enabled high-quality object synthesis, yet most generated assets remain static, limiting their use in physical simulation and embodied learning. A simulation-ready articulated asset must specify not only its geometry and appearance, but also how its components are organized, how they move, and which physical properties govern their interactions. Existing approaches often recover articulation after geometry generation or represent geometric parts and kinematic links at the same granularity, making it difficult to preserve fine details while maintaining a compact physical structure. We present a bottom-up autoregressive framework that models fine-grained geometric components and rigid kinematic links as distinct but connected levels, enabling the unified generation of geometry, articulation, and simulation-relevant physical attributes, including joint dynamics. To address the scarcity of fully annotated articulated data, we develop a scalable data pipeline and co-train the model using large-scale part-aware geometry and simulation-ready assets. Experiments demonstrate state-of-the-art performance in geometric fidelity, articulation accuracy, and simulation executability. Furthermore, manipulation policies trained on trajectories collected from our generated assets transfer successfully to real-world objects, demonstrating their value for sim-to-real robot learning.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.