Aster: Architecting Sim-Ready Worlds from Real Scenes for Embodied AI

Embodied AI increasingly relies on simulation for training and evaluation, creating a growing demand for scalable Real2Sim pipelines that transform real-world observations into simulation-ready environments. However, existing 3D reconstruction and digital twin approaches primarily optimize for visual fidelity, often producing unstructured meshes or cluttered scenes that are difficult to deploy in embodied simulators. We argue that Real2Sim should be formulated as a simulation-ready world generation problem rather than a geometry reconstruction problem. To this end, we present ASTER, a framework for architecting sim-ready worlds from real scenes for embodied AI. ASTER integrates simulation-oriented world understanding, world construction, and physics-based refinement to recover structured scene layouts and object assets, predict and assign physically plausible properties, and use simulation feedback to improve scene stability and executability. This design enables efficient world generation without extensive per-scene rendering. Evaluation of reconstructed scenes shows fewer collisions and higher stability, while policies trained in ASTER worlds achieve similar performance in matched ground-truth simulation scenes. In a challenging real-robot setting, adding ASTER-generated data increased observed real-robot success from 0/6 to 4/6 trials, providing a cheap way to scale up data for real-world robot deployment.