I-OctoScene: Multi-agent 3D Scene Synthesis with Multimodal Interactive Instructions
Abstract
3D Scene Synthesis (3DSS) has emerged as a key enabler of embodied AI, providing diverse and controllable 3D environments for training and evaluation. Although recent approaches have achieved notable progress in 3DSS, they are still constrained by closed and monolithic paradigms, limiting synthesis personalization and quality. To address these issues, we propose I-OctoScene, an Interactive Multi-Agent 3DSS framework that generates 3D indoor scenes through multi-agent collaboration guided by progressive external interactions. Specifically, I-OctoScene introduces a Progressive Multimodal Interactive Refinement module (PMIR) to support the interpretation and integration of external feedback, enabling controllable scene generation aligned with personalized user requirements. In addition, an Intent-Aligned Tool Coordination mechanism (IATC) leverages Adaptive Tooling Skills (ATSs) to systematically orchestrate tool usage, facilitating complex external intent resolution and coordinated tool collaboration, ultimately improving synthesis quality. Extensive quantitative and visualization experiments across diverse room-type settings demonstrate the superiority of our proposed method.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.