- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes VibeWorlding, a multimodal agent framework for constructing 3D open worlds. The authors introduce VWE-Bench, a benchmark comprising 2,616 3D assets, 323 seed worlds, and 6,828 queries. They further develop VibeWorlding-Gym, which integrates a unified 3D tool environment, a dual-constraint verifier, and SFT and GRPO training to improve agents' 3D scene construction capabilities. Experimental results demonstrate that the proposed approach significantly improves the performance of open-source MLLMs, enabling them to achieve performance comparable to several frontier closed-source models.
Critical strengths
-
The research problem is meaningful. This work integrates open-ended 3D scene construction and multi-turn editing into a unified framework. The proposed benchmark and interactive environment provide a valuable platform for systematically evaluating the 3D understanding and manipulation capabilities of multimodal agents.
-
The experimental evaluation is relatively comprehensive. The authors compare multiple frontier MLLMs and conduct ablation studies, human evaluations, and generalization experiments to validate the effectiveness of their approach. The performance improvements achieved through reinforcement learning are substantial.
Critical weaknesses
-
The methodological novelty could be further clarified. The proposed approach mainly combines existing techniques, including SFT, GRPO, and multimodal tool use, and shares certain similarities with related works such as SceneReVis. The authors should better highlight the key technical differences and unique contributions of their method.
-
The evaluation reliability has certain limitations. Physical feasibility is primarily assessed using AABB-based approximate collision detection, which may not accurately capture complex 3D collisions. Additionally, Verified and Unverified queries employ different scoring schemes, and the rationale for combining them into the Overall Pass@1 metric requires further justification.
Questions for authors
-
Compared with existing reinforcement learning approaches for 3D scene construction, such as SceneReVis, what are the key innovations of this work in terms of training strategies and multimodal feedback mechanisms?
-
Would the physical feasibility evaluation results remain consistent if more precise 3D collision detection methods were employed? Additionally, could the authors further justify the combination of different scoring schemes for Verified and Unverified queries into the Overall Pass@1 metric?
Minor Suggestions for Evaluation Reporting
Thank you again for the revisions. After reviewing the updated manuscript, I have two minor suggestions that could further improve the presentation and experimental analysis.
Additionally, the Pass@1 label in Figure 5 could be updated to reflect the revised metric definition. These are minor suggestions and do not affect my current recommendation of Weak Acceptance.