- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes WoRld As a Program, a framework for spatial reasoning and 3D scene editing with multimodal language models. It associates visually tagged objects with reconstructed 3D point clouds, exposes these objects as variables, and generates executable programs for geometric computation or scene modification. An adaptive mechanism selects between direct visual reasoning and program-based reasoning. The model is trained through spatial grounding followed by spatial program learning. Experiments cover three video spatial reasoning benchmarks and EditRoom-DB, supplemented by component ablations and analyses of grounding and program correctness.
Critical strengths
- The framework provides a clear connection between visual object references and executable geometric representations. Persistent object identifiers make the generated programs interpretable and allow geometric computations to operate on a shared scene representation.
- The component ablations and oracle-substitution experiments provide useful diagnostic evidence. In particular, replacing predicted object grounding with oracle grounding increases accuracy from 68% to 92%, whereas replacing generated programs with oracle programs increases it to 77%. This helps identify grounding as an important remaining bottleneck.
Critical weaknesses
- The incremental contribution over existing program-based spatial reasoning methods needs stronger experimental isolation.Direct comparisons with closely related methods such as pySpatial and VADAR, or controlled implementations of their interfaces using the same backbone, geometry, and supervision, would help establish the specific benefit of WRAP’s persistent visual–geometric binding.
- The reliability analysis does not sufficiently characterize errors in the reconstructed scene representation. WRAP depends on depth estimation, segmentation, and instance merging, but the experiments mainly vary reconstruction frame count or substitute oracle grounding. Controlled tests involving depth noise, incomplete masks, occlusions, and incorrect identity merging would clarify when persistent grounded variables remain reliable and when errors propagate into program execution.
Questions for authors
- How do the experiments isolate the contribution of the proposed method from the effects of training data and model capacity?
- Do the editing metrics penalize unintended modifications to objects that were not supposed to be edited?
- How is the correctness of generated programs assessed beyond successful execution?