- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, the authors focus on multi-view spatial reasoning problem and propose a See Once, Ask Many (SOAM) framework. In this framework, the input images are first translated into reusable and object-anchored neural scene graph. Based on this scene graph, VLM will generate programs for each input question for explicit calculation of specific spatial properties. Experiments on VIEW2SPACE-v1 and SPAR-Bench show the effectiveness of this framework.
Critical strengths
-
Instead of leveraging a neural network-based model as previous methods, the authors use coding agents to design specific programs to deal with multi-view spatial reasoning problems, which is more explainable and can take advantage of the outstanding coding ability of current agents.
-
Once the scene graph is constructed, different questions could be answered within the same scene graph, which will increase the efficiency.
Critical weaknesses
-
Because the program backend is based on external geometry models but their prediction can be noisy and incorrect, I wonder how do the authors deal with such problem.
-
To my understanding, the whole framework is trained with labeled object states and task-specific output formats. Therefore, I wonder how could the training scale with more data and whether it will fail in open-ended scenes or naturally phrased questions.
Questions for authors
- The first step of the framework is detecting objects and clustering the detection of the same object. I wonder what tracking/detection model is leveraged and how does the framework perform when this step fails.