- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this paper, the authors propose to use query-based pretrained geometry model to extract geometric cues for spatial reasoning. Different from previous methods that extract fixed features of geometry models first and obtain question-relevant clues then, the proposed AskGeo framework directly queries the decoder of pretrained geometry model to extract question-relevant geometric features, making the resultant features contain more important information. On both static and dynamic spatial reasoning benchmarks, the proposed framework achieves the best performance.
Critical strengths
To prove the effectiveness of the proposed framework, the authors conduct comprehensive experiments. First, they compare with different baselines across both static and dynamic spatial reasoning benchmarks, showing the effectiveness of extracting geometric features directly in a query-based approach rather than based on fixed geometric features. Second, they conduct numerous ablations to validate the design of different components.
Critical weaknesses
-
I think there should be an additional ablation study to evaluate whether the general understanding ability of the final model degrades on benchmarks such as Video-MME.
-
I wonder whether the model trained on DSR-Train could improve performance also on similar benchmarks such as VLM4D and STI-Bench.
Questions for authors
Please refer to the weaknesses.
2 comments
The authors address part of my concerns. I keep my rating for weak acceptance.
Response to Reviewer Gw9y
We thank the reviewer for the positive assessment and the useful suggestions. Changes in the revised PDF are shown in teal.
W1 & W2: General video understanding and generalization to other spatio-temporal benchmarks.
Following the suggestions, we evaluate the models trained on DSR-Train zero-shot, without any further training, on Video-MME (general video understanding) and on VLM4D and STI-Bench (spatio-temporal reasoning). All models are evaluated on the official test sets with the official evaluation protocols, using the same evaluation setting. The results have been added to App. A.5 (Table 11).
On Video-MME, AskGeo scores 63.7, compared with 61.4 for GSM and 55.0 for GeoSR, indicating that it preserves general video understanding. On VLM4D and STI-Bench, AskGeo improves over GSM by 8.0 and 7.5 points and over GeoSR by 13.9 and 8.0 points, respectively, suggesting that the gains from training on DSR-Train carry over to other spatio-temporal benchmarks.
We thank the reviewer again and hope that our responses have addressed the reviewer’s concerns.