- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this paper, the authors propose a two-stage training framework for spatial reasoning models. In the first stage, a VLM will be trained to locally estimate the 3D coordinate of one given reference and globally estimate the 3D coordinates of existing objects. This pretraining stage equips model with local and global awareness. In the second stage, the VLM is trained for spatial question answering problem with chain-of-thought and revise its CoT when the answer derived with geometry does not align with ground truth. On different spatial reasoning benchmarks, the final model achieves the best performance.
Critical strengths
-
In this paper, the authors propose an assumption that the former works neglect that the CoT of one spatial reasoning model could be enhanced if pretrained with geometric supervision, which can be valuable for further research in this field.
-
The authors propose to leverage reliability estimation to mitigate the noisy geometric estimation during data generation to make the model learn whether to rely on the human-generated CoT or revise by itself.
-
The authors conduct comprehensive experiments to prove the effectiveness of different components.
Critical weaknesses
-
In Section 3.1, the authors clarify the problem format for model pretraining. However, I cannot find the source dataset that these pretraining data are constructed on and how many data are used for pretraining.
-
I wonder how the final performance changes when the number of pretraining data scales. This can make whether the pretraining data is important clearer.
-
I wonder whether the general understanding performance, such as performance on Video-MME degrades after training under your framework. It is not desirable to sacrifice the general understanding ability for spatial reasoning ability.
Questions for authors
Please refer to the weaknesses.
5 comments
Thank You and a Gentle Follow-up
Dear Reviewer JC16,
Thank you again for your constructive suggestions. Following your feedback, we clarified the sources and sizes of the pretraining datasets, added experiments using different amounts of pretraining data, and evaluated general video understanding on Video-MME. We hope these additions help address your questions about the role of pretraining data and the effect of our framework on general understanding.
With less than two days remaining in the discussion period, we wanted to kindly follow up and ask whether you have any remaining questions or would appreciate further clarification. We would be happy to provide additional explanations and discuss any outstanding points.
Thank you sincerely for your time and thoughtful feedback.
Best,
Authors
Thank You and a Gentle Follow-up Before the Discussion Deadline
Dear Reviewer JC16,
We sincerely thank you again for your constructive comments. Following your suggestions, we have provided point-by-point responses to all of your questions and concerns, supported by additional experiments and detailed clarifications.
Specifically, we clarified the pretraining data sources and sizes, added experiments varying the amount of pretraining data, and provided a Video-MME evaluation to examine general video understanding.
Your comments have been valuable in guiding these additional experiments and clarifications. We have incorporated these improvements into the revised manuscript (main paper and appendix) uploaded to the submission system and highlighted the changes in green.
With less than 20 hours remaining in the discussion period, we would be grateful if you could review our rebuttal and share a brief follow-up comment. We would be happy to address any remaining questions or concerns.
Thank you again for your time and thoughtful suggestions.
Best regards,
Authors
Sincerely thank you for your constructive comments and suggestions. We appreciate your recognition of the potential value of geometric pretraining for spatial CoT reasoning and our comprehensive experiments. Below, we provide clarifications and additional experimental results, responding point by point to the weaknesses (W1–W3) in your review.
W1: Sources and size of the pretraining data
We use 503,735 reconstruction QA pairs derived from the official ScanNet training split. The breakdown is shown in the table below. We have added the source, size, and local/global breakdown of the QA-RP pretraining data to 'Datasets and Benchmarks' in Sec. 4 (page 7) of the revised manuscript.
W2: Effect of pretraining data scale
Following your suggestion, we trained models on randomly sampled 25% and 50% subsets and on the full QA-RP dataset, followed by second-stage SFT using the same data and training settings. All three models were evaluated on ReVSI using the same protocol. As shown in the table below, performance improves consistently as the amount of pretraining data increases, with the full-data model outperforming the 25% model by 5.5 points. We have added the experiment on pretraining data scale to Appendix A.4 (page 20, Tab. 15) of the revised manuscript.
W3: Preservation of general understanding ability
Following your suggestion, we compare the baseline with and without our two proposed components: QA-Native Reconstruction Pretraining (QA-RP) and spatial CoT with visual compensation (CoT-VC) on the Video-MME benchmark. As shown in the table below, adding QA-RP and CoT-VC improves the ReVSI score from 52.4 to 62.8, while Video-MME accuracy remains comparable (58.4 vs. 58.5). Thus, our designs improve spatial reasoning without an observed decrease in Video-MME accuracy relative to this baseline. We have added the Video-MME comparison and the corresponding ReVSI results to Appendix A.4 (page 20, Tab. 16) of the revised manuscript.
Thank you again for your time and insightful suggestions. We have incorporated the above clarifications and additional experimental results into the revised manuscript (main paper and appendix) uploaded to the submission system and highlighted the changes in green. If you have any further questions or suggestions, please let us know. We are glad to address them.