acceptodds
Under review as a conference paper at ICLR 2027

SpatialSpeak: QA-Native Reconstruction with Local and Global Context for Spatial Chain-of-Thought Reasoning

21 views

Abstract

Vision-language models (VLMs) can benefit from geometric priors for multi-view spatial reasoning, yet answer-only training does not directly supervise the intermediate geometric estimates and their use in deriving quantitative spatial answers. We hypothesize that spatial chain-of-thought (CoT) supervision becomes more effective when the VLM first jointly learns complementary local geometry and global scene context through multi-view reconstruction. We introduce SpatialSpeak, a two-stage framework that connects QA-native reconstruction pretraining with spatial CoT learning. In Stage I, QA-Native Reconstruction Pretraining (QA-RP) combines marked-point 3D queries for fine-grained local geometry with object-center queries for global scene context across views. Both tasks are formulated as text-based question answering, allowing geometric estimation and subsequent reasoning to share the same autoregressive output interface. In Stage II, spatial CoT with Visual Compensation (CoT-VC) trains the model to express question-relevant geometric estimates and use them to derive answers, with reliability assessment and visual compensation supporting answer refinement when needed. On ReVSI, QA-RP increases the gain from CoT-VC from 2.6 to 6.9 points, and ablations show that both local and global reconstruction supervision are beneficial. SpatialSpeak achieves state-of-the-art results on ReVSI, VSI-Bench, and SPAR-Bench, with a ReVSI score of 62.8 that exceeds the strongest compared baseline by 8.7 points. Code and pretrained models will be released.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Spatial reasoning, multimodal language models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_JC16OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose a two-stage training framework for spatial reasoning models. In the first stage, a VLM will be trained to locally estimate the 3D coordinate of one given reference and globally estimate the 3D coordinates of existing objects. This pretraining stage equips model with local and global awareness. In the second stage, the VLM is trained for spatial question answering problem with chain-of-thought and revise its CoT when the answer derived with geometry does not align with ground truth. On different spatial reasoning benchmarks, the final model achieves the best performance.

Critical strengths
  1. In this paper, the authors propose an assumption that the former works neglect that the CoT of one spatial reasoning model could be enhanced if pretrained with geometric supervision, which can be valuable for further research in this field.

  2. The authors propose to leverage reliability estimation to mitigate the noisy geometric estimation during data generation to make the model learn whether to rely on the human-generated CoT or revise by itself.

  3. The authors conduct comprehensive experiments to prove the effectiveness of different components.

Critical weaknesses
  1. In Section 3.1, the authors clarify the problem format for model pretraining. However, I cannot find the source dataset that these pretraining data are constructed on and how many data are used for pretraining.

  2. I wonder how the final performance changes when the number of pretraining data scales. This can make whether the pretraining data is important clearer.

  3. I wonder whether the general understanding performance, such as performance on Video-MME degrades after training under your framework. It is not desirable to sacrifice the general understanding ability for spatial reasoning ability.

Questions for authors

Please refer to the weaknesses.

5 comments

Sincerely thank you for your constructive comments and suggestions. We appreciate your recognition of the potential value of geometric pretraining for spatial CoT reasoning and our comprehensive experiments. Below, we provide clarifications and additional experimental results, responding point by point to the weaknesses (W1–W3) in your review.


W1: Sources and size of the pretraining data

We use 503,735 reconstruction QA pairs derived from the official ScanNet training split. The breakdown is shown in the table below. We have added the source, size, and local/global breakdown of the QA-RP pretraining data to 'Datasets and Benchmarks' in Sec. 4 (page 7) of the revised manuscript.

Training component  QA pairs
Local reconstruction  391,344
Global context  112,391
Total  503,735

W2: Effect of pretraining data scale

Following your suggestion, we trained models on randomly sampled 25% and 50% subsets and on the full QA-RP dataset, followed by second-stage SFT using the same data and training settings. All three models were evaluated on ReVSI using the same protocol. As shown in the table below, performance improves consistently as the amount of pretraining data increases, with the full-data model outperforming the 25% model by 5.5 points. We have added the experiment on pretraining data scale to Appendix A.4 (page 20, Tab. 15) of the revised manuscript.

Pretraining data  Avg.
25%  57.3
50%  59.8
100% (Full)  62.8

W3: Preservation of general understanding ability

Following your suggestion, we compare the baseline with and without our two proposed components: QA-Native Reconstruction Pretraining (QA-RP) and spatial CoT with visual compensation (CoT-VC) on the Video-MME benchmark. As shown in the table below, adding QA-RP and CoT-VC improves the ReVSI score from 52.4 to 62.8, while Video-MME accuracy remains comparable (58.4 vs. 58.5). Thus, our designs improve spatial reasoning without an observed decrease in Video-MME accuracy relative to this baseline. We have added the Video-MME comparison and the corresponding ReVSI results to Appendix A.4 (page 20, Tab. 16) of the revised manuscript.

Model  ReVSI  Video-MME
w/o QA-RP and CoT-VC  52.4  58.4
SpatialSpeak (Full)  62.8  58.5

Thank you again for your time and insightful suggestions. We have incorporated the above clarifications and additional experimental results into the revised manuscript (main paper and appendix) uploaded to the submission system and highlighted the changes in green. If you have any further questions or suggestions, please let us know. We are glad to address them.

AuthorsOpenReview ↗

Thank You and a Gentle Follow-up

Dear Reviewer JC16,

Thank you again for your constructive suggestions. Following your feedback, we clarified the sources and sizes of the pretraining datasets, added experiments using different amounts of pretraining data, and evaluated general video understanding on Video-MME. We hope these additions help address your questions about the role of pretraining data and the effect of our framework on general understanding.

With less than two days remaining in the discussion period, we wanted to kindly follow up and ask whether you have any remaining questions or would appreciate further clarification. We would be happy to provide additional explanations and discuss any outstanding points.

Thank you sincerely for your time and thoughtful feedback.

Best,

Authors

AuthorsOpenReview ↗

Thank You and a Gentle Follow-up Before the Discussion Deadline

Dear Reviewer JC16,

We sincerely thank you again for your constructive comments. Following your suggestions, we have provided point-by-point responses to all of your questions and concerns, supported by additional experiments and detailed clarifications.

Specifically, we clarified the pretraining data sources and sizes, added experiments varying the amount of pretraining data, and provided a Video-MME evaluation to examine general video understanding.

Your comments have been valuable in guiding these additional experiments and clarifications. We have incorporated these improvements into the revised manuscript (main paper and appendix) uploaded to the submission system and highlighted the changes in green.

With less than 20 hours remaining in the discussion period, we would be grateful if you could review our rebuttal and share a brief follow-up comment. We would be happy to address any remaining questions or concerns.

Thank you again for your time and thoughtful suggestions.

Best regards,

Authors

Reviewer_KWyXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SpatialSpeak studies whether explicitly learning geometry helps VLMs benefit more from spatial chain-of-thought supervision. The method uses a Qwen3-VL-4B backbone with visual and VGGT geometry features. In the first stage, the model learns to predict marked-point coordinates and object centers through text-based QA. In the second stage, it is trained on geometric reasoning traces constructed from 3D annotations, with reliability labels and a conditional answer-refinement template. The paper reports improvements on ReVSI, VSI-Bench, and SPAR-Bench, together with component ablations and a ScanNet reconstruction evaluation.

i find the combination of reconstruction pretraining and spatial CoT supervision useful, and the ReVSI improvement is substantial. My main reservation is the interpretation of visual compensation. I could not tell whether the “w/o VC” ablation keeps the same final answer targets, so I am not yet convinced that the gain demonstrates reliable error detection and visual correction. I currently lean toward weak rejection, but clarification of this comparison, together with the missing data and evaluation details, could change my assessment.

Critical strengths
  1. I liked the result in Table 1. CoT-VC gives a 2.6-point gain without reconstruction pretraining and a 6.9-point gain with it. This is a useful finding: the two stages seem to work better together than either does alone. Removing either the local or global reconstruction task also hurts performance.
  2. The ReVSI result is strong: 62.8 versus 54.1 for the best competing method in the table. The improvements on VSI-Bench and SPAR-Bench make the empirical case more convincing. I also appreciate that several main baselines were evaluated using their official checkpoints with the same number of input frames.
Critical weaknesses
  1. I am not sure Table 4 isolates the benefit of visual compensation. In Eq. 6, a LOW example ends with the ground-truth answer after a short refinement statement. What answer is used for that same example when VC is removed? If the answer target changes, the improvement could come partly from better answer supervision. The paper needs to resolve this before attributing the gain to reliability assessment and visual correction. There is also no direct test of whether the model’s HIGH/LOW predictions actually distinguish correct from incorrect geometric estimates.

  2. I had trouble working out the exact training and reconstruction evaluation setup. The paper gives hyperparameters, but not the number of local/global training examples, their mixture, or the size of the CoT dataset. It is also unclear whether removing a pretraining task reduces the training budget. For Table 5, the method predicts one queried point at a time, but the evaluation uses point clouds; the sampling and aggregation procedure is not explained. These details matter for reproducing the results and interpreting the comparisons. The QA benchmarks also need an explicit account of scene-level separation from the training data, beyond the held-out scenes mentioned for reconstruction.

Questions for authors

1.In the “w/o VC” experiment, what happens to LOW examples? Are they kept, and do they have exactly the same final answer targets as in the full model? If not, could you report a comparison with the examples and answer targets held fixed?

2.How well do the predicted HIGH/LOW labels identify errors at test time? For examples marked LOW, does the final answer improve over the answer obtained from the preceding geometric calculation? A before/after comparison, including cases where correction makes the answer worse, would help me assess this part of the method.

3.Could you give the training sample counts and mixture ratios, explain the scene-level separation from the QA evaluation sets, and state whether the ablations use matched training budgets? For Table 5, please also explain how the point clouds are generated from point queries and how sampling is handled across methods.

4 comments

Sincerely thank you for the constructive comments and suggestions. We appreciate your recognition of the value of combining reconstruction pretraining with spatial CoT and of our strong results. Below, we address W1–W2 and Q1–Q3 with clarifications and experiments.


W1: VC, answer supervision, and test-time reliability

(a) Answer supervision in the w/o VC ablation

LOW samples are retained, but their targets differ: the original w/o VC uses geometry-derived answers, whereas Full uses ground-truth answers after refinement. Following your suggestion, we retrained a GT-answer-matched w/o VC model using the same training examples and exactly the same final-answer targets as w/ VC, including GT answers for LOW examples. We removed only the HIGH/LOW reliability labels and visual-refinement statements from the training CoT, keeping all other training settings unchanged.

As shown below, ‘w/ VC’ achieves 62.8 on ReVSI versus 59.0 for this control, retaining a 3.8-point advantage under the same final-answer supervision. This supports a benefit of the VC training design beyond differences in final-answer supervision. VC trains the model to generate reliability labels and visual-refinement statements as part of its CoT, with the aim of guiding subsequent answer refinement. We have clarified the final-answer targets in the original w/o VC ablation and added the comparison with identical targets to Sec. 4.1 (page 8) of the revised manuscript.

VC variant  Avg.
w/o VC (GT-answer-matched)  59.0
Full method (w/ VC)  62.8 (+3.8)

(b) Accuracy of reliability labels

Following your suggestion, we evaluate whether the predicted reliability labels correctly assess the model-generated geometric conclusions before refinement on ReVSI, using the same reliability criterion as in Sec. 3.3 of the main paper. The full model achieves 71.1% reliability-label accuracy, compared with 65.9% without QA-RP. We have added the test-time evaluation of HIGH/LOW reliability labels to Sec. 4.1 (page 8) of the revised manuscript.


W2: Data, training budgets, reconstruction, and scene separation

(a) Training sample counts and mixture ratios

The dataset sizes are listed below. Within QA-RP, local reconstruction and global context account for 77.69% and 22.31% of the QA records, respectively. The CoT-VC includes 51,779 QA pairs. We will release all the training data together with code and model checkpoints. We have added the sample counts and the local/global mixture ratios to 'Datasets and Benchmarks' in Sec. 4 (page 7) of the revised manuscript.

Training component  QA pairs  Scene overlap with evaluation benchmarks
Local reconstruction (QA-RP stage)  391,344  0
Global context (QA-RP stage)  112,391  0
Total QA-RP stage  503,735  0
CoT-VC  51,779  0

(b) Training-budget matching in the ablations

All reconstruction scenes are already included in the fine-tuning data. Thus, removing a pretraining task leaves the overall training-scene coverage unchanged but reduces the total number of samples seen during training. To study whether more training on the existing SFT data can explain the overall benefit of QA-RP, we extended SFT for the variant without QA-RP to match the total number of samples seen by the full method (w/ QA-RP) during training.

As shown below, extended fine-tuning improves the ReVSI score from 55.0 to 56.2, but it remains below Full’s 62.8. This supports the value of QA-RP beyond simply extending training on the existing SFT data. We have added the extended SFT comparison with the total number of training samples seen matched to that of the full model in Appendix A.4 (page 19, Tab. 14) of the revised manuscript.

Setting  Avg.
w/o QA-RP  55.0
w/o QA-RP, extended fine-tuning  56.2
Full method (w/ QA-RP)  62.8

W2: Data, training budgets, reconstruction, and scene separation

(c) Point-cloud generation, sampling, aggregation procedure in Tab. 5

The single-point interface produces a dense reconstruction through repeated point queries. Specifically, for each scene, we keep the input frames fixed and change only the marked query point. We select frames with a frame-index stride of 30 and sample image points at 20-pixel intervals. Each query returns the marked point’s 3D coordinates as text in the first-frame camera coordinate system. Since all predictions of the same scene share this coordinate system, we combine them across sampled pixels and frames to form the dense reconstruction evaluated in Tab. 5. All compared methods use the same input frames and are evaluated at the same sampled pixel locations. We will also release the reconstruction code and predicted point clouds. We have added the pointmap generation, aggregation, and shared sampling protocol for Tab. 5 to Appendix A.4 (page 20) of the revised manuscript.

(d) Scene-level separation from QA evaluation sets

We have carefully compared the scene IDs in our reconstruction and CoT training data against those in the evaluation benchmarks. The scenes between our training and the evaluation benchmarks (VSI-Bench, ReVSI-Bench, SPAR-Bench, and the reconstruction evaluation set) have no overlap. All reconstruction and CoT training examples come from the official ScanNet training split (1201 scenes). No validation or test scenes are used for training. We have clarified the scene-level separation between the QA-RP and spatial CoT training data and the evaluation benchmarks in Sec. 4 (page 7) of the revised manuscript.


Q1: Retention of LOW samples and final-answer targets

Please see part (a) of our response to W1 in your review, where we clarify how LOW examples are handled and report the requested comparison with the training examples and GT final-answer targets held fixed.


Q2: Before/after comparison on LOW examples

We report the accuracy of the reliability labels in part (b) of our response to W1 in your review. Here, we further follow your suggestion and report the requested before/after comparison on LOW examples on ReVSI. Specifically, for the same LOW examples from the same trained model, we compare the answers obtained from the preceding geometric calculation with the final answers after visual refinement in the table below. As we can see, refinement improves the average score of low-case samples.

Answer stage  Avg. (LOW cases)
Before refinement: geometry-derived answer  23.2
After refinement: final answer  34.7

Although refinement improves the average score, it can also worsen individual answers in some failure cases. For example, when asked to count the lamps and sofas, the model initially derives the correct answer of 4 from geometry (2 lamps and 2 sofas). It labels this answer LOW and changes it to 3 after visual refinement. We have added the before/after comparison and the failure example to Appendix A.4 (page 19, Tab. 12) of the revised manuscript.


Q3: Data, scene separation, budgets, and reconstruction

Please see parts (a)–(d) of our response to W2 in your review, where we address these questions in detail.


Thank you again for your time and insightful suggestions. We have incorporated the above clarifications and additional experimental results into the revised manuscript (main paper and appendix) uploaded to the submission system and highlighted the changes in green. If you have any further questions or suggestions, please let us know. We are glad to address them.

AuthorsOpenReview ↗

Thank You and a Gentle Follow-up

Dear Reviewer KWyX,

Thank you again for your constructive feedback. Following your suggestions, we added a VC ablation with identical final-answer targets, evaluated the reliability labels, and compared LOW-case answers before and after refinement, including a failure example. We also provided a training-budget comparison and clarified the data composition, scene separation, and reconstruction evaluation procedure. We hope these experiments and discussions help address your concerns.

With less than two days remaining in the discussion period, we wanted to kindly follow up and ask whether any points would benefit from further clarification or discussion. We would be happy to address any remaining questions.

Thank you sincerely for your time and valuable suggestions.

Best,

Authors

AuthorsOpenReview ↗

Thank You and a Gentle Follow-up Before the Discussion Deadline

Dear Reviewer KWyX,

We sincerely thank you again for your constructive comments. Following your suggestions, we have provided point-by-point responses to all of your questions and concerns, supported by additional experiments and detailed clarifications.

Specifically, we added the requested ablation with identical final-answer targets, reliability evaluation, and a before/after comparison on LOW cases, including a failure example. We also provided an extended fine-tuning comparison to examine the training-budget concern and clarified the data and evaluation setup.

Your comments have been valuable in guiding these additional experiments and clarifications. We have incorporated these improvements into the revised manuscript (main paper and appendix) uploaded to the submission system and highlighted the changes in green.

With less than 20 hours remaining in the discussion period, we would be grateful if you could review our rebuttal and share a brief follow-up comment. We would be happy to address any remaining questions or concerns.

Thank you again for your time and thoughtful suggestions.

Best regards,

Authors

Reviewer_E4R7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SpatialSpeak, a two-stage framework for spatial reasoning with multi-view images. The first stage uses QA-native reconstruction pretraining with local point-coordinate queries and global object-level queries. The second stage introduces spatial chain-of-thought reasoning with a reliability-based visual compensation mechanism. Experiments on ReVSI, VSI-Bench, and SPAR-Bench show strong performance, including 62.8 on ReVSI and 76.0 on SPAR-Bench.

Critical strengths
  1. Coherent reconstruction-to-reasoning design. The same autoregressive QA interface is used for geometric reconstruction and downstream reasoning. The ReVSI ablations support the value of both stages: the full model scores 62.8, compared with 55.9 without CoT-VC and 55.0 without QA-RP.

  2. Strong results across multiple benchmarks. SpatialSpeak outperforms the reported baselines on ReVSI, VSI-Bench, and SPAR-Bench. The complementary local/global query ablations also provide useful evidence for the proposed reconstruction supervision.

Critical weaknesses
  1. The claimed visual compensation mechanism is not directly validated. For LOW-reliability samples, Eq. (6) supplies the ground-truth answer after a generic visual-refinement statement. The paper does not evaluate whether the model actually uses visual evidence or whether the improvement mainly comes from additional answer supervision. This weakens the central claim of reliable geometry-aware reasoning.

  2. Important data and evaluation details are missing. The paper does not report the exact sizes and scene splits of the reconstruction and CoT datasets, or their possible overlap with evaluation benchmarks. It also does not explain how the single-point textual prediction interface produces the dense point maps evaluated in Table 5. These omissions limit reproducibility and confidence in the comparisons.

Questions for authors
  1. What are the exact sample counts, scene splits, and data sources for the local, global, and spatial-CoT training sets? Is there any scene or video overlap with the evaluation benchmarks?

  2. How are the dense point maps in Table 5 generated from the marked-point autoregressive prediction interface?

  3. Can the authors report the accuracy or calibration of the intermediate geometric estimates and HIGH/LOW reliability labels? What evidence shows that LOW cases use visual information rather than only learning from the supplied ground-truth answer?

  4. Was the threshold (\tau=0.3) selected using a separate validation set or based on ReVSI test performance?

6 comments

Sincerely thank you for your constructive comments and suggestions. We appreciate your recognition of our coherent reconstruction-to-reasoning design and strong performance across multiple benchmarks. Below, we provide clarifications and additional experimental results, responding point by point to the weaknesses (W1–W2) and the questions (Q1–Q4) in your review.


W1: Validation of visual compensation (VC)

We understand your concern: in the original ablation, removing VC also changes the final-answer supervision for LOW examples, making it difficult to determine whether the improvement comes from the VC training design or from better answer supervision. To separate these effects, we trained a GT-answer-matched w/o VC control using the same training examples and exactly the same final-answer targets as w/ VC, including GT answers for LOW examples. We removed only the HIGH/LOW reliability labels and visual-refinement statements from the training CoT, keeping all other training settings unchanged.

As shown below, ‘w/ VC’ achieves 62.8 on ReVSI versus 59.0 for this control, retaining a 3.8-point advantage under the same final-answer supervision. This supports a benefit of the VC training design beyond differences in final-answer supervision. VC trains the model to generate reliability labels and visual-refinement statements as part of its CoT, with the aim of guiding subsequent answer refinement. We have clarified the original VC ablation and added the comparison with identical final-answer targets to Sec. 4.1 (page 8) of the revised manuscript.

VC variant  Avg.
w/o VC (GT-answer-matched)  59.0
Full method (w/ VC)  62.8 (+3.8)

W2: Training-data and reconstruction-evaluation details

(a) Training data, quantities, and scene separation

The exact counts are listed in the table below. The evaluation benchmarks include VSI-Bench, ReVSI-Bench, SPAR-Bench, and the reconstruction evaluation set. All reconstruction and CoT training examples come from the official ScanNet training split (1201 scenes). No validation or test scenes are used for training. We have carefully checked all the scenes and there is no overlap between training and evaluation benchmarks. We will release all the training data together with code and model checkpoints. We have added the training data sources, sample counts, and scene separation details to 'Datasets and Benchmarks' in Sec. 4 (page 7) of the revised manuscript.

Training component  QA pairs  Scene overlap with evaluation benchmarks
Local reconstruction (QA-RP stage)  391,344  0
Global context (QA-RP stage)  112,391  0
Total QA-RP stage  503,735  0
CoT-VC  51,779  0

(b) Dense point-map prediction in Tab. 5

The single-point interface produces a dense reconstruction through repeated point queries. Specifically, for each scene, we keep the input frames fixed and change only the marked query point. We select frames with a frame-index stride of 30 and sample image points at 20-pixel intervals. Each query returns the marked point’s 3D coordinates as text in the first-frame camera coordinate system. Since all predictions of the same scene share this coordinate system, we combine them across sampled pixels and frames to form the dense reconstruction evaluated in Tab. 5. All compared methods use the same input frames and are evaluated at the same sampled pixel locations. We will also release the reconstruction code and reconstruction results. We have added the dense pointmap construction and the shared sampling protocol for Tab. 5 to Appendix A.4 (page 20) of the revised manuscript.


Q1: Training data and separation from evaluation benchmarks

Please see part (a) of our response above to W2 in your review for the training data sources, exact dataset sizes, scene splits, and benchmark-overlap checks.


Q2: Dense point-map generation for Tab. 5

Please see part (b) of our response above to W2 in your review, where we explain how repeated single-point predictions are combined to generate the dense point maps in Tab. 5.

Q3: Accuracy, reliability, and visual evidence

(a) Accuracy of intermediate geometric estimation and reliability labels

Following your suggestion, we directly evaluate intermediate geometric estimates in generated CoT traces for ReVSI questions of ScanNet by measuring the Euclidean distance between the predicted 3D object coordinates and their corresponding ground truth. Our full model achieves a mean error of 0.272 m. To examine the contribution of QA-Native Reconstruction Pretraining (QA-RP), we compare our full model with the variant trained without this stage on the same estimates. The comparison is shown in the table below. QA-RP reduces the mean error by 40%, supporting its contribution to more accurate intermediate geometric estimation. This is consistent with the QA-RP ablation in Tab. 1 of the main paper, where removing QA-RP reduces the overall ReVSI score from 62.8 to 55.0.

Besides, we follow your suggestion and evaluate whether the predicted reliability labels correctly assess the model-generated geometric conclusions before refinement on ReVSI, using the same reliability criterion as in Sec. 3.3 of the main paper. The full model achieves 71.1% reliability-label accuracy, compared with 65.9% without QA-RP. We have added the evaluations of intermediate geometry and reliability-label accuracy to Sec. 4.1 (page 8) of the revised manuscript.

QA-RP variant  Mean Euclidean error (m) ↓\downarrow
w/o QA-RP  0.453
Full method (w/ QA-RP)  0.272
Improvement  40% error reduction

(b) Additional answer supervision versus visual evidence

Please see our response above to W1 in your review for the ablation with the same final-answer supervision.

Here, to further study whether final-answer generation utilizes visual information, we conduct a region-masking experiment on the LOW cases. We first save the reasoning text generated from the original images up to the start of the final answer. For each masking condition, we provide the modified images and the same question, reuse exactly the same saved reasoning text as a fixed prefix, and autoregressively generate only the final-answer tokens during visual refinement. Thus, the preceding reasoning remains identical across conditions, and the modified images are used only to generate the final answer during visual refinement. Specifically, 'Target-region masking' masks out the question-relevant object regions. 'Random-region masking' masks out randomly selected regions outside the target regions. 'No masking' utilizes the original unmasked images.

As shown in the table below, random-region masking yields 34.1, close to the 34.7 obtained without masking, whereas target-region masking reduces the score to 24.4. This represents a 10.3-point drop relative to no masking and a 9.7-point gap relative to random-region masking. With the question and preceding reasoning held fixed, the larger decrease under target-region masking provides exploratory evidence that visual information contributes to final-answer generation in LOW cases. We have added the LOW-case region-masking protocol and results to Appendix A.4 (page 19, Tab. 13) of the revised manuscript.

Visual input for final-answer generationAvg. (LOW cases)
Target-region masking24.4
Random-region masking34.1
No masking34.7

Q4: Selection of the reliability threshold

The final threshold τ\tau was selected based on ReVSI test performance in the threshold ablation (Tab. 3 of the main paper). τ\tau is used only during training-data construction to assign HIGH/LOW reliability labels and construct the corresponding CoT targets. At inference, the model generates reliability labels and answers autoregressively, without applying an explicit threshold.

Nevertheless, the benefit of CoT-VC holds across all three tested thresholds: τ=0.1\tau = 0.1, 0.30.3, and 0.50.5 achieve 59.6, 62.8, and 59.5, respectively, compared with 55.9 without CoT-VC (Tab. 3 of the main paper). The two alternative thresholds retain gains of 3.7 and 3.6 points. At both alternative thresholds, the ReVSI score exceeds 54.1, the best competing score in Tab. 6 of the main paper. These results show that the effectiveness and performance advantage are not confined to the selected threshold of τ=0.3\tau = 0.3. We have clarified how the threshold was selected and used to construct the training data in Sec. 4.1 (page 8) of the revised manuscript.


Thank you again for your time and insightful suggestions. We have incorporated the above clarifications and additional experimental results into the revised manuscript (main paper and appendix) uploaded to the submission system and highlighted the changes in green. If you have any further questions or suggestions, please let us know. We are glad to address them.

AuthorsOpenReview ↗

Thank You and a Gentle Follow-up

Dear Reviewer E4R7,

Thank you again for your constructive comments. Following your suggestions, we added an ablation with matched final-answer supervision, quantitative evaluations of intermediate geometry and reliability labels, and a region-masking experiment examining the use of visual information in LOW cases. We also clarified the training data and reconstruction evaluation protocol. We hope these additions help address your concerns.

With less than two days remaining in the discussion period, we wanted to kindly follow up and ask whether you have any remaining concerns or would find further clarification helpful. We would be happy to provide additional explanations.

Thank you sincerely for your time and thoughtful feedback.

Best,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.