acceptodds
Under review as a conference paper at ICLR 2027

SpatialCORE: Confidence-Aware Grounded Spatial Reasoning in Large Vision–Language Models

Abstract

Large Vision-Language Models (LVLMs) have made remarkable progress across visual perception tasks, yet spatial reasoning remains a persistent weakness, especially for questions that require reasoning over visual space. Recent spatial-reasoning methods incorporate generated grounding, where models predict bounding boxes, masks, or other localization outputs for task-relevant objects as part of their reasoning trace. However, these approaches typically optimize final-answer correctness alone, allowing correct answers to be rewarded even when the model does not reason from confidently localized task-relevant objects. We introduce **SpatialCORE** **(Spatial**ly **CO**nfident **RE**asoning), a post-training framework that turns the model's own confidence in generated grounding into a learning signal for spatial reasoning. Its central idea is to reinforce grounding that is both accurate and confident, encouraging the model to reason from confidently localized task-relevant objects. SpatialCORE realizes this through a *self-regulating spatial reward* that weights each predicted grounding, i.e., bounding box's matching quality by its coordinate-token confidence. An answer gate further ties grounding optimization to final-answer correctness. SpatialCORE achieves state-of-the-art results among open-source and specialized spatial reasoning models across diverse benchmarks, and transfers effectively in zero-shot settings to unseen data distributions. The source code is available in the supplementary material.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large vision-language models, spatial reasoning, generated grounding, reinforcement learning, uncertainty estimation, multimodal reasoning

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_Y3qrOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a post-training strategy for VLMs that includes a reward based on the confidence of model's visual grounding output. Through experiments, the author shows this strategy achieves strong performance compared to mentioned open-source models and spatial reasoning models, demonstrating the effectiveness of including confidence.

Critical strengths

The idea of including confidence of visual grounding into the design of reward is reasonable. The results shows strong performance and improvement on SpatiaLab suggests the transferability of the proposed strategy.

Critical weaknesses
  1. Although the idea of including confidence is interesting, but it is not novel. For example, [1] [2] have already investigated the inclusion of confidence into the reward signal and link it with visual grounding. The comparison with these paper should be included in the manuscript and explicitly mention the difference other than the different calculation of confidence.
  2. Based on the results in Tab. 2, the improvement of SpatialCORE-8B compared to Qwen3-VL-8B-Thinking+GRPO is little, with an average of only 1.7.
  3. As illustrated in Eq. 5, since the uncertainty is calculated based on digit-token entropy, the uncertainty between coordiantes 100 and 900 can receive the same score as uncertainty between 100 and 101, which both involve one uncertain digit and two certain digits.
  4. For Fig. 4, the author should include comparison with a model trained with the same grounding supervision without confidence weighting instead of compare to an unadapted backbone.

[1] Wenyi Xiao, Xinchi Xu, Leilei Gan. "VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning".

[2] Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, Deepak Pathak. "Maximizing Confidence Alone Improves Reasoning"

Questions for authors

Please refer to weakness 1, 3, 4.

2 comments

W1

Although the idea of including confidence is interesting, but it is not novel. For example, [1] [2] have already investigated the inclusion of confidence into the reward signal and link it with visual grounding. The comparison with these paper should be included in the manuscript and explicitly mention the difference other than the different calculation of confidence.

[1] Wenyi Xiao, Xinchi Xu, Leilei Gan. "VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning".

[2] Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, Deepak Pathak. "Maximizing Confidence Alone Improves Reasoning"

Answer:

We thank the reviewer for highlighting these studies. While these studies share some techniques with SpatialCORE, they differ fundamentally in their learning objectives. To our knowledge, SpatialCORE is the first to turn the model's confidence in its generated grounding into a learning signal for spatial reasoning.

VL-Calibration [1].

VL-Calibration trains the model to report confidence that reflects its visual certainty and answer correctness. Its “visual grounding” signal measures how predictions change when image regions are masked. This indicates whether the model relies on the image, but does not establish whether it correctly locates the objects needed for spatial reasoning.

Its calibration objective evaluates whether reported confidence matches the calibration target; it does not directly evaluate the accuracy of predicted object locations. SpatialCORE provides this feedback by comparing predicted bounding boxes with pseudo-GT boxes and giving greater credit to accurate boxes generated with greater confidence. The difference lies in the learning objective: extending VL-Calibration to reward accurate, confident object locations would require grounding-quality feedback, beyond a different confidence estimator.

RENT [2].

RENT studies text-only mathematical problem solving and scientific question answering; it does not investigate visual grounding. Further, its reward depends on confidence alone, without checking whether the generated answer is correct.

In our spatial-reasoning setting, confidence alone does not establish whether predicted bounding boxes correctly locate task-relevant objects. Equally confident boxes can have different localization quality yet receive the same confidence reward. SpatialCORE makes confidence's reward contribution depend on grounding quality, evaluated against pseudo-GT boxes. The answer gate further ties the spatial reward to final-answer correctness. These are substantive differences in the learning objective, beyond how confidence is calculated.

We have added both studies and these distinctions to the Related Work section of the revised manuscript.

W2

Based on the results in Tab. 2, the improvement of SpatialCORE-8B compared to Qwen3-VL-8B-Thinking+GRPO is little, with an average of only 1.7.

Answer:

SpatiaLab is evaluated entirely zero-shot: no SpatiaLab samples are used during post-training, and its scenes and task designs differ from OmniSpatial. Standard GRPO improves the backbone by 1.43 percentage points (44.78% to 46.21%), while explicit grounding SFT falls below the backbone (44.21%; Table 8), showing that additional grounding supervision does not guarantee improved transfer. SpatialCORE improves the backbone by 3.13 points to 47.91%, more than doubling GRPO's zero-shot gain using the same post-training data. Its additional 1.70-point improvement over GRPO shows that the benefit extends to an unseen distribution, complementing the 2.54-point improvement over GRPO on OmniSpatial.

Table 8: Accuracy (%) after explicit grounding SFT and SpatialCORE post-training. SpatiaLab is evaluated entirely zero-shot.

VariantOmniSpatialSpatiaLab
Base43.9044.78
Base + SFT45.2644.21
SpatialCORE48.4647.91

Base denotes Qwen3-VL-8B-Thinking.

W3

As illustrated in Eq. 5, since the uncertainty is calculated based on digit-token entropy, the uncertainty between coordiantes 100 and 900 can receive the same score as uncertainty between 100 and 101, which both involve one uncertain digit and two certain digits.

Answer:

Equal entropy does not mean equal reward, because each sampled box is still scored by its overlap with the reference. Coordinate-token entropy determines the confidence weight; IoU evaluates the box's geometric overlap with the reference (Eqs. 3, 5–7). The entropy term therefore weights the reward rather than measuring spatial error itself.

Taking 100 as the correct coordinate in the reviewer's example, uncertainty between 100 and 900 produces rollouts that choose either value. When choosing 900 substantially reduces the box's overlap with the reference, that box contributes less to the spatial reward. In contrast, choosing between 100 and 101 generally preserves nearly the same box and IoU. With comparable probabilities of generating the incorrect coordinate, the 100–900 case therefore receives a lower expected spatial reward despite having the same entropy. IoU captures how the generated coordinates affect localization quality and incorporates this into the spatial reward.

W4

For Fig. 4, the author should include comparison with a model trained with the same grounding supervision without confidence weighting instead of compare to an unadapted backbone.

Answer:

We complement Figure 4 with a matched comparison of SpatialCORE with and without confidence weighting. Figure 4 compares SpatialCORE with the unadapted backbone; the added comparison isolates confidence weighting. Both models use the same cold-start SFT checkpoint, grounding-first prompts, pseudo-GT boxes, vision and language LoRA settings, and training budget. The ablated model retains the spatial reward and answer gate; only confidence weighting is removed.

Both models are evaluated on the same OmniSpatial test questions. Predicted bounding boxes are ranked by coordinate-token uncertainty, and the same proportion of highest-confidence boxes is retained for each model. Localization error is measured as 100(1-IoU), including zero-overlap predictions so that confident localization failures remain counted.

When all predictions are retained, localization errors are similar: 64.7 for SpatialCORE and 65.9 without confidence weighting. Among the most confident 25%, the errors are 18.8 and 26.8, respectively. This matched comparison suggests that confidence weighting makes the model's confidence a better indicator of box accuracy, supporting improved confidence–localization reliability. The comparison is now included in Section 4.3, Table 3 of the revised manuscript.

Table 9: Localization error at matched confidence-retention proportions on OmniSpatial. Brackets indicate 95% confidence intervals.

Highest-confidence boxes retained (%)SpatialCORE Error ↓SpatialCORE w/o confidence weighting Error ↓Error reduction ↑ with confidence weighting
2518.8 [6.3, 38.9]26.8 [8.4, 46.7]+8.0 [-0.4, +19.5]
5041.7 [25.2, 57.3]43.8 [28.3, 59.3]+2.1 [-0.9, +8.0]
7557.1 [45.1, 68.8]59.7 [47.8, 70.2]+2.6 [-0.9, +6.4]
10064.7 [56.1, 73.4]65.9 [57.3, 74.3]+1.1 [-0.3, +3.5]

Error reduction is the confidence-removed model's error minus SpatialCORE's error, computed before rounding.

Reviewer_xtU2OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a reinforcement learning reward design for explicit object grounding in vision-language models. It uses pseudo-labels to assess bounding-box accuracy and coordinate-token entropy to distinguish confidence among predictions of similar localization quality, encouraging more accurate grounding with more informative confidence estimates. The method combines this spatial reward with answer and format rewards for GRPO training. Experiments on OmniSpatial and zero-shot SpatiaLab show improvements over the backbone and a standard GRPO baseline.

Critical strengths

This paper addresses a useful problem, i.e., correct final answers can coexist with poor intermediate grounding. Combining localization quality with generation confidence provides a practical training signal.

Critical weaknesses
  1. The paper does not specify whether it shares the same SFT initialization, grounding-first format, and LoRA settings as SpatialCORE. Since the appendix shows that cold-start training is important for maintaining grounding behavior, these details are necessary to attribute the gains to the proposed reward. The confidence-weighting ablation partly addresses this concern, but only on the Spatial Interaction subset.

  2. The impact of grounding errors on final answers is insufficiently analyzed. The paper provides qualitative examples, but does not report final-answer accuracy across different IoU ranges, the proportions of incorrect grounding with correct answers and correct grounding with incorrect answers, or how answers change when intermediate boxes are perturbed, replaced, or removed at inference time. These analyses would clarify error propagation and whether answers depend on the generated grounding. The training-time pseudo-label corruption experiment does not directly address these questions.

  3. The inference cost is unclear. The paper reports a maximum completion length of 3,072 tokens, but provides no actual output-length or latency comparisons with the baselines. Reporting grounding and reasoning token counts, along with accuracy under matched token budgets, would clarify the method’s accuracy–cost trade-off.

Questions for authors

please see weaknesses.

4 comments

The paper does not specify whether it shares the same SFT initialization, grounding-first format, and LoRA settings as SpatialCORE.

Answer:

The main-table GRPO baseline and SpatialCORE share the same SFT initialization (cold start), grounding-first output format, and LoRA configuration. They also use identical prompts, training data, training budget, and evaluation settings, with the same vision and language parameters updated. The GRPO baseline uses answer-accuracy and format rewards; SpatialCORE adds the self-regulating spatial reward and answer gate. The main-table comparison therefore controls for initialization and output format. These shared training settings are now clarified in Section 4.2 of the revised manuscript.

Since the appendix shows that cold-start training is important for maintaining grounding behavior, these details are necessary to attribute the gains to the proposed reward. The confidence-weighting ablation partly addresses this concern, but only on the Spatial Interaction subset.

Answer:

We have extended the confidence-weighting comparison beyond the original Spatial Interaction subset, and the gain from confidence weighting holds on the full OmniSpatial test set and on zero-shot SpatiaLab. Both models are trained on the full OmniSpatial training split from the same cold-start SFT checkpoint, using identical grounding-first prompts, vision and language LoRA settings, and training budgets. The ablated model retains the spatial reward and answer gate; only confidence weighting is removed.

Table 5: Final-answer accuracy (%) with and without confidence weighting on the full OmniSpatial test set and zero-shot SpatiaLab.

VariantOmniSpatialSpatiaLab
SpatialCORE w/o confidence weighting46.8946.78
SpatialCORE48.4647.91

Removing confidence weighting lowers accuracy from 48.46% to 46.89% on OmniSpatial and from 47.91% to 46.78% on zero-shot SpatiaLab. Since the two models differ only in confidence weighting, these matched comparisons identify additional accuracy gains of 1.57 and 1.13 percentage points from confidence weighting, extending the evidence beyond the original subset.

The same pair of models also differs in localization among the most confident predictions. We rank predicted bounding boxes by coordinate-token uncertainty and retain the same proportion for each model. Localization error is 100(1-IoU), including zero-overlap predictions. Errors are similar when all predictions are retained (64.7 vs. 65.9), but are 18.8 vs. 26.8 among the most confident 25% (Table 6). The accuracy ablation is included in Appendix C.1, and the localization analysis in Section 4.3, Table 3 of the revised manuscript.

Table 6: Localization error at matched confidence-retention proportions on OmniSpatial. Brackets indicate 95% confidence intervals.

Highest-confidence boxes retained (%)SpatialCORE Error ↓SpatialCORE w/o confidence weighting Error ↓Error reduction ↑ with confidence weighting
2518.8 [6.3, 38.9]26.8 [8.4, 46.7]+8.0 [-0.4, +19.5]
5041.7 [25.2, 57.3]43.8 [28.3, 59.3]+2.1 [-0.9, +8.0]
7557.1 [45.1, 68.8]59.7 [47.8, 70.2]+2.6 [-0.9, +6.4]
10064.7 [56.1, 73.4]65.9 [57.3, 74.3]+1.1 [-0.3, +3.5]

Error reduction is the confidence-removed model's error minus SpatialCORE's error, computed before rounding.

W2

The impact of grounding errors on final answers is insufficiently analyzed. The paper provides qualitative examples, but does not report final-answer accuracy across different IoU ranges, the proportions of incorrect grounding with correct answers and correct grounding with incorrect answers, or how answers change when intermediate boxes are perturbed, replaced, or removed at inference time. These analyses would clarify error propagation and whether answers depend on the generated grounding. The training-time pseudo-label corruption experiment does not directly address these questions.

Answer:

Inference-time interventions show that perturbing, replacing, or removing generated bounding boxes changes the model's final answers. We edit the generated boxes and let SpatialCORE continue reasoning on the same questions. Answer-change rates increase from 10.8% for boxes perturbed to IoU 0.75 with their original generated boxes to 16.2% at IoU 0.25. Replacing boxes with wrong-location boxes changes 25.5% of answers, and removing boxes changes 28.9%, compared with 6.8% under an unchanged re-run control. These analyses are now included in Appendix C.4 of the revised manuscript.

The analysis also separates grounding correctness from final-answer correctness. Among 652 questions with pseudo-GT boxes, 31.6% have correct answers despite grounding IoU below 0.5, while 20.9% have incorrect answers despite IoU of at least 0.5. Some correct answers become incorrect after the box edits, while some incorrect answers become correct. Overall final-answer accuracy remains within approximately 1.5 percentage points of the control run with unchanged boxes. Generated boxes therefore influence the model’s answers, while grounding errors do not translate into proportional loss in final-answer accuracy.

W3

The inference cost is unclear. The paper reports a maximum completion length of 3,072 tokens, but provides no actual output-length or latency comparisons with the baselines. Reporting grounding and reasoning token counts, along with accuracy under matched token budgets, would clarify the method's accuracy–cost trade-off.

Answer:

SpatialCORE improves accuracy over GRPO under the same evaluation settings while generating fewer tokens and showing comparable latency. Following the reviewer's suggestion, we measured output lengths and inference latency for the Qwen3-VL-8B-Thinking backbone, its standard GRPO variant, and SpatialCORE. All three use the same maximum completion length of 3,072 tokens and the same decoding settings as in the reported accuracy evaluations.

SpatialCORE generates 164.3 tokens per response on average, compared with 178.1 for GRPO and 1,959.9 for the backbone. The substantial reduction from the backbone occurs with both post-trained models, consistent with post-training helping the model answer spatial questions through more concise grounding and reasoning traces. Compared with GRPO, SpatialCORE uses 7.7% fewer output tokens, with fewer tokens devoted to both grounding and reasoning. On the same 200 randomly sampled questions, processed one request at a time, mean latency is 1.93 seconds for SpatialCORE and 2.02 seconds for GRPO. SpatialCORE's higher accuracy therefore accompanies lower average token usage and comparable latency. These inference-cost results are now included in the Section 4.3, Table 5 of the revised manuscript.

Table 7: Mean generated-token counts and inference latency under matched evaluation settings

ModelGroundingReasoningTotalLatency (s)
Base382.11,561.51,959.923.81
GRPO55.5105.6178.12.02
SpatialCORE46.7100.5164.31.93

Base denotes Qwen3-VL-8B-Thinking. Grounding counts bounding-box lines; Total also includes the transition sentence and final answer.

I thank the authors for their response, which fully addresses my concerns.

Thank you for your helpful feedback. We are glad our responses have fully addressed your concerns, and we appreciate your suggestions for improving the paper.

Reviewer_vtgAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper propose Spatialcore, a new RL methods framework for improving 2D spatial reasoning. The main idea is to leverage model's confidence with bbox location quality. The confidence is evaluated form coordinate-token entropy. Experiments on OmniSpatial and SpatialLab show gains over baseline models and pure grpo training.

Critical strengths

The paper is well written, and the motivation is clear. The paper is well-structured and self-contained. The proposed confidence-aware methods is easy to integrate. The authors evaluate their approach on two spatial reasoning benchmarks and provide several ablation studies. The results show consistent improvements, and the ablation study shows that confidence weighting contributes to the performance gains.

Critical weaknesses

My major concern is that the paper does not distinguish improvements in visual grounding from improvements in spatial reasoning. The proposed spatial reward primarily optimizes bounding box localization quality and coordinate-token confidence, rather than directly supervising the correctness of spatial relationships or reasoning steps. Also, since the spatial reward relies on bounding boxes generated by external grounding model, it is not clear why the proposed RL framework is necessary to obtain these improvements. The purposed method is also needed to be evaluated beyond 2D tasks for more general spatial reasoning to show the general ability in spatial reasoning.

Questions for authors
  1. How could the method general beyond 2D tasks in spatial reasoning? ( videos, multi view images)
  2. Can the authors provide experiments that distinguish improvements in object localization from improvements in spatial reasoning? For example, would SpatialCORE still outperform standard GRPO when both models are provided with identical, accurate bounding boxes?
  3. Since pseudo-ground-truth bounding boxes are already available, have the authors considered comparing against supervised fine-tuning with explicit grounding supervision?
4 comments

Q1

How could the method general beyond 2D tasks in spatial reasoning? ( videos, multi view images)

Answer:

SpatialCORE could be extended to videos and multi-view images by labeling each predicted bounding box with its frame or view, while retaining the same reward. Our Qwen3-VL backbone already accepts both input types [1], and the proposed extension adapts how generated grounding is labeled and matched.

Each predicted bounding box is compared only with reference boxes from the same frame or view. The self-regulating spatial reward then works as in the single-image case: it weights each box's localization quality by coordinate-token confidence and combines the results across frames or views, while the answer gate still ties this reward to final-answer correctness.

  • Videos: Boxes labeled by frame let the reasoning trace compare where objects are at different moments, for example to judge how one object moves relative to another. Open-o3-Video [2] already generates timestamped boxes during video reasoning and rewards their spatial and temporal alignment.

  • Multi-view images: Boxes labeled by view let the reasoning trace combine evidence about the same objects seen from different viewpoints. MM-Spatial [3] studies spatial reasoning from multiple images of a scene with camera information.

Our Limitations and Future Work paragraph already identifies multi-view context and depth as future extensions of this work.

References

  1. Qwen3-VL Technical Report.
  2. Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence.
  3. MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs.

Q2

Can the authors provide experiments that distinguish improvements in object localization from improvements in spatial reasoning? For example, would SpatialCORE still outperform standard GRPO when both models are provided with identical, accurate bounding boxes?

Answer:

SpatialCORE outperforms standard GRPO even when both models receive the same, more accurate bounding boxes at inference. In the requested experiment on the OmniSpatial test set, we provided both models with pseudo-GT boxes (instead of asking the model to generate their own) and allowed them to continue reasoning. This allowed us to compare final-answer accuracy with the supplied object locations held fixed.

Table 3: OmniSpatial accuracy (%) with model-generated BBoxes or the same supplied BBoxes for both models. Both models use Qwen3-VL-8B-Thinking.

ModelModel-generated BBoxesSupplied BBoxes
Standard GRPO45.9247.16
SpatialCORE48.4650.10

The more accurate object locations supplied by pseudo-GT boxes raise accuracy to 50.10% for SpatialCORE and 47.16% for standard GRPO. The accuracy gap widens from 2.54 to 2.94 percentage points. Moreover, SpatialCORE using its own generated boxes outperforms standard GRPO supplied with pseudo-GT boxes (48.46% vs. 47.16%). The key result is not simply that pseudo-GT boxes improve SpatialCORE’s accuracy, but that SpatialCORE still outperforms standard GRPO under the same supplied grounding, supporting a contribution to spatial reasoning beyond improved localization. This experiment is now included in Section 4.3 of the revised manuscript.

Since pseudo-ground-truth bounding boxes are already available, have the authors considered comparing against supervised fine-tuning with explicit grounding supervision?

Answer:

Following the reviewer's suggestion, we trained an SFT baseline with explicit grounding supervision. It uses the full OmniSpatial training split, the same backbone, and the same vision and language LoRA settings as SpatialCORE. Target responses include pseudo-GT bounding boxes, reasoning traces generated by GPT-4o, and final answers. Training uses standard next-token cross-entropy loss over the complete target response.

Table 4: Accuracy (%) after SFT with explicit grounding supervision and SpatialCORE post-training. SpatiaLab is evaluated zero-shot.

VariantOmniSpatialSpatiaLab
Base43.9044.78
Base + SFT45.2644.21
SpatialCORE48.4647.91

Base denotes Qwen3-VL-8B-Thinking.

SFT with explicit supervision for both grounding and reasoning is not sufficient to improve zero-shot generalization in this setting. SFT gains 1.36 percentage points on OmniSpatial, but falls 0.57 points below the backbone on SpatiaLab. Learning the supplied bounding boxes and reasoning traces therefore improves performance within the training distribution without producing a corresponding gain on unseen scenes and task designs. SpatialCORE outperforms the SFT variant by 3.20 points on OmniSpatial and 3.70 points on SpatiaLab. These gains support its use of pseudo-GT boxes to evaluate the model's own generated grounding, with the answer gate linking the resulting spatial reward to final-answer correctness. This SFT comparison is now included in Appendix C.3 of the revised manuscript.

Thanks for the detailed responses and the additional experiments. Most of my concerns have been addressed. 
I have one small question about the new Spearman correlation analysis: are the correlations computed over all predicted boxes pooled across questions, with the bootstrap performed at the question level? It would be helpful to clarify this briefly.

 Overall, I appreciate the authors' efforts in addressing the feedback and believe the additional experiments have made the paper stronger. Most of my concerns have been resolved, and I maintain my original rating.

We thank the reviewer for the constructive feedback and are glad that most concerns have been addressed. Yes, Spearman correlations are computed separately for each model over predicted BBoxes pooled across questions, including zero-overlap predictions. Bootstrapping is performed at the question level, resampling the same questions for both models and retaining all BBoxes from each selected question. We recompute both pooled correlations and their difference in each of 10,000 bootstrap replicates.

Reviewer_2vsxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

SpatialCORE investigates how to make spatial QA training simultaneously attend to answer correctness, relevant object locations, and the model’s confidence when generating those locations. The authors compute full-vocabulary prediction entropy at the positions where coordinate digits are generated, convert it to per-box confidence weights, and combine these with localization quality, automatic reference-box reliability, and answer correctness to form the training reward. The confidence here pertains to coordinate generation, not to the entire reasoning chain or final answer.Training first initializes with 1,000 samples containing boxes, reasoning, and answers, then adjusts the language and vision components via reinforcement learning. At inference time, the model independently generates box text, reasoning, and answers without an external detector. Overall,I find this paper’s main increment should be positioned as the confidence-weighting scheme for coordinate generation and associated training design.

Critical strengths
  1. The added weighting has direct internal ablation support. In Table 3’s subset-matched GRPO run, the full model scores 56.33; removing confidence weighting while retaining localization reward yields 52.33. Other variants also check visual-side updates, reference-box reliability, and answer gating. This is the most direct evidence for the added design, supporting its training value in the tested setting.

  2. The full scheme shows transfer gains and automatic supervision is empirically checked. Relative to vanilla GRPO on the same base, OmniSpatial improves from 45.92 to 48.46; the held-out SpatiaLab improves from 46.21 to 47.91, showing gains extend beyond training benchmarks. The authors audited 262 retained boxes in 200 training samples, finding 251 localized to relevant objects. Retraining after randomly corrupting 40% of valid reference boxes yields 46.70,which is below the original 48.46 but above the unadapted base 43.90. These checks support supervision quality and robustness to the tested corruption.

Critical weaknesses

First, the rationale for additionally rewarding confidence remains incomplete. Appendix B explains how weighting favors lower-entropy trajectories at comparable localization quality, but not why this preference improves grounding reliability. Coordinate-token entropy is not itself a correctness probability. Figure 4 supports improved confidence–localization correspondence, but lacks the matched weight-removed model needed to attribute that change to the new term. Table 3’s 56.33 versus 52.33 supports training value, without establishing that confidence better distinguishes accurate from inaccurate boxes when visual evidence is insufficient.

Second, evidence for the confidence weight’s independent benefit remains limited in scope. Whether vanilla GRPO shares the 1,000-sample initialization, prompts, visual-side updates, data, and budget is not fully specified. Removing visual-side updates in Table 3 lowers performance by 3.33 percentage points on that subset, demonstrating another consequential training factor. The main tables therefore support the full scheme, while the confidence term’s direct ablation evidence primarily covers Spatial Interaction. Broader matched evidence could establish its empirical value; current evidence remains insufficient for that.

Questions for authors
  1. Why reward greater coordinate certainty at comparable localization quality? Appendix B explains the preference, but not its reliability benefit. Could you compare the full and confidence-weight-removed models on the same objects, reporting localization error rates at the same retained proportion of highest-confidence boxes, particularly where visual evidence is insufficient? Please specify Figure 4’s sample selection and reference-box source.

  2. Are the main-table baseline and Table 3 variants matched apart from the stated changes? Please clarify initialization, prompts, data, vision/language updates, training budget, and evaluation settings. This would clarify which gains support the confidence term and which support the full training scheme.

With credible comparisons, I would consider raising the score from 2 to 3 based on either improved confidence–localization reliability, or narrowed reliability claims backed by stronger, stable QA gains attributable to confidence weighting beyond the tested subset.

9 comments
AuthorsOpenReview ↗

Q1/W1: Rationale and Evidence for Confidence-Aware Grounding

Why reward greater coordinate certainty at comparable localization quality? Appendix B explains the preference, but not its reliability benefit.

Answer:

SpatialCORE rewards greater confidence at comparable localization quality to encourage the model to reason from confidently predicted object locations. During spatial reasoning, SpatialCORE predicts bounding boxes for the relevant objects. If these generated boxes were rewarded using IoU alone, equally accurate boxes would receive the same credit, regardless of how confidently the model predicted them. The self-regulating spatial reward distinguishes these boxes by giving more credit to the one predicted with greater confidence.

Spatial reasoning requires the model to compare the positions, distances, and relationships of relevant objects. Our hypothesis is that predicting their locations accurately and confidently helps the model make these comparisons with less uncertainty about where the objects are.

We agree with the reviewer that Appendix B.2 can more clearly explain why rewarding accurate, confident generated grounding may benefit spatial reasoning, and have expanded this rationale in the revised manuscript. For a fixed matched prediction–reference pair, its contribution to the matching-quality term is

\omega_{i,j} R_{\mathrm{BBox}}^{(j,k)} = \left[\beta+(1-\beta)(1-H_{i,j})\right] R_{\mathrm{BBox}}^{(j,k)}. \tag{1}

Holding the assignment and matching quality fixed, its sensitivity to coordinate-token uncertainty is

\frac{\partial\left(\omega_{i,j}R_{\mathrm{BBox}}^{(j,k)}\right)}{\partial H_{i,j}} = -(1-\beta)R_{\mathrm{BBox}}^{(j,k)}. \tag{2}

Thus, the reward contribution gained from reducing uncertainty scales with the matched BBox quality. The same reduction in uncertainty produces a larger increase for a well-matched box and a smaller increase for a poorly matched box; it produces no increase when the matching quality is zero. Confidence therefore strengthens the reward contribution in proportion to the quality of the generated grounding.

This preference also carries through the spatial reward. With the matched pairs and recall fixed,

\frac{\partial P_i}{\partial H_{i,j}} = -\frac{1-\beta}{|\mathcal{M}_i^{\ast}|}R_{\mathrm{BBox}}^{(j,k)}, \qquad \frac{\partial R_{\mathrm{spatial}}^{(i)}}{\partial P_i} = \frac{(1+\alpha^2)\mathrm{Recall}_i^2}{\left(\alpha^2P_i+\mathrm{Recall}_i\right)^2} \geq 0. \tag{3}

where the second expression applies when the denominator is positive. Since the answer gate is positive, this preference is preserved in the composite reward when answer and format rewards are held fixed.

We have clarified in the revised manuscript that reliability refers to how accurately the model's most confident predicted bounding boxes locate the relevant objects. Appendix B.2 explains why the reward favors accurate, confident generated grounding, while Section 4.3 evaluates the relationship between confidence and localization quality.

AuthorsOpenReview ↗

Q1/W1: Rationale and Evidence for Confidence-Aware Grounding (Continued)

Could you compare the full and confidence-weight-removed models on the same objects, reporting localization error rates at the same retained proportion of highest-confidence boxes, particularly where visual evidence is insufficient?

Answer:

SpatialCORE's predicted boxes are more accurate when the model is more confident. To examine whether confidence weighting contributes to this relationship, we compared SpatialCORE with and without confidence weighting on the same OmniSpatial test questions, keeping all other training settings identical. Each model's predicted bounding boxes are ranked by coordinate-token uncertainty, retaining the same proportion of highest-confidence boxes. Mean localization error is measured as 100(1-IoU), including zero-overlap predictions so that confidently incorrect boxes remain counted, a particular concern when visual evidence is insufficient.

When all predictions are retained, localization errors are similar: 64.7 for SpatialCORE and 65.9 without confidence weighting. Among the most confident 25%, the errors are 18.8 and 26.8, respectively. These results suggest that confidence weighting makes the model's confidence a better indicator of box accuracy, supporting improved confidence–localization reliability. this comparison has been added to Section 4.3, Table 3 of the revised manuscript.

Table 1: Localization error at matched confidence-retention proportions on OmniSpatial. Brackets indicate 95% confidence intervals.

Highest-confidence boxes retained (%)SpatialCORE Error ↓SpatialCORE w/o confidence weighting Error ↓Error reduction ↑ with confidence weighting
2518.8 [6.3, 38.9]26.8 [8.4, 46.7]+8.0 [-0.4, +19.5]
5041.7 [25.2, 57.3]43.8 [28.3, 59.3]+2.1 [-0.9, +8.0]
7557.1 [45.1, 68.8]59.7 [47.8, 70.2]+2.6 [-0.9, +6.4]
10064.7 [56.1, 73.4]65.9 [57.3, 74.3]+1.1 [-0.3, +3.5]

Error reduction is the confidence-removed model's error minus SpatialCORE's error, computed before rounding.

Please specify Figure 4's sample selection and reference-box source.

Answer:

Figure 4 uses 652 questions from the OmniSpatial test split, each with at least one valid reference box. These are drawn from the Spatial Interaction and Perspective Taking categories. References are pseudo-GT boxes generated offline on these test images using GPT-4o-mini phrase extraction and Grounding DINO, retaining detections with validity at least 0.3.

For each model, we extract valid predicted bounding boxes generated before </think>. Each predicted box is evaluated using its maximum IoU against any reference box for that question. Figure 4 includes only predictions with positive overlap: 2,226 of 4,409 baseline boxes and 804 of 962 SpatialCORE boxes. Entropy quartiles are computed separately within each model. The sample selection and reference-box source are now clarified in Section 4.3 of the revised manuscript.

AuthorsOpenReview ↗

Q2/W2: Matched Training and Evaluation Settings

Are the main-table baseline and Table 3 variants matched apart from the stated changes? Please clarify initialization, prompts, data, vision/language updates, training budget, and evaluation settings.

Answer:

Yes. Within each comparison, the models share the initialization, prompts, training data and budget, and evaluation settings, apart from the explicitly stated changes. The main-table GRPO baseline and SpatialCORE start from the same cold-start SFT checkpoint, use the same grounding prompt, and update the same vision and language LoRA parameters. The baseline uses answer-accuracy and format rewards during GRPO; SpatialCORE adds the self-regulating spatial reward and answer gate. These matched training and evaluation settings are now clarified in Sections 4.2–4.3 and Appendix A.4 of the revised manuscript.

The ablation variants in Table 6 of the revised manuscript (Table 3 in the original submission) follow the same controlled setup on the Spatial Interaction subset, with each variant removing only its named component. Vision and language updates are therefore also matched, except in the explicitly designated vision-LoRA ablation. The main-table comparison measures the gains from adding the self-regulating spatial reward and answer gate to GRPO. The ablation without confidence weighting isolates the additional benefit of using confidence in the spatial reward.

This would clarify which gains support the confidence term and which support the full training scheme.

… stronger, stable QA gains attributable to confidence weighting beyond the tested subset.

Answer:

Confidence weighting improves SpatialCORE's final-answer accuracy across a broader range of spatial questions. To assess its contribution to the gains in Tables 1–2, we extend the Spatial Interaction ablation in Table 6 to the full OmniSpatial test set and zero-shot SpatiaLab. The ablated variant removes only confidence weighting. Unlike the existing standard GRPO (Base+GRPO) baseline, it retains the spatial reward and answer gate, with all other training and evaluation settings matched.

On the full OmniSpatial test set, SpatialCORE improves over GRPO by 2.54 percentage points, of which 1.57 points are lost without confidence weighting. On zero-shot SpatiaLab, 1.13 points of the 1.70-point gain are lost. Removing confidence weighting therefore reduces approximately 62% and 66% of the observed gains, respectively. Its contribution to final-answer accuracy extends beyond the original Spatial Interaction subset to the full test set and an unseen benchmark. This extended ablation is now included in Appendix C.1 of the revised manuscript.

Table 2: Spatial reasoning accuracy (%). Base denotes Qwen3-VL-8B-Thinking. Post-trained variants share the SFT initialization and training/evaluation settings, apart from the stated reward changes.

VariantOmniSpatialSpatiaLab
Base43.9044.78
Base + GRPO45.9246.21
SpatialCORE w/o confidence weighting46.8946.78
SpatialCORE48.4647.91

Comments on the paper

2 comments

We thank all reviewers for their time, effort, and constructive feedback, which have helped us strengthen the paper. We have updated the manuscript with additional experiments and clarifications, with additions and revisions highlighted in blue\color{blue}{\text{blue}}. We welcome further questions or suggestions and would be happy to clarify any aspect of our responses or revised manuscript during the discussion.

AuthorsOpenReview ↗

Summary of Responses and Revisions

Dear ACs, SACs, and Reviewers,

Thank you for your time and constructive feedback. We appreciate the reviewers’ recognition of the value of using model confidence to reward accurate object localization (Reviewers xtU2 and Y3qr), the evidence that confidence weighting improves answer accuracy (Reviewers 2vsx and vtgA), and the gains on a benchmark held out from training (Reviewers 2vsx and Y3qr). Reviewer vtgA also highlighted the ease of adding SpatialCORE’s reward to an existing training setup.

The additional experiments and revisions address the main concerns raised during discussion.

  1. Contribution of confidence weighting (Reviewers 2vsx, xtU2, and Y3qr). We added a Spatial Interaction experiment that gives every box the same weight. A fixed weight improved accuracy, and using each box’s confidence added 2 percentage points in this comparison. Confidence weighting also improved accuracy on OmniSpatial and SpatiaLab under matched training settings.
  2. Reliability of predicted boxes (Reviewers 2vsx and Y3qr). We compared the most confident quarter of boxes from each model. With confidence weighting, the share that missed the reference entirely fell from 13.2% to 8.8%. The reductions were statistically significant whether we counted boxes with no overlap or included those with very little overlap.
  3. Role of object locations in reasoning (Reviewers vtgA and xtU2). We gave both models the same boxes, and SpatialCORE still outperformed standard GRPO (50.10% vs. 47.16%). Moving or removing its predicted boxes changed some answers, showing that the model uses those locations. We also clarified how SpatialCORE differs from earlier work by giving confidence more influence in the training reward for more accurate boxes (Reviewer Y3qr).

The revised manuscript includes these comparisons and clarifications, with changes highlighted in blue. Thank you again for helping us strengthen the paper.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.