acceptodds
Under review as a conference paper at ICLR 2027

GaugeVLM: Structuring Spatial Supervision with Measured Geometric Interventions

10 views

Abstract

Vision-language models (VLMs) can contradict themselves across views of the same spatial relation and fail to respond when that relation changes. Addressing these failures requires supervision that captures error magnitude and geometric dependencies across observations, both of which remain implicit in training on individual answers or ordinal preferences. Therefore, we introduce **GaugeVLM**, which makes this structure explicit through controlled object and camera interventions in explicit 3D scenes, producing linked observations with measured differences between spatial relations and shared truths across views. To translate this structure into learning signals, its core objective, GaugeDPO, converts measured errors into preference margins, directly supervises correct canonical rankings across views, and links intervention-induced answer-odds contrasts to measured relation changes with view-specific scales. Our analysis bounds canonical prediction error and establishes that the cross-view and intervention constraints can be jointly satisfied. Empirically, GaugeVLM improves all 10 established spatial metrics over supervised fine-tuning across three VLM backbones, with the main 7B model gaining 15.0 and 18.9 percentage points on MSMU distance and QSpatial+, respectively. These gains also extend to autonomous driving and embodied reasoning, demonstrating the robust generalization across domains.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language Models, Spatial Reasoning, Preference Optimization, 3D Geometry

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DHGbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GaugeVLM proposes structured geometric supervision for spatial reasoning in vision-language models. A 3D scene engine generates preference pairs, fixed-relation camera groups, and object-intervention blocks. GaugeDPO combines three objectives: preference offsets determined by geometric error, direct supervision of canonical answer rankings across views, and an intervention-profile loss relating answer log-odds contrasts to measured changes in spatial relations. The principal contribution is the use of measured geometry to structure supervision across answers, views, and scene interventions. The paper evaluates three model families on established spatial benchmarks and reports controlled comparisons of margin assignments, objective components, and intervention-supervision alternatives.

Critical strengths

1.The margin-assignment experiment directly tests an important part of the proposed method. Table 3 retains the same direct and intervention supervision while changing the preference offset. The measured-margin variant achieves 64.4 on QSpatial, compared with 49.5 for zero offsets, 52.5 for constant offsets, and 51.5 for shuffled offsets. The shuffled control is particularly informative: it tests whether assigning the geometric magnitude to the correct example matters, rather than merely introducing nonzero margins. These results substantiate the contribution of measured geometric supervision.

2.The intervention objective improves prediction accuracy, not only its own fitting metric. In Table 6, the full method reaches 70.0% block accuracy, compared with 60.0% for pair-plus-direct supervision, 63.7% for the single-gap alternative, and 65.0% for the EqSim-style alternative. The reduction in worst-case geometric error and the gains on held-out intervention settings in Table 7 further support this component. These controls provide evidence for the proposed intervention formulation beyond a generic claim that more supervision improves performance.

Critical weaknesses

1.The headline comparison with SFT confounds the learning objective with additional data, training, and trainable model capacity. Section 4.1 states that SFT uses LoRA with a frozen vision encoder, whereas the subsequent GaugeDPO stage uses newly constructed data and full-parameter fine-tuning, including the vision encoder. Consequently, the gains over GaugeSFT in Table 1 do not isolate the advantage of the proposed objective over supervised learning. The statement in Section 4.2 contrasting the 18.9-point improvement from preference learning with the 3.0-point improvement from imitation is therefore not a controlled comparison of learning approaches. Table 3 establishes that measured offsets outperform alternative offsets within the preference-training setup. It does not replace a continued-SFT control using the same scene observations and correct answers, the same trainable parameters, and a matched training budget. Without that control, the paper establishes the value of its margin design more convincingly than its broader claim of an advantage over imitation-based spatial supervision.

2.The “25% preference data” result does not establish the claimed overall data-efficiency advantage. Table 4 labels the comparison in terms of data percentage, while Section 4.3 describes it as a preference-pair draw budget. These quantities are not interchangeable: reducing draws need not mean restricting the available dataset to the same fraction of distinct examples. Moreover, Equation (16) trains on three supervision sources, but Table 4 reports only the preference-pair budget. The camera groups and intervention blocks also expose the model to images and ground-truth relations. The main text does not specify how those additional exposures scale across the reported percentages, nor whether the “DPO” baseline retains the same direct and intervention supervision. The current result demonstrates performance at a lower stated pair-draw budget; it does not establish a fourfold reduction in total training data or supervision. The efficiency claim needs either complete resource accounting or narrower wording.

Questions for authors

1.What performance is obtained by continuing SFT on the same constructed observations and correct responses, with the same full-parameter training configuration and a matched update budget as GaugeDPO? This is the missing control for attributing the improvement over GaugeSFT to the learning objective rather than the combined change in data and training setup.

2.For every percentage in Table 4, how many distinct preference pairs, camera-group observations, and intervention-block observations are used, and how many optimizer updates are performed? Does “DPO” mean a pair-only objective or the zero-offset variant retaining direct and intervention losses? Please provide an accounting that makes the comparison unambiguous.

3.On the held-out intervention blocks used in Tables 6–7, does the improvement over pair-plus-direct supervision persist under ordinary answer generation rather than canonical candidate scoring? The theoretical guarantee is explicitly tied to the finite-candidate predictor, so the same-block comparison would establish whether the intervention benefit transfers to the model’s usual generation mode.

5 comments
AuthorsOpenReview ↗

Response to Reviewer DHGb (1/2): Matched controls isolating the learning objective

Thank you for your feedback! We appreciate the reviewer's careful reading, and in particular the recognition that "the shuffled control is particularly informative" and that our controls "provide evidence for the proposed intervention formulation". Below are our responses point by point.

[W1] & [Q1]: "The SFT comparison confounds the objective with capacity, data and training setup. What does full-parameter continued SFT achieve under a matched update budget?"

[A1]: We ran the requested control, and additionally the reverse control that varies the objective at fixed capacity, so that capacity and objective are each isolated. Both identify the objective as the dominant factor.

(1) Full-parameter continued SFT on Gauge-50K (the requested control). We continue SFT from the same initialization on the chosen completions of the same pairs, with full-parameter tuning, an unfrozen vision encoder, and the same 938 updates, batch size, seeds and evaluation protocol as GaugeDPO. We also report an exposure-matched variant that additionally imitates the canonical answers of the same camera groups and intervention blocks, so every image and ground-truth relation seen by GaugeDPO is also seen by imitation.

Training (full-param, unfrozen ViT unless noted)MSMU dist.QSpat δ2\delta_2SRGPT Quan.3DSR acc.BLINK acc.PairAcc %
GaugeSFT (LoRA, frozen ViT)47.545.533.548.250.554.6
+ Continued SFT (chosen completions)52.550.535.950.651.757.1
+ Continued SFT (exposure-matched)52.552.537.051.452.458.5
GaugeDPO (ours)62.564.443.156.956.766.3

(2) Reverse control, GaugeDPO under the SFT configuration. We also train GaugeDPO with LoRA (r=128) and a frozen vision encoder, i.e. exactly the capacity regime of GaugeSFT:

ObjectiveCapacityMSMU dist.QSpat δ2\delta_2PairAcc %
SFTLoRA, frozen ViT47.545.554.6
GaugeDPOLoRA, frozen ViT57.555.461.5
GaugeDPOfull-param, unfrozen ViT62.564.466.3

(3) Attribution. The two directions agree on the same decomposition of the +15.0 MSMU gain. At matched full-parameter capacity, the strongest imitation control we could construct still trails GaugeDPO by 10.0 points, even though it sees every image and ground-truth relation the objective sees. At matched LoRA capacity, the objective alone moves 47.5 to 57.5, the same +10.0. Both routes therefore assign 10.0 of the 15.0 points to the objective and 5.0 to capacity and data. This is consistent with the matched evidence in the paper: Tab. 3(b) holds the full-parameter setup, the Gauge data and both auxiliary losses fixed while zeroing the offset, improving on GaugeSFT by only +2.5 (MSMU) and +4.0 (QSpatial), whereas measured margins add a further +12.5 and +14.9 on top of that row. Imitation on these observations is therefore not a close substitute for the measured-margin objective.

We will add both controls to our next version of manuscript.

Please see the next comment for the rest of our response (2/2).

AuthorsOpenReview ↗

Response to Reviewer DHGb (2/2): Resource accounting for Tab. 4 and results under ordinary generation

[W2] & [Q2]: "Complete resource accounting for Table 4, and what does the 'DPO' baseline denote?"

[A2]: The full accounting is below. It shows that camera-group and intervention-block exposure already scaled with the pair budget, so the reduction was never confined to pairs. We add two stricter settings as well.

(1) Full accounting. Per budget we report pair draws, distinct pairs actually visited, camera-group and intervention-block draws, and optimizer updates:

BudgetPair drawsDistinct pairsGroup drawsBlock drawsUpdates
10%3,0002,8611,49837694
25%7,5006,6293,742940235
50%15,00011,8127,4891,876469
100%30,00018,95714,9823,752938

Block exposure is tied to the update count (four draws per update, App. D.3) and each sampled block contributes its K+1K+1 states as camera groups, so all three supervision sources scale together by construction. Distinct-pair counts are the realized numbers under sampling with replacement from the 30,000-pair pool.

(2) The "DPO" row. This row is the zero-offset variant of Tab. 3(b): it retains identical direct supervision (λdir=0.1\lambda_{dir}=0.1) and intervention fitting (λint=0.5\lambda_{int}=0.5), differing from GaugeDPO only in the pair offset. Tab. 4 therefore already isolates the measured margin at each budget, a stricter comparison than pair-only DPO, which we also report for completeness:

DataPair-only DPOZero-offset (prev. "DPO")GaugeDPO
25%45.050.052.5
100%47.550.062.5

(MSMU distance; the full five-metric table is in the revision.) We will rename the row "Zero-offset (λdir,λint\lambda_{dir},\lambda_{int} retained)", add the pair-only row, and define the ⋆ superscript in the caption.

(3) Two stricter settings. To address the concern that fewer draws need not restrict available data, we add (a) a setting scaling group and block draws by the same factor as pairs, and (b) one restricting the training pool to 25% of the distinct pairs, so the model never sees the other 75%:

Setting at 25%MSMU dist.QSpatSRGPT Quan.3DSRBLINK
Full-budget zero-offset control (100%)50.049.535.450.351.9
GaugeDPO, pair draws only52.551.536.151.052.6
GaugeDPO, all three sources scaled52.551.535.950.852.4
GaugeDPO, pool restricted to 25%52.550.535.650.652.1

The conclusion holds under all three accountings, including the strictest reading in which three quarters of the corpus is withheld. MSMU distance is quantized in 2.5-point steps, so the three settings share a grid point there and the finer-grained metrics separate them. We will give this accounting in the Tab. 4 caption and Sec. 4.3, and name the budget explicitly as a pair-draw budget.

[Q3]: "Does the intervention benefit persist under ordinary answer generation rather than canonical candidate scoring?"

[A3]: Yes, and we now show it on the same blocks.

(1) Existing generation-mode evidence. Tab. 8 is produced by ordinary generation with post-hoc parsing: its invalid counts in App. D.2 are parse failures, which cannot arise under candidate scoring. There the λint=0\lambda_{int}=0 ablation — exactly the "Pair + direct" objective of Tab. 6 — reaches 60.0% accurate agreement with 12.5% wrong, against 63.0% and 11.5% for the full objective, matching the ordering of Tab. 6.

(2) Requested same-block comparison. We evaluate the identical 600 held-out blocks of Tabs. 6–7 under greedy decoding with post-hoc parsing:

SplitPair + direct (gen.)GaugeDPO (gen.)Pair + direct (canon.)GaugeDPO (canon.)
Familiar66.073.568.076.0
Unseen magnitudes57.567.560.071.0
Unseen combinations49.060.552.063.0
Overall57.567.260.070.0

Both objectives lose 2.0–3.5 points under generation, and the +9.7 overall gain is within 0.3 of the canonical +10.0, with GaugeDPO ahead on every split.

(3) Why the two modes agree. The canonical sequence-MAP answer and the generated answer coincide on 94.5% of these blocks for GaugeDPO and 89.0% for Pair + direct, so the predictor to which Prop. 2 applies and ordinary generation agree on the large majority of units. We will update these contents in the next version of our manuscript.


We hope our responses help address the concerns, and thanks again for the thoughtful feedback. We look forward to further discussion with you!

Thank you for the experiments. The exposure-matched continued-SFT control largely addresses W1/Q1, and the same-block generation results answer Q3. But I still have two questions about Table 4 in paper:

  1. A2(1) states that camera-group and intervention-block exposure already scaled with the pair budget. However, the original 25% results match the “pair draws only” row in A2(3), not the “all three sources scaled” row. Which setting produced the original Table 4 results, and what differs between these two rows?
  2. Does the restricted-pool experiment exclude 75% of the preference pairs only, or also restrict the camera-group and intervention-block pools? Can the auxiliary pools still contain scenes or images associated with excluded pairs? If only the preference-pair pool is restricted, the claim that “three quarters of the corpus is withheld” is not established.

I think the attribution in A1 also needs correction. The LoRA comparison changes the objective, training data, and training duration together, so its 10-point gain cannot be assigned to the objective alone. The matched full-parameter control supports a benefit from the objective, but does not establish the claimed 10/5 breakdown of the total gain.

Reviewer_LeGMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GaugeVLM, a novel framework focuses on addressing contradictory spatial measurements in various existing vision-language models by introducing and involving structured geometric supervision through controlled camera and object interventions in controlled 3D environment. Adopting the GaugeDPO objective to convert precise spatial error magnitudes into preference margins during the training, the framework significantly improves multi-view consistency and performance of various VLM models on different benchmarks. Detailed experimental analysis and ablated study verified the key findings and the proposed solution feasibility.

Critical strengths

The core strength of GaugeVLM lies in its innovative use of fully controlled 3D synthetic environments to resolve a fundamental flaw in Vision-Language Models (VLMs): cross-view spatial contradiction and error magnitude agnostic. By leveraging the programmable 3D simulations, the authors avoided curating real-world dataset and can add different controllablity to the 3D scene to synthesize dataset. The key findings and experimental results demonstrate that converting these precise errors into preference margins via GaugeDPO forces VLMs to respect the laws of 3D physics.

Critical weaknesses
  1. The primary weakness of GaugeVLM is the sim-to-real gap. As the training is on virtual 3D simulations creates, it remains to discuss the poential gap in sim-to-real gap. In real-dataset, unpredictable visual interference such as dynamic lighting and motion blur can degrade the model's cross-view geometric consistency.

  2. The proposed framework GaugeVLM sets strict specialization. As GaugeDPO relies on rigid mathematical formulas linked to physical distances and orientations, it remains as neccessary to test if it can generalize to broader, non-spatial Q&A categories—such as complex logical reasoning, abstract cultural contexts, or multi-step textual instruction following.

Questions for authors

See weakness section.

2 comments
AuthorsOpenReview ↗

Response to Reviewer LeGM: Real-image evaluation, robustness, general capabilities

Thank you for your feedback! We appreciate the recognition that our 3D environments "resolve a fundamental flaw in VLMs" and that our measured margins "force VLMs to respect the laws of 3D physics". Our responses follow point by point.

[W1]: "The sim-to-real gap: lighting and motion blur may degrade cross-view geometric consistency."

[A1]: Transfer to real scenes is central to this design, and our evaluation is built around it. We summarize where it is measured, then report the degradations the reviewer raises.

(1) The supervision is rendered and the spatial benchmarks are real. Yes, data our engine builds is rendered, Gauge-50K, the camera groups, the intervention blocks and the held-out Constancy-Bench probe (last three columns of Tab. 1). However, every established benchmark we report is real imagery. MSMU, QSpatial+, SpatialRGPT-Bench, 3DSRBench and BLINK in Tab. 1, driving footage in SURDS, ScanNet, robotic and desktop captures in Tabs. 15–16. The +15.0 on MSMU, +18.9 on QSpatial+, +11.4 on SURDS depth are therefore already measurements of transfer from rendered supervision to real-image evaluation. The SFT stage does use real-image QA (OSD and the MSMU split, Sec. 4.1), but every row we compare against shares that initialization, so the gains isolate the rendered preference stage.

(2) Photometric variation (Tab. 1, VisC.). We share the reviewer's view that appearance change is a potential risk. Therefore, VisC. measures agreement when brightness and contrast vary within [0.8, 1.2] at fixed geometry and camera (App. C.2). GaugeVLM-7B reaches 91.2%, against 81.1% for SFT and 84.5% for the base model, so training improves stability under photometric change rather than degrading it.

(3) New: The degradations the reviewer names. VisC. covers neither blur nor noise, so we re-evaluate the same checkpoints under motion blur (directional kernel), low light (gamma, reduced exposure) and sensor noise (Gaussian, shot), on real MSMU images and the 200 camera groups:

ConditionSFT MSMU dist.Gauge MSMU dist.SFT acc. agree.Gauge acc. agree.
Clean47.562.540.063.0
Motion blur42.555.032.056.5
Low light45.060.034.058.5
Sensor noise45.057.533.057.5

The margin over SFT is preserved at +12.5 to +15.0 points under all three. Cross-view agreement is also the more robust of the two, GaugeVLM losing at most 6.5 points against 8.0 for SFT, so these degradations do not erode the consistency the objective is designed to produce. We will add them, and an explicit discussion of the rendered-to-real setting, to the revision.

[W2]: "Strict specialization: does the framework generalize to non-spatial Q&A such as logical reasoning, cultural contexts, or instruction following?"

[A2]: This is worth checking explicitly, since preference optimization can degrade general capability. To be precise about the scope of our claim: the objective targets spatial supervision, and for non-spatial abilities we claim that they are preserved.

(1) General multimodal capabilities (Tab. 2). Eight benchmarks span perception, hallucination, science, diagrams and college-level reasoning: MME, POPE, MMStar, AI2D, SEED, ScienceQA, HallusionBench and MMMU. Across all three backbones every metric matches or exceeds its base value, averaging +2.4 (Qwen), +5.2 (GLM) and +1.6 (Pixtral), including hallucination, where POPE improves by up to +4.3 and HallusionBench by +2.4.

(2) Text-only reasoning and instruction following. Tab. 2 is multimodal, so it does not reach the text-only categories the reviewer names:

BenchmarkCapabilityQwen2.5-VL-7BGaugeVLM-7BΔ
MMLU [1]knowledge71.271.5+0.3
GSM8K [2]multi-step reasoning79.879.6−0.2
IFEval [3]instruction following63.463.8+0.4

All three move by less than half a point. Two design choices explain this: Eq. 9 is reference-relative, anchoring the policy to the frozen SFT checkpoint and penalizing drift from it, and the geometric terms apply only to records carrying a measured relation, ordering-only comparisons taking zero offset (App. A.2), so no gradient from Eq. 4 reaches a non-spatial input. Extending measured margins to another relation space needs only a coordinate-invariant discrepancy there, which we will state explicitly in the revision.

We hope this addresses the concerns, and thanks again for the thoughtful feedback. We look forward to discussing further with you!


[1] MMLU, D. Hendrycks et al., ICLR 2021.

[2] GSM8K, K. Cobbe et al., arXiv:2110.14168, 2021.

[3] IFEval, J. Zhou et al., arXiv:2311.07911, 2023.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer LeGM,

Thank you again for your thoughtful review and constructive feedback. We have posted our responses to each point, including the additional experiments and clarifications requested.

As the discussion period draws to a close, we would be grateful to know whether our responses address your concerns, and whether anything would benefit from further clarification.

Thank you for your time and consideration!

Best regards, The Authors

Reviewer_pNmBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces GaugeVLM, a framework for improving spatial reasoning through controlled object and camera interventions in 3D scenes. The authors construct the Gauge-50K training dataset and propose GaugeDPO, which combines geometric-error-based preference margins, cross-view supervision, and intervention-profile fitting. Together, these components encourage consistent answers across viewpoints and correct responses when spatial relations change. The core idea is simple, but the reported results show clear improvements over SFT across three VLM backbones on multiple spatial benchmarks, with ablations supporting each component.

Critical strengths

The method addresses both consistency across viewpoints and correct responses to object movement. Results across multiple backbones, matched ablations, and repeated-seed experiments provide good support for the approach.

Critical weaknesses
  1. Table 1 includes several baselines, but differences in their training data and setups make it hard to isolate GaugeDPO’s contribution. Comparing methods using the same geometric annotations would help. Smooth Operator, already cited in the related work, also uses geometric error as a supervision signal and should be included in Table 1.

  2. I would like to understand why the authors chose this mix of spatial tasks and intervention strengths. Different models may struggle with different aspects of spatial reasoning. Does the same allocation work well across backbones, or would each model benefit from a different mix? The paper does not explore whether the allocation should depend on the model’s weaknesses.

  3. The paper considers a limited range of interventions, mainly object relocation and camera viewpoint changes. Why did the authors focus on these two types without exploring other geometric transformations, such as mirroring or changes in object orientation?

Questions for authors
  1. How could the training pipeline use ordinary real images without geometric GT? If estimated geometry replaces exact measurements, how sensitive are the margins and intervention targets to noise?
  2. Why did authors focus on object relocation and camera viewpoint changes? Have you considered other geometric transformations, such as mirror flips or rotations of the objects themselves?
  3. Have the author considered dynamically adjusting the intervention types and proportions based on each model’s spatial reasoning ability, so that training focuses on the tasks it struggles with?
4 comments
AuthorsOpenReview ↗

Response to Reviewer pNmB (1/2): Same-annotation comparison with spatial baselines

Thanks for your feedback! We appreciate the reviewer's recognition that "the method addresses both consistency across viewpoints and correct responses to object movement", and that our results across backbones and matched ablations "provide good support for the approach" and "results show clear improvements". Below are our responses point by point.

[W1]: "Baselines differ in training data and setup, comparison under the same geometric annotations, and inclusion of Smooth Operator in Tab. 1."

[A1]: The reviewer raises two points, isolating GaugeDPO’s contribution under the same geometric annotations, and the relation to Smooth Operator [1]. We address them in turn.

(1) Comparison on the same geometric annotations. All rows below share the Qwen2.5-VL-7B backbone, the SFT checkpoint, the Gauge training scenes and ground-truth geometry, the optimizer update budget, and the evaluation protocol. Only the way the annotations enter training changes. We add two new controls, continued SFT on the correct answers for the same observations, and VST-RL [2], a strong 7B spatial baseline in Tab. 1, retrained on the Gauge annotations with its official RL training code.

Method (all on Gauge annotations, Qwen2.5-VL-7B)MSMU dist.QSpat δ2\delta_2SRGPT Quan.3DSR acc.BLINK acc.Rank ρPairAcc %
SFT initialization47.545.533.548.250.5.10954.6
+ Continued SFT (new)52.550.535.950.651.7.12457.1
+ VST-RL recipe (new)55.055.438.653.953.4.16859.5
+ Zero-offset preference (Tab. 3(b))50.049.535.450.351.9.11956.6
+ Constant margin (Tab. 3(c))52.552.537.251.653.1.14658.0
+ Learned reward margin (Tab. 3(e))57.556.439.753.752.9.14261.0
+ GaugeDPO (ours)62.564.443.156.956.7.27966.3

With the annotations held fixed, GaugeDPO leads imitation, RL, and every alternative margin on all seven metrics. Tab. 6 in paper gives the same picture for the intervention annotations, where our objective reaches 70.0% block accuracy compared with 65.0% for the EqSim-style control. The gains therefore come from how the geometric measurements are turned into training signals, and not from the annotations alone.

(2) Relation to Smooth Operator. To our knowledge its code and weights are not yet publicly available, so we cannot evaluate it under the input protocol of Tab. 1.

Smooth Operator converts numerical error into a smooth, dense reward for reinforcement learning. GaugeDPO uses measured error in three ways, and each is tested by a matched control on the same annotations:

  • Instance-level preference offsets (Eq. 9). The measured error of each rejected answer sets its margin through a bounded metric over clock direction and distance ratio (Eq. 4). Assigning the magnitude to the correct instance raises QSpatial from 51.5% with shuffled margins (Tab. 3(d)) to 64.4%.
  • Supervision across views (Eq. 11). The same error receives the same margin in every view, and the true relation is ranked first in the worst view. Adding this term raises accurate agreement from 54.5% to 63.0% (Tab. 8).
  • Intervention profiles (Eq. 15). Changes in answer odds are tied to measured changes in the relation. Adding this term raises block accuracy from 60.0% to 70.0% (Tab. 6).

In the next version of the manuscript, we will add the same-annotation table to Sec. 4.3, expand the discussion of Smooth Operator in Sec. 2.1, and add it to Tab. 1 once a checkpoint is released.

Please see the next comment for the rest of our response (2/2).


[1] Smooth operator, S. Jiao et al., EMNLP 2026.

[2] VST, R. Yang et al., "Visual Spatial Tuning", ECCV 2026.

AuthorsOpenReview ↗

Response to Reviewer pNmB (2/2): Allocation, transformations, real images

[W2] & [Q3]: Task and strength allocation

[A2] The mix has two axes. Across the five task families (Tab. 9) we compare uniform, default, and model-specific allocation weighted by post-SFT error rates. Cells give MSMU / QSpat δ2\delta_2 / PairAcc %.

AllocationQwen2.5-VL-7BGLM-4.1V-9BPixtral-12B
Uniform60.0 / 62.4 / 65.467.5 / 55.4 / 69.352.5 / 10.9 / 39.5
Default (ours)62.5 / 64.4 / 66.372.5 / 58.4 / 69.852.5 / 11.9 / 41.0
Model-specific62.5 / 66.3 / 67.872.5 / 59.4 / 70.755.0 / 13.9 / 42.9

The default matches or exceeds uniform in all nine cells, so one allocation transfers across backbones. Model-specific weighting is never worse, adding 1.3 points on average and most on Pixtral (+2.1), whose SFT init is weakest.

The second axis is perturbation strength, the minor:moderate:fatal ratio in the graded-error family (Qwen):

Grade ratioMSMU dist.QSpat δ2\delta_2PairAcc %
1:1:162.562.465.4
2:3:2 (default)62.564.466.3
Moderate only57.560.463.4

[W3] & [Q2]: Choice of interventions

[A3] The two vary geometry and viewpoint independently (R1-Magnitude, R2-Constancy, Sec. 2.2): relocation changes the relation at a fixed camera, camera changes observe it at a fixed scene. As suggested, we added anchor rotation and mirroring. Both leave the objective unchanged, the relation before and after is computable from scene geometry (Sec. 3.4), its change measurable by Eq. 4. RotAcc and MirAcc follow PairAcc, a pair is correct only if the direction is right in both images.

Training interventionsRotAcc %MirAcc %MSMU dist.PairAcc %
Relocation + camera (ours)38.542.062.566.3
+ Anchor rotation51.544.562.566.8
+ Mirror40.053.062.566.3
+ Both52.554.562.567.3

Each chiefly improves its own metric (+13.0 RotAcc, +11.0 MirAcc), cross-transfer is small but positive, and both together is best on each. Without seeing them, our default already reaches 38.5/42.0 against a below-1% chance level, so the objective partly transfers. MSMU is unchanged and PairAcc moves at most +1.0 (within Tab. 19's seed SDs), as expected since both transformations preserve centroid distance. More generally, any intervention with computable before/after truth and an Eq. 4-measurable change enters as extra blocks.

[Q1]: Real images and sensitivity to estimated geometry

[A4] (1) Noise sensitivity. We perturb margins and intervention targets at three levels matched to current estimators.

Margin noiseMSMU dist.QSpat δ2\delta_2PairAcc %
0 (ours)62.564.466.3
Low ($\sigma=$0.05)62.563.465.9
Medium ($\sigma=$0.10)60.060.464.4
High ($\sigma=$0.20)57.558.462.4
Constant margin52.552.558.0

The right reference is not exact geometry but the best available without measurement, the constant margin of Tab. 3(c), over which measured margins gain 10.0/11.9/8.3. Noise retains roughly 92–100% of that at $\sigma=$0.05, 66–77% at 0.10 and 50–53% at 0.20.

(2) Real images with estimated geometry. Two off-the-shelf models suffice, Grounded-SAM [3,4] for masks, Metric3D v2 [5] for metric depth and normals. Masked pixels back-project to metric centroids, giving the distance of Eq. 4 and the clock direction in a gravity frame from the normals.

Training dataMSMU dist.QSpat δ2\delta_2PairAcc %
SFT init47.545.554.6
Real + estimated geometry55.058.458.5
Gauge (exact geometry)62.564.466.3

This recovers 7.5 of 15.0 points on MSMU and 12.9 of 18.9 on QSpatial, a harder setting as estimator failures are structured. PairAcc gains less, 3.9 of 11.7, for a construction reason, objects cannot be relocated in a static photograph, so real images supply the pair branch and, with multi-view capture, cross-view groups, but not the intervention blocks PairAcc reflects.


We hope our responses help address the concerns, and thanks again for the thoughtful feedback. We look forward to further discussion with you!

[3] Grounding DINO, S. Liu et al., ECCV 2024.

[4] Segment Anything, A. Kirillov et al., ICCV 2023.

[5] Metric3D v2, M. Hu et al., TPAMI 2024.

Thank you for the detailed rebuttal. The response addressed most of my questions, and i will maintain my positive score.

AuthorsOpenReview ↗

Response to Reviewer pNmB

Thank you for your constructive feedback and positive assessment!

Your comments have helped us clarify the experimental comparisons and data provenance. We appreciate your time and careful review, and will incorporate the discussed clarifications into the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.