acceptodds
Under review as a conference paper at ICLR 2027

Learning and Reasoning with Geometry: How Generation Helps Spatial Understanding

Abstract

Image and video generators can produce spatially consistent visual content, but how to use this capability to improve spatial understanding remains underexplored. We study this problem from two perspectives: geometric supervision during training and explicit geometric evidence during inference. To this end, we introduce a unified geometric understanding and generation model (GeoUG) that integrates depth and multi-view generation with spatial understanding. First, we design a multi-scale depth codec that enables metric-depth generation using a pretrained visual autoencoder, and show that joint training yields task-dependent gains in spatial understanding. Second, we introduce 3DCueBench to control geometric difficulty while balancing selected 2D cues, and find that generated depth provides larger reasoning gains on questions requiring fine-grained geometric discrimination. Building on this finding, we develop adaptive interleaved reasoning, in which GeoUG first drafts an answer and then decides whether to generate geometric evidence to confirm or revise it. We train this decision through reinforcement learning, balancing accuracy gains over retaining the draft against generation cost. GeoUG achieves circular accuracies of 97.9%, 90.8%, and 94.2% on CV-Bench-3D, 3DCueBench, and DA-2K with fewer generation calls than always generating.

Primary area:
generative models
Keywords:
spatial understanding, unified understanding and generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WgeWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how geometric generation can support spatial understanding, both as supervision during training and as explicit evidence during inference. The authors introduce GeoUG, built on BAGEL, with metric-depth and viewpoint generation capabilities. A multi-scale depth codec enables metric-depth prediction through the existing visual generation interface. For inference, a draft-conditioned policy learns whether to generate depth and obtain geometric measurements before retaining or revising its initial answer. The paper also introduces 3DCueBench, which balances selected 2D cues across geometric difficulty levels. Experiments show task-dependent benefits from generative co-training and improved single-image spatial reasoning with selective depth generation.

Critical strengths
  1. The paper presents a coherent approach connecting geometric supervision with inference-time geometric evidence. The draft-conditioned policy achieves accuracy close to always generating while reducing depth-generation calls, and its advantage over random selection supports the usefulness of the learned acquisition policy.
  2. 3DCueBench and the accompanying analysis provide a useful examination of spatial evaluation. Balancing selected cue-consistent and cue-conflicting examples across difficulty levels helps distinguish geometric discrimination from reliance on simple image-plane relationships.
Critical weaknesses
  1. The additional value of generative modeling over access to explicit geometric information remains insufficiently established. The experiments demonstrate that the proposed generation-assisted pipeline improves over direct answering. However, this comparison does not sufficiently distinguish the benefit of generating geometry from the benefit of obtaining geometric information. Non-generative alternatives could provide depth maps, numerical readouts, or geometry-encoder features to a comparably trained understanding model. This distinction matters because DA3 readout already achieves an average score of 92.8, compared with GeoUG’s 94.3, and essentially matches GeoUG on 3DCueBench. Appendix F.3 is relevant, but supplying measurements to existing VLMs does not provide a controlled comparison with a comparably trained geometry-augmented model; the appendix also acknowledges prompting and resolution differences. Consequently, the results establish a useful pipeline, but leave its incremental contribution relative to alternative forms of geometric integration unclear.
  2. The evidence for broader spatial understanding is limited to single-image reasoning and small sets of views. The strongest inference-time results concern single-image depth and distance comparisons. The co-training experiments on MindCube and the MMSI camera-motion subset provide some multi-view evidence, but do not establish the effectiveness of the generation-assisted reasoning procedure over substantially longer image sequences. Such settings introduce requirements beyond per-image depth quality, including cross-view alignment, geometric consistency, and integration of partially observed scene information. The relatively modest multi-view co-training gains do not resolve these questions. This limits the demonstrated breadth of the approach and leaves uncertainty about how far its benefits extend beyond the evaluated comparisons.
Questions for authors
  1. Could the authors provide a controlled comparison with a non-generative geometry-augmented model, using comparable spatial supervision and training conditions? For example, how does GeoUG compare with a model trained to consume depth maps, DA3 readouts, or geometry-encoder features and selectively use that information?
  2. How would the proposed generation-assisted reasoning procedure operate over longer sequences, such as 16–32 frames requiring cross-view spatial integration? Is there evidence that its benefits persist, and how would geometric inconsistencies across generated depth maps be handled?
  3. Given the strong DA3 readout results, what practical advantage does the generative route offer at comparable computational cost? Measurements of actual inference latency and training requirements would help assess whether the observed accuracy gains justify this route beyond reducing calls relative to always generating within GeoUG.
4 comments

Q1: Value of generative geometry

Thank you for distinguishing the value of generating geometry from that of accessing geometric information. We have added the requested trained, non-generative geometry-augmented baseline and evaluated it on all three benchmarks. This addresses a gap in our original evidence: direct answering and the off-the-shelf VLM comparisons in Appendix F.3 did not provide this trained comparison.

Our BAGEL–DA3 baseline uses frozen DA3 to estimate depth and returns numerical geometric measurements to the reasoning model. Spatial SFT trains the model to use these measurements, and RL trains it to acquire them selectively. We align the understanding initialization, main spatial questions and answer supervision, and RL recipe with GeoUG. This tests whether a trained model with access to external geometry can recover GeoUG's performance.

MethodCV-Bench-3D3DCueBenchDA-2KAverage
DA3 deterministic readout (Table 1)95.390.992.192.8
Trained BAGEL–DA3, adaptive96.5082.6287.1488.75
Trained BAGEL–DA3, forced acquisition96.0088.3889.5191.29
GeoUG, adaptive (paper)97.990.894.294.3

Full-benchmark circular accuracy (%); average across the three benchmarks.

1. GeoUG outperforms the trained external-geometry alternative, even when acquisition is forced. GeoUG exceeds adaptive BAGEL–DA3 by approximately 5.6 points on average. Forcing DA3 acquisition improves the baseline from 88.75% to 91.29%, but GeoUG still leads by approximately 3.0 points, with higher accuracy on every benchmark. The remaining gap therefore cannot be attributed solely to the baseline choosing not to acquire geometry. This comparison strengthens the evidence beyond an improvement over direct answering: GeoUG also outperforms a model explicitly trained to access and use external geometric measurements.

2. The strong deterministic DA3 result helps distinguish geometry quality from learned use of geometry. We retain this baseline prominently: its 92.8% average exceeds both trained BAGEL–DA3 variants, and GeoUG's advantage over it is 1.5 points, with essentially matched performance on 3DCueBench. Thus, our claim is not that DA3 lacks useful geometric information. Rather, access to that information does not by itself reproduce GeoUG's performance in the evaluated learned reasoning pipeline. The deterministic readout and trained baseline answer different questions: the former measures what task-specific rules can extract from DA3, while the latter tests a model trained to acquire and reason with its measurements.

3. The demonstrated advantage belongs to the complete integration, rather than to geometry generation in isolation. GeoUG retains a VAE-encoded representation of its generated depth alongside numerical query results; the evaluated BAGEL–DA3 baseline receives numerical measurements alone. GeoUG therefore provides the reasoning model with both a depth representation and explicit measurements. This contextual representation, together with joint generation–understanding training, offers a plausible explanation for the performance gap, but the current comparison does not isolate their individual effects. We will make this input distinction explicit rather than attribute the entire gain to the generative depth source.

The new experiment supports a concrete conclusion: under the aligned training settings described above, the complete GeoUG system outperforms the evaluated trained non-generative numerical-tool alternative, including when that alternative always acquires geometry.

Q2: Long-sequence reasoning and cross-view consistency

Thank you for raising this concern. We added 16-frame VSI-Bench experiments to test two distinct questions: whether geometric co-training transfers to video spatial understanding, and whether generated geometry can assist inference across multiple frames.

1. Training-time transfer to 16-frame spatial understanding. We compare four models, all of which answer directly from RGB frames without generating geometry at inference.

ModelAvg.Object countingObject sizeAbsolute distanceRelative directionRoute planning
Original BAGEL31.9923.8833.7524.6545.7331.96
Understanding-only42.8640.8367.8229.5246.2129.90
+ Multi-view generation43.9940.8570.2129.5147.4131.96
+ Depth generation42.6140.9667.0031.0246.2327.84

The task-dependent transfer observed in our original experiments extends to the 16-frame setting. Relative to understanding-only training, depth co-training improves absolute-distance estimation by 1.50 points, while multi-view co-training improves relative direction, route planning, and object size by 1.20, 2.06, and 2.39 points, respectively. Multi-view co-training also increases average performance from 42.86 to 43.99. These results support choosing geometric objectives according to the target spatial skill, rather than expecting every objective to improve every task.

2. Inference-time assistance with cross-view consistency checks. We compare direct and geometry-assisted reasoning on 88 geometry-eligible absolute-distance questions from 58 scenes, using the same GeoUG checkpoint and 16 RGB frames. Eligibility follows fixed grounding and geometric-support checks, not answer correctness.

GeoUG generates metric depth independently per frame using MMDC. Auxiliary object-region and camera-pose predictions align the corresponding surface points in a shared coordinate system. To handle inconsistencies between independently predicted depth maps, we apply occlusion-aware cross-view reprojection checks, retaining points supported by another depth map and rejecting ambiguous detections or insufficient geometric support. The model answers using the retained visible-surface distance estimates, without ground-truth depth or answer-based geometric correction.

Inference condition on the same 88 questionsAbsolute-distance MRA (×100)
Direct RGB reasoning31.82
GeoUG-generated depth assistance60.45

The 28.63-point gain shows that generated geometry can assist 16-frame reasoning when combined with explicit alignment and consistency filtering. This result applies to the eligible subset in an externally orchestrated diagnostic, rather than a full-benchmark evaluation of an autonomous policy. Qualitative examples appear in Appendix A (Long-sequence inference).

Q3: Practical cost and benefit compared with DA3

Our goal is to understand how generation can help spatial understanding. GeoUG learns to generate geometric evidence and decide when to use it to answer spatial questions.

The DA3 readout baseline uses task-specific scripts to extract the supplied coordinates, compute depths or distances, and choose an answer from those values. In GeoUG, the model constructs the measurement queries and produces the answer. DA3 readout is a strong baseline for these structured tasks, but comparing the full reasoning systems would require a DA3-augmented model trained under comparable conditions.

GeoUG learns to generate depth with 6k additional training steps on 8 GPUs, starting from its pretrained generative backbone. DA3 starts from DINOv2 and reports 200k steps on 128 H100 GPUs, with DA3-Giant taking approximately 10 days. GeoUG also supports several geometric generation tasks within the same model, though we have so far tested their benefit to reasoning only through depth.

Our current implementation is substantially slower than DA3 readout. However, on 32 randomly sampled 3DCueBench questions, adaptive reasoning reduces mean latency from 63.28 s to 53.68 s, saving 9.60 s per request (15.2%) compared with forced generation using the same checkpoint, including drafting and decision overhead. We measured both modes on one A100-80GB GPU at batch size 1, with identical inference settings. After two warmup requests, each question was evaluated three times per mode using the original option order. Adaptive reasoning skips generation on 18.75% of requests, while both modes answer 89/96 requests correctly. This reduces GeoUG’s inference cost, but does not make it faster than DA3. We acknowledge this limitation and will report the absolute timings and full measurement protocol in the supplementary material.

Thank you for the additional experiments and clarifications. They address my main concerns, although the baseline comparison and multi-frame evaluation still have some limitations. I maintain my weak-accept recommendation.

Thank you for taking the time to review our additional experiments and clarifications. We’re glad they helped address your main concerns. We appreciate your constructive feedback, and we’ll take care to explain the remaining limitations more clearly in the revised manuscript. Thank you again!

Reviewer_mDq7OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces GeoUG, a unified model that uses geometric generation to improve spatial understanding during both training and inference. It combines depth and view-generation supervision with a draft-conditioned policy that selectively generates depth for reasoning. The paper also introduces 3DCueBench to evaluate spatial reasoning while controlling selected 2D shortcuts.

Critical strengths
  1. The paper distinguishes geometric generation as training supervision from its use as inference-time evidence, providing useful analysis of when each benefits spatial understanding.
  2. The proposed benchmark balances selected 2D cues, while the adaptive reasoning experiments demonstrate near-always-generate accuracy with fewer generation calls.
Critical weaknesses
  1. Using depth generation during training to improve spatial understanding and generating visual evidence during inference have already been explored in previous work. For example, UniMRG incorporates depth representations during training, MVoT generates visual intermediate states during reasoning. The authors should clarify how their method differs from these approaches. Comparisons under similar experimental settings would also be helpful, particularly with COOPER, which is likewise based on BAGEL.

  2. What is the computational overhead of generating images during inference? Does it substantially increase the computation required? My understanding is that image generation involves multiple denoising steps and can be considerably slower than generating text tokens. The paper should discuss and quantify this overhead.

Questions for authors
  1. Could you clarify how your method differs from similar prior work, particularly approaches that already incorporate depth representations during training or generate intermediate representations during inference? If the experimental settings are similar, could you provide quantitative comparisons?

  2. How much additional computation and inference time does image generation require compared with directly answering in text?

4 comments

Q1: Comparison with UniMRG, MVoT, and COOPER

We thank the reviewer for highlighting these closely related works. We agree that auxiliary depth generation and visual intermediate reasoning have precedents. GeoUG contributes (1) explicit metric verification using generated geometry, (2) findings on when geometric generation improves spatial understanding, and (3) online learning of generation utility.

MethodWhat is generatedUse of generation at inferenceExplicit training for when to generate
UniMRGRGB reconstruction, depth, segmentationNoNo
MVoTIntermediate visual statesVisual reasoningNo
COOPERDepth, segmentationInterleaved visual reasoningRL: offline visual-gain labels shape exploration reward
GeoUG (ours)Metric depth; view generationMetric measurements verify the draftRL: online draft-relative benefit, minus generation cost

1. Compared with MVoT and COOPER: explicit metric verification. MVoT uses generated visual states to support reasoning, while COOPER generates depth and segmentation maps during interleaved reasoning. GeoUG contributes a metric representation and a measurement-based verification procedure: MMDC preserves metric depth through a fixed, invertible encoding, and geometric queries turn decoded depth into measurements that confirm or revise the draft. This is particularly relevant to object-to-object distance, which cannot be determined simply by ranking camera-relative depths.

2. Compared with UniMRG: examining task-dependent benefits of geometric generation. UniMRG uses auxiliary RGB reconstruction, depth, and segmentation tasks to improve understanding. GeoUG further investigates which geometric generation objectives benefit which spatial abilities, separately comparing metric-depth, camera-conditioned view, and exo-to-ego generation against understanding-only training. We find that the benefits depend on the task: on MMSI camera motion, multi-view generation improves accuracy whereas depth generation lowers it. This provides a concrete finding about both positive and negative transfer, showing why the choice of geometric generation objective matters.

3. Compared with COOPER: online generation utility and quantitative results. COOPER already learns adaptive interleaving, with fixed visual-gain labels computed before RL shaping its exploration reward. GeoUG instead evaluates the current draft and estimates generation-branch success through forced-generation rollouts. Its decision signal therefore tracks whether generation improves the current model's answer enough to justify its cost. We additionally identify that trajectory-level group-relative advantages cancel the shared generation cost when every rollout generates. Our dedicated decision advantage compares generation against retaining the draft, preserving a benefit-and-cost signal even when sampled rollouts choose the same branch. Empirically, GeoUG exceeds random selection at matched generation rates by 3.5 / 4.7 / 5.7 points on CV-Bench-3D / 3DCueBench / DA-2K, supporting its ability to select useful generation. The offline/online distinction concerns the visual-gain signal, not COOPER's entire RL procedure or total reward.

To complement this methodological comparison, we evaluated the released full COOPER model on the CV-Bench-3D and 3DCueBench. Circular accuracy requires both orders to be correct.

BenchmarkMethodDepthDistanceOverallGeneration rate
CV-Bench-3DCOOPER64.257.360.848.4
CV-Bench-3DGeoUG (ours)98.597.397.950.3
3DCueBenchCOOPER50.341.345.856.1
3DCueBenchGeoUG (ours)9586.590.867.4

GeoUG achieves substantially higher spatial reasoning accuracy than the BAGEL-based COOPER baseline. On CV-Bench-3D, it reaches 97.9% versus 60.8% circular accuracy at similar generation rates (50.3% versus 48.4%); on 3DCueBench, it reaches 90.8% versus 45.8%.

COOPER's observed confusion between camera depth and object-to-object distance illustrates a challenge beyond producing visual evidence: using it to answer the intended geometric question. GeoUG addresses this challenge through metric encoding and explicit measurement queries, which support checking and revising the draft.

GeoUG contributes which geometry to learn, how to verify answers with it, and when generation is worth its cost, supported by task-dependent transfer, metric verification, and online draft-relative decision learning.

Q2: Generation overhead and latency savings

Geometric generation improves answer accuracy at an additional inference cost; adaptive interleaved reasoning reduces that cost by generating selectively. We quantified this trade-off on 32 randomly sampled 3DCueBench questions on one A100 GPU at batch size 1. After two warmup requests, each question was evaluated three times per mode, using the original option order.

ModeGeneration rateMean latencyAnswer accuracy
Direct text answer0%11.22 s81.25%
Forced depth generation100%63.28 s92.71%
Adaptive reasoning81.25%53.68 s92.71%

Forced generation adds 52.06 s over direct answering, while improving accuracy by 11.46 percentage points on this subset. Depth generation and decoding alone take 41.97 s. Adaptive reasoning retains the observed accuracy gain while saving 9.60 s per request (15.2%), including its decision overhead.

Savings depend on generation frequency. This timing subset invokes generation on 81.25% of requests, versus 67.4% in the full 3DCueBench evaluation. Using the measured skip/generation branch costs (11.84/63.34 s), a simple model estimates 26.4% latency savings at a 67.4% generation rate, or 32.5–40.6% at rates of 50–60%, assuming unchanged branch costs. These are conditional estimates, distinct from the measured 15.2% reduction.

Thus, generation remains costly, but adaptive reasoning offers a measurable accuracy–latency benefit over always generating.

I thank the authors for their rebuttal. Although I consider the generation cost too high compared to that of text responses, the authors have honestly reported this result. I have raised my score.

Thank you! We appreciate your constructive comments and thoughtful feedback.

Reviewer_dxW1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how geometric generation can improve spatial understanding from two perspectives: geometric supervision during training and explicit geometric evidence during inference. The authors propose GeoUG, a unified understanding and generation framework that integrates metric-depth and multi-view generation with spatial understanding. The main contributions include a Multi-scale Metric Depth Codec (MMDC) for metric-depth generation and an adaptive interleaved reasoning strategy, which uses reinforcement learning to determine whether additional geometric evidence is needed based on an initial draft answer. The paper also introduces 3DCueBench to evaluate spatial reasoning under different geometric difficulties while balancing selected 2D visual cues. Experiments on several spatial understanding and depth estimation benchmarks demonstrate that GeoUG achieves competitive performance while reducing depth generation calls compared with always generating.

Critical strengths
  1. The proposed framework provides an interesting way to connect geometric generation with spatial understanding. By investigating geometric generation during both training and inference, the paper demonstrates how generation can serve as additional supervision and explicit evidence for spatial reasoning.

  2. The proposed adaptive interleaved reasoning strategy is interesting. By conditioning the generation decision on an initial draft answer, the method can reduce unnecessary depth generation while maintaining competitive accuracy. The experimental results and ablation studies demonstrate the effectiveness of the proposed decision strategy.

  3. The introduction of 3DCueBench provides a useful evaluation of spatial reasoning by controlling geometric difficulty and balancing selected 2D visual cues. This helps investigate whether the model relies on simple image-plane relationships rather than geometric information.

Critical weaknesses
  1. Insufficient comparison with closely related methods: The experiments lack comparisons with closely related geometry-enhanced vision-language models, such as G²VLM (CVPR 2026) and 3DThinker (CVPR 2026), which also incorporate 3D geometric information into spatial understanding and reasoning. In particular, G²VLM also integrates geometric modeling and spatial reasoning within a unified framework and supports interleaved reasoning. Additional comparisons on common spatial reasoning benchmarks, along with a more detailed discussion of their methodological differences, would help clarify the novelty and advantages of GeoUG over existing approaches.

  2. Limited improvements from MMDC: In Table 5, MMDC provides only marginal improvements over single-disparity encoding (7.24 vs 7.47), while larger gains come from increasing the training data (6.23). The contribution of MMDC requires further clarification.

  3. Insufficient metric-depth evaluation: The depth estimation results in Table 4 are reported with affine alignment, which does not directly demonstrate the accuracy of absolute metric-depth prediction.

  4. Limited computational efficiency analysis: The paper mainly evaluates efficiency through generation rates. Additional results on inference latency and computational cost would better demonstrate the practical advantages of adaptive reasoning.

Questions for authors
  1. Can the authors provide comparisons with G²VLM, 3DThinker, or other closely related methods?

  2. How sensitive is MMDC to the choice of depth scales (2, 8, and 32 meters)?

  3. Can the authors report metric-depth estimation results without affine alignment?

  4. How much inference latency and computational cost does adaptive reasoning actually save?

  5. Can GeoUG generalize to more comprehensive spatial reasoning benchmarks, such as VSI-Bench?

5 comments

Q1: Comparison with G²VLM and 3DThinker

We thank the reviewer for highlighting these closely related works. G²VLM, 3DThinker, and GeoUG all incorporate geometry into spatial reasoning. GeoUG’s main distinction is adaptive interleaved reasoning, i.e., learning whether to generate explicit geometric evidence, conditioned on an initial answer and balancing the expected improvement against generation cost. It either retains this draft or generates depth and queries geometric measurements to confirm or revise it.

AspectG²VLM3DThinkerGeoUG
Geometry used for QAPre-encoded geometric features accessed through attentionGenerated geometry-aligned latent tokensGenerated depth and explicit geometric measurements
Reasoning workflowGeometry encoding → answer3D latents → textual reasoning → answerDraft → decide → optionally generate and measure → confirm/revise
Whether to acquire geometryEncoded before answeringLatents generated before textual reasoningLearned decision balancing improvement over the draft against generation cost

Direct comparison: We evaluate the available official checkpoints on the same benchmarks. GeoUG outperforms the G^2VLM and 3DThinker.

Official checkpoint / variantCV-Bench-3D3DCueBench
G²VLM released baseInternRobotics/G2VLM-2B-MoT19.0015.00
3DThinker stage 1jankin123/3DThinker-Mindcube50.3349.12
GeoUG (adaptive)Existing manuscript results97.990.8

We will include the completed comparisons and clarified methodological discussion in the revised manuscript.

Q2: Contribution of MMDC

MMDC outperforms all tested single-scale configurations. To clarify its independent contribution, we vary the single-disparity scale cc under the same training and evaluation settings. MMDC achieves a lower average AbsRel than even the best tested single scale (7.24 vs. 7.38; a 2.0% relative reduction). The best single scale also varies across benchmarks, supporting the motivation for a multi-scale representation.

EncodingScale cc (m)Avg. AbsRel ↓
Single-disparity37.459
Single-disparity57.474
Single-disparity87.421
Single-disparity167.384
Single-disparity327.911
MMDC(2, 8, 32)7.24

We acknowledge that this gain is modest and distinguish it from the larger improvement obtained by expanding the training data.

Depth-range analysis: the aggregate score masks larger gains at near range. To examine where MMDC helps, we evaluate paired images from five depth benchmarks (NYU-v2, KITTI, ETH3D, Sintel, and DIODE). For each image and method, we perform affine alignment once over all valid pixels, then evaluate pixels grouped by ground-truth metric depth. The boundaries are the geometric means between adjacent MMDC scales (2, 8, and 32 m). We report pooled image-macro AbsRel ×100 (↓) and δ₁ (%) (↑).

Depth rangeSingle-scale AbsRel ↓MMDC AbsRel ↓Relative reductionSingle-scale δ₁ ↑MMDC δ₁ ↑
Near (<4 m)12.13711.2627.21%89.9990.13
Mid (4–16 m)10.48210.4120.66%86.4386.82
Far (≥16 m)22.56822.0852.14%67.3167.54

MMDC improves both metrics in all three broad depth ranges, with a particularly clear reduction in near-range error. The 7.21% relative AbsRel reduction below 4 m shows that the aggregate result alone understates the benefit in this range. The improvement also extends to far-range depth, where AbsRel decreases by 2.14%, supporting the motivation for representing depth at multiple scales.

Together with the single-scale ablation above, these results establish an independent contribution of MMDC: it outperforms every tested single-scale configuration overall and improves performance across the three broad depth ranges under matched training settings. The larger gain from additional data does not negate the benefit of the representation itself. We will include these analyses to make this distinction clear.

Q3: Sensitivity to depth scales

We thank the reviewer for this question. MMDC remains stable under the different depth scales. We trained an alternative model with scales (3, 12, 48) m instead of (2, 8, 32) m, preserving their ratios, and compare the two models at the same 3,000-step training budget with matched training data. Each model is decoded using its own training scales. We use the same evaluation protocol, prompt, and sampling settings for both models.

Following the paper, we report affine-aligned AbsRel ×100 (↓) and δ₁ in percent (↑). Each cell below shows AbsRel / δ₁; AVG is the unweighted mean across the five datasets.

MethodAVGNYU-v2KITTIETH3DSintelDIODE
MMDC (2, 8, 32), (paper)7.24 / 92.573.35 / 98.386.51 / 95.344.62 / 96.9916.31 / 77.235.40 / 94.88
MMDC (3, 12, 48)7.24 / 92.523.38 / 98.456.38 / 95.294.61 / 96.8816.29 / 77.205.54 / 94.78

Mean AbsRel is 7.24 for both scale configurations at the reported precision, while mean δ₁ changes from 92.57% to 92.52% (−0.05 percentage points). These results indicate limited sensitivity to the absolute scale values under this tested shift. The default-scale row is taken directly from the paper; the alternative-scale row reports the additional experiment.

Q4: Absolute metric-depth accuracy

We agree that affine-aligned evaluation alone does not establish absolute metric-depth accuracy. Other generative models (Marigold, Lotus) that predict per-image normalized relative depth in [0,1][0,1] cannot directly recover depth in meters without additional scale information. MMDC enables metric-depth output through a fixed, invertible encoding that preserves physical scale, allowing our model to decode predictions directly into meters.

We therefore evaluate metric depth without affine alignment. The table reports AbsRel ×100 ↓ / δ1 (%) ↑, with reported DA-V2 and Depth Pro results included for reference.

MethodNYUv2KITTIETH3DDIODE
DA-V216.40 / 80.9010.60 / 88.6036.10 / 36.3041.20 / 22.10
Depth Pro10.70 / 91.9023.50 / 38.3038.50 / 32.8031.90 / 37.70
GeoUG-Depth (ours)18.47 / 71.759.89 / 91.5420.58 / 61.7625.57 / 55.18

On KITTI, ETH3D, and DIODE—all of which include outdoor scenes—our results have lower AbsRel and higher δ1 than the reported DA-V2 and Depth Pro results.

Q5: Inference efficiency

Thank you for this suggestion. Adaptive reasoning reduces average inference latency by 15.2%, including drafting and decision overhead, compared with forced depth generation using the same checkpoint.

We measured both modes on 32 randomly sampled 3DCueBench questions using one A100-80GB GPU at batch size 1, with identical inference settings. After two warmup requests, each question was evaluated three times per mode. Mean latency decreased from 63.28 s to 53.68 s, while both modes answered 89/96 requests correctly under the original option order.

The timing breakdown explains this reduction: depth generation and decoding account for 66.3% of the forced-generation runtime. Adaptive reasoning avoids this computation on 18.75% of requests, yielding a measurable net saving after accounting for the reasoning overhead. Thus, the benefit extends beyond fewer generation calls to lower actual inference latency.

We acknowledge that geometric generation remains expensive in our current implementation. Our claim is that adaptive reasoning reduces this cost within GeoUG.

Q6: Generalization to VSI-Bench

Thank you for suggesting VSI-Bench. We have added a comprehensive evaluation to test whether the task-dependent benefits of geometric co-training extend to video spatial reasoning. We evaluate four models on VSI-Bench with 16 sampled frames. The table below reports results:

ModelAvg.Object countingObject sizeAbsolute distanceRelative directionRoute planning
Original BAGEL31.9923.8833.7524.6545.7331.96
Understanding-only42.8640.8367.8229.5246.2129.9
+ Multi-view generation43.9940.8570.2129.5147.41 (+1.20)31.96
+ Depth generation42.6140.9667.0031.02 (+1.50)46.2327.84

The results support our central finding that transfer depends on the geometric task. Relative to understanding-only training, depth generation improves absolute-distance estimation by 1.50 points, while multi-view generation improves relative-direction reasoning and route planning by 1.20 and 2.06 points, respectively, alongside a 2.39-point improvement in object-size estimation. These results extend our original analysis to video spatial understanding: geometric generation during training can benefit corresponding spatial skills.

Thank you for your thoughtful feedback. If you have a chance to revisit our response, we’d be grateful if you could consider the new results in your overall assessment. Thanks again!

Comments on the paper

1 comment

Dear Reviewer dxW1,

As the rebuttal discussion period is drawing to a close, we kindly encourage you to review the authors' responses to your previous concerns and provide any further feedback or clarification as appropriate.

Thank you for your time and cooperation.

Best regards,

Your Area Chairs

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.