acceptodds
Under review as a conference paper at ICLR 2027

DepthThinker: Towards Real-World Spatial Perception with Metric Depth Reasoning

11 views

Abstract

Accurate depth perception is essential for vision-language models (VLMs) in navigation, embodied interaction, and autonomous driving, where decisions depend on precise distances and spatial constraints. However, existing depth training methods largely rely on regression losses or token-level cross-entropy, providing limited supervision for how depth estimates should be used in spatial reasoning. As a result, fluent explanations may still be inconsistent with scene geometry and physical constraints, limiting their reliability in unfamiliar environments. We introduce DepthThinker, a VLM designed to support test-time scaling in spatial perception through extended, depth-aware reasoning. Following joint supervised fine-tuning on depth and reasoning tasks, we introduce D-GRPO, which uses verifiable task rewards and feedback on reasoning quality to improve the accuracy and consistency of spatial judgments. To evaluate how depth perception supports practical spatial understanding and decision-making, we introduce WorldDepthBench, a benchmark of 3,360 question-answers covering depth perception, geometric reasoning, and action decisions, alongside a general spatial understanding category. With only 2B parameters, DepthThinker achieves a mean of 0.915 across seven datasets, surpassing reported results from depth-specialized and proprietary VLMs, and reaches 0.927 in a four-dataset comparison with specialist depth estimators. On WorldDepthBench, it achieves 45.77% accuracy, improving over joint supervised fine-tuning by 6.85 percentage points and outperforming Qwen3-VL-8B-Thinking by 3.30 points. These results show that combining depth supervision with reasoning and verifiable rewards improves spatial understanding and the use of depth information in practical decisions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language Models, Real-World Spatial Perception, Metric Depth Grounding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_62C4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies how metric depth supervision can improve spatial reasoning in VLMs. The authors combine joint depth-and-reasoning SFT with D-GRPO, which weights task rewards using a reasoning-quality score. They also introduce WorldDepthBench to test depth-dependent judgments under physical constraints. The contribution lies both in the training recipe and benchmark.

Critical strengths

The motivation is clear, and WorldDepthBench covers useful physical constraints. RL improves accuracy by 6.85 points over SFT while largely preserving average depth accuracy.

Critical weaknesses
  1. Comparisons with related spatial reasoning RL methods are missing. The paper should include methods that use geometry-aware rewards or model-based feedback to improve spatial reasoning. These comparisons would help clarify the benefit of D-GRPO.

  2. The quality-gate ablation is incomplete. The current experiments do not isolate its contribution. A comparison with task-reward-only GRPO, is needed.

  3. The method is validated on only one 2B backbone. Its effectiveness across model families and scales remains unclear. Testing another backbone or a larger model would help establish whether the gains extend beyond the current setting.

Questions for authors
  1. Why use the external VLM to score responses instead of distilling its reasoning traces?

  2. What alternative reward designs did you consider?

7 comments
AuthorsOpenReview ↗

Official Response to Reviewer 62C4 (Part 1/4)

We thank the reviewer for the clear summary and for acknowledging the motivation of WorldDepthBench and the RL gains. Our responses follow.

W1/Q2. About Related Spatial-Reasoning RL Methods and Alternative Rewards

Comparisons with related spatial reasoning RL methods are missing. The paper should include methods that use geometry-aware rewards or model-based feedback to improve spatial reasoning. These comparisons would help clarify the benefit of D-GRPO.

What alternative reward designs did you consider?

Thank you for the suggestion. We have added comparisons with three alternative reward designs covering numerical depth supervision, smooth numerical rewards, and model-based reasoning feedback:

  1. DepthLM-style GRPO. Following one of the numerical reward formulations evaluated in DepthLM [1], we compute negative absolute relative error for numerical tasks:

    =-\frac{1}{N}\sum_{i=1}^{N}\frac{|\hat d_i-d_i|}{d_i}.$$ To make this reward compatible with multiplicative quality gating, we apply a fixed bounded transformation: $$\widetilde R_{\mathrm{AbsRel}} =\frac{1}{1-R_{\mathrm{AbsRel}}} =\frac{1}{1+\frac{1}{N}\sum_{i=1}^{N}\frac{|\hat d_i-d_i|}{d_i}}.$$ The transformed reward lies in $(0,1]$, equals 1 for an exact prediction, and decreases smoothly as the prediction error increases. We then compute the final numerical-task reward as $$R_{\mathrm{final}}^{\mathrm{AbsRel}} =G\cdot\widetilde R_{\mathrm{AbsRel}}.$$ This bounded transformation is our task-specific adaptation of the negative AbsRel reward. It preserves the ordering of numerical prediction accuracy while ensuring that, for a fixed prediction error, higher reasoning quality yields a higher final reward. We have also retained the ranking rewards for multi-point missions, as well as MCQ rewards.
  2. GRPO + SNRA. We adopt the Smooth Numerical Reward Activation (SNRA) operator from Smooth Operator [2] within our standard GRPO implementation. SNRA maps numerical prediction errors to dense rewards through a parameterized sigmoid:

    =\frac{1}{N}\sum_{i=1}^{N}\frac{2}{1+\exp(k(t)e_i)}, \qquad e_i=\left(\frac{\hat d_i-d_i}{d_i+10^{-8}}\right)^2.$$ We use a predefined linear schedule, $$k(t)=2+18\frac{t-1}{159}, \qquad t\in\{1,\ldots,160\},$$ where $t$ is the actual training update index. This schedule increases the sigmoid sharpness from 2 at update 1 to 20 at update 160. Since the SNRA reward already lies in $[0,1]$, no additional range transformation is required. We apply the same reasoning-quality gate $G$ to obtain the gated numerical reward: $$R_{\mathrm{gated}}^{\mathrm{SNRA}}(t) =G\cdot R_{\mathrm{SNRA}}(t).$$ The original ranking rewards for multi-point tasks and the MCQ rewards remain unchanged.
  3. Embodied-R-style reward adaptation. Inspired by the answer re-derivation feedback in Embodied-R [3], we use a frozen reference model to assess consistency between the reasoning and final answer. We retain our original metric, ranking, and MCQ task rewards and replace the quality gate with an additive reward:

    =R_{\mathrm{task}}+0.1I_{\mathrm{valid}}+0.25C,$$ where $I_{\mathrm{valid}}$ indicates that the response passes completion, format, and numerical/schema checks. Reference feedback is applied only to valid responses with non-empty reasoning and a correct original answer. A frozen copy of the initial SFT model re-derives an answer from the public question, reasoning text, and required answer format. It receives no image, hidden ground truth, or separate original final-answer field; values or conclusions already present in the reasoning text remain unchanged. The reference runs at temperature 0 with a 1,024-token limit, and its answer is checked locally. We set $C=1$ if the reference produces a correct MCQ answer or if every numerical prediction satisfies $$\max\!\left(\frac{\hat d_i}{d_i},\frac{d_i}{\hat d_i}\right)<1.25;$$ otherwise, $C=0$. The same correctness criterion determines whether the original response is eligible for reference feedback. Thus, numerical consistency requires both the original and re-derived answers to satisfy the same tolerance, rather than exact equality between their predicted values.

Results are reported below, with WorldDepthBench accuracy in percentages and mean δ1 averaged across the seven depth-evaluation datasets.

MethodWDB overallPerc.Geo.Act.Gen.Mean δ1
SFT38.9351.9835.4225.9445.830.914
D-GRPO (full)45.7757.1941.7734.0654.370.915
DepthLM-style GRPO37.8049.0635.8324.7945.210.904
GRPO + SNRA40.0954.2738.4423.6547.920.905
Embodied-R reward adaptation40.5152.9238.5426.5647.500.904
AuthorsOpenReview ↗

Official Response to Reviewer 62C4 (Part 2/4)

D-GRPO achieves the highest accuracy across all four WorldDepthBench categories and the highest mean δ1. Its overall accuracy exceeds DepthLM-style GRPO, GRPO + SNRA, and the Embodied-R reward adaptation by 7.97, 5.68, and 5.26 percentage points, respectively.

The differences are particularly pronounced in action decisions, where D-GRPO reaches 34.06%, compared with 23.65–26.56% for the alternative reward designs. D-GRPO also preserves the SFT checkpoint’s metric perception, with mean δ1 increasing from 0.914 to 0.915, whereas the alternatives reach 0.904–0.905. Thus, its reasoning gains are accompanied by maintained depth accuracy.

These results support the combined design of D-GRPO: verifiable rewards supervise metric accuracy, depth ordering, and decisions under physical constraints, while the thinking-quality gate assesses how the response connects visual evidence and depth estimates to its conclusion. The component ablations in our response to W2 further isolate the contributions of these rewards and the gate.

We will include these comparisons, the shared experimental settings, and the details of each reward adaptation in the revision.

References

[1] Zhipeng Cai et al. DepthLM: Metric Depth from Vision Language Models. arXiv:2509.25413, 2025.

[2] Siwen Jiao et al. Smooth Operator: Smooth Verifiable Reward Activates Spatial Reasoning Ability of Vision-Language Model. EMNLP 2026; arXiv:2601.07695.

[3] Baining Zhao et al. Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning. arXiv:2504.12680, 2025.

W2. About the Thinking-Quality Gate Ablation

The quality-gate ablation is incomplete. The current experiments do not isolate its contribution. A comparison with task-reward-only GRPO, is needed.

Thank you for the suggestion. We have completed a matched comparison between full D-GRPO and task-reward-only GRPO, together with ablations of the individual reward components. All RL variants start from the same SFT checkpoint and use the same training data, sampling settings, and training budget. In the task-reward-only variant, we remove the thinking-quality gate while retaining all verifiable task rewards, directly isolating the gate’s contribution.

The results are summarized below. WorldDepthBench scores are accuracy (%), and mean δ1 is averaged across the seven depth-evaluation datasets.

VariantWDB overallPerc.Geo.Act.Gen.Mean δ1
SFT38.9351.9835.4225.9445.830.914
D-GRPO (full)45.7757.1941.7734.0654.370.915
w/o thinking-quality gate (task-reward-only GRPO)39.4950.4236.5627.1948.130.906
w/o ranking reward42.0853.1341.4626.6752.080.909
w/o metric reward37.2951.0433.8520.2150.830.901
w/o multiple-choice reasoning reward39.6453.2337.1921.8852.920.903

Full D-GRPO exceeds task-reward-only GRPO by 6.28 percentage points in overall accuracy, with improvements across all four categories. It also maintains the SFT checkpoint’s metric-depth accuracy (mean δ1: 0.914 → 0.915), whereas removing the gate lowers mean δ1 to 0.906. This matched comparison supports the gate’s contribution to spatial reasoning performance while preserving metric perception.

The remaining ablations show that each reward component also contributes to the full model’s performance. Removing the metric or ranking reward lowers both overall accuracy and mean δ1. Removing the multiple-choice reasoning reward reduces action-decision accuracy from 34.06% to 21.88%, supporting the importance of explicitly rewarding decisions under physical constraints.

We will include these completed ablations and their shared experimental settings in the revision.

AuthorsOpenReview ↗

Official Response to Reviewer 62C4 (Part 3/4)

W3. About the Effect of Model Scale

The method is validated on only one 2B backbone. Its effectiveness across model families and scales remains unclear. Testing another backbone or a larger model would help establish whether the gains extend beyond the current setting.

Thank you for the suggestion. We have extended our framework to Qwen3-VL-8B-Thinking to test whether its benefits extend to a larger backbone. We examine metric-depth perception after SFT and spatial reasoning after D-GRPO separately.

(a) Metric-depth perception after SFT.

Using the same depth-supervision data, increasing the backbone from 2B to 8B improves mean δ1 from 91.4% to 91.7% across the seven evaluation datasets:

BackboneMean δ1 after depth-supervised SFT (%)
Qwen3-VL-2B-Thinking91.4
Qwen3-VL-8B-Thinking91.7

This modest 0.3-percentage-point gain suggests that the 2B backbone already provides sufficient capacity for accurate metric-depth estimation under our training setup.

(b) Spatial reasoning after D-GRPO.

Following D-GRPO, DepthThinker-8B-RL outperforms both DepthThinker-2B-RL and DepthThinker-8B-SFT across all four WorldDepthBench categories:

ModelWDB overallPerc.Geo.Act.Gen.
DepthThinker-2B-RL45.7757.1941.7734.0654.37
DepthThinker-8B-SFT46.9352.8144.2735.9462.50
DepthThinker-8B-RL49.7960.2145.1037.0863.75

Compared with DepthThinker-2B-RL, the 8B model improves overall accuracy by 4.02 percentage points, including gains of 3.33 percentage points in geometric reasoning and 3.02 percentage points in action decisions. D-GRPO also raises the 8B model’s overall accuracy from 46.93% to 49.79%, a gain of 2.86 percentage points over its SFT counterpart, with improvements across all four categories. Relative to the original Qwen3-VL-8B-Thinking baseline (42.47%), DepthThinker-8B-RL achieves an overall gain of 7.32 percentage points, further supporting the effectiveness of our framework at the larger scale.

These results suggest that additional model capacity helps translate depth estimates into stronger spatial reasoning and action decisions under physical constraints. They support the applicability of our framework at both the 2B and 8B scales within the Qwen3-VL family, while evaluation across other model families remains a direction for future work. We will include these 8B results and the corresponding analysis in the revision.

Reviewer_dRAfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether vision-language models can use metric depth to reason about spatial relationships and physical constraints. The proposed model, DepthThinker, starts from Qwen3-VL-2B-Thinking and combines depth supervision with reasoning demonstrations during supervised fine-tuning. This is followed by D-GRPO, which rewards accurate depth predictions, correct ordering, and correct spatial answers, while using a separate VLM to assess reasoning quality. The paper also introduces WorldDepthBench, containing 3,360 questions covering depth perception, geometric reasoning, action decisions, and general spatial understanding.

Critical strengths
  1. I find the connection between depth perception and physical constraints well motivated. Predicting a distance and using that distance correctly are different requirements, and the benchmark makes this distinction concrete through reachability, clearance, and stopping-distance questions. The authors also appropriately distinguish these static-image decisions from actual closed-loop execution.
  2. The comparison between the SFT and RL checkpoints provides meaningful evidence that the complete training procedure helps. Accuracy improves from 38.93% to 45.77%, with gains across all four categories and eleven of twelve core subtypes, while metric depth accuracy remains largely unchanged. This is a useful result even though the model does not outperform larger baselines in every category.
Critical weaknesses
  1. The contribution of the thinking-quality gate remains unclear. The SFT-to-RL comparison shows that the complete procedure helps, but there is no matched task-reward-only GRPO baseline. Moreover, the evaluator lacks reference geometry, so correct answers with inaccurate intermediate distances may still receive positive rewards. I would like to see evidence that the gate improves geometric correctness rather than simply encouraging plausible explanations.
  2. The depth superiority claims need a more consistent comparison. Tables 1 and 2 use previously reported baseline results while retaining their original evaluation protocols. Although the authors acknowledge some differences, it remains unclear whether the images, queries, splits, and preprocessing are equivalent across methods. Evaluating the closest baselines under a common protocol, or qualifying the claims accordingly, would make the results more convincing.
Questions for authors
  1. How does full D-GRPO compare with task-reward-only GRPO when both start from the same SFT checkpoint and use the same training budget? Is there evidence that the gate reduces repetitive or geometrically incorrect reasoning?
  2. Have intermediate depth estimates and constraint calculations been checked against reference geometry on a sample of responses? Such an analysis would help distinguish improved answer accuracy from improved geometric reasoning.
  3. Which depth baselines share the exact evaluation images, coordinate queries, splits, and preprocessing? Can the closest methods be compared under a common protocol?
5 comments
AuthorsOpenReview ↗

Official Response to Reviewer dRAf (Part 1/3)

We thank the reviewer for the careful reading and constructive questions. We address each point below; new results and clarifications will be added to the revised PDF.

W1 / Q1. About the Thinking-Quality Gate and Reference Geometry

The contribution of the thinking-quality gate remains unclear. The SFT-to-RL comparison shows that the complete procedure helps, but there is no matched task-reward-only GRPO baseline. Moreover, the evaluator lacks reference geometry, so correct answers with inaccurate intermediate distances may still receive positive rewards. I would like to see evidence that the gate improves geometric correctness rather than simply encouraging plausible explanations.

Thank you for the comment. We clarify why the gate was introduced, present matched comparisons, and distinguish its role from verification against reference geometry.

(a) Motivation: preventing the model from bypassing reasoning.

We introduced the thinking-quality gate after observing a shortcut during RL: the model could obtain increasing task rewards while abandoning the required reasoning. In paired development runs, task-reward-only RL progressively shortened responses, eventually producing empty thinking blocks followed by direct answers by approximately 30 updates. This occurred despite an SFT initialization that included reasoning demonstrations.

The gate therefore provides feedback on the student’s current reasoning, assessing whether it meaningfully connects visual evidence and depth estimates to the stated constraints and its conclusion. For tasks requiring reasoning, an empty thinking block receives zero combined reward. The gate assesses reasoning content without explicitly rewarding length, allowing concise, supported reasoning to receive full credit.

(b) Matched gate and reward ablations.

We have completed a matched comparison between full D-GRPO and task-reward-only GRPO, together with ablations of the individual reward components. All RL variants use the same SFT initialization, training data, sampling settings, and training budget. The task-reward-only variant retains all verifiable task rewards and removes only the gate.

We additionally test a geometry-assisted variant that supplies task-relevant ground-truth geometry to the same VLM judge. WorldDepthBench results are accuracy (%).

VariantWDB overallPerc.Geo.Act.Gen.Mean δ1
SFT38.9351.9835.4225.9445.830.914
D-GRPO (full)45.7757.1941.7734.0654.370.915
D-GRPO with GT geometry supplied to the judge45.3056.6741.9833.0253.750.914
w/o thinking-quality gate (task-reward-only GRPO)39.4950.4236.5627.1948.130.906
w/o ranking reward42.0853.1341.4626.6752.080.909
w/o metric reward37.2951.0433.8520.2150.830.901
w/o multiple-choice reasoning reward39.6453.2337.1921.8852.920.903

Removing the gate reduces overall accuracy by 6.28 percentage points, from 45.77% to 39.49%, with decreases across all four categories. Mean depth δ1 also falls from 0.915 to 0.906. This isolates the gate’s contribution to downstream performance and preservation of metric perception.

(c) Reference geometry for the judge.

Our SFT initialization provides a strong metric-depth foundation, achieving a mean δ1 of 0.914 on standalone depth estimation. We introduced the thinking quality gate to address a different problem observed during task-only RL: the model could bypass reasoning despite this perceptual foundation. The gate therefore encourages the model to connect visual evidence and depth estimates to the task constraints and its conclusion.

To test whether explicit geometry further helps, we additionally supplied task-relevant ground-truth geometry to the original VLM judge, keeping the initialization, training data, sampling settings, and training budget unchanged. This variant slightly improves Geo. accuracy from 41.77% to 41.98%, but overall accuracy changes from 45.77% to 45.30%. Thus, adding reference geometry to the judge provides no additional overall benefit under the tested setting.

Our independent response audit also shows that intermediate depth estimates improve during D-GRPO even without reference geometry being supplied to the judge: trace-level δ1 increases from 0.4341 to 0.4876, while the median absolute log-ratio decreases from 0.5483 to 0.4341 (see Q2). These results support retaining the current gate and using reference geometry to independently assess intermediate reasoning. The audit measures the effect of the full training procedure; the matched gate ablation above provides the evidence for the gate's individual contribution.

AuthorsOpenReview ↗

Official Response to Reviewer dRAf (Part 2/3)

Q2. About Intermediate Geometry

Have intermediate depth estimates and constraint calculations been checked against reference geometry on a sample of responses? Such an analysis would help distinguish improved answer accuracy from improved geometric reasoning.

Thank you for suggesting this analysis. We have now conducted an additional audit of intermediate depth estimates and constraint calculations against reference geometry. This analysis was not included in the original submission.

(a) Audit protocol and coverage.

We evaluate three models—SFT, task-reward-only GRPO, and D-GRPO—on the same 1,605 depth-grounded questions, comprising 3,690 reference points. To assess the contribution of the thinking quality gate, we include task-reward-only GRPO as a matched baseline. Both RL variants use the same SFT initialization, training data, sampling settings, and training budget; the task-reward-only baseline omits the thinking quality gate.

For depth estimation, we extract every camera-to-point distance stated in the reasoning block that can be matched to a reference point, and compare it with the corresponding reference distance. Within each model, δ1 and the median absolute log-ratio are computed over the same model-specific set of matched distance mentions, with equal weight per mention. The evaluated questions and reference points are shared across models, but the extracted mentions may differ in number and content. Missing estimates are excluded from these error metrics, and reference-point coverage is reported separately.

The SFT, task-reward-only GRPO, and D-GRPO responses contain 20,171, 18,476, and 18,107 matched distance mentions, respectively. These mentions cover 2,549, 2,561, and 2,566 distinct reference points out of the shared set of 3,690, corresponding to coverage rates of 69.1%, 69.4%, and 69.5%, respectively.

For constraint calculations, we independently reapply each decision rule using the numerical values stated by the model itself, and check whether those values support its selected option. Examples include comparing a stated target distance with a reach radius, or a stated stopping distance with the available free space. Before auditing model responses, we validate the rule implementation against reference geometry: it reproduces the released answers for all checked braking and metric-range questions (240/240 and 162/162, respectively). The rules are fully recoverable from 802 SFT responses, 807 task-only GRPO responses and 808 D-GRPO responses, which form the respective decision-audit subsets.

(b) Intermediate depth accuracy and geometric support improve under D-GRPO.

The results are summarized below.

MetricSFTTask-Reward-Only GRPOD-GRPO
Answer accuracy on the 1,605 audited questions43.93%45.67%49.72%
δ1 of distances stated in reasoning, all matched mentions43.41%41.92%48.76%
Median absolute log-ratio, all matched mentions ↓0.54830.52410.4341
Share of correct, rule-auditable answers supported by the stated geometry64.6%68.3%75.1%

The depth-error metrics use all matched distance mentions, with equal weight per mention. The geometry-support rate is computed only among correct answers for which the decision rules are fully recoverable.

Compared with SFT, D-GRPO improves answer accuracy by 5.79 percentage points and intermediate-depth δ1 by 5.35 percentage points, while reducing the median absolute log-ratio from 0.5483 to 0.4341. These results show that the improvement in final-answer accuracy is accompanied by measurable improvements in the intermediate depth estimates expressed in reasoning traces.

The matched task-reward-only baseline further clarifies the contribution of the thinking quality gate. Task-reward-only GRPO improves answer accuracy over SFT, but its intermediate-depth δ1 decreases from 43.41% to 41.92%, despite a modest reduction in median error. D-GRPO improves over this baseline by 4.05 percentage points in answer accuracy and 6.84 percentage points in intermediate-depth δ1, and achieves a lower median absolute log-ratio (0.4341 vs. 0.5241).

Among correct, rule-auditable answers, the share supported by the model’s own stated geometry also rises to 75.1%, compared with 64.6% for SFT and 68.3% for task-reward-only GRPO. Together, these findings provide evidence that the thinking quality gate contributes to both more accurate intermediate depth estimates and stronger geometric support for correct decisions.

AuthorsOpenReview ↗

Official Response to Reviewer dRAf (Part 3/3)

W2 / Q3. About the Consistency of Depth Evaluation

The depth superiority claims need a more consistent comparison. Tables 1 and 2 use previously reported baseline results while retaining their original evaluation protocols. Although the authors acknowledge some differences, it remains unclear whether the images, queries, splits, and preprocessing are equivalent across methods.

Which depth baselines share the exact evaluation images, coordinate queries, splits, and preprocessing? Can the closest methods be compared under a common protocol?

Thank you for highlighting this issue. We clarify the depth convention, provide an ablation of its effect, and specify the scope of the protocol alignment below. We will also qualify the depth-comparison claims to reflect the use of previously reported baseline results.

(a) Rationale for ray-depth.

We use ray-depth, following DepthLM and VLM³, to maintain a consistent depth convention with these closely related baselines. Ray-depth measures the Euclidean distance from the camera center to a queried 3D point, whereas Z-depth measures the point’s coordinate along the optical axis.

Ray-depth also matches the physical quantities required by several of our downstream tasks. For example, determining whether a target lies within a camera-centered spherical workspace requires its Euclidean distance from the camera. We therefore use ray-depth consistently throughout training and evaluation.

(b) Ablation of the depth convention.

To examine sensitivity to the depth definition, we train SFT models with ray-depth and Z-depth supervision and evaluate each using the corresponding reference-depth convention. Both achieve similar metric-depth accuracy across the seven evaluation datasets:

Depth definitionDDADNuSc.ETH3DSUNiBimsNYUScanNet++Average δ1
Z-depth0.8180.9040.9150.9210.9530.9470.9170.911
Ray-depth (ours)0.8210.9000.9320.9120.9620.9440.9280.914

The similar mean δ1 values (0.911 vs. 0.914) indicate that our method can learn either representation effectively when supervision and evaluation use a consistent definition. We retain ray-depth for its alignment with DepthLM/VLM³ and its relevance to our downstream tasks.

For DepthVLM, we retain its published Z-depth scores as reference results and explicitly identify this convention in the comparison tables. The ablation above addresses our model’s sensitivity to the depth definition.

(c) Shared settings and independently constructed evaluation sets.

Our SFT and D-GRPO checkpoints are evaluated on exactly the same images, coordinate queries, splits, and preprocessing. Their comparison therefore measures the effect of RL under a common evaluation protocol.

For comparisons with external baselines, we follow the published DepthLM/VLM³ evaluation conventions, including focal normalization and 8,192 point queries per dataset, and construct our evaluation sets according to their published split procedures.

For the remaining baselines in Tables 1 and 2, we follow the same reporting practice as DepthLM and VLM³, using previously reported results while retaining their original evaluation protocols. We will explicitly state this reporting convention, identify the source of each baseline result, and annotate known protocol differences in the revised tables.

Accordingly, we will describe Tables 1 and 2 as comparisons against reported baseline results under their respective protocols, annotate known differences such as DepthVLM’s Z-depth convention, and qualify the superiority claims. We will also add a protocol summary covering depth definitions, split construction, query sampling, and preprocessing so that readers can assess the scope of each comparison.

Reviewer_GL2UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces DepthThinker, a model post-trained from Qwen3-VL-2B, which is capable of accurately predicting metric depth and performing spatial reasoning based on that depth information. The core approach integrates perceptual foundation with verifiable rewards. The authors propose a two-stage training framework: first, establishing a robust foundation for metric perception and cold-start the reasoning process through joint SFT that incorporates depth supervision; and second, employing the D-GRPO, which utilizes RL augmented by geometric rewards and a thinking quality gate. Furthermore, the authors introduce WorldDepthBench, a benchmark designed to evaluate a model's proficiency in transitioning from depth perception to actionable decision-making. Experimental results demonstrate that DepthThinker outperforms specialized models in metric depth estimation across seven evaluation datasets and surpasses 8B-parameter models in spatial reasoning tasks.

Critical strengths
  1. The paper enhances spatial reasoning and decision capabilities by improving a model’s proficiency in metric depth estimation. It establishes a foundational perception capability through joint SFT, followed by the use of RL to bolster spatial reasoning and decision-making. The resulting model, DepthThinker, demonstrates exceptional performance in both metric depth estimation and spatial reasoning.

  2. The D-GRPO framework introduces a hybrid reward strategy comprising a spatial decision reward and a thinking quality gate. The spatial decision reward incorporates both a metric accuracy reward and a ranking reward. The thinking quality gate is applied multiplicatively to the spatial decision reward, ensuring that positive feedback is granted only when the answer is correct and the underlying reasoning is coherent.

  3. The paper introduces WorldDepthBench, which addresses a gap in existing benchmarks by accounting for decision under spatial constraints. With its rigorous taxonomic design and meticulous data processing, this benchmark provides a valuable new resource for the analysis of spatial reasoning models.

Critical weaknesses
  1. The two-stage post-training of DepthThinker, i.e., first establishing perception foundation and then enhancing reasoning capabilities, aligns closely with existing post-training paradigms for VLMs, limiting its overall novelty.

  2. The analysis of the role of rewards within D-GRPO is incomplete; it lacks an ablation study to validate the individual contributions of each spatial reward and the thinking quality gate.

  3. WorldDepthBench reuses 11 existing open-source benchmarks (Appendix B.1), offering limited original contribution in terms of evaluation data construction. Furthermore, it lacks evaluations based on video or multi-view image inputs, which are critical for spatial perception and embodied AI scenarios.

Questions for authors
  1. Why are the 2D coordinates in the images normalized to the range [0, 2000]?

  2. The size of DepthThinker is only 2B, yet it utilizes an exceptionally large number of training samples (i.e., 102.4M) during the joint SFT. Could the authors clarify why such a massive dataset is required, and whether the model's performance correlates with the number of samples used? Additionally, how many samples were employed during the RL phase?

  3. The significant performance gains observed in DepthThinker over the 2B base model may be attributed to the limited capacity of the smaller base model. How would the performance compare if applied to a larger model (e.g., 7B)? I am not requesting additional fine-tuning experiments, but rather an analysis that could help clarify the underlying issues and potential outcomes.

  4. There is a lack of implementation details for DepthThinker, for example training time.

8 comments
AuthorsOpenReview ↗

Official Response to Reviewer GL2U (Part 1/5)

We thank the reviewer for the careful reading and constructive questions. We address each point below; new results and clarifications will be added to the revised PDF.

W1. About the Novelty of DepthThinker

The two-stage post-training of DepthThinker, i.e., first establishing perception foundation and then enhancing reasoning capabilities, aligns closely with existing post-training paradigms for VLMs, limiting its overall novelty.

Thank you for the comment. We agree that SFT followed by RL is an established training paradigm. Our contribution lies in developing reasoning grounded in accurate metric-depth perception, through the following designs:

  1. A metric-depth foundation for subsequent reasoning. Large-scale depth supervision establishes the perceptual capability needed to reason about distances and physical constraints. Under the same D-GRPO setup, the depth-supervised initialization achieves 43.84% on WorldDepthBench at step 100, compared with 33.90% for the control without large-scale depth initialization (Fig. 4). This 9.94-percentage-point gap supports the importance of the perceptual foundation for subsequent reasoning.

  2. Depth-specific rewards and reasoning-quality feedback. D-GRPO combines verifiable rewards for metric accuracy, depth ordering, and decisions under physical constraints. The thinking-quality gate additionally assesses whether the reasoning supports the answer. This addresses an observed failure mode of task-reward-only GRPO: the model can obtain increasing rewards while bypassing the required reasoning, as shown in our response to W2 below. Direct depth-estimation prompts also constitute 40% of each RL batch to maintain explicit supervision on metric perception.

  3. Targeted evaluation with WorldDepthBench. WorldDepthBench evaluates both metric perception and its application to spatial reasoning and decisions. Its constraint-based questions require models to use distances to determine reachability, assess clearance, and select feasible actions, making the consequences of depth errors visible in the final answers. We further clarify the benchmark’s construction and originality in our response to W3.

We will revise the contribution paragraph to make these depth-specific contributions explicit.

Q1. About the Coordinate Normalization

Why are the 2D coordinates in the images normalized to the range [0, 2000]?

Thank you for the question. We follow the coordinate convention of our baseline, VLM³: Vision Language Models Are Native 3D Learners (Cai et al., 2026), normalizing the horizontal and vertical image axes independently to [0, 2000]. This keeps our query interface consistent with the baseline and provides a common coordinate representation across different image resolutions. We apply the same convention throughout training and evaluation.

This range also provides finer localization precision. The sensor-derived images in WorldDepthBench have longer sides of 640–1,920 pixels. With nearest-integer rounding, normalization to [0, 2000] introduces at most 0.48 pixels of quantization error per axis for these images, compared with 0.96 pixels for [0, 1000]. Halving this error is particularly useful for query points near object boundaries, where a small coordinate shift can cross a depth discontinuity and identify a surface at a substantially different depth.

We will clarify this rationale and the per-axis normalization procedure in the revision.

References

[1] Zhipeng Cai et al. VLM³: Vision Language Models Are Native 3D Learners. arXiv:2605.30561, 2026

AuthorsOpenReview ↗

Official Response to Reviewer GL2U (Part 2/5)

W2. About the Reward and Thinking-Quality Gate Ablations

The analysis of the role of rewards within D-GRPO is incomplete; it lacks an ablation study to validate the individual contributions of each spatial reward and the thinking quality gate.

Thank you for the suggestion. We have completed ablations of the thinking-quality gate and each reward component. Below, we present these results and a diagnostic analysis of reasoning behavior with and without the gate.

(a) Component ablations.

All ablation variants start from the same SFT checkpoint and use the same training data, sampling settings, and training budget as full D-GRPO, allowing us to assess each component under matched conditions.

Full D-GRPO improves WorldDepthBench overall accuracy from 38.93% to 45.77%, while preserving metric-depth estimation accuracy (mean δ1: 0.914 → 0.915). The results are summarized below. WorldDepthBench scores are accuracy (%), and mean δ1 is averaged across the seven depth-evaluation datasets.

VariantWDB overallPerc.Geo.Act.Gen.Mean δ1
SFT38.9351.9835.4225.9445.830.914
D-GRPO (full)45.7757.1941.7734.0654.370.915
w/o thinking-quality gate (task-reward-only GRPO)39.4950.4236.5627.1948.130.906
w/o ranking reward42.0853.1341.4626.6752.080.909
w/o metric reward37.2951.0433.8520.2150.830.901
w/o multiple-choice reasoning reward39.6453.2337.1921.8852.920.903

Full D-GRPO achieves the highest overall accuracy and mean δ1 among these variants. Removing the gate reduces overall accuracy by 6.28 percentage points, from 45.77% to 39.49%, and lowers mean δ1 from 0.915 to 0.906. Removing the metric or ranking reward also reduces both measures, supporting the contribution of absolute-distance and relative-order supervision. Without the multiple-choice reasoning reward, action-decision accuracy drops from 34.06% to 21.88%, highlighting the importance of directly rewarding the application of geometry to task constraints.

(b) Reasoning collapse without the gate.

The task rewards evaluate final predictions and decisions, but do not directly assess the reasoning behind them. In paired development runs with identical initialization, data, and optimization settings, task-reward-only RL learned to bypass the required reasoning: responses collapsed to an empty thinking block followed by a direct answer.

MetricTask reward onlyTask reward × gate (development version)
Mean response length, update 1942 tokensApproximately 880–940 tokens
Mean response length, update ~1058 tokensApproximately 840–850 tokens
Mean response length, later updatesApproximately 15 tokens from update ~30 through 1,182834–1,007 tokens through update 309
Training reward0.67 → 0.880.67 → 0.91

Despite the increasing training reward, the task-only run’s validation δ1 decreased from 0.91 to 0.88. The disappearance of the reasoning trace, together with declining validation accuracy, shows that optimizing task rewards alone can encourage a shortcut that undermines the intended depth-grounded reasoning behavior. The development gate prevented this collapse over its observed training window.

These diagnostic runs used a development version of the gate; the final VLM-based gate is evaluated in the matched ablation above. The final gate assesses whether the response coherently connects visual evidence and depth estimates to the stated constraints and its conclusion. For tasks that require reasoning, an empty thinking block receives zero combined reward. This directly removes the incentive to obtain positive reward through a correct answer with no reasoning, while the verifiable task rewards continue to supervise numerical and decision correctness.

We will include these completed ablations, their shared experimental settings, and the diagnostic analysis in the revision.

AuthorsOpenReview ↗

Official Response to Reviewer GL2U (Part 3/5)

W3. About WorldDepthBench

(a) About the Originality of WorldDepthBench

WorldDepthBench reuses 11 existing open-source benchmarks (Appendix B.1), offering limited original contribution in terms of evaluation data construction.

We reuse public images and depth/geometry annotations because they provide the metric references needed to derive verifiable answers, while reducing the cost of collecting and annotating new 3D data. Building on these annotations, 2,726 of the 3,360 questions are newly constructed by our pipeline, accounting for 81.1% of the full benchmark and 94.7% of the core tasks. The remaining 634 questions adapt annotations from existing spatial benchmarks.

Our contribution is to construct evaluation questions that connect metric-depth estimation with reasoning under explicit physical constraints. We design question templates, physical parameters, candidate actions, and distractors, and compute reference answers from the underlying geometry. These tasks extend evaluation from estimating distances or recognizing spatial relations to determining whether those distances satisfy task requirements. For example, correctly ordering targets by distance does not establish whether they lie within a specified reach radius. This design makes depth errors consequential for the final answer, allowing the benchmark to reveal differences in how effectively models use metric depth in reasoning and decision-making.

We also balance answer classes within shared physical-parameter settings and control route appearance to reduce shortcuts (Appendix B.3). We will clarify the construction process in the revision.

(b) About Video and Multi-view Evaluation

Furthermore, it lacks evaluations based on video or multi-view image inputs, which are critical for spatial perception and embodied AI scenarios.

We have added supplementary evaluations on multi-view and video inputs and clarify their sources, sample sizes, and evaluation protocols below.

Data and visual inputs. The evaluated multi-view subset contains 500 questions: 250 cross-view range comparisons constructed from ScanNet++ and 250 surround-view clearance questions constructed from nuScenes. Cross-view questions provide two RGB images from the same scene, with reference ranges computed by back-projecting queried points using source depth, intrinsics, and camera poses and measuring their distances from the specified anchor camera. Surround-view questions provide six keyframe camera images in a stated order; reference obstacle distances are defined as the minimum horizontal distance from the ego origin to annotated 3D box corners.

The video evaluation contains 180 questions from 68 ScanNet++ iPhone scenes, with 60 questions each on camera displacement, camera path length, and out-of-view target range. Each clip supplies 16 uniformly sampled, ordered frames at 640 × 480 resolution, presented as video input. Reference displacement is computed from endpoint camera centers, path length from the full source pose trajectory, and out-of-view range from the first-frame target’s reconstructed 3D position and the final camera center.

Evaluation protocol. Models receive RGB images or video, questions, physical assumptions, and answer options. Ground-truth depth, camera poses, calibration matrices, 3D annotations, and reference answers are used only for construction and scoring. Queried points use textual coordinates normalized independently to [0, 2000] along each image axis.

All three checkpoints use the same aspect-ratio-preserving image preprocessing, with per-image pixel caps of 1,000,000 for cross-view and 400,000 for surround-view inputs. Video inputs use the stored 640 × 480 frames and request all 16 frames. All checkpoints use their checkpoint chat templates, a common task system prompt, explicit thinking, temperature 1.0, top-p 0.95, top-k 20, seed 42, one response per question, and a 16,384-token completion budget. We report exact selected-option accuracy; unfinished, truncated, or unparseable responses count as incorrect. Cross-view questions have two options, while surround-view and video questions have four, corresponding to chance accuracies of 50%, 25%, and 25%, respectively.

These evaluations examine whether the benefits of our training extend to spatial reasoning with multiple visual inputs. The results are shown below (accuracy, %):

ModelCross-viewSurround-viewVideo
Qwen3-VL-2B-Thinking41.614.431.7
DepthThinker-2B (SFT)48.417.234.4
DepthThinker-2B (SFT + D-GRPO)51.620.835.6

Both SFT and subsequent D-GRPO improve accuracy across all three settings. The D-GRPO exceeds the base model by 10.0, 6.4, and 3.9 percentage points on cross-view, surround-view, and video inputs, respectively. These results show our training also benefits depth-grounded spatial reasoning with multiple visual inputs.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response by Authors

We would like to thank all reviewers for their careful evaluation and constructive suggestions. We have uploaded a revised version of our paper and highlighted all additions and revised passages in blue. The main revisions are summarized below:

  1. We clarify the depth-specific contributions of DepthThinker, the coordinate normalization, and the complementary roles of reasoning-trace distillation during SFT and VLM-based feedback during RL. (Sections 1 and 3; Appendices A.1, A.2, and A.4; Reviewers GL2U and 62C4)
  2. We add matched ablations of the thinking-quality gate and each task-reward component, together with a geometry-assisted judge variant and an analysis of reasoning collapse in development runs. Full D-GRPO achieves 45.77% overall accuracy, compared with 39.49% for task-reward-only GRPO. (Section 5.6.1, Table 4; Appendix C.5; Reviewers GL2U, dRAf, and 62C4)
  3. We add an independent audit of intermediate depth estimates and constraint calculations on 1,605 depth-grounded questions. We report extraction coverage and metric denominators, providing evidence of improved intermediate geometry and stronger support for correct decisions. (Appendix C.7, Tables 13–14; Reviewer dRAf)
  4. We add comparisons with DepthLM-style GRPO, GRPO with Smooth Numerical Reward Activation, and an Embodied-R-style reward adaptation, with details of their implementation under our task setting. (Appendix C.6, Table 12; Reviewer 62C4)
  5. We extend the framework to Qwen3-VL-8B-Thinking using the same RL budget, sampling settings, and evaluation protocol as 2B. DepthThinker-8B achieves 49.79% overall accuracy, improving over its SFT counterpart by 2.86 percentage points. (Section 5.6.2, Table 5; Reviewers GL2U and 62C4)
  6. We clarify the originality of WorldDepthBench: 2,726 questions are newly constructed, accounting for 81.1% of the full benchmark. We also add supplementary evaluations on 500 multi-view questions and 180 video questions, with detailed evaluation protocols; all supplementary evaluation scenes are excluded from SFT and RL training. (Section 4; Appendices B.1 and C.10; Reviewer GL2U)
  7. We add SFT data-scaling results and clarify the distinction between training records, point-depth labels, the RL prompt pool, and actual prompt draws. We also provide training time, hardware, and evaluator serving details. (Appendices A.1, A.4–A.6, and C.8; Reviewer GL2U)
  8. We clarify the scope of external depth comparisons and add a protocol summary and a ray-depth versus Z-depth ablation. We explicitly distinguish the shared evaluation inputs of our SFT/RL checkpoints from the respective published protocols of external baselines. (Appendices C.1 and C.9; Tables 10 and 16; Reviewer dRAf)

We hope that these revisions address the reviewers’ concerns and improve the clarity and completeness of the paper. We sincerely thank all reviewers for their time and valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.