acceptodds
Under review as a conference paper at ICLR 2027

DepthEvidence: Unifying Metric Depth Prediction and Geometric Reasoning in Multimodal Language Models

Abstract

Spatial reasoning with metric constraints requires linking objects to geometric measurements and preserving their numerical content during language reasoning. We present DepthEvidence, a 4B model that uses its own dense metric predictions as object-grounded evidence for language generation. A camera-conditioned decoder predicts full-resolution metric depth using multi-scale visual features and high-resolution RGB refinement. A dense-to-language interface converts predicted depths and decoder features into object-aligned continuous geometry tokens anchored to object identifiers. Geometric supervision encourages metric information to remain recoverable before and after language-context interaction, while instruction tuning supports object measurement and compositional reasoning. We introduce a Depth-VQA benchmark evaluating object-depth queries, relative comparisons, and decisions combining spatial and numerical constraints. Across nine datasets, DepthEvidence achieves the highest average dense among evaluated methods, competitive with specialized estimators. It also leads the evaluated methods in instance-level metric depth estimation and overall accuracy on both relative and metric reasoning tracks, while broadly preserving general VQA performance and improving spatial understanding relative to the base model.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models, Metric Depth Estimation, 3D Spatial Reasoning, Depth-Aware Vision-Language Models, Visual Question Answering, Object-Centric Geometric Reasoning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_WEPDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DepthEvidence, a 4B multimodal language model that uses its own metric-depth predictions as object-aligned evidence for language generation. The method contains a camera-conditioned dense decoder for full-resolution metric-depth prediction and a dense-to-language interface that pools predicted depths and decoder features within referenced object regions. The resulting descriptors are converted into continuous geometry tokens anchored to explicit object identifiers. Auxiliary objectives supervise the recoverability of object-level metric statistics before and after language-context interaction, while a ranking loss encourages correct relative ordering. The paper additionally introduces Depth-VQA, which organizes depth understanding into dense metric prediction, instance-level metric measurement, and compositional reasoning involving relative relations, numerical gaps, ranges, thresholds, and spatial filters. Across nine dense-depth datasets, DepthEvidence obtains the highest average δ1\delta_1 among the evaluated methods while remaining competitive with specialized estimators. It also achieves the strongest reported aggregate results on instance-level estimation and the two Depth-VQA reasoning tracks. Matched no-interface comparisons, supervision ablations, agent baselines, and general-VQA evaluations are used to analyze the contribution and capability retention.

Critical strengths

1.The paper identifies and addresses an important missing interface between geometric prediction and language reasoning. Existing depth-aware VLMs may predict depth and text in parallel without ensuring that language generation actually uses the predicted metric field. DepthEvidence addresses this gap through object-aligned geometry tokens tied to explicit identifiers and supervised for metric recoverability after contextualization. This is a focused and technically meaningful contribution rather than a superficial combination of two tasks.

2.The empirical study includes strong matched controls and covers prediction, reasoning, and capability retention. The comparison between the full model and “DepthEvidence w/o interface” isolates the interface more cleanly than comparisons against unrelated baselines. The full interface improves Relative Overall accuracy from 0.5838 to 0.6270 and Metric Overall accuracy from 0.7692 to 0.8504. Table 6 further decomposes the effects of raw-depth, contextual-depth, and ranking supervision. The paper also evaluates nine-dataset dense-depth generalization, agent pipelines, general VQA retention, and spatial understanding, providing evidence that the gains are not obtained by simply sacrificing the base model’s broader capabilities.

Critical weaknesses

1.The metric reliability and independence of Depth-VQA require stronger validation. For outdoor data, the paper states that sparse depth is first completed so that every image has a dense depth map. Object means, percentiles, ranges, and threshold decisions are then derived from these completed maps. The main paper does not identify the completion method, quantify its metric error, or explain how uncertainty in unobserved regions is propagated into benchmark labels. Manual verification can identify incorrect masks or inconsistent questions, but it may not establish that a generated object mean such as 1.31 m is metrically correct. The benchmark is also relatively small, with 757 Level-1 instances, 370 Relative questions, and 376 Metric questions. The paper should clarify strict separation between training and benchmark scenes, instances, and generated question patterns, and ideally report uncertainty or confidence intervals. This matters because the paper’s main reasoning claims depend on the reliability and independence of these labels.

2.The causal role of the model’s own predicted dense depth is not fully isolated. The interface descriptor combines decoder features, box coordinates, and summarized predicted-depth statistics. Table 6 shows that adding the interface with answer supervision alone produces almost no improvement; most of the measured gain appears after direct supervision of the geometry tokens with ground-truth object-depth statistics. This is consistent with the proposed method, but it does not by itself establish that inference-time answers depend on the model’s predicted depth map. A stronger causal test would separately evaluate depth-statistics-only and decoder-feature-only interfaces, replace the predicted depth with shuffled or perturbed depth, and compare predicted-depth evidence with a ground-truth-depth upper bound. Without such interventions, it remains possible that the interface relies substantially on decoder features, object/location priors, or task-specific instruction tuning rather than the explicit metric field emphasized in the paper’s central claim.

Questions for authors

1.What method is used to complete sparse outdoor depth when constructing Depth-VQA, and how is the metric accuracy of the completed values validated? In particular, how are object-level means and 5th/95th percentiles handled in regions with little or no original sensor support? Please also confirm whether benchmark scenes, object instances, and generated questions are strictly disjoint from the instruction-tuning data.

2.Can the authors provide a causal intervention demonstrating that the answers depend on the predicted dense metric field? For example, what happens if the predicted depth statistics are shuffled across objects, perturbed in scale, or removed while retaining decoder features? Conversely, what is the performance of a depth-statistics-only interface and of an interface supplied with ground-truth depth? These results would materially affect my confidence in the paper’s central claim.

3.How many independent linguistic templates or generation patterns are represented in the benchmark, and how sensitive are the results to paraphrasing or altered numerical ranges? Since training and evaluation questions are synthetically generated, evidence of robustness to unseen wording and constraint compositions would help distinguish geometric reasoning from template adaptation.

3 comments

Thank you for your constructive feedback on evidence use and benchmark validation.

W1. Label reliability and independence

Q1 details completion and validation, and 9WCi Q1 explains scene/video separation. Validation combines held-out sensor testing with manual inspection of metric geometry.

W2. Role of predicted depth

Fixed-Full shuffling covers all 746 Level-2 questions and reduces the Overall scores by 8.65 / 15.38 pp, adding evidence of the inference-time role of object-aligned statistics (Q2 / R5).

Q1. Completion and label validation

We express Waymo, Argoverse 2, nuScenes and DDAD depths as metric optical-axis Z. Within 0–250 m, we compare eight candidates (C1) in 20 m bins (final bin 240–250 m), using 400 frames per source–range. A base model is selected by equally weighted bin-level AbsRel; bin-specific replacements require adequate coverage and scene-resampling-supported RMSE improvement. Object statistics use valid depths within instance masks.

Each source has 100 independent-scene test frames, with 80% input / 20% held-out sensor points, plus contiguous-region withholding. After fixing selection, we compare MAE, RMSE, AbsRel, P95, δ₁ and coverage. We additionally inspect 120 frames per source (60 random, 60 targeted) using RGB, depth and LiDAR-overlaid point clouds for scale, boundaries and structure. Benchmark scenes and video sequences are excluded from both dense-depth and instruction-tuning training, following Appendix C.2 and the separation described in 9WCi Q1.

Table C1. Depth-completion candidates evaluated

CandidateInput type
OMNI-DC v1.0RGB + sparse metric depth
NLSPN KITTI DCRGB + sparse metric depth
MetricAnything Student DepthMapRGB + focal length
Depth Anything V2 Metric VKITTI LargeRGB
Depth Anything 3 DA3METRIC-LARGERGB + focal length
UniDepth V2 ViT-L/14RGB + camera intrinsics
Metric3D v2 ViT-LRGB + camera intrinsics
Depth Anything V2 Relative Large + LiDAR alignmentRGB + sparse metric depth

Q2. Ablations and depth intervention

R1 reports separately retrained Stats-only / Features-only. We further fix Full and shuffle the six fields mean, standard deviation, minimum, maximum, center and log-mean as a bundle across objects before the projector. Features, boxes, location attributes, object IDs, images and questions retain their original inputs. All 746 questions are scored against the original labels.

R5 gives 54.05% / 69.66% Relative / Metric, drops of 8.65 / 15.38 pp from published Full, and gap δ₁ of 52.08% / 36.17%. This supports sensitivity to object-aligned statistics. Clean scores are the published references.

Table R5. Measured shuffle results against the Full reference

MetricFull published clean (%)Shuffle measured (%)Drop (pp)Shuffle successes / N
Relative Overall Accuracy ↑62.7054.058.65200 / 370
Metric Overall Accuracy ↑85.0469.6615.38163 / 234
Metric Pair (Numerical Depth Gap) δ₁ ↑58.3352.086.2525 / 48
Spatial Metric Pair (Numerical Depth Gap) δ₁ ↑40.4336.174.2634 / 94

Note: Scores are percentages; drops are percentage points. Full clean values are the published references. All 746 questions received the intervention, with disjoint scoring groups of 370 / 234 / 48 / 94 questions. The table reports aggregate scores.

Q3. Wording and constraint generalization

We evaluate Full on original and rephrased versions of the same questions, preserving images, objects, numbers, semantics and answer format. This tests linguistic variation within the original numerical conditions and reasoning constraints. For example, “Which vehicle on the left is closest?” becomes “Among vehicles on the left side, select the one nearest to the camera.” Relative / Metric is 62.43% / 84.62%, drops of only 0.27 / 0.42 pp (R9), supporting stability under the tested rephrasings.

Table R9. Performance under meaning-preserving question rephrasing

TaskNFull reference Acc (%) ↑Rephrased Acc (%) ↑Change (pp)Rephrased correct / N
Relative Overall37062.7062.43−0.27231 / 370
Metric Overall (discrete questions)23485.0484.62−0.42198 / 234

Note: Images, objects, numbers, semantics and answer format are preserved. Changes use the supplied Full reference.

Dear Reviewer WEPD,

Thank you again for your constructive feedback. In our rebuttal, we added a fixed-checkpoint depth-statistics shuffling experiment, statistics-only and features-only ablations, and a paraphrase evaluation. We also clarified the depth-completion procedure and scene/sequence separation.

We would greatly appreciate your thoughts on the new evidence, particularly regarding the inference-time use of predicted depth. We are happy to discuss any remaining concerns.

Thank you for the additional experiments and clarifications.However, the rebuttal describes held-out sensor validation, but I could not find the resulting errors or coverage rates in the supplementary material. What are these results, particularly for object regions with little sensor support?

Reviewer_QtRVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents DepthEvidence, a 4B-parameter multimodal model built on Qwen3.5-4B. Its central idea is that dense metric-depth predictions should serve as explicit, object-grounded evidence for language reasoning, rather than only as a parallel output. To this end, the method uses a camera-conditioned decoder to predict full-resolution metric depth, and a dense-to-language interface that pools the predicted depth and decoder features into a sequence of continuous tokens for each object in the scene. The authors also introduce Depth-VQA, a benchmark of 1,503 questions split into Level 1 (instance-level metric depth) and Level 2 (multi-object compositional reasoning). Experimentally, DepthEvidence achieves the highest average dense δ1 across nine datasets (Level 0) and the highest overall accuracy on Depth-VQA, while showing little degradation on general VQA benchmarks relative to the base model.

Critical strengths
  1. Well-motivated problem formulation. Existing depth-aware VLMs either predict depth as a parallel output (e.g., DepthVLM) or answer point queries (e.g., DepthLM), and in neither case are the language answers required to depend on the predicted depth field. This paper explicitly closes that loop at the object level. In addition, the Level 0/1/2 decomposition cleanly separates three distinct capabilities: dense prediction, measurement, and compositional reasoning.
  2. Controlled and informative ablations. The “w/o interface” variant isolates the contribution of the dense-to-language interface. Table 6 contains a particularly telling result: when the interface is trained with L_ans alone, Relative Overall Acc is exactly unchanged (0.5838). This is good evidence that the model exploits the geometry tokens only once they receive geometric supervision, which directly supports the paper’s core argument.
  3. A unified model that preserves general capability. DepthEvidence is a single 4B model that achieves competitive dense metric depth while retaining strong general-VQA performance.
Critical weaknesses
  1. The Level-0 headline claim is fragile. According to Table 1, DepthEvidence’s margin over UniDepth V2 on the nine-dataset average is only 0.0027, and DepthEvidence actually trails UniDepth V2 on 7 of the 9 datasets. The average lead comes almost entirely from SUN RGB-D, where UniDepth V2 scores an anomalously low 0.3969, compared with 0.8420 for DepthEvidence. Excluding SUN RGB-D, the eight-dataset averages are roughly 0.792 for DepthEvidence versus 0.844 for UniDepth V2, which reverses the ranking. Because UniDepth V2 scores roughly 0.86 on the other indoor datasets, its SUN RGB-D result looks like an evaluation artifact rather than a genuine failure, and it should be investigated. Until it is, the claim of the highest average dense δ1 should be qualified.
  2. Two ablations needed to support the central claim are missing.
  • Decoder read-out baseline. The object descriptor s_i already contains pooled statistics of the predicted depth, and the Level-1 targets are the same statistics computed over object masks. It is therefore unclear whether the language pathway adds anything beyond the decoder for measurement questions. A direct comparison is needed: how do the VLM’s Level-1 answers compare with simply reading the box- or mask-pooled statistic from DepthEvidence’s own predicted depth map?
  • Text-serialization baseline. The paper argues for continuous geometry tokens, but never tests the most obvious alternative: writing the pooled predicted statistics into the prompt as text (e.g., “obj_1: mean 1.31 m, nearest 1.12 m”). Without this comparison, the advantage of continuous tokens over explicit numerical text remains unestablished.
  1. The baseline evaluation protocol is under-specified in ways that affect interpretation. Two results suggest output-format or answer-parsing failures rather than reasoning failures: DepthLM-12B scores 0.0000 on every Level-2 accuracy column, and Qwen3.5-4B has a Level-1 RMSE of 206 m. If these numbers reflect parsing failures, the comparison with DepthEvidence overstates its advantage. The paper should describe how answers are extracted from each baseline, report parse-failure rates, and mark cells where a baseline cannot meaningfully answer as N/A rather than 0.
Questions for authors
  1. (Weakness 1) What explains UniDepth V2’s score of 0.3969 on SUN RGB-D, given that it reaches roughly 0.86 on the other indoor datasets? Could you verify the evaluation setup for this dataset (e.g., camera intrinsics, depth scaling, or valid-pixel masking)?
  2. (Weakness 2) What are the Level-1 δ1, RMSE, and AbsRel obtained by directly reading the pooled statistic from DepthEvidence’s own predicted depth map, using (a) box pooling and (b) ground-truth mask pooling?
  3. (Weakness 2) How does a variant that serializes the same pooled statistics as plain text in the prompt compare with the continuous geometry tokens on Level 1 and Level 2?
  4. (Weakness 3) In Table 3, DepthLM-12B obtains 0.0000 on every Level-2 accuracy column, yet non-zero δ1 on the two numerical-gap columns. Does this reflect answer-format or parsing failures (e.g., never emitting a valid object identifier) rather than incorrect reasoning? Please report parse-failure rates for all baselines, describe any answer-extraction or retry procedure, and consider marking inapplicable cells as N/A.
Reviewer_9WCiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DepthEvidence, a multimodal large language model based on Qwen3.5-4B. Its key advantage is that, despite its compact 4B-parameter scale, it uses the dense depth maps it predicts as object-level evidence, enabling it to answer spatial questions.

The methodology and innovations are primarily threefold.

First, a dense depth decoder is employed: it utilizes camera intrinsic parameters—integrated via FiLM—as conditioning, and incorporates a high-resolution RGB refinement branch to predict full-resolution metric depth (thereby achieving the objectives of "Level 0").

Second, tokens are supervised using an auxiliary depth regression loss both before and after being incorporated into the language context. The paper introduces a "dense-to-language" interface that pools depth statistics and decoder features within a given object bounding box, projecting them into K=2K=2 continuous geometric tokens per object, which are then placed following the corresponding object identifier.

Third, the paper establishes two benchmarks, designated as "Level 1" and "Level 2." Experimental results demonstrate that the proposed method outperforms all competing approaches on both levels. Notably, the authors provide a rigorously aligned ablation study—evaluating the model without this interface—which confirms that general Visual Question Answering (VQA) capabilities remain largely intact.

Critical strengths

The approach presented in this paper, implemented as a 4B-parameter VLM, demonstrates strong dense depth estimation capabilities. Specifically, the data show that DepthVLM-4B achieves an average δ1\delta_1 score of 0.6673 across nine datasets, while DepthEvidence reaches 0.7972. This outperforms DepthLM-12B—a model with three times the number of parameters—by 0.1033, surpassing it on 8 out of the 9 datasets. The mean performance is nearly identical for both indoor and outdoor environments, indicating that conditioning the model on camera intrinsic parameters facilitates effective cross-domain generalization.

As shown in Table 4, the proposed method demonstrates strong spatial understanding and general-purpose capabilities. This is particularly significant, as these results are based on benchmarks not created by the authors themselves. Specifically, the data show improvements across several metrics: ERQA increased from 54.00 to 57.25, EmbSpatialBench increased from 81.30 to 83.14, and Ego3D accuracy rose from 62.82 to 67.99, while RMSE dropped from 13.17 to 6.83 (−48%). Furthermore, there are modest gains in general-purpose benchmarks, including MMMU, MMBench, HallusionBench, and AI2D. Collectively, these results highlight the advantages of the proposed approach.

The experimental methodology is sound, and the ablation studies are particularly well-conducted. Examining the ablation results—for instance, in Table 5—it is clear that the decoder components have been thoroughly evaluated. The data confirm that both the SSI loss and the high-resolution (HR) RGB branch contribute to performance gains across all four datasets, with the best results achieved when they are combined. Furthermore, the authors provide a plausible explanation (lines 448–453) for why the HR branch yields greater gains in indoor environments (+0.0842 vs. +0.0568); these are all notable strengths of the work.

The baseline coverage is comprehensive, encompassing a variety of general-purpose VLMs, depth-aware VLMs, other agent pipelines, and dedicated depth estimation models. The problem formulation is also clearly defined. Specifically, recent depth-aware VLMs either output depth maps directly or treat geometry merely as an implicit feature. However, "predicting depth" does not necessarily mean that the model actually utilizes this depth information during reasoning. This paper poses a critical question: can a model associate its predicted metric depth field with specific objects while preserving the underlying numerical information throughout the process of linguistic reasoning? This is a well-chosen, precise line of inquiry.

Critical weaknesses

The experimental results require further clarification and strengthening. First, it must be demonstrated whether the observed performance gains can indeed be attributed to the "token interface"—or, as the paper suggests, to enhanced reasoning capabilities. Furthermore, since all baselines are zero-shot, it is necessary to provide a comparison against models trained on in-distribution data.

As the author notes, both agent pipelines “remain below DepthEvidence without the interface.” Consequently, the conclusion that the combination of an external estimator and a VLM is inferior to our integrated design is confounded by the variable of whether or not fine-tuning was applied. This is a very critical issue.

This creates significant confusion. The core argument of this paper is that the interface enables the model to treat its own geometric predictions as evidence for its reasoning. However, the existing evidence could just as easily be explained by fine-tuning the model on in-distribution, templated problems—where numerical values are fed into the model via a learned projector—a mechanism that would contribute far less.

Therefore, we may need to supplement the core design's baseline. Potential approaches include—but are not limited to—the following: using the same backbone and dataset, but incorporating sis_i as numerical text values ​​into the prompt before fine-tuning; comparing against region-based deep features (similar to the approach in SpatialRGPT); or bypassing the LLM entirely by directly utilizing pooled values ​​from the model’s own dense predictions within the bounding boxes. Ultimately, it must be determined whether continuous tokens are truly necessary.

Furthermore, the Level-0 conclusions rely on a single dataset; as shown in the existing tables, the current evidence is limited to one favorable dataset and only a few dozen test cases. For instance, DepthEvidence outperforms UniDepth V2 on only 2 out of 9 datasets—achieving the best result solely on the SUN RGB-D dataset—while performing worse across all 4 outdoor datasets (with an outdoor mean of 0.797 versus 0.866).

Questions for authors

There are several points that require clarification, which are crucial:

  1. Are the training and test sets for Depth-VQA partitioned at the scene or sequence level? The objective here is to demonstrate performance under an in-distribution training setting and to compare this against a zero-shot baseline. It should be clarified whether the Depth-VQA results can truly show that the observed performance stems from the continuous token interface or from actual "reasoning" capabilities.

  2. Could you provide subgroup results or paired significance tests for the interface-driven gains? Also, what are the answer-parsing failure rates across the baselines? These requests are intended to make the experimental conclusions more rigorous.

  3. Since the model has already calculated that the "average depth of obj_1 is 1.31 m," the simplest approach is to incorporate this statement directly into the prompt and then fine-tune the same model using the same data. You don't need a projection network or continuous tokens.

  4. Are most of the test questions quite easy? As I understand it, the model’s average relative error for depth estimation is 16%—which, at a depth of 1.3 m, translates to about 20 cm. Given this, the model could not consistently get questions with a 2 cm margin of error correct. I would appreciate it if the authors could clarify whether the test questions themselves are simply too easy.

  5. A crucial point concerns the evaluation protocol for the SUN RGB-D dataset. This requires clarification: while UniDepth V2 achieves 0.86 on other indoor datasets, it reaches only 0.397 on SUN RGB-D. Are the intrinsic parameters, cropping/rescaling settings, and depth upper limits used in the evaluation consistent with those in the original paper? Furthermore, if you exclude SUN RGB-D from the analysis (0.792 vs. 0.844), do the conclusions still hold?

Reviewer_XwuNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents DepthEvidence a 4B vision language model that unifies dense metric depth prediction and object level geometric reasoning. The model builds on Qwen3.5-4B and adds a camera conditioned depth decoder using multiscale visual features and high resolution RGB refinement. The main proposed component is a dense to language interface that extracts depth statistics and decoder features for referenced objects, projects them into continuous geometry tokens, and makes these tokens available to the language model during answer generation. Auxiliary geometric objectives supervise depth information before and after language contextualization.

The authors also introduce Depth-VQA, a benchmark consisting of 1,503 questions covering instance metric depth estimation, relative depth reasoning, and compositional questions involving spatial and metric constraints. The model is evaluated against specialized depth estimators, general purpose VLMs, depth aware VLMs, and external depth estimator plus VLM pipelines.

DepthEvidence achieves 79.72% on nine dense depth datasets and improves over its matched no interface baseline on relative reasoning and on accuracy scored metric reasoning. The model also improves several spatial understanding benchmarks while largely retaining general VQA capabilities. The results are promising, although the gains are not uniform across all datasets and tasks.

Critical strengths

One I would say is the effective integration of metric depth prediction and object level reasoning. The paper tackles an important problem and the proposed system provides a practical way to make dense geometric predictions available to language generation. The matched ablation shows meaningful improvements, particularly on metric constrained reasoning (76.92 -> 85.04%). The reduction in Level 1 AbsRel from 0.2351 to 0.1629 is also encouraging. These results suggest that explicitly exposing geometric evidence can help with numerical spatial tasks.

Another one is comprehensive experimental study. The authors evaluate dense depth prediction across nine datasets and introduce a benchmark covering several types of object level depth reasoning. They also provide useful ablations of the decoder components and geometric supervision losses. The additional control in Appendix D.1 is valuable and shows that direct metric supervision without the interface does not recover the performance of the full model.

Critical weaknesses

On the novelty front, the main components are relatively established. Dense depth prediction within VLMs has been explored by DepthVLM, while region grounded geometric reasoning and geometry language fusion have been studied by SpatialRGPT and SpatialStack. The paper's main novelty is the particular way predicted depth statistics and decoder features are projected into object aligned tokens, together with auxiliary supervision. The ablations are useful but do not fully establish why this representation is needed. In Table 6, adding the interface with only the answer loss produces almost no improvement. Most of the gains appear after introducing additional geometric regression objectives. Appendix D.1 helps separate the impact of metric supervision, but a stronger baseline would provide the same object depth statistics from DepthEvidence's own depth decoder as structured text to an equally trained language model.

Another issue is that the interface includes both depth statistics and fused visual features. The experiments do not clearly isolate which of these carries the improvements. Since the geometric losses do not backpropagate into the dense decoder, the results establish effective feature conditioning rather than fully joint optimization of depth estimation for downstream reasoning.

Another weakness is that there is limited evidence that the benchmark measures robust metric reasoning beyond the training distribution. Depth-VQA is useful but relatively small and largely built from automatically generated questions and depth derived annotations. The benchmark labels depend on depth completion and automatically generated instance masks, which raises questions about the precision of the metric ground truth, particularly around object boundaries and narrow numerical thresholds.

More importantly, Metric Overall Accuracy is computed over only 234 of the 376 metric questions. The 142 numerical gap questions are evaluated separately. The full model obtains δ1 of 58.33% on Metric Pair numerical gaps and 40.43% on Spatial Metric Pair. These are meaningful improvements over the baselines but also show that accurate numerical reasoning remains difficult. As a result the reported 85.04% metric accuracy should not be interpreted as performance across all metric tasks.

The dense depth results are also somewhat mixed. The nine dataset average is only 0.27 percentage points above UniDepth V2, and specialized models perform better on several individual datasets. Similarly, general VQA retention is not completely uniform, with regressions on MMMU-Pro, RealWorldQA, and CharXiv. These do not invalidate the results but the claims should be more carefully qualified.

Questions for authors

1)Can you compare the continuous geometry interface against an equally trained textual geometry baseline? In particular, use the same DepthEvidence depth decoder to generate object level depth statistics and provide these as structured text to the same Qwen backbone, using comparable depth instruction tuning and supervision. This would help determine whether the continuous representation provides a meaningful advantage beyond access to accurate depth measurements.

2)Can you separate the contribution of predicted metric depth from the decoder features?

  1. How reliable are the numerical targets and how does the model perform across the full metric benchmark? Can you provide the error distribution of the underlying completed depth targets and clarify whether manual verification checks physical metric accuracy or just consistency of the generated questions? I would also appreciate absolute error metrics for numerical depth gaps, and performance stratified by gap size, object depth, and dataset. This seems particularly important given the narrow metric intervals used in some examples.

4)How robust is the system to imperfect object grounding and camera intrinsics?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.