- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes a lightweight model for supervised metric monocular depth estimation. The architecture follows the encoder-DPT design of foundation models, with a MobileViT encoder and a pruned dpt decoder. Two components are proposed: Deep-shifted Skip Connections, that take the skip features only from deeper encoder layers instead of the usual multi-scale ones, and a metric head with fixed log-spaced bins, a small adjuster and a residual term built from the bin widths. Models have few parameters and are trained and tested in-domain on NYUv2 and KITTI. The paper reports better results than previous lightweight methods (mainly on KITTI), a zero-shot metric evaluation, an analysis of the indoor-outdoor gap with cropping/stretching of the input, and runtime on a NVIDIA Orin.
Critical strengths
- The topic is relevant. Bringing the encoder-DPT design of foundation models to models with a few million parameters is a practical problem.
- Strong in-domain results on KITTI for the model size.
- The ablations cover all the main components (encoder, skip type, decoder channels, number of bins, head vs ZoeDepth head).
- DSC is tested on several encoders and on two existing models (GuideDepth, Lite-Mono). The observation that shallow skips can hurt small encoders could be useful, if better analyzed.
- It also tries to look beyond in domain accuracy. It includes a zero-shot metric evaluation and a diagnostic analysis of the indoor-outdoor gap. The conclusions are not fully supported (see below), but the direction is interesting.
- The paper is overall easy to follow, and figure 1 explains the architecture clearly.
Critical weaknesses
- All models are trained and evaluated only in-domain on NYUv2 or KITTI. The lightweight monocular depth field is moving towards compact models that work zero-shot everywhere, trained on large-scale data, usually by distillation from foundation models. In this setting the main thesis of the paper cannot be verified. What makes foundation models strong (data scale and supervision) is not transferred at all, only the encoder-DPT structure is. The paper results show this. This is a pity, because the architecture may be competitive when trained at scale, but there is no evidence.
- The only evidence for “beyond naive compression” is one row in Table 8 (pruned DAv1). How DINOv2 is pruned (depth? width?), how it is initialized and how it is trained is not described. 0.750 on KITTI is below FastDepth, which suggests a badly configured baseline. Today the standard way to compress a foundation model is distillation, not pruning without a teacher.
- The HieR formulation is not consistent: the indices in eq 3-6 do not match each other or table 5, the residual term is always positive and does not depend on the selected bin, key values (dmin/dmax) and the training loss are not given, and the adjuster is not ablated.
- Some claims are contradicted by the tables, for instance: 1) “Robust zero-shot generalization” and “lightweight networks have stronger generalizability”: in table 6 Lite-Mono is better on DIODE (0.671 vs 0.204) and VKITTI2, GuideDepth is better on SUN RGB-D, and RePDepth-S is worse than XXS on SUN and DIODE Indoor. Moreover, there are no zero-shot qualitative results. 2) “The indoor-outdoor gap stems from dataset bias, not capacity”: in tables 10 and 11, DAv1 has a very small gap (0.978 vs 0.965) and RePDepth a large one (0.919 vs 0.790). This points to capacity and data
- The appendix claims sharp details and thin structures. But the depth is an expectation over bins which gives intermediate values at discontinuities, and DSC removes the high-res features. Both go against sharp edges. No edge metric is given. Please report them and show point clouds near discontinuities.
- There is no validation set. All ablations (bins, DSC layers, channels) are reported on the test sets and the final configuration is chosen from them.
- Self-supervised baselines evaluated with median scaling are in the same table as supervised metric models. Which numbers are re-run and which are copied? Which KITTI gt, crop and cap are used?
- Table 7: “Origin” and “Stretch” are not defined precisely. Stretching changes the effective focal length, so a drop is expected for a metric model. Scale aligned metrics are needed to separate scale and structure errors.
- Regarding the efficiency: -- frameworks are mixed. ZipDepth is reported only in PyTorch, while TensorRT, ONNX and torch.compile versions are available. -- LMDepth, the main baseline, is missing. VRAM is given only for RePDepth. The “Accuracy” column contains the numeric precision. -- The accuracy of the TensorRT FP16 model is not reported. -- There is nothing about export to phone (ONNX/TFLite/CoreML, NPU support of the MobileViT unfold/fold ops), cpu latency, int8 or energy. -- Without accuracy in the right regime, this section does not show a better accuracy-efficiency trade-off.
- The related work covers only lightweight MDE. General MDE literature is almost missing: supervised (BTS, AdaBins, NeWCRFs, PixelFormer), self-supervised, zero-shot relative / foundation models (MiDaS, Marigold, Depth Pro, MoGe, infiniDepth), metric (ZoeDepth, Metric3D, UniDepth), video depth consistency etc..
- The log-spaced anchors are the spacing-increasing discretization of DORN, which is cited but not credited. Bin-based methods (AdaBins, LocalBins, BinsFormer, IEBins) should be discussed.
- Missing lightweight works: PyDNet (IROS 2018), MiDaS v3.1, which already pairs DPT with lightweight encoders (close to contribution 1)
- ZipDepth is cited, but it is called a “task-specific custom architecture” in contrast with foundation frameworks. This is not clear, since it is distilled from a foundation model at scale. DepthART (ACM 2026) is not cited. Both are concurrent and not required as baselines, but they should be discussed.
- More recent hybrid encoders (EfficientViT, FastViT, RepViT) are not compared.
Some minor points:
- Missing details: encoder pretraining (?), augmentations, LR schedule, NYU training subset, KITTI gt, number of bins in the main model, “loss-aware depth refinement” is never defined.
- The difference between XXS and S (width only) is not explained and XS is not tested. In XXS the decoder is almost half of the parameters, which should be discussed given the claim that the encoder determines performance
- The order of the ablation questions in 4.3 does not match the order of the sections. Table 3 caption says “decoders” but the table compares encoders. Table 8 is placed far from Sec. 4.3.4.
- the FastDepth KITTI row in tab 1 has the NYU numbers of tab. 2
- DaNet has the same numbers at low and high resolution in tab2
- GuideDepth MSC layers [0,0,0,0]
- 5.8M vs 5.86M and 1.85M vs 1.86M
- FastDepth high-res is cited as Tu et al.
- Some claims are too strong (“exceptional performance”, “all competing approaches fail”).
- There is no limitations section.
Questions for authors
- How were hyperparameters and checkpoints selected? Is there a validation split?
- Details of the pruned DAv1 baseline: pruning strategy, initialization, training.
- The training loss, the dmin/dmax values, how R relates to the 64/16 configuration, why the residual is not conditioned on the selected bin, and an ablation with and without the adjuster.
- The criterion for DSC, the stride and channels of each selected layer, and a sweep over configurations.
- The evaluation protocol of each method in Table 6 (median scaling or metric), and the starting checkpoint of the KITTI model.
- How inputs are resized or cropped, and the comparison at the same pixel budget at low resolution.
- Latency at 384×1152 with the same stack for all methods, the accuracy of the TensorRT FP16 model, and results on phone/CPU.
- Do you plan to release code and weights?
My current score is weak reject. The architecture could be interesting, and the in-domain results on KITTI are good for the size. But the paper studies a setting (in-domain only, NYU and KITTI) that is behind the current direction of lightweight monocular depth, and the main missing part (large-scale training and zero-shot evaluation) is difficult to solve in a rebuttal. Several claims are also not supported by the paper’s own tables and the method description has some inconsistencies.
5 comments
Answer
Thanks to the authors for the answers. The clarifications on the KITTI protocol, the removal of the self-supervised methods and the extended related work are useful. My main concern is still the same. The models are trained and tested only in-domain on NYUv2 and KITTI. There is no large-scale training and no zero-shot evaluation. Some answers are also not clear to me. The baseline in table 8 seems to be a small ViT trained in-domain, not a compressed foundation model. So it is not clear how it supports the "beyond naive compression" claim. Also, in the answer to another reviewer the authors say that coarse and fine bins are the same, but in table 5 the final model uses 64/16. I am happy to read more clarifications from the authors.
Thank you for sharing your perspective. We would like to gently clarify two points regarding our evaluation and core research motivation, while also providing an update on our ongoing large-scale training efforts:
Q1) First, we would like to note that our manuscript already includes zero-shot generalization evaluations (presented in Table 6), rather than evaluating solely in-domain on NYUv2 and KITTI. These experiments verify the cross-dataset transferability of our model without any target-domain fine-tuning. Second, regarding the absence of large-scale distillation training, our primary research objective is to investigate how architectural design can enable compact networks to learn genuine depth reasoning even from limited training data. Rather than relying on massive-scale pre-training or heavy teacher distillation to memorize geometric patterns, we aim to uncover how the network itself infers depth. This is directly evidenced by our analysis in Figure 3 and Table 7, which demonstrates that our model exhibits distinct depth reasoning behaviors adapted to the underlying geometric priors of different datasets. Training on standard benchmarks like NYUv2 and KITTI allows us to cleanly isolate and validate this architectural inductive bias without confounding factors from large-scale external data. This experiment uncovers a previously overlooked phenomenon: the model exhibits distinctly different reasoning mechanisms across different datasets, which further advances the understanding of interpretability in monocular depth estimation (MDE) models. Therefore, we believe that our perspective offers substantial novelty and meaningful insights to the community.
Q2) We apologize for our previous vague and incomplete response. What we intended to convey is that we initially set both the coarse bins and fine bins to 64 in an attempt to achieve higher precision. Our design philosophy is to hierarchically partition the model's estimation range, yielding a theoretical resolution of max_depth / (coarse_bins × fine_bins). Therefore, using 64 fine bins theoretically enables finer granularity and higher accuracy. However, our empirical results demonstrate that using 16 fine bins achieves the optimal trade-off between accuracy and model size.
We would like to clarify that the pruned foundation model in Table 8 was not obtained through a post-training "pruning process". To create this lightweight baseline, we scaled down the architecture of the original vit_small backbone by reducing the embedding dimension, the network depth, and the number of attention heads. The specific architectural differences are as follows: Original vit_small: patch_size=16, embed_dim=384, depth=12, num_heads=6 Our compact vit_xxs: patch_size=14, embed_dim=256, depth=6, num_heads=4 Since our model is evaluated under a supervised learning paradigm, we applied the same training strategy to the scaled-down foundation model for a fair comparison. Specifically, we retrained the foundation model using a warm-start initialization from the scaled-down pre-trained weights. With this setup, the performance on KITTI reached 81.8/0.129; however, this strategy did not yield a significant impact on the NYUv2 dataset.
Thank you for your constructive feedback. First, while shallow features (1/2 and 1/4 resolutions) have high spatial resolution, they lack semantic context and contain substantial RGB texture noise, which often introduces texture-copy artifacts. Consistent with recent findings (e.g., ZipDepth), discarding 1/2 and 1/4 features does not degrade boundary sharpness when deeper semantic representations are effectively leveraged. Our DSC prioritizes deeper features with larger receptive fields to accurately distinguish true geometric boundaries from surface textures.
Thank you for raising this critical point regarding rigorous experimental protocol. We completely agree that selecting hyperparameters and architectural configurations directly on the test set risks test-set overfitting. The primary reason for our original setup is that the standard benchmarks in monocular depth estimation—specifically the official NYUv2 split and KITTI Eigen split—only define training and testing sets without an official validation split. To remain strictly consistent with prior works such as Depth Anything v1/v2, we followed their exact data split protocol.
Thank you for pointing out this fairness concern. We agree that mixing median-scaled self-supervised methods with supervised metric models is inconsistent. In the revised manuscript, we have removed all self-supervised baselines from Tables (replacing them with recent supervised methods) to ensure a strictly apples-to-apples comparison. Furthermore, we clarify that all baseline results in Tables are taken directly from their original publications without re-evaluation. For our KITTI evaluation protocol, we use the Eigen split raw LiDAR ground truth, apply the standard Eigen crop, and set the maximum depth cap to 80 meters.
Thank you for your insightful geometric analysis. In the revised manuscript, we have precisely defined "Origin" (native aspect ratio via cropping/padding) and "Stretch" (resizing/stretching the lower ground region to fill the target input resolution, which alters the aspect ratio and effective focal length).Interestingly, our experiments reveal a notable dataset-dependent divergence. On NYUv2, stretching causes a severe performance drop, which aligns well with your reasoning—indoor scenes lack distant vanishing points and rely heavily on undistorted vertical FoV to infer metric scale. Conversely, on KITTI, stretching the ground region minimally affects metric accuracy. We attribute this robustness to the strong perspective and structural priors of driving scenes (e.g., consistent road-plane geometry and canonical vehicle sizes), which enable the model to anchor the global scale despite local stretching. These results confirm that on NYUv2, the drop under "Stretch" is predominantly driven by global scale shift rather than structural distortion.
Thank you for your constructive feedback and for suggesting these seminal works. We agree that covering the broader MDE landscape better contextualizes our lightweight design. In the revised Section 2 (Related Work), we have expanded our literature review to systematically discuss: (1) supervised and generalizable metric models (e.g., BTS, AdaBins, NeWCRFs, PixelFormer, ZoeDepth, Metric3D, UniDepth); (2) self-supervised and video depth consistency methods; and (3) zero-shot relative and foundation models (e.g., MiDaS, Marigold, Depth Pro, MoGe, InfiniDepth). We further highlight how their high computational overhead motivates our compact, deployment-friendly architecture. All suggested references have been properly cited in the revised manuscript.