acceptodds
Under review as a conference paper at ICLR 2027

RePDepth: Redesigning Lightweight Monocular Depth Estimation Beyond Naive Compression

Abstract

Monocular Depth Estimation (MDE) plays a vital role in enabling intelligent systems to perceive 3D structures from a single image. However, while high-performing frameworks rely on computationally heavy Vision Transformer (ViT) encoders, the performance of existing lightweight models still lag behind. To bridge this gap, we introduce a lightweight model named Representation-Preserving Depth (RePDepth), demonstrating a viable pathway to adapt the Large Foundation Model (LFM) architecture for lightweight architectures with minimal performance degradation. RePDepth is built upon the encoder-DPT decoder architecture, leveraging a ViT-CNN hybrid encoder named MobileViT to simultaneously achieve global receptive fields and high inference efficiency. Crucially, revealing that conventional Multi-scale Skip Connections (MSC) are sub-optimal in multiple lightweight settings, we adopt Deep-shifted Skip Connections (DSC) to route features with larger receptive fields. We further design a lightweight metric depth head, HieR module, based on a bin-centric approach with coarse-to-fine refinement. In-domain evaluations demonstrate that RePDepth outperforms existing lightweight MDE methods, achieving competitive accuracy with a marginal 3% performance drop on the KITTI dataset compared to LFMs, while significantly reduce the model size by around 200 times (with minimum of 1.85M parameters). Finally, extensive experiments provide deeper insights into the widely observed indoor-outdoor performance gap, validate our model's robust zero-shot metric depth estimation across diverse datasets, and highlight its edge deployment capabilities.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer Vision, Monocular Depth Estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Dz9UOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a lightweight model for supervised metric monocular depth estimation. The architecture follows the encoder-DPT design of foundation models, with a MobileViT encoder and a pruned dpt decoder. Two components are proposed: Deep-shifted Skip Connections, that take the skip features only from deeper encoder layers instead of the usual multi-scale ones, and a metric head with fixed log-spaced bins, a small adjuster and a residual term built from the bin widths. Models have few parameters and are trained and tested in-domain on NYUv2 and KITTI. The paper reports better results than previous lightweight methods (mainly on KITTI), a zero-shot metric evaluation, an analysis of the indoor-outdoor gap with cropping/stretching of the input, and runtime on a NVIDIA Orin.

Critical strengths
  • The topic is relevant. Bringing the encoder-DPT design of foundation models to models with a few million parameters is a practical problem.
  • Strong in-domain results on KITTI for the model size.
  • The ablations cover all the main components (encoder, skip type, decoder channels, number of bins, head vs ZoeDepth head).
  • DSC is tested on several encoders and on two existing models (GuideDepth, Lite-Mono). The observation that shallow skips can hurt small encoders could be useful, if better analyzed.
  • It also tries to look beyond in domain accuracy. It includes a zero-shot metric evaluation and a diagnostic analysis of the indoor-outdoor gap. The conclusions are not fully supported (see below), but the direction is interesting.
  • The paper is overall easy to follow, and figure 1 explains the architecture clearly.
Critical weaknesses
  • All models are trained and evaluated only in-domain on NYUv2 or KITTI. The lightweight monocular depth field is moving towards compact models that work zero-shot everywhere, trained on large-scale data, usually by distillation from foundation models. In this setting the main thesis of the paper cannot be verified. What makes foundation models strong (data scale and supervision) is not transferred at all, only the encoder-DPT structure is. The paper results show this. This is a pity, because the architecture may be competitive when trained at scale, but there is no evidence.
  • The only evidence for “beyond naive compression” is one row in Table 8 (pruned DAv1). How DINOv2 is pruned (depth? width?), how it is initialized and how it is trained is not described. 0.750 on KITTI is below FastDepth, which suggests a badly configured baseline. Today the standard way to compress a foundation model is distillation, not pruning without a teacher.
  • The HieR formulation is not consistent: the indices in eq 3-6 do not match each other or table 5, the residual term is always positive and does not depend on the selected bin, key values (dmin/dmax) and the training loss are not given, and the adjuster is not ablated.
  • Some claims are contradicted by the tables, for instance: 1) “Robust zero-shot generalization” and “lightweight networks have stronger generalizability”: in table 6 Lite-Mono is better on DIODE (0.671 vs 0.204) and VKITTI2, GuideDepth is better on SUN RGB-D, and RePDepth-S is worse than XXS on SUN and DIODE Indoor. Moreover, there are no zero-shot qualitative results. 2) “The indoor-outdoor gap stems from dataset bias, not capacity”: in tables 10 and 11, DAv1 has a very small gap (0.978 vs 0.965) and RePDepth a large one (0.919 vs 0.790). This points to capacity and data
  • The appendix claims sharp details and thin structures. But the depth is an expectation over bins which gives intermediate values at discontinuities, and DSC removes the high-res features. Both go against sharp edges. No edge metric is given. Please report them and show point clouds near discontinuities.
  • There is no validation set. All ablations (bins, DSC layers, channels) are reported on the test sets and the final configuration is chosen from them.
  • Self-supervised baselines evaluated with median scaling are in the same table as supervised metric models. Which numbers are re-run and which are copied? Which KITTI gt, crop and cap are used?
  • Table 7: “Origin” and “Stretch” are not defined precisely. Stretching changes the effective focal length, so a drop is expected for a metric model. Scale aligned metrics are needed to separate scale and structure errors.
  • Regarding the efficiency: -- frameworks are mixed. ZipDepth is reported only in PyTorch, while TensorRT, ONNX and torch.compile versions are available. -- LMDepth, the main baseline, is missing. VRAM is given only for RePDepth. The “Accuracy” column contains the numeric precision. -- The accuracy of the TensorRT FP16 model is not reported. -- There is nothing about export to phone (ONNX/TFLite/CoreML, NPU support of the MobileViT unfold/fold ops), cpu latency, int8 or energy. -- Without accuracy in the right regime, this section does not show a better accuracy-efficiency trade-off.
  • The related work covers only lightweight MDE. General MDE literature is almost missing: supervised (BTS, AdaBins, NeWCRFs, PixelFormer), self-supervised, zero-shot relative / foundation models (MiDaS, Marigold, Depth Pro, MoGe, infiniDepth), metric (ZoeDepth, Metric3D, UniDepth), video depth consistency etc..
  • The log-spaced anchors are the spacing-increasing discretization of DORN, which is cited but not credited. Bin-based methods (AdaBins, LocalBins, BinsFormer, IEBins) should be discussed.
  • Missing lightweight works: PyDNet (IROS 2018), MiDaS v3.1, which already pairs DPT with lightweight encoders (close to contribution 1)
  • ZipDepth is cited, but it is called a “task-specific custom architecture” in contrast with foundation frameworks. This is not clear, since it is distilled from a foundation model at scale. DepthART (ACM 2026) is not cited. Both are concurrent and not required as baselines, but they should be discussed.
  • More recent hybrid encoders (EfficientViT, FastViT, RepViT) are not compared.

Some minor points:

  • Missing details: encoder pretraining (?), augmentations, LR schedule, NYU training subset, KITTI gt, number of bins in the main model, “loss-aware depth refinement” is never defined.
  • The difference between XXS and S (width only) is not explained and XS is not tested. In XXS the decoder is almost half of the parameters, which should be discussed given the claim that the encoder determines performance
  • The order of the ablation questions in 4.3 does not match the order of the sections. Table 3 caption says “decoders” but the table compares encoders. Table 8 is placed far from Sec. 4.3.4.
  • the FastDepth KITTI row in tab 1 has the NYU numbers of tab. 2
  • DaNet has the same numbers at low and high resolution in tab2
  • GuideDepth MSC layers [0,0,0,0]
  • 5.8M vs 5.86M and 1.85M vs 1.86M
  • FastDepth high-res is cited as Tu et al.
  • Some claims are too strong (“exceptional performance”, “all competing approaches fail”).
  • There is no limitations section.
Questions for authors
  • How were hyperparameters and checkpoints selected? Is there a validation split?
  • Details of the pruned DAv1 baseline: pruning strategy, initialization, training.
  • The training loss, the dmin/dmax values, how R relates to the 64/16 configuration, why the residual is not conditioned on the selected bin, and an ablation with and without the adjuster.
  • The criterion for DSC, the stride and channels of each selected layer, and a sweep over configurations.
  • The evaluation protocol of each method in Table 6 (median scaling or metric), and the starting checkpoint of the KITTI model.
  • How inputs are resized or cropped, and the comparison at the same pixel budget at low resolution.
  • Latency at 384×1152 with the same stack for all methods, the accuracy of the TensorRT FP16 model, and results on phone/CPU.
  • Do you plan to release code and weights?

My current score is weak reject. The architecture could be interesting, and the in-domain results on KITTI are good for the size. But the paper studies a setting (in-domain only, NYU and KITTI) that is behind the current direction of lightweight monocular depth, and the main missing part (large-scale training and zero-shot evaluation) is difficult to solve in a rebuttal. Several claims are also not supported by the paper’s own tables and the method description has some inconsistencies.

5 comments

We would like to clarify that the pruned foundation model in Table 8 was not obtained through a post-training "pruning process". To create this lightweight baseline, we scaled down the architecture of the original vit_small backbone by reducing the embedding dimension, the network depth, and the number of attention heads. The specific architectural differences are as follows: Original vit_small: patch_size=16, embed_dim=384, depth=12, num_heads=6 Our compact vit_xxs: patch_size=14, embed_dim=256, depth=6, num_heads=4 Since our model is evaluated under a supervised learning paradigm, we applied the same training strategy to the scaled-down foundation model for a fair comparison. Specifically, we retrained the foundation model using a warm-start initialization from the scaled-down pre-trained weights. With this setup, the performance on KITTI reached 81.8/0.129; however, this strategy did not yield a significant impact on the NYUv2 dataset.

Thank you for your constructive feedback. First, while shallow features (1/2 and 1/4 resolutions) have high spatial resolution, they lack semantic context and contain substantial RGB texture noise, which often introduces texture-copy artifacts. Consistent with recent findings (e.g., ZipDepth), discarding 1/2 and 1/4 features does not degrade boundary sharpness when deeper semantic representations are effectively leveraged. Our DSC prioritizes deeper features with larger receptive fields to accurately distinguish true geometric boundaries from surface textures.

Thank you for raising this critical point regarding rigorous experimental protocol. We completely agree that selecting hyperparameters and architectural configurations directly on the test set risks test-set overfitting. The primary reason for our original setup is that the standard benchmarks in monocular depth estimation—specifically the official NYUv2 split and KITTI Eigen split—only define training and testing sets without an official validation split. To remain strictly consistent with prior works such as Depth Anything v1/v2, we followed their exact data split protocol.

Thank you for pointing out this fairness concern. We agree that mixing median-scaled self-supervised methods with supervised metric models is inconsistent. In the revised manuscript, we have removed all self-supervised baselines from Tables (replacing them with recent supervised methods) to ensure a strictly apples-to-apples comparison. Furthermore, we clarify that all baseline results in Tables are taken directly from their original publications without re-evaluation. For our KITTI evaluation protocol, we use the Eigen split raw LiDAR ground truth, apply the standard Eigen crop, and set the maximum depth cap to 80 meters.

Thank you for your insightful geometric analysis. In the revised manuscript, we have precisely defined "Origin" (native aspect ratio via cropping/padding) and "Stretch" (resizing/stretching the lower ground region to fill the target input resolution, which alters the aspect ratio and effective focal length).Interestingly, our experiments reveal a notable dataset-dependent divergence. On NYUv2, stretching causes a severe performance drop, which aligns well with your reasoning—indoor scenes lack distant vanishing points and rely heavily on undistorted vertical FoV to infer metric scale. Conversely, on KITTI, stretching the ground region minimally affects metric accuracy. We attribute this robustness to the strong perspective and structural priors of driving scenes (e.g., consistent road-plane geometry and canonical vehicle sizes), which enable the model to anchor the global scale despite local stretching. These results confirm that on NYUv2, the drop under "Stretch" is predominantly driven by global scale shift rather than structural distortion.

Thank you for your constructive feedback and for suggesting these seminal works. We agree that covering the broader MDE landscape better contextualizes our lightweight design. In the revised Section 2 (Related Work), we have expanded our literature review to systematically discuss: (1) supervised and generalizable metric models (e.g., BTS, AdaBins, NeWCRFs, PixelFormer, ZoeDepth, Metric3D, UniDepth); (2) self-supervised and video depth consistency methods; and (3) zero-shot relative and foundation models (e.g., MiDaS, Marigold, Depth Pro, MoGe, InfiniDepth). We further highlight how their high computational overhead motivates our compact, deployment-friendly architecture. All suggested references have been properly cited in the revised manuscript.

Reviewer_Dz9UOpenReview ↗

Answer

Thanks to the authors for the answers. The clarifications on the KITTI protocol, the removal of the self-supervised methods and the extended related work are useful. My main concern is still the same. The models are trained and tested only in-domain on NYUv2 and KITTI. There is no large-scale training and no zero-shot evaluation. Some answers are also not clear to me. The baseline in table 8 seems to be a small ViT trained in-domain, not a compressed foundation model. So it is not clear how it supports the "beyond naive compression" claim. Also, in the answer to another reviewer the authors say that coarse and fine bins are the same, but in table 5 the final model uses 64/16. I am happy to read more clarifications from the authors.

Thank you for sharing your perspective. We would like to gently clarify two points regarding our evaluation and core research motivation, while also providing an update on our ongoing large-scale training efforts:

Q1) First, we would like to note that our manuscript already includes zero-shot generalization evaluations (presented in Table 6), rather than evaluating solely in-domain on NYUv2 and KITTI. These experiments verify the cross-dataset transferability of our model without any target-domain fine-tuning. Second, regarding the absence of large-scale distillation training, our primary research objective is to investigate how architectural design can enable compact networks to learn genuine depth reasoning even from limited training data. Rather than relying on massive-scale pre-training or heavy teacher distillation to memorize geometric patterns, we aim to uncover how the network itself infers depth. This is directly evidenced by our analysis in Figure 3 and Table 7, which demonstrates that our model exhibits distinct depth reasoning behaviors adapted to the underlying geometric priors of different datasets. Training on standard benchmarks like NYUv2 and KITTI allows us to cleanly isolate and validate this architectural inductive bias without confounding factors from large-scale external data. This experiment uncovers a previously overlooked phenomenon: the model exhibits distinctly different reasoning mechanisms across different datasets, which further advances the understanding of interpretability in monocular depth estimation (MDE) models. Therefore, we believe that our perspective offers substantial novelty and meaningful insights to the community.

Q2) We apologize for our previous vague and incomplete response. What we intended to convey is that we initially set both the coarse bins and fine bins to 64 in an attempt to achieve higher precision. Our design philosophy is to hierarchically partition the model's estimation range, yielding a theoretical resolution of max_depth / (coarse_bins × fine_bins). Therefore, using 64 fine bins theoretically enables finer granularity and higher accuracy. However, our empirical results demonstrate that using 16 fine bins achieves the optimal trade-off between accuracy and model size.

Reviewer_54F6OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RePDepth, a lightweight monocular depth estimation (MDE) model. It adopts the encoder–DPT decoder architecture widely used in large foundation models and redesigns it for resource-constrained settings. The model consists of four components: a MobileViT encoder (a hybrid CNN–ViT backbone), a pruned DPT decoder, Deep-shifted Skip Connections (DSC) that draw skip features from the deeper stages of the encoder, and a Hierarchy Regression (HieR) metric depth head that combines logarithmically spaced bin anchors with residual-based refinement. The authors report improved performance over existing lightweight models on KITTI and NYUv2. They also analyze zero-shot generalization, deployment efficiency on edge devices, and the performance gap between indoor and outdoor scenes.

Critical strengths

None

Critical weaknesses
  1. Limited Technical Novelty :
  • The adoption of MobileViT amounts to selecting an existing off-the-shelf backbone rather than introducing a new architectural contribution.
  • The proposed Deep-shifted Skip Connection (DSC) changes only the layer indices from which skip features are extracted. It is therefore closer to a hyperparameter choice than to a new module.
  • The HieR module combines fixed logarithmic bins, a BinsFormer-style tanh-based adjustment, and a bin-width residual. As such, it does not appear to depart substantially from prior bin-based approaches such as LocalBins, BinsFormer, and ZoeDepth. The paper offers limited insight from a representation learning perspective, which is a central focus of ICLR.
  1. Concerns regarding the fairness of comparisons :
  • Lite-Mono and BoRe-Depth in Table 1 are self-supervised methods. Comparing them with supervised metric depth models in the same table to claim state-of-the-art performance may not be appropriate.
  • The FastDepth low-resolution row in Table 1 (KITTI: 0.778/0.942/0.987/0.142/0.591) is identical to the NYUv2 results reported in Table 2. This appears to be a copying error. In addition, the high-resolution FastDepth row is attributed to Tu et al.
  • The KITTI evaluation protocol is not specified. Details such as the type of ground truth (raw LiDAR or annotated depth), the use of Garg or Eigen crop, and the maximum depth cap (e.g., 80 m) should be clarified.
  • Input resolutions, training data, and pretraining differ across the compared methods. It is also unclear whether the baseline results were reproduced by the authors or taken from the original papers.
  1. Overstatements between claims and results:
  • Zero-shot generalization: RePDepth performs considerably worse than GuideDepth on SUN RGB-D (AbsRel 0.793 / δ1 0.218) and than Lite-Mono on DIODE Outdoor (δ1 0.194 vs. 0.671). This contradicts the claims of “robust zero-shot” performance and “stronger generalizability.”
  • NYUv2: RePDepth-XXS underperforms LMDepth on all metrics, and RePDepth-S gains only 0.004 in δ1 with roughly twice the parameters. The claim of outperforming existing methods thus seems overstated. Gap to large foundation models: The claimed ~3% gap to LFMs is not supported by LFM results on KITTI in the main tables, and the appendix reports a different figure (~4%).
  • Practical advantage: According to the publicly reported results of Depth Anything V1, Depth-Anything-Small (24.8M) already runs in about 3 ms on an RTX 4090 with TensorRT, including pre- and post-processing. Given this, the practical appeal of the proposed method remains unclear.
Questions for authors
  1. Could the authors clarify the KITTI evaluation protocol (e.g., ground-truth type, Garg/Eigen crop, and maximum depth cap) and indicate whether the baseline results were reproduced or taken from the original papers?
  2. Excluding self-supervised methods, could the authors provide comparisons with supervised lightweight baselines, such as Depth-Anything-V2-Small or distillation-based compact models?
  3. Could the authors describe the pruning procedure and training settings used for the pruned Depth Anything V1 baseline in Table 8?
  4. Given the relatively weak zero-shot results in Table 6, what is the basis for the conclusion that carefully designed lightweight networks have “stronger generalizability”?
  5. A direct comparison with Depth-Anything-Small on the same edge hardware (e.g., Jetson Orin Nano) would help clarify the practical benefit of the proposed method.
  6. The paper presents its design as a general pathway for adapting foundation MDE architectures to the lightweight regime, but this is validated only on Depth Anything V1. Could the authors discuss whether the proposed approach is applicable to other foundation models, such as Depth Anything V2, Depth Pro, Metric3D, or MoGe, and provide any corresponding results? If certain models are not compatible with the approach, an explanation of why would also be helpful.
3 comments

Thank you for your valuable feedback. First, we respectfully disagree with your view that "DSC changes only the layer indices from which skip features are extracted. It is therefore closer to a hyperparameter choice than to a new module." The purpose of traditional skip connections is to fuse features of different resolutions, which includes not only coarse and fine feature representations but also the larger receptive fields and stronger representational capacity brought by deeper networks. Such a conclusion is somewhat arbitrary and would even negate the effectiveness of all previous works. Furthermore, the number of parameters expanded by DSC is very small and can be compressed by channel pruning; therefore, it does not necessarily result in a higher parameter count. Thus, the parameter selection hypothesis does not hold. At the same time, based on mobileViT, the skip connection after the ViT possesses a global receptive field and a stronger representational capacity, which naturally led us to design it as a DSC. Second, this topic itself is not about introducing a novel module; its purpose is to shed light on the role of deeper skip connections with larger receptive fields in lightweight MDE networks, which has already been experimentally verified on several previous methods in our paper.

The focus of this paper is on designing a lightweight overall network architecture. The proposed bin method does not aim to revolutionize previous bin methods, but rather to minimize redundant designs as much as possible to achieve a lightweight architecture without degrading performance. To support our argument, we attach here the results of replacing the metric depth head on Depth Anything v1 with ours, and its performance is on par with the original Depth Anything v1. The results is as follows:

\begin{table}[h!] \centering \caption{An ablation investigation on the impact of the proposed HieR module within Depth Anything V1 framework on NYU Depth V2 dataset.} \resizebox{\textwidth}{!}{ \begin{tabular}{l|cccccc} \hline \hline \textbf{Methods} & δ1↑\mathbf{\delta_1}\uparrow & δ2↑\mathbf{\delta_2}\uparrow & δ3↑\mathbf{\delta_3}\uparrow &
Abs Rel$\downarrow$ & RMSE$\downarrow$ & log10↓log_{10}\downarrow \ \hline

DAv1 & 0.98385 & 0.998 & 1.0 & 0.057 & 0.207 & 0.025 \\hline DAv1 + HieR Module
&\textbf{0.98406} & 0.998 & 0.999 & 0.057 & 0.208 & 0.025 \ \hline \hline \end{tabular} } \label{tab:original_verification} \vspace{-0.5cm} \end{table}

Considering self-supervised methods is inappropriate, we decide to remove those results and replacing them with other, more recent methods.

We sincerely thank you for helping us spot the data error in the table. We will correct the actual performance of FastDepth and highlight the changes in red in the revised PDF. Our protocol on the KITTI dataset is identical to that of Depth Anything v1/v2 (the largest depth is set to 80m). It is quite normal for different methods to operate at different resolutions, as various network designs have their own specific requirements for input and output sizes. Forcing all models to use the same input size would likely cause previous methods to lose their structural advantages, further defeating the purpose of a fair comparison.

Our benchmark results are based on previous literature, and we will indicate the sources in the article.

Depth Anything V1 in Table 8 adopts exactly the same training settings as RepDepth. We would like to clarify that the model in Table 8 was not obtained through a post-training "pruning process". To create this lightweight baseline, we scaled down the architecture of the original vit_small backbone by reducing the embedding dimension, the network depth, and the number of attention heads. The specific architectural differences are as follows: Original vit_small: patch_size=16, embed_dim=384, depth=12, num_heads=6 Our compact vit_xxs: patch_size=14, embed_dim=256, depth=6, num_heads=4

While we point out we don't have any description of a 'general pathway', we would like to showcase a representative example. Most state-of-the-art ViT-based monocular depth estimation models—including the Depth Anything series—share a fundamental architectural paradigm: an Encoder followed by a DPT-style Decoder and depth head. By incorporating our MobileViT-inspired encoder with the DPT decoder, we demonstrate a design pathway that naturally extends across this entire family of methods. Thus, using Depth Anything V1 serves as a representative implementation to showcase the effectiveness of this generalized approach.

Thank you for your response. However, some concerns remain unaddressed.

Additionally, I have further questions based on your responses to other reviewers:

Q1) Table 8 refers to the baseline as "Pruned DAv1," but according to the rebuttal, it was obtained by scaling down the architecture rather than by pruning. Could you revise the name accordingly and clarify what "naive compression" refers to in this context?

Q2) Tables 1 and 2 report only the number of parameters, and the input resolutions differ across the compared methods. Could you report FLOPs for all methods at their evaluation resolutions, and ideally also at a common resolution, to allow a fair assessment of computational cost?

Q3) In your response to Reviewer kf6y, you stated that "adopting DSC achieves higher performance while simultaneously reducing computational overhead and memory consumption compared to traditional multi-layer features." You also mentioned to me that "the number of parameters expanded by DSC is very small and can be compressed by channel pruning; therefore, it does not necessarily result in a higher parameter count." However, in Table 4, DSC increases the parameter count for some encoders (e.g., MobileNet v3: 5.24M → 6.83M), and no FLOPs or memory measurements are provided. Could you clarify this point and report the FLOPs?

Q4) In your response to Reviewer kf6y, you stated that "the number of coarse and fine bins should be the same in our design". However, Sec. 4.3.3 states that "the design with 64 coarse bins with 16 hierarchy residual achieved highest performance," and Table 5 lists the main configuration as 64/16, with the 64/64 setting performing worse. Could you clarify how the 16 hierarchy residuals are obtained ?

Appreciate for further response. We have updated the pdf accordingly and here are the answer to the questions:

Q1) We consider to rename all the "Pruned DAv1" as "Downsized DAv1". Naive compression was also explained, following by a detailed explanation within parentheses. All changes were marked red.

Q2) We have add one more column 'FLOPS' in tab 1 and 2.

Q3) We adopt power-of-two channel scaling factors (i.e., 1/2, 1/4, and 1/8), which accounts for the minor discrepancy in model size before and after incorporating Depthwise Separable Convolutions (DSC). Without channel reduction, the model size would increase substantially without yielding a commensurate gain in final performance. Conversely, using non-power-of-two channel compression could degrade model performance,as channel counts that are not multiples of two often suffer from suboptimal hardware/representational efficiency, thereby compromising the fairness of the comparison. We will update the FLOPs and memory consumption metrics in Table 4 in the revised manuscript.

Q4) We apologize for our previous vague and incomplete response. What we intended to convey is that we initially set both the coarse bins and fine bins to 64 in an attempt to achieve higher precision. Our design philosophy is to hierarchically partition the model's estimation range, yielding a theoretical resolution of max_depth / (coarse_bins × fine_bins). Therefore, using 64 fine bins theoretically enables finer granularity and higher accuracy. However, our empirical results demonstrate that using 16 fine bins achieves the optimal trade-off between accuracy and model size.

Reviewer_kf6yOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a parameter-efficient algorithm for monocular depth estimation. Although the gains over other small/lightweight vision models are well established, comparisons of accuracy–efficiency trade-offs against larger models and some recent lightweight SOTA models are not sufficiently rigorous. Certain analyses, particularly the deployment evaluation, are useful for embodied applications, but the novelty of Deep-shifted Skip Connections (DSC) remains questionable.

Critical strengths

** Parameter efficiency is promising, particularly on KITTI. ** Practical deployment evaluation is useful and provides valuable results. ** The empirical evaluation of feature selection and skip connections is in-depth.

Critical weaknesses

** Reorganizing skip connections to favor deeper encoder layers is not a novel idea [1, 3]. DPT [2] already investigates the selection of skip connections from different encoder layers. Specifically, Table 6 of DPT reports that the best results are achieved by combining skip connections from shallow and deep layers. Given that the proposed method is based on the DPT architecture, the novelty of DSC is questionable. Although the authors provide empirical evidence supporting deeper connections in lightweight architectures, the underlying idea has already been studied. References [1] https://arxiv.org/abs/2402.08276 [2] https://openaccess.thecvf.com/content/ICCV2021/papers/Ranftl_Vision_Transformers_for_Dense_Prediction_ICCV_2021_paper.pdf [3] https://arxiv.org/pdf/2208.13441

** Four of the six numbered equations contain indexing inconsistencies or ambiguities that require clarification. Equation (3): The same index is used for both bin indexing and feature summation. These should be distinguished for clarity. Equation (4): The authors define (m) as the number of multi-scale features and (n) as the number of coarse bins. However, both summations iterate over (m). The summation bounds should be clarified, particularly given the different numbers of coarse and fine bins. Additionally, Equation (4) uses the original bin anchors rather than the adjusted anchors from Equation (3). Are the learned adjustments actually used in the final prediction? Equation (5): The indexing is out of range at (i=n-1), since it requires (b_n), which is undefined. The valid range should be from 0 to (n-2). Equation (6): The residual hierarchy repeats the last bin width, but the indexing is unclear. More importantly, Table 5 reports 64 coarse bins and 16 fine bins, while Equations (5)–(6) define a residual hierarchy containing (n) elements. The relationship between the 64 residual widths and the 16 fine bins is not explained. How are the fine bins constructed and used in the final prediction?

** Tables 1 and 2 report δ1\delta_1, δ2\delta_2, δ3\delta_3, AbsRel, and additional error metrics, including RMSE, SqRel, or log10. However, Table 6 reports only AbsRel and δ1\delta_1. Given the paper's claims of robust zero-shot generalization and its more comprehensive in-domain evaluation, the authors should justify the reduced set of metrics or provide additional results for a more complete assessment.

** Several relevant recent methods are not included in the main quantitative comparisons, including Depth Anything V2/V3, Depth Pro, UniDepthV2, VGGT/VGGT Omega, and ZipDepth. While not all these models are directly comparable under the same evaluation settings, their relevance to the claimed accuracy–efficiency trade-offs should be discussed, with quantitative comparisons where appropriate. ZipDepth is particularly relevant. It is included in Table 9 for efficiency comparison, reporting 66 FPS under PyTorch FP16, compared with 22 FPS for RePDepth-S under PyTorch FP32. Although the precision settings differ, ZipDepth is not included in the main in-domain or zero-shot accuracy comparisons, where it could provide a more recent and competitive lightweight baseline.

Questions for authors

Please check the weaknesses.

1 comment

While the concept of depth skip connections (DSC) is not entirely novel, our work distinguishes itself by applying it to the field of monocular depth estimation (MDE) under strict constraints on parameter count and computational complexity. The significance of this approach lies in guiding future lightweight MDE research to carefully consider the placement of skip connections during model design.

Our experiments maintain strict control over model scale. Under approximately identical model sizes, adopting DSC achieves higher performance while simultaneously reducing computational overhead and memory consumption compared to traditional multi-layer features. Furthermore, incorporating channel compression allows for superior performance with an even smaller model footprint. DPT only discusses skip connections for a single encoder at the scale of large models and lacks further investigation into the generalizability of this phenomenon across other methods. For instance, given that DPT utilizes a pure ViT network, it remains unaddressed whether the combination of shallow and deep skip connections applies to other foundation models, lightweight models, or CNN/ViT hybrid architectures. These questions are not explored in DPT. Adopting a combination of shallow and deep layers is unsuitable for lightweight models because processing at the original resolution consumes substantial computational and spatial resources. This is also indicated in ZipDepth, for they downsample the feature to 1/4 resolution. The discussion of skip connections in DPT appears primarily intended to justify its own architectural design and occupies only a minor portion of the paper. Nevertheless, we are willing to supplement our manuscript with comparative experiments evaluating the combination of deep and shallow layers.

Beyond demonstrating generalizability across multiple methods, the design of DSC is also reflected in its synergy with the ViT module of our proposed encoder to achieve enhanced feature extraction capabilities. In other words, DSC is not employed merely due to network depth; rather, to obtain higher performance, we determined that placing DSC after the ViT module yields a larger receptive field. This represents a rational design choice based on our specific model characteristics and general architectural principles. Although the provided references [1, 3] present similar conclusions, we note that one of these papers focuses on segmentation tasks, making its findings not directly transferable to the MDE domain. The FSCN proposed in the other paper functions more as an adaptive skip connection mechanism, which consumes extensive computational resources and is incompatible with lightweight model design ($\leq=$30M parameters).

We thank the reviewer for pointing out the notational flaws in the formulas. We have updated and clarified the definitions of indexes as follows: i, j are used for feature fusion. k is used for bin index. n is used as the overall number of the coarse and fine bins. The difference between coarse and fine bin is clearified with upper limit in definition. For instance, 0, ..., n-1or n-2. lowercase L is used for multi feature summation in bin adaptor. The number of coarse and fine bins should be the same in our design.

We will correct this issue in the revised manuscript.We appreciate the feedback regarding the tables. The utilization of key metrics such as δ1\delta_1 and Abs Rel for multi-dataset evaluation is not unique to our study and has been adopted in prior works [da1, da2, zipdepth]. Including all evaluation metrics in Table 6 would result in excessive visual clutter; therefore, we utilized a more streamlined set of indicators. We are willing to incorporate additional metrics subject to page limit constraints. Furthermore, Zipdepth yields relative depth results, whereas our method produces metric depth. Consequently, Zipdepth does not involve bin design, which accounts for the observed discrepancy in inference speed. Because relative depth methods hold a significant advantage over metric depth methods in zero-shot comparisons, we did not include a comparison with Zipdepth in this specific section. Finally, the PyTorch version of our model currently lacks code-level optimization, a limitation that is effectively mitigated by TensorRT quantization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.