- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper tracles the problem of high-resolution depth estimation. The core idea is to retain depth-aware latent representations as guidance while generating the final depth map directly in pixel space. The pixel branch generating depth at the target resolution bypasses the original VAE decoder, which gets rid of substantial inference overhead. The trained model achieves SoTA performance on Synth4K.
Critical strengths
- EagleDepth achieves the SoTA on the Synth4K benchmark.
- Exploring to replace the VAE decoder with other advanced techique like PiD for depth is an interesting direction.
Critical weaknesses
- Exploring to replace the VAE decoder with other advanced techique like PiD for depth is an interesting direction (It's a strength I listed above). But the solution is less informative. As far as I understand from the paper, the proposed solution is merely tuning the PiD for depth estimation without any depth specific adaptation. All claims are somehow overlapped with the PiD and I do not see much new knowledge that this paper can bring to the feild.
- The authors use FLUX.2-klein4B (Labs, 2025) as the base latent and fine-tune PiD for diffusion decoder. FLUX.2-klein4B is a very strong base model and it's nothing related to the contributions in this paper. It will raise the concern that EagleDepth achieves the SoTA only becaue it uses a stronger model as the base model. As shown in Tab.4, after using a weaker latent model, the performance is no longer best.
- The training data is also somehow different from previous work with more synthetic data. This can also lead to concerns about comparison.
Questions for authors
Please reply to my Critical Weaknesses. I would like to know if my judgement about the techique contribution is reasonable. I would also like to hear the authors' opinions about the influence of scaling up the base model capacity and data volumn. These points will influence my final grade.
2 comments
Response to Reviewer wD4G (denoted as R3) - Part 2
W3-3 Differences in training data and the use of more synthetic data.
Response Thank you for your comment. Different depth estimation methods use different training datasets, and we seek a balance between training efficiency and performance. Among our training datasets, Rawmantic3D is the only one that we introduce to depth estimation for the first time, motivated by our high-resolution training objective. All other datasets have been used in previous depth estimation methods. As detailed in Appendix B, we filter and subsample the datasets to obtain 536K RGB–depth pairs, a training set smaller than that used by most methods (e.g. DA3 and MoGe-2).
To provide a fairer comparison, we include additional reduced-data training settings in the comparison below:
| Method | Training data | ETH3D | DIODE |
|---|---|---|---|
| Marigold | 74K | 6.5 / 96.0 | 10.0 / 90.7 |
| Lotus-2 | 59K | 5.1 / 97.5 | 6.1 / 95.2 |
| PPD | 125K | 3.2 / 99.1 | 4.1 / 97.7 |
| MoGe-2 | 8.9M | 3.5 / 98.7 | 4.7 / 96.4 |
| Ours | Marigold-like | 3.6 / 98.8 | 4.8 / 96.9 |
| Ours | PPD-like | 3.4 / 98.9 | 4.1 / 97.9 |
| Ours (1024) | 536K | 3.1 / 99.2 | 3.9 / 97.9 |
These results show that our method remains competitive when trained on the same amount of data.
We also need to emphasize that, although PPD itself is trained on relatively few samples, it relies on pretrained vision foundation models (DA2 and MoGe-2) to provide priors. These models have been trained on millions of images with depth or 3D geometry supervision. In contrast, our method only uses 536K RGB–depth pairs for depth/geometry supervision.
Q3-1 Technical contribution and the influence of scaling up base model capacity and training data volume.
Response Thank you for your thoughtful assessment. Our goal is to develop a depth estimation model that makes accurate predictions and generalizes well. This involves engineering choices about the training data and the base model. At the same time, our method offers some novel contributions and brings substantial performance gains, as discussed in our response to W3-1.
Regarding base model capacity and training data volume, we agree that scaling them up can improve performance, consistent with scaling laws. Nevertheless, our core contribution also plays a crucial role in achieving these results, as discussed in our responses to W3-2 and W3-3.
Response to Reviewer wD4G (denoted as R3) - Part 1
W3-1Differences from PiD and the paper's contributions.ResponseThanks for your comment. As stated in the paper, our core design is to adapt a pixel diffusion decoder to decode low-resolution depth latents into detailed, high-resolution depth maps. Our work offers two main contributions:We demonstrate the effectiveness of pixel diffusion decoding for depth estimation, with particularly strong results in fine-detail recovery.
We identify and address challenges in adapting pixel diffusion decoding to depth estimation (a key focus of your question). Specifically, our method differs from the original PiD in two aspects:
(a) Patch continuity. Each patch token corresponds to a 16 × 16 pixel region that may span sharply different foreground and background depths (such depth transitions are often much more abrupt than the corresponding appearance changes in natural RGB images). Decoding these tokens without local spatial mixing can produce visible patch discontinuities near object boundaries, as shown in Figure 4. We introduce a continuity-promoting module to suppress these artifacts.
(b) RGB input. In text-to-image generation, the pixel diffusion decoder takes noise as its pixel input and relies on high-resolution priors learned during training to supply fine details. For depth estimation, we instead use the high-resolution RGB image as the pixel input, allowing the decoder to also obtain fine-detail information directly from the observed image.
W3-2SOTA performance may be attributable solely to the latent base model, FLUX.2-klein-4B.ResponseThank you for your comment. The pixel diffusion decoder complements the overall depth accuracy provided by coarse depth-latent predictions by recovering high-resolution details. The quality of the latent depth model is therefore important. We also acknowledge that our optimization of the latent model is an engineering contribution rather than a fundamentally new method. Nevertheless, EagleDepth's performance is not attributable solely to the latent model. Table 4 of the main paper demonstrates two benefits of pixel diffusion decoding:Even with a weaker latent model such as Lotus-2, our method remains competitive. For example, in terms of overall depth accuracy, ours (Lotus-2) achieves a mean δ1 of 79.3 over Synth4K-1/2, approaching PPD (83.3) and URGT (84.3) while outperforming all other baselines. In terms of fine-detail recovery, pixel diffusion decoding improves the mean boundary F1 from 17.0 to 21.1, outperforming PPD (16.7).
In addition, PPD, the strongest compared baseline, also relies on external pretrained models. We provide the following comparison using different external models:
ETH3D and DIODE report AbsRel ↓ / δ1 ↑; Synth4K reports AbsRel ↓ / δ1 ↑ / AbsRelH ↓ / δ1H ↑ / boundary F1 ↑.