- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents a mixture-density depth representation that explicitly models the depth ambiguity near the boundary. Experiments show the flying points are significantly reduced, and the quantitative errors such as AbsRel are also reduced.
Critical strengths
- From table 1, the method seems to be very effective, reducing Chamfer Distance (CD) significantly while increasing little computational budget.
- The method is well motivated and solves the transparent object depth and sky depth.
Critical weaknesses
- Limited novelty: the main idea of mixture depth density seems to be too similar to the prior work SMDNet [1]. Increasing the number of clusters or applying to different backbones seem to be trivial.
- During inference, the cluster index of each pixel is selected (by argmax) independently. Will this result in uneven and bumpy surfaces when nearby pixels are assigned into different clusters? It would be beneficial to add metrics such as smoothness during evaluation.
[1] SMD-Nets: Stereo Mixture Density Networks
Questions for authors
- Could the author explain more substantial difference with respect to SMDNet?
3 comments
Thank you again for your review. As the discussion period is ending soon, we wanted to check whether our response and the revised paper address your two concerns, the difference from SMD-Nets and whether per-pixel selection create uneven, bumpy surfaces. If anything is still unclear, we would be glad to clarify it in the remaining time.
Best regards,
The authors
Thank you for your reply
Thank you for your reply. My concern regarding the uneven and bumpy surfaces has been addressed. However, I still find the novelty insufficient. A similar mixture density formulation was already proposed in SMD-Net. As noted in line 266 of your paper ("Our mixture-density depth representation is architecture-agnostic"), your proposed modifications for scaling to large pretrained depth models—such as the staged finetuning recipe—appear incremental. Therefore, I will maintain my rating.
We thank the reviewer for the comments. New results are in the revised PDF (new text in blue; new tables and figures in App. E).
Q1. Difference from SMD-Nets.
Thank you for pointing this out. We have discussed SMD-Nets in the introduction and related work. SMD-Nets models stereo disparity with a two-component mixture. Our work differs from and extends it in two ways:
With this recipe, about 11k finetuning steps on 4 GPUs (compared with 128 H100s used to train DA3) are enough to outperform the original pretrained models at boundaries. MDA also even outperforms methods designed specifically for this problem, such as PPD for flying points (Tables 1 and 4).
We refer to the blog [1] from Michael J. Black discussing the different forms that novelty can take. Our work fits the described "novelty". First, "a new use of an old method can be novel if nobody ever thought to use it this way": Mixture densities are well known, but to our knowledge they have not been used to remove flying points in modern depth models, or to represent co-existing transparent layers and unbounded sky depth. Second, "if a paper has a simple idea that works better than the state of the art, then it is most likely not trivial": MDA changes only the output layer, yet with simple finetuning it improves the boundary quality of state-of-the-art models.
W2. Can per-pixel selection create uneven, bumpy surfaces?
Per-pixel selection causes only subtle bumps. A bump appears only when neighboring pixels on the same surface select components that predict different depths. In our experiments, we find that this is rare away from boundaries, because the components there almost always predict the same depth and it barely matters which one is selected. Only 0.2% of non-boundary pixels switch to a component whose depth differs by more than 1%. Fig. R1 (App. E.6) visualizes this on planar regions. In the zoomed-in normal maps, MDA is about as smooth as the pretrained model. The selected component sometimes changes inside a plane (e.g., the carpet in the third row). Because the components predict nearly the same depth there, the normals show no seam.
Quantitative results confirm this. Normal consistency (NC), the cosine similarity between predicted and GT surface normals, is sensitive to small bumps, so bumpy surfaces would lower it. MDA has higher NC than the single-head DA3 finetuned in the same setting (Table R3, App. E.3, also listed below). On VGGT, MDA achieves NC on par with or better than the single-head VGGT (Table 5):
References
[1] M. J. Black. Novelty in Science. Medium, 2022. https://medium.com/@black_51980/novelty-in-science-8f1fd1a0a143