- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a discriminative single-pass model for monocular depth estimation. The idea is to separate two roles: a large-patch ViT gives global context, and a second network works on full-resolution pixel features, guided by that context, to predict the final depth. The authors say that standard ViT + convolutional decoder models lose fine structures because depth is rebuilt from low res features. They claim better local geometry than discriminative models, competitive global accuracy, and lower latency than generative models. The model is trained only on synthetic data and evaluated zero-shot on several benchmarks, with both global and boundary metrics.
Critical strengths
- The problem is relevant (fine structures and flying points in point clouds), and the motivation is clear. The paper is well written and easy to follow.
- The idea is simple and works in a single forward pass, with no iterative sampling. Thin structures (eg poles) look better in many qualitative examples.
- The evaluation is wide: many baselines, two benchmarks, and boundary metrics also on real data.
- The authors are honest in the appendix: they compare with the closest recent works (Moge3, SurGe), they show the center-crop comparison with PointDiT, and they show worse results where they exist (for example Moge2 on global accuracy). The limitations are stated.
- Compared with MDA (a much larger model), the boundary quality is similar with lower latency. Compared with generative pixel-space methods, the latency is clearly lower at the tested resolution.
- Training and evaluation protocols are described in detail and the authors promise code and weights.
Critical weaknesses
-
The comparison with the baselines is not controlled. PXDepth is trained only on synthetic data with its own loss, and starts from moge2, while the baselines have other data and other training. The ablation (table 3b) has a controlled comparison with a conv decoder, but at small scale, and the gain is small. So it is not clear how much of the gain in table 1-2 comes from the architecture. HiRoom CD is 54.9 in tab.2 and 72.5 in table 3, so the training setup changes the results more than the decoder. I would like to see MOGe2 (conv decoder) trained with the same data, loss and budget, in the main tables.
-
It is not shown that the proposed idea is what matters. The only baseline is the moge2 decoder, which is much cheaper, so a bigger model could do better for any design. There is no baseline with similar compute and a simple design (for example rgb-guided upsampling, or a decoder with full-resolution skip connections on the same ViT). There is also no test that separates "full-resolution features" from "more transformer layers after the ViT" (for example the same predictor on a low-resolution stream, or with blurred RGB input). In table 3c the three guidance variants may have a different number of parameters, and no parameter count is given. If so, the gain of the guidance can be only capacity.
-
The efficiency claim is shown only at low resolution. Latency is measured at 518x518 (48.9 ms), but at 1022x770 it is 193 ms. I expect the cost of the attention grow with the square of the number of tokens, so 2K or 4K images may be very expensive. This is the setting where fine structures matter more. The model is also 2x times slower than Moge, and the generative baselines are tested only with 3-4 steps (what about 1?). The paper should show latency, memory and accuracy vs resolution for all methods, in the same conditions, and parameters and FLOPs for all methods.
-
The comparison with PointDiT depends on the protocol. With the center crop protocolPointDiT-large is better than PXDepth on average. In tabel 1 PointDiT is resized without keeping the aspect ratio and loses. So the claim of better accuracy than PointDiT is not valid in the other protocol, and this result is only in the appendix. Also PXDepth starts from MoGe-2, while PointDiT uses DINOv3 features and is trained from scratch, and the DINOv3 variant of PXDepth is clearly worse.
-
Global accuracy is below the closest model...PXDepth is below moge2 (its own initialization) on 6 of 8 datasets in delta1. The differences are larger on real driving data . This could be the cost of synthetic training, but the paper only says mog2 has more diverse data and does not test it. The word "competitive" is correct for a small gap but it should be discussed more.
-
The boundary gain over the best baseline is small. against moge2, Infinidepth, PPD and PointDiT the gain is large, but against MDA mean CD is 70.6 vs 72.8, Acc is worse and on 7Scenes (the only real one) MDA is also better in CD. Since CD is the average of acc and completeness the CD win may come from completeness. Please report completeness.
-
the metrics do not directly measure the main claim. The claim is about structural distortion and local geometry, but the paper reports global metrics and boundary acc/CD (after ICP, on GT edge pixels). As far as I know, recent works on the same problem also use boundary F1/SEE (see marigold v2), local relative error and normal error. Distortion of thin structures or smooth surfaces and flying pixels are not measured directly. The MDA protocol is fine as a starting point, but alone it is not enough.
Some considerations about the eval protocol:
- The inference resolution of the proposed method for the accuracy results is not stated. If it is about 1022x770, the latency in table 2 is not the right one. It is also not clear if the predicted masks are used in the boundary evaluation (and the masks of the baselines).
- The naming is inconsistent: the caption of fig1 calls InfiniDepth, MDA and PPD "structure-aware", but tables 1-2 put them in "Global Prediction" and only "Ours" in "Structure Aware". So the sentence "better global accuracy than structure-aware methods" compares the method with itself.
- Fig. 1 does not include moge2 (better delta1 and faster) and mixes numbers from two benchmarks. PointDiT is missing in the qualitative figures. MDA has a strange outlier on DDAD (rel 21.8) that lowers its mean..please check.
Related work / comparisons issues:
- The natural baselines for "low-resolution features to full resolution" (guided or joint upsampling, learned convex upsampling) are not discussed.
- PointDiT, Moge3 and Surge should be in the main tables, not in the appendix (I do not penalize the last two if they are concurrent).
Some minor points:
- Missing details: batch size, data augmentation, initialization of the modulation layers, size of the feed-forward layers, etc.
- The paper says latent diffusion models lose structure because of the VAE, but no such model is compared. A recent latent-DiT depth model that the paper already cites (for example Marigoldv2 or Lotus2) would test this claim.
- The context features come from the sum of 4 layers (D.1), but the main text does not say it.
Questions for authors
- To show that the gain comes from the design: MoGe-2 (conv decoder) and PXDepth trained with the same data, loss and budget, in the main tables. Also a baseline with similar compute and a simple design (RGB-guided decoder), and the same predictor with a low-resolution stream. What is the real source of the boundary gain?
- Efficiency in the settings where the method should help: latency, memory and accuracy vs resolution (up to at least 2K), parameters and FLOPs of all methods, same measurement protocol, and the generative methods with fewer steps.
- Metrics that measure the claim: boundary F1, local error or normals, flying pixels, and completeness separately. Does the gain over MDA stay?
- PointDiT with its own protocol in the main tables. Does the claim of better accuracy still hold, also with a DINOv3-initialized PXDepth?
- How do you explain the drop on real data (DDAD, KITTI) with respect to MoGe-2? Can a small amount of real or pseudo-labeled data fix it?
- At which resolution are the accuracy numbers computed, and are predicted masks used in the boundary evaluation?
My current score is weak reject, mainly because the gains are not clearly attributable to the design (data, loss, capacity), and because the efficiency claim holds only at low resolution. The idea is reasonable and the real-data boundary results are interesting. I can raise the score if the rebuttal gives the controlled comparison with MoGe-2 and the simple-design baselines at similar compute.
POST REBUTTAL: In light of the authors’ rebuttal and the additional clarifications provided, I have decided to raise my rating from Weak Reject to Weak Accept. The rebuttal satisfactorily addresses my main concerns.
10 comments
Response to Reviewer k4Bv (2/4)
W3 & Q2: Efficiency and Model Capacity Comparison Across Different Resolutions and Generative Sampling Steps
A: Thank you for suggesting these comparisons, which help make our evaluation fairer and more comprehensive.
We report parameter counts, FLOPs, latency, and memory usage across different input resolutions and numbers of generative sampling steps on an RTX 5880, as shown in Table R2 and Table R3. PXDepth achieves lower latency than the multi-step generative baselines at both resolutions. The results have been added to Table 12 in Appendix F.2 of the revised paper.
Our current high-resolution training stage targets approximately 1K resolution, comparable to the setting used by PPD, while PointDiT operates at a resolution of 512 × 512. Scaling structure-preserving depth prediction to 2K and beyond remains challenging, particularly when balancing structural fidelity and computational efficiency. We plan to explore higher-resolution training and inference in future work.
Table R2. Efficiency comparison at 518 × 518 resolution on an RTX 5880.
| Category | Method | Params (M) | FLOPs (T) | Latency (ms) | Memory (GiB) |
|---|---|---|---|---|---|
| Global Accuracy | DA V2 | 335.3 | 1.3 | 20.4 | 1.7 |
| DepthPro | 952.0 | 19.2 | 223.7 | 3.7 | |
| MoGe-2 | 330.9 | 1.4 | 20.5 | 1.6 | |
| Structure Aware | InfiniDepth | 307.2 | 1.6 | 61.6 | 1.9 |
| PointDiT-L (Step=1) | 771.5 | 1.5 | 51.8 | 3.8 | |
| PointDiT-L (Step=2) | 771.5 | 2.2 | 76.0 | 3.8 | |
| PointDiT-L (Step=3) | 771.5 | 2.9 | 104.6 | 3.8 | |
| PointDiT-L (Step=4) | 771.5 | 3.7 | 125.5 | 3.8 | |
| PPD (Step=1) | 804.2 | 3.2 | 72.2 | 4.0 | |
| PPD (Step=2) | 804.2 | 5.7 | 118.0 | 4.1 | |
| PPD (Step=3) | 804.2 | 8.3 | 163.0 | 4.1 | |
| PPD (Step=4) | 804.2 | 10.8 | 206.9 | 4.1 | |
| MDA | 1355.7 | 4.0 | 100.1 | 6.3 | |
| Ours | 687.0 | 3.3 | 48.9 | 2.9 |
Table R3. Efficiency comparison at 1022 × 770 resolution on an RTX 5880.
| Category | Method | Params (M) | FLOPs (T) | Latency (ms) | Memory (GiB) |
|---|---|---|---|---|---|
| Global Accuracy | DA V2 | 335.3 | 4.9 | 56.5 | 2.4 |
| DepthPro | 952.0 | 19.2 | 223.7 | 3.7 | |
| MoGe-2 | 330.9 | 5.1 | 71.5 | 2.4 | |
| Structure Aware | InfiniDepth | 307.2 | 5.3 | 163.7 | 2.1 |
| PPD (Step=1) | 804.2 | 15.6 | 287.6 | 4.5 | |
| PPD (Step=2) | 804.2 | 28.4 | 451.3 | 4.5 | |
| PPD (Step=3) | 804.2 | 41.2 | 621.7 | 4.5 | |
| PPD (Step=4) | 804.2 | 53.9 | 783.9 | 4.5 | |
| MDA | 1355.7 | 14.2 | 222.9 | 8.6 | |
| Ours | 687.0 | 15.4 | 190.0 | 3.5 |
W6, W7 & Q3: Metrics that measure the claim: boundary F1, local error or normals, flying pixels, and completeness separately.
A: We thank the reviewer for suggesting these additional evaluation metrics, which enable a more comprehensive assessment of local structural fidelity. Following this suggestion, we additionally report completeness, boundary F1, Local Rel, Local , and Normal Error, together with the original Acc and CD metrics.
Specifically, boundary F1 evaluates the alignment of depth discontinuities in 2D. Acc, completeness, and CD measure geometric fidelity near object boundaries. Local Rel and Local assess local geometric fidelity. Normal Error evaluates surface orientation consistency. These metrics are evaluated on 7Scenes, HiRoom, NRGBD, iBims-1, and HAMMER.
As shown in Table R4, PXDepth achieves the best completeness, CD, boundary F1, Local Rel, and Local , while maintaining competitive Acc and Normal Error. These results provide more direct evidence that PXDepth better preserves local structures and reduces structural distortions. We have added the average results to Table 2 and the per-dataset breakdowns to Tables 19 and 20 of the revised paper.
Table R4. Comprehensive evaluation of boundary fidelity and local geometric fidelity on five datasets.
| Category | Method | Acc (mm) | Completeness (mm) | CD (mm) | F1 () | Local Rel | Local | Normal () |
|---|---|---|---|---|---|---|---|---|
| Global Prediction | DA V2 | 102.7 | 131.3 | 117.0 | 16.65 | 2.06 | 61.95 | 21.91 |
| DepthPro | 99.9 | 122.7 | 111.3 | 15.98 | 2.03 | 63.78 | 24.14 | |
| MoGe-2 | 90.1 | 115.1 | 102.6 | 13.27 | 2.11 | 63.99 | 22.16 | |
| Structure Aware | PPD | 80.2 | 155.0 | 117.6 | 19.44 | 2.05 | 63.84 | 37.38 |
| InfiniDepth | 92.9 | 140.2 | 116.5 | 19.32 | 2.17 | 60.26 | 24.59 | |
| MDA | 64.7 | 104.7 | 84.7 | 18.22 | 2.47 | 63.26 | 27.04 | |
| SurGe | 71.9 | 97.5 | 84.7 | 18.20 | 2.03 | 69.60 | 20.23 | |
| MoGe-3 | 79.6 | 110.4 | 95.0 | 15.40 | 2.01 | 66.46 | 21.13 | |
| Ours | 66.5 | 94.7 | 80.6 | 22.59 | 1.91 | 71.22 | 22.04 |
Response to Reviewer k4Bv (3/4)
W4 & Q4: Comparison with PointDiT and initialization of PXDepth
A: Thank you for pointing out this issue, which helps us make a fairer comparison with PointDiT. We have moved the comparison with PointDiT under its official center-crop protocol from the appendix to Tables 3 and 4 in the main paper, and removed the PointDiT results obtained under the mismatched evaluation protocol from Tables 1 and 2. PointDiT-L is restricted to a fixed input resolution of 512 × 512, whereas PXDepth supports varying input resolutions and aspect ratios. Under this shared evaluation protocol, PXDepth achieves better global depth accuracy and local geometric fidelity on the MDA Benchmark, although PointDiT-L achieves slightly better average global depth accuracy on the MoGe Benchmark, as shown in Tables 3 (Table R5) and 4 (Table R6). Regarding initialization, MoGe-2 initialization accelerates convergence, but the DINOv3-initialized PXDepth also produces structure-preserving depth predictions after convergence.
Table R5. Center-crop comparison with PointDiT on the MoGe Benchmark.
| Method | Rel | |
|---|---|---|
| PointDiT-L (3 steps) | 5.2 | 95.3 |
| Ours | 5.5 | 95.0 |
Table R6. Center-crop comparison with PointDiT on the MDA Benchmark.
| Method | Rel | Acc | CD | F1 | Local Rel | Local | Latency (ms) | |
|---|---|---|---|---|---|---|---|---|
| PointDiT-L (3 steps) | 3.4 | 97.1 | 77.9 | 75.4 | 22.90 | 1.94 | 68.44 | 104.6 |
| Ours | 3.0 | 97.5 | 56.6 | 63.3 | 27.72 | 1.83 | 71.96 | 48.9 |
W5 & Q5: How do you explain the drop on real data (DDAD, KITTI) with respect to MoGe-2? Can a small amount of real or pseudo-labeled data fix it?
A: We attribute this gap mainly to our synthetic-only training, while MoGe-2 benefits from more diverse training data. This is also supported by our controlled comparison in Table 6 (as shown in Table R7), where MoGe-2-Synth shows lower global depth accuracy than the original MoGe-2. Incorporating real-world or pseudo-labeled data is a promising direction to mitigate this gap, which we plan to explore in future work. We have further discussed the global accuracy gap between MoGe-2 and PXDepth in Section 5.3 of the revised paper.
Table R7. Effect of synthetic-only training on global depth accuracy and local geometric fidelity.
| Method | Rel | Acc (mm) | CD (mm) | Local Rel | Local | |
|---|---|---|---|---|---|---|
| MoGe-2 | 5.13 | 95.59 | 90.1 | 102.6 | 2.11 | 63.99 |
| MoGe-2-Synth | 5.97 | 94.54 | 98.9 | 104.2 | 2.11 | 65.29 |
| Ours | 5.75 | 94.84 | 66.5 | 80.6 | 1.91 | 71.22 |
Q6: At which resolution are the accuracy numbers computed, and are predicted masks used in the boundary evaluation?
A: Sorry for the confusion. In the original paper, we already specified the dataset-specific evaluation resolutions in Appendix E.2 (Table 10). We evaluate accuracy at these resolutions. For boundary evaluation, PXDepth and all baselines use ground-truth masks rather than predicted masks.
Response to Reviewer k4Bv (1/4)
We appreciate the time and effort you devoted to this detailed review. Your comments have helped us strengthen the architectural validation and provide a more comprehensive assessment of geometric fidelity and computational efficiency.
A: Thank you for suggesting this controlled experiment, which is essential for validating our core claim. We retrain MoGe-2 with a randomly initialized convolutional decoder (MoGe-2-Synth) to ablate the effect of synthetic-only training. We also compare PXDepth with a baseline combining the MoGe-2 backbone and a full-resolution RGB-guided convolution-based decoder, with a total parameter count comparable to that of PXDepth. The RGB-guided decoder adopts a U-Net-style architecture with RGB skip connections, while global context from the MoGe-2 backbone is injected into the bottleneck through adaptive normalization. Further details are provided in Appendix D.5 and Figure 10. Both baselines use the same data, loss functions, and training budget as PXDepth.
We evaluate Rel and δ1 on the MoGe Benchmark, and Acc, CD, boundary F1, Local Rel, and Local δ0.01 on 7Scenes, HiRoom, NRGBD, iBims-1, and HAMMER. Rel and δ1 assess global depth accuracy. Acc and CD evaluate 3D boundary geometry, boundary F1 measures 2D boundary alignment, and Local Rel and Local δ0.01 assess local geometric fidelity. The results have been added to Table 6 of the revised paper.
As shown in Table R1 and Fig. 7, the RGB-guided convolution-based baseline improves local geometric fidelity, indicating that increased model capacity and RGB skip connections can benefit local structure preservation. However, structural distortions caused by convolutional decoding still remain. Similarly, synthetic-only training slightly improves MoGe-2's local structures but cannot fundamentally resolve these distortions. In contrast, PXDepth achieves better boundary and local geometric fidelity while preserving fine-grained structures. These results support the contribution of our pixel-space design beyond increased model capacity, full-resolution skip connections, or synthetic-only training.
Table R1. Controlled comparison of PXDepth with RGB-guided decoder baselines.