- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes ProtoDN, an instance-level framework benefiting monocular 3D detection. Experiments on KITTI show the benefit of the approach.
Critical strengths
- Writing: The writing is easy to follow.
- Evaluation: Experiments on KITTI show the benefits of the approach.
Critical weaknesses
Central Claims
- Since the primary contribution of this paper is using instance-level information for challenging objects under occlusion, long-range observation (L018, L), it needs to be compared quantitatively on datasets which do have sufficient long-range dataset such as Argoverse-v2, ManTruckscenes and a more recent AevaScenes datasets. KITTI has very less objects in long-range observation range.
- Another central claim in the paper is the training stability of geometry-guided methods compared to distribution-driven methods (L050). I do not know if the appearance-aware geometry-guided methods make the convergence better or worse?. Please provide two plots showing AP3D 70 and AP3D 50 of your baseline, baseline + Geometry-Guided and ProtoDN from 1st to 250 epoch in steps of 10 epoch. Ideally, you should plot all these three methods over 3 random seeds. This will help us justify the convergence claim of ProtoDN.
Evaluation Protocol and Metrics:
- Are the KITTI test and val models listed in Table 1 trained on single class (Car) or 3 classes (Car, Pedestrian and Cyclist) ? If it was done in a single class, could you provide the ProtoDN KITTI Val/Test numbers for 3 classes?
- The last three rows show that ablation studies Tab. 3 shows that the impact of Visual Enhancement and Dynamic Update Module is minor. This is specially important since the Mono3D on KITTI is notably sensitive to initialization. I request that the authors report the median performance across three random seeds on the KITTI Val split at 250th epoch as in [A] to ensure the results are statistically significant and the modules are indeed useful. In other words, I do not want any checkpoint selection to be done.
- The IoU3D threshold of 0.7 of KITTI is a strict metric and is very sensitive. To provide a more comprehensive view of performance, please report KITTI Val results using a 0.5 threshold, consistent with MonoDLE [22] and DEVIANT [D].
Generalization and Scalability
- It would also be great if authors can compare on multi-camera benchmarks such as the nuScenes dataset leaderboard. I would advise the authors to quantitatively compare against SparseBEV [B], and HoP [C] on the nuscenes leaderboard.
- SeaBird [D] suggests that perspective-view detectors struggle with the large objects while BEV detectors are sub-optimal without dice loss. Since the new paper introduces the side-view, please compare the TriViewNet idea against PBEV+SeaBird on the KITTI-360 Val split and leaderboard.
- Evaluate on the Waymo Open Dataset to test robustness against camera pitch/roll noise (e.g., San Francisco's hilly terrain), similar to the evaluations in MonoUni and DEVIANT [A].
- Please provide cross-dataset performance metrics (e.g., mean absolute depth error) by testing the KITTI-trained model on the nuScenes frontal validation set, comparing it against MonoUni, and MonoCoP.
Misc:
- This paper would be difficult to reproduce without the code. Do the authors plan to release the code for all datasets and provide pre-trained, especially KITTI leaderboard models and logs.
References
- [A] DEVIANT, Depth Equivariant Network for monocular 3D object detection, Kumar et al., ECCV 2022.
- [B] SparseBEV, Liu et al., ICCV 2023
- [C] HoP: Temporal enhanced training of multi-view 3d object detector via historical object prediction, Zong et al, ICCV 2023
- [D] SeaBird, Segmentation in BEV with dice loss improves monocular 3D detection of large objects, Kumar et al., CVPR 2024.
Questions for authors
Please see the weaknesses section.
6 comments
Response to Reviewer t3vc (Part 2)
- Another central claim in the paper is the training stability of geometry-guided methods compared to distribution-driven methods (L050). I do not know if the appearance-aware geometry-guided methods make the convergence better or worse?. Please provide two plots showing 70 and 50 of your baseline, baseline + Geometry-Guided and ProtoDN from 1st to 250 epoch in steps of 10 epoch. Ideally, you should plot all these three methods over 3 random seeds. This will help us justify the convergence claim of ProtoDN.
We thank the reviewer for raising this concern. We would like to clarify that our statement regarding training stability highlights the benefits of query denoising in providing more direct supervision for query learning, thereby improving the training process. Both geometry-guided and distribution-driven approaches are denoising mechanisms, distinguished primarily by their noise construction strategies. We do not claim that either approach has an advantage over the other in terms of training stability. To address the reviewer's concern about the effect of appearance awareness on model convergence, we conducted additional experiments comparing our geometry-guided baseline with our method, ProtoDN, using three different random seeds. The convergence plots have been added to the revised manuscript in Figure 4 (Lines 810–823). As shown in the plots, ProtoDN exhibits convergence trends comparable to the baseline under both IoU thresholds, without noticeably larger performance fluctuations or additional training instability. These results suggest that incorporating appearance awareness does not compromise the convergence behavior of the original geometry-guided denoising framework.
- Are the KITTI test and val models listed in Table 1 trained on single class (Car) or 3 classes (Car, Pedestrian and Cyclist) ? If it was done in a single class, could you provide the ProtoDN KITTI Val/Test numbers for 3 classes?
We thank the reviewer for requesting clarification on the training settings and three-class evaluation. The KITTI validation and test results reported in Table 1 are obtained from models trained on the Car class only. To further evaluate the generalizability of ProtoDN across object categories, we additionally trained a three-class model and report the Pedestrian and Cyclist 3D AP results on the KITTI validation set below. Although the improvements over MonoDGP on Pedestrian are relatively limited, ProtoDN achieves substantial gains on Cyclist. Meanwhile, compared with the baseline, ProtoDN consistently improves performance across all difficulty levels for both Pedestrian and Cyclist. These results suggest that the proposed method can also benefit categories beyond Car. We have also included the three-class evaluation results in Table 7 (Lines 721–728) of the supplementary material in the revised manuscript.
| Method | Ped. Easy | Ped. Mod. | Ped. Hard | Cyc. Easy | Cyc. Mod. | Cyc. Hard |
|---|---|---|---|---|---|---|
| MonoDGP | 13.77 | 10.06 | 7.96 | 12.21 | 6.61 | 5.95 |
| Baseline | 13.04 | 10.08 | 7.56 | 16.22 | 8.47 | 7.23 |
| ProtoDN | 13.50 | 10.27 | 7.97 | 17.13 | 8.88 | 8.47 |
Response to Reviewer t3vc (Part 3)
4.The last three rows show that ablation studies Tab. 3 shows that the impact of Visual Enhancement and Dynamic Update Module is minor. This is specially important since the Mono3D on KITTI is notably sensitive to initialization. I request that the authors report the median performance across three random seeds on the KITTI Val split at 250th epoch as in [A] to ensure the results are statistically significant and the modules are indeed useful. In other words, I do not want any checkpoint selection to be done.
We thank the reviewer for the valuable feedback regarding the effectiveness of the proposed modules. Following the requested evaluation protocol, we conducted additional ablation experiments using three different random seeds and report the median performance on the KITTI validation set at epoch 250. The results shows that the ProtoDN model achieves the best performance across all six metrics. Removing Visual Enhancement or Dynamic Update decreases 3D AP on the Moderate split by 0.73 and 0.55, respectively. These results provide further evidence that both modules contribute to the overall performance improvements, with their combination achieving the best results.
We note that reporting the median performance across three random seeds at 250th epoch is seldom used in this field. This evaluation protocol may lead to lower absolute performance, as further corroborated by the additional baseline results. In our paper, all compared methods follow the same checkpoint-selection strategy to ensure a fair comparison.
| Method | Joint Diff. | Visual Enh. | Dynamic Upd. | AP$_{BEV}$ Easy | AP$_{BEV}$ Mod. | AP$_{BEV}$ Hard | AP$_{3D}$ Easy | AP$_{3D}$ Mod. | AP$_{3D}$ Hard |
|---|---|---|---|---|---|---|---|---|---|
| Baseline | 39.37 | 28.96 | 25.73 | 30.22 | 22.68 | 19.82 | |||
| ProtoDN | ✓ | ✓ | 39.93 | 29.68 | 26.14 | 30.47 | 23.50 | 20.10 | |
| ProtoDN | ✓ | ✓ | 40.16 | 29.47 | 26.12 | 31.07 | 23.32 | 20.22 | |
| ProtoDN | ✓ | ✓ | ✓ | 40.68 | 30.19 | 27.22 | 32.34 | 24.05 | 20.58 |
- The IoU3D threshold of 0.7 of KITTI is a strict metric and is very sensitive. To provide a more comprehensive view of performance, please report KITTI Val results using a 0.5 threshold, consistent with MonoDLE [22] and DEVIANT [D].
We thank the reviewer for suggesting additional evaluations at a lower IoU threshold. Following the evaluation settings of MonoDLE and DEVIANT, we report the KITTI validation results at IoU = 0.5, as shown in the table below. The results show that ProtoDN outperforms MonoDLE and DEVIANT by clear margins, while also achieving consistent improvements over our baseline in both 3D AP and BEV AP across all difficulty levels. These results further support the effectiveness of ProtoDN under a less restrictive IoU threshold.
| Method | AP$_{BEV}$ Easy | AP$_{BEV}$ Mod. | AP$_{BEV}$ Hard | AP$_{3D}$ Easy | AP$_{3D}$ Mod. | AP$_{3D}$ Hard |
|---|---|---|---|---|---|---|
| MonoDLE | 60.73 | 46.87 | 41.89 | 55.41 | 43.42 | 37.81 |
| DEVIANT | 65.28 | 49.63 | 43.50 | 61.00 | 46.00 | 40.18 |
| Baseline | 72.58 | 56.00 | 51.20 | 68.79 | 52.65 | 47.98 |
| ProtoDN (Ours) | 73.73 | 56.63 | 52.13 | 69.76 | 53.21 | 48.77 |
Response to Reviewer t3vc (Part 1)
We thank the reviewer for suggesting an evaluation of our method's long-range detection capability. Given the time and computational resource constraints, we conducted additional experiments on the Waymo Open Dataset, which provides a suitable benchmark for evaluating long-range detection performance and enables direct comparisons with existing monocular 3D object detection methods. Following the evaluation protocol reported in MonoCoP, we report performance across different distance ranges to assess the effectiveness of ProtoDN for long-range detection. As shown in the following table, ProtoDN outperforms the compared methods in both the 30–50 m and 50 m+ ranges. These results provide further evidence of the effectiveness of ProtoDN for long-range monocular 3D object detection.