acceptodds
Under review as a conference paper at ICLR 2027

ProtoDN: Appearance-Aware Denoising with Dynamic Visual Prototypes for Monocular 3D Object Detection

Abstract

Monocular 3D object detection is inherently challenging due to the depth ambiguity and incomplete observations from a single view. Recent denoising-based monocular 3D detectors focus on improving query learning and training stability. However, their denoising strategies remain largely centered on distribution modeling and geometric guidance, leaving instance-level visual information underexploited and limiting the modeling of challenging objects under occlusion, long-range observation, and truncation. To address this issue, we propose \method, an appearance-aware denoising framework with dynamic visual prototypes. We first introduce a dynamic visual prototype memory to aggregate class-specific visual priors. Building on this memory, we develop two core modules: (i) Geometry–Visual Joint Difficulty Modeling integrates geometric and visual cues to estimate instance difficulty and modulates denoising perturbations accordingly, enabling visually informed difficulty-aware denoising. (ii) Prototype-Guided Visual Enhancement leverages visual prototypes as instance-relevant appearance priors and explicitly injects them into denoising representations, compensating for insufficient visual information during reconstruction. Extensive experiments demonstrate superior performance of our method over recent approaches, with particularly notable improvements on challenging instances.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Monocular 3D Object Detection, Denoising Training, Difficulty-Aware Learning, Visual Prototypes

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_t3vcOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes ProtoDN, an instance-level framework benefiting monocular 3D detection. Experiments on KITTI show the benefit of the approach.

Critical strengths
  • Writing: The writing is easy to follow.
  • Evaluation: Experiments on KITTI show the benefits of the approach.
Critical weaknesses

Central Claims

  • Since the primary contribution of this paper is using instance-level information for challenging objects under occlusion, long-range observation (L018, L), it needs to be compared quantitatively on datasets which do have sufficient long-range dataset such as Argoverse-v2, ManTruckscenes and a more recent AevaScenes datasets. KITTI has very less objects in long-range observation range.
  • Another central claim in the paper is the training stability of geometry-guided methods compared to distribution-driven methods (L050). I do not know if the appearance-aware geometry-guided methods make the convergence better or worse?. Please provide two plots showing AP3D 70 and AP3D 50 of your baseline, baseline + Geometry-Guided and ProtoDN from 1st to 250 epoch in steps of 10 epoch. Ideally, you should plot all these three methods over 3 random seeds. This will help us justify the convergence claim of ProtoDN.

Evaluation Protocol and Metrics:

  • Are the KITTI test and val models listed in Table 1 trained on single class (Car) or 3 classes (Car, Pedestrian and Cyclist) ? If it was done in a single class, could you provide the ProtoDN KITTI Val/Test numbers for 3 classes?
  • The last three rows show that ablation studies Tab. 3 shows that the impact of Visual Enhancement and Dynamic Update Module is minor. This is specially important since the Mono3D on KITTI is notably sensitive to initialization. I request that the authors report the median performance across three random seeds on the KITTI Val split at 250th epoch as in [A] to ensure the results are statistically significant and the modules are indeed useful. In other words, I do not want any checkpoint selection to be done.
  • The IoU3D threshold of 0.7 of KITTI is a strict metric and is very sensitive. To provide a more comprehensive view of performance, please report KITTI Val results using a 0.5 threshold, consistent with MonoDLE [22] and DEVIANT [D].

Generalization and Scalability

  • It would also be great if authors can compare on multi-camera benchmarks such as the nuScenes dataset leaderboard. I would advise the authors to quantitatively compare against SparseBEV [B], and HoP [C] on the nuscenes leaderboard.
  • SeaBird [D] suggests that perspective-view detectors struggle with the large objects while BEV detectors are sub-optimal without dice loss. Since the new paper introduces the side-view, please compare the TriViewNet idea against PBEV+SeaBird on the KITTI-360 Val split and leaderboard.
  • Evaluate on the Waymo Open Dataset to test robustness against camera pitch/roll noise (e.g., San Francisco's hilly terrain), similar to the evaluations in MonoUni and DEVIANT [A].
  • Please provide cross-dataset performance metrics (e.g., mean absolute depth error) by testing the KITTI-trained model on the nuScenes frontal validation set, comparing it against MonoUni, and MonoCoP.

Misc:

  • This paper would be difficult to reproduce without the code. Do the authors plan to release the code for all datasets and provide pre-trained, especially KITTI leaderboard models and logs.

References

  • [A] DEVIANT, Depth Equivariant Network for monocular 3D object detection, Kumar et al., ECCV 2022.
  • [B] SparseBEV, Liu et al., ICCV 2023
  • [C] HoP: Temporal enhanced training of multi-view 3d object detector via historical object prediction, Zong et al, ICCV 2023
  • [D] SeaBird, Segmentation in BEV with dice loss improves monocular 3D detection of large objects, Kumar et al., CVPR 2024.
Questions for authors

Please see the weaknesses section.

6 comments
AuthorsOpenReview ↗

Response to Reviewer t3vc (Part 1)

  1. Since the primary contribution of this paper is using instance-level information for challenging objects under occlusion, long-range observation (L018, L), it needs to be compared quantitatively on datasets which do have sufficient long-range dataset such as Argoverse-v2, ManTruckscenes and a more recent AevaScenes datasets. KITTI has very less objects in long-range observation range.

We thank the reviewer for suggesting an evaluation of our method's long-range detection capability. Given the time and computational resource constraints, we conducted additional experiments on the Waymo Open Dataset, which provides a suitable benchmark for evaluating long-range detection performance and enables direct comparisons with existing monocular 3D object detection methods. Following the evaluation protocol reported in MonoCoP, we report performance across different distance ranges to assess the effectiveness of ProtoDN for long-range detection. As shown in the following table, ProtoDN outperforms the compared methods in both the 30–50 m and 50 m+ ranges. These results provide further evidence of the effectiveness of ProtoDN for long-range monocular 3D object detection.

IoU$_{3D}$DifficultyMethodAP$_{3D}$ All0–3030–5050–∞APH$_{3D}$ All0–3030–5050–∞
0.7L1GUPNet2.286.150.810.032.276.110.800.03
DEVIANT2.696.950.990.022.676.900.980.02
MonoUNI3.208.610.870.133.168.500.860.12
MonoDETR2.115.990.730.122.105.940.730.12
MonoDGP2.416.670.840.122.396.620.840.12
MonoCoP2.727.441.07‾\underline{1.07}0.16‾\underline{0.16}2.707.381.06‾\underline{1.06}0.16‾\underline{0.16}
ProtoDN (Ours)2.73‾\underline{2.73}8.13‾\underline{8.13}1.180.232.71‾\underline{2.71}8.08‾\underline{8.08}1.170.23
0.7L2GUPNet2.146.130.780.022.126.080.770.02
DEVIANT2.526.930.950.022.506.870.940.02
MonoUNI3.048.590.850.123.008.480.840.12
MonoDETR1.985.960.710.101.975.920.700.10
MonoDGP2.266.650.810.102.246.590.810.10
MonoCoP2.557.411.03‾\underline{1.03}0.14‾\underline{0.14}2.537.351.02‾\underline{1.02}0.14‾\underline{0.14}
ProtoDN (Ours)2.56‾\underline{2.56}8.10‾\underline{8.10}1.140.202.54‾\underline{2.54}8.05‾\underline{8.05}1.130.20
0.5L1GUPNet10.0224.784.840.229.9424.594.780.22
DEVIANT10.9826.855.130.1810.8926.645.080.18
MonoUNI10.9826.634.040.5710.7326.303.980.55
MonoDETR9.6823.784.721.009.6023.584.670.99
MonoDGP10.0624.015.060.999.8423.735.010.98
MonoCoP11.7627.59‾\underline{27.59}6.03‾\underline{6.03}1.48‾\underline{1.48}11.6527.35‾\underline{27.35}5.97‾\underline{5.97}1.46‾\underline{1.46}
ProtoDN (Ours)11.48‾\underline{11.48}28.686.651.6311.37‾\underline{11.37}28.396.591.62
0.5L2GUPNet9.3924.694.670.199.3124.504.620.19
DEVIANT10.2926.754.950.1610.2026.544.900.16
MonoUNI10.3826.573.950.5310.2426.243.890.51
MonoDETR†9.0823.704.550.879.0023.494.510.86
MonoDGP†9.4323.924.880.869.3223.654.840.85
MonoCoP11.0327.49‾\underline{27.49}5.82‾\underline{5.82}1.29‾\underline{1.29}10.9327.25‾\underline{27.25}5.76‾\underline{5.76}1.27‾\underline{1.27}
ProtoDN (Ours)10.77‾\underline{10.77}28.576.421.4210.66‾\underline{10.66}28.296.361.41
AuthorsOpenReview ↗

Response to Reviewer t3vc (Part 2)

  1. Another central claim in the paper is the training stability of geometry-guided methods compared to distribution-driven methods (L050). I do not know if the appearance-aware geometry-guided methods make the convergence better or worse?. Please provide two plots showing AP3DAP_{3D} 70 and AP3DAP_{3D} 50 of your baseline, baseline + Geometry-Guided and ProtoDN from 1st to 250 epoch in steps of 10 epoch. Ideally, you should plot all these three methods over 3 random seeds. This will help us justify the convergence claim of ProtoDN.

We thank the reviewer for raising this concern. We would like to clarify that our statement regarding training stability highlights the benefits of query denoising in providing more direct supervision for query learning, thereby improving the training process. Both geometry-guided and distribution-driven approaches are denoising mechanisms, distinguished primarily by their noise construction strategies. We do not claim that either approach has an advantage over the other in terms of training stability. To address the reviewer's concern about the effect of appearance awareness on model convergence, we conducted additional experiments comparing our geometry-guided baseline with our method, ProtoDN, using three different random seeds. The convergence plots have been added to the revised manuscript in Figure 4 (Lines 810–823). As shown in the plots, ProtoDN exhibits convergence trends comparable to the baseline under both IoU thresholds, without noticeably larger performance fluctuations or additional training instability. These results suggest that incorporating appearance awareness does not compromise the convergence behavior of the original geometry-guided denoising framework.

  1. Are the KITTI test and val models listed in Table 1 trained on single class (Car) or 3 classes (Car, Pedestrian and Cyclist) ? If it was done in a single class, could you provide the ProtoDN KITTI Val/Test numbers for 3 classes?

We thank the reviewer for requesting clarification on the training settings and three-class evaluation. The KITTI validation and test results reported in Table 1 are obtained from models trained on the Car class only. To further evaluate the generalizability of ProtoDN across object categories, we additionally trained a three-class model and report the Pedestrian and Cyclist 3D AP results on the KITTI validation set below. Although the improvements over MonoDGP on Pedestrian are relatively limited, ProtoDN achieves substantial gains on Cyclist. Meanwhile, compared with the baseline, ProtoDN consistently improves performance across all difficulty levels for both Pedestrian and Cyclist. These results suggest that the proposed method can also benefit categories beyond Car. We have also included the three-class evaluation results in Table 7 (Lines 721–728) of the supplementary material in the revised manuscript.

MethodPed. EasyPed. Mod.Ped. HardCyc. EasyCyc. Mod.Cyc. Hard
MonoDGP13.7710.067.9612.216.615.95
Baseline13.0410.087.5616.228.477.23
ProtoDN13.5010.277.9717.138.888.47
AuthorsOpenReview ↗

Response to Reviewer t3vc (Part 3)

4.The last three rows show that ablation studies Tab. 3 shows that the impact of Visual Enhancement and Dynamic Update Module is minor. This is specially important since the Mono3D on KITTI is notably sensitive to initialization. I request that the authors report the median performance across three random seeds on the KITTI Val split at 250th epoch as in [A] to ensure the results are statistically significant and the modules are indeed useful. In other words, I do not want any checkpoint selection to be done.

We thank the reviewer for the valuable feedback regarding the effectiveness of the proposed modules. Following the requested evaluation protocol, we conducted additional ablation experiments using three different random seeds and report the median performance on the KITTI validation set at epoch 250. The results shows that the ProtoDN model achieves the best performance across all six metrics. Removing Visual Enhancement or Dynamic Update decreases 3D AP on the Moderate split by 0.73 and 0.55, respectively. These results provide further evidence that both modules contribute to the overall performance improvements, with their combination achieving the best results.

We note that reporting the median performance across three random seeds at 250th epoch is seldom used in this field. This evaluation protocol may lead to lower absolute performance, as further corroborated by the additional baseline results. In our paper, all compared methods follow the same checkpoint-selection strategy to ensure a fair comparison.

MethodJoint Diff.Visual Enh.Dynamic Upd.AP$_{BEV}$ EasyAP$_{BEV}$ Mod.AP$_{BEV}$ HardAP$_{3D}$ EasyAP$_{3D}$ Mod.AP$_{3D}$ Hard
Baseline39.3728.9625.7330.2222.6819.82
ProtoDN✓✓39.9329.6826.1430.4723.5020.10
ProtoDN✓✓40.1629.4726.1231.0723.3220.22
ProtoDN✓✓✓40.6830.1927.2232.3424.0520.58
  1. The IoU3D threshold of 0.7 of KITTI is a strict metric and is very sensitive. To provide a more comprehensive view of performance, please report KITTI Val results using a 0.5 threshold, consistent with MonoDLE [22] and DEVIANT [D].

We thank the reviewer for suggesting additional evaluations at a lower IoU threshold. Following the evaluation settings of MonoDLE and DEVIANT, we report the KITTI validation results at IoU = 0.5, as shown in the table below. The results show that ProtoDN outperforms MonoDLE and DEVIANT by clear margins, while also achieving consistent improvements over our baseline in both 3D AP and BEV AP across all difficulty levels. These results further support the effectiveness of ProtoDN under a less restrictive IoU threshold.

MethodAP$_{BEV}$ EasyAP$_{BEV}$ Mod.AP$_{BEV}$ HardAP$_{3D}$ EasyAP$_{3D}$ Mod.AP$_{3D}$ Hard
MonoDLE60.7346.8741.8955.4143.4237.81
DEVIANT65.2849.6343.5061.0046.0040.18
Baseline72.5856.0051.2068.7952.6547.98
ProtoDN (Ours)73.7356.6352.1369.7653.2148.77
Reviewer_mDr7OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes ProtoDN, which maintains dynamic class-specific visual prototypes for denoising-based monocular 3D detection. Instance-prototype similarity adjusts geometric perturbations, while retrieved prototype features enrich denoising queries. These operations are used only during training. Results on KITTI show improvements over the baseline, particularly on the Hard validation subset.

Critical strengths
  1. Visual prototypes are used only during training, allowing the method to improve the detector without retaining prototype retrieval or query enhancement at inference.
  2. Experiments show that ProtoDN improves KITTI validation Hard AP3D from 21.05 to 22.09, improving detection on challenging examples.
Critical weaknesses
  1. Eq. (7) treats the maximum instance-prototype similarity as visual certainty, but an unusual-looking object is not necessarily difficult to detect. How does this measure distinguish uncommon appearance from unreliable visual evidence, which may call for different perturbation strengths?
  2. In Eq. (8), the visual factor can only reduce or preserve the original noise magnitude. Table 4 links visual similarity to detection quality, but does not establish whether instance-specific modulation is better than simply reducing noise. A fixed-scaling control with a comparable average perturbation magnitude would help answer this.
  3. Both prototypes and instance features evolve during training. Is visual certainty computed before or after updating the prototypes with the current batch? If it is computed afterward, could an instance’s contribution to the update inflate its own certainty?
Questions for authors
  1. Prototype updates use similarity-weighted RoI features and an exponential moving average. Are these mechanisms sufficient to limit the influence of occluder or background features from heavily occluded instances?
3 comments
AuthorsOpenReview ↗

Response to Reviewer mDr7 (Part 1)

  1. Eq. (7) treats the maximum instance-prototype similarity as visual certainty, but an unusual-looking object is not necessarily difficult to detect. How does this measure distinguish uncommon appearance from unreliable visual evidence, which may call for different perturbation strengths?

We agree that uncommon appearance does not necessarily imply unreliable visual evidence, and maximum prototype similarity cannot fully distinguish between these cases. It is important to clarify that when the reliability of visual evidence is uncertain, adopting a relatively conservative perturbation represents a reasonable design trade-off. For uncommon but visually clear instances, smaller perturbations may reduce the difficulty of their denoising task but do not remove their denoising supervision. For genuinely challenging instances, smaller perturbations help prevent noisy queries from deviating excessively from their ground-truth targets. Meanwhile, other reliable instances can still support denoising learning through stronger perturbations, so locally conservative perturbations do not necessarily make the overall training strategy overly conservative. Furthermore, perturbation strengths are jointly determined by visual and geometric certainty, avoiding reliance on prototype similarity alone.

  1. In Eq. (8), the visual factor can only reduce or preserve the original noise magnitude. Table 4 links visual similarity to detection quality, but does not establish whether instance-specific modulation is better than simply reducing noise. A fixed-scaling control with a comparable average perturbation magnitude would help answer this.

We thank the reviewer for suggesting a fixed-scaling control experiment. To address this concern, we conducted an additional experiment in which instance-specific visual certainty was replaced with a fixed scaling factor aligned with the average perturbation magnitude. As shown in the table below, ProtoDN consistently outperforms Fixed Scaling across all three difficulty levels. Moreover, Fixed Scaling performs even worse than our baseline, suggesting that uniformly reducing perturbation magnitudes is insufficient and may limit the model's ability to learn effective denoising from stronger perturbations. These results provide strong evidence that instance-specific visual certainty modulation is more effective than simply reducing noise.

MethodAP$_{BEV}$EasyAP_{BEV}\ Mod.AP$_{BEV}$ HardAP$_{3D}$ EasyAP$_{3D}$ Mod.AP$_{3D}$ Hard
Fixed Scaling39.9429.4125.7931.0023.3220.08
Baseline41.5030.0127.4333.7224.2821.05
ProtoDN (Ours)42.5231.6228.1233.5624.5722.09
AuthorsOpenReview ↗

Response to Reviewer mDr7 (Part 2)

  1. Both prototypes and instance features evolve during training. Is visual certainty computed before or after updating the prototypes with the current batch? If it is computed afterward, could an instance’s contribution to the update inflate its own certainty?

We thank the reviewer for highlighting the potential self-reinforcement issue in prototype updates. In our implementation, visual certainty is computed before the prototype memory is updated with the current batch. All instances within the batch are evaluated using the same pre-update prototype memory, thereby avoiding potential self-reinforcement effects caused by incorporating an instance's own features into the prototypes before computing similarity. We have clarified this update order in the revised manuscript (Lines 232–235) to better explain the prototype-based certainty estimation process.

  1. Prototype updates use similarity-weighted RoI features and an exponential moving average. Are these mechanisms sufficient to limit the influence of occluder or background features from heavily occluded instances?

We thank the reviewer for raising this important concern about the robustness of prototype updates. To mitigate this issue, our dynamic prototype update mechanism combines similarity-weighted aggregation across instances with high-momentum EMA updates. First, prototype updates aggregate features from multiple instances based on their similarities to the learned prototypes. Heavily occluded instances with relatively low similarity to a given prototype tend to contribute less to its update, thereby reducing the influence of potentially noisy features. Second, we employ EMA updates with a momentum of 0.99, ensuring that historical prototypes dominate each update and limiting the impact of individual noisy instances. These two mechanisms effectively suppress the influence of heavily occluded instances, maintaining the representativeness and stability of the prototype memory.

Thanks for the clarification. My concerns are addressed. I will keep my rating.

Reviewer_3WwpOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ProtoDN, an appearance-aware denoising framework for monocular 3D object detection. The authors argue that existing denoising methods mainly rely on geometric information while overlooking instance-level visual cues. To address this, they introduce a dynamic visual prototype memory that maintains class-specific appearance representations. These prototypes are used in two ways: combining geometric and visual certainty to adaptively control denoising perturbations, and providing additional appearance information to noisy queries through prototype-guided enhancement.

Critical strengths
  • I find the paper clear and easy to follow. The motivation is well explained, and the proposed method is straightforward.
  • The experimental results show improvements over previous methods on the KITTI dataset.
Critical weaknesses
  1. Limited novelty.

My main concern is the limited novelty of the proposed method. The proposed method seems to be mainly a combination of existing techniques. MonoDLGD already introduces geometry-guided difficulty-aware denoising, while MonoCLUE has explored prototype-based visual representations for monocular 3D detection. In my opinion, the proposed method is quite incremental, and I do not see a substantial methodological contribution beyond combining these existing approaches. Moreover, the prototype construction relies on conventional techniques such as K-means clustering and EMA updates, with little novelty in the formulation itself.

  1. Insufficient justification for several design choices. I find several components of the proposed method insufficiently motivated or explained:
  • The authors use the maximum cosine similarity between an instance feature and its class-specific prototypes to measure visual certainty. However, it is not clear why this similarity should reliably reflect the difficulty or uncertainty of 3D detection. Could the authors provide a clearer justification for this assumption?
  • Since the ground-truth instance features are already available, what is the advantage of retrieving and aggregating visual prototypes instead of directly injecting the instance features into the denoising queries?
  • Equation (8) applies the same visual-certainty score to both projected-box and depth perturbations. Could the authors explain why a single certainty score is appropriate for these two different attributes?
  • Equation (13) scales the appearance prior by visual certainty, meaning that less certain instances receive weaker visual enhancement. This seems somewhat inconsistent with the motivation of compensating for insufficient visual information in challenging instances. Could the authors clarify the reasoning behind this design choice?
  1. Marginal performance gains.

Although the proposed method achieves competitive results, the improvements over recent 2026 baselines are marginal. Given this marginal improvement, it is difficult to assess whether the additional components provide a meaningful advantage. I would also like to see MonoDLGD included in the computational efficiency comparison in terms of GFLOPs and parameter count, to better understand the trade-off between performance and complexity.

  1. Lack of sensitivity analysis.

Several important hyperparameters, including the warm-up period, ramp-up duration, and appearance-prior injection strength, appear to be empirically selected without sufficient justification. I would like to see a sensitivity analysis of these parameters to better understand how they affect performance and whether the method is robust to different settings.

Questions for authors

Questions for the Authors

  1. Novelty: Could the authors clarify the main methodological differences between ProtoDN and existing approaches such as MonoDLGD and MonoCLUE? In particular, what is the key contribution beyond combining difficulty-aware denoising with prototype-based visual representations?

  2. Visual certainty: What is the justification for using maximum cosine similarity as a measure of visual certainty? Is there any evidence that this score reliably reflects instance-level detection difficulty?

  3. Prototype-based enhancement: Since the ground-truth instance features are already available, why not directly inject these features into the denoising queries? Could the authors compare this simpler approach with the proposed prototype-based enhancement?

  4. Certainty weighting: Equation (8) uses the same visual-certainty score for both projected-box and depth perturbations, while Equation (13) gives less visual enhancement to instances with lower certainty. Could the authors explain the motivation behind these choices, particularly the latter, given the goal of improving detection for challenging instances?

  5. Performance and efficiency: Given the relatively small improvements over MonoDLGD, could the authors report results across multiple random seeds to demonstrate their consistency? Also, why is MonoDLGD not included in the computational efficiency comparison in Table 2?

  6. Hyperparameter sensitivity: Could the authors provide a sensitivity analysis for the warm-up period, ramp-up duration, and appearance-prior injection strength? It is currently unclear how sensitive the method is to these settings.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 3Wwp (Part 1)

  1. Novelty: Could the authors clarify the main methodological differences between ProtoDN and existing approaches such as MonoDLGD and MonoCLUE? In particular, what is the key contribution beyond combining difficulty-aware denoising with prototype-based visual representations?

We thank the reviewer for the insightful comments on the novelty of ProtoDN.

Difference from MonoDLGD. MonoDLGD primarily estimates instance difficulty based on geometric uncertainty and adaptively perturbs projected bounding boxes and depth accordingly. In contrast, ProtoDN introduces prototype-based visual certainty and progressively integrates it with geometric certainty through multiplicative fusion. This allows the perturbation magnitude to account for both geometric reliability and instance-level appearance consistency, which are not considered in MonoDLGD.

Difference from MonoCLUE. Although both methods employ visual memory, their objectives and integration mechanisms are fundamentally different. MonoCLUE employs segmentation-guided clustering to extract local object-part features, with the aim of enhancing visual representations. In contrast, ProtoDN constructs class-specific prototypes from ground-truth RoI features and specifically incorporates them into the denoising training process. The prototype memory serves two complementary purposes: 1) implicitly estimating visual certainty to modulate geometry-guided perturbations, and 2) explicitly retrieving appearance priors and injecting them into perturbed denoising queries. Therefore, ProtoDN does not simply use prototypes to enhance object representations; rather, it establishes a prototype-guided denoising mechanism that jointly addresses perturbation reliability and the lack of appearance priors.

The key contribution beyond the combination of existing methods. We acknowledge that ProtoDN draws upon and integrates ideas from difficulty modeling and visual prototype memory. However, this work aims to address an underexplored issue in existing denoising methods: instance-level visual information is insufficiently exploited during the denoising process, limiting the effective modeling of challenging objects under occlusion, long-range observation, and truncation. To address this issue, ProtoDN incorporates visual information into instance-level difficulty modeling and appearance prior construction, providing an appearance-aware denoising learning mechanism for challenging objects.

  1. Visual certainty: What is the justification for using maximum cosine similarity as a measure of visual certainty? Is there any evidence that this score reliably reflects instance-level detection difficulty?

We thank the reviewer for providing an opportunity to clarify the justification for our visual certainty measure and its relevance to instance-level detection difficulty.

The maximum cosine similarity measures the extent to which an instance's visual features can be represented by the typical visual prototypes of its category. If an instance exhibits high similarity to at least one visual prototype, this indicates that its visual features are typical and representative, and thus it is assigned a high visual certainty. Conversely, if its features have low similarity to all visual prototypes, its appearance cannot be adequately represented by the learned typical visual patterns. This may result from occlusion, truncation, or appearance variations, which reduce the reliability of the available visual evidence and consequently lead to lower visual certainty.

We further investigate the relationship between visual certainty and detection quality in Table 4 of our original submission by comparing two groups of instances with similarly high geometric certainty. Despite their comparable geometric certainty, the group with lower visual certainty exhibits larger depth errors (0.419 m vs. 0.252 m) and 3D center errors (0.441 m vs. 0.289 m), as well as lower 3D IoU (0.469 vs. 0.587). These results indicate that visual certainty based on maximum cosine similarity provides complementary instance-level information beyond geometric certainty and is closely associated with the instance-level detection difficulty.

AuthorsOpenReview ↗

Response to Reviewer 3Wwp (Part 2)

  1. Prototype-based enhancement: Since the ground-truth instance features are already available, why not directly inject these features into the denoising queries? Could the authors compare this simpler approach with the proposed prototype-based enhancement?

We agree that directly injecting features from ground-truth instances into denoising queries is an intuitive alternative. However, the purpose of injecting visual features into denoising queries is to provide reliable appearance priors. Directly injecting ground-truth instance features may introduce instance-specific deviations caused by factors such as occlusion and background interference, thereby bringing noise and bias into the denoising queries. In contrast, our method constructs appearance priors through a Top-k Soft Fusion mechanism. These priors can provide more stable visual references while retaining instance-specific relevance through similarity-based weighting.

To validate the effectiveness of this design, we conducted an additional experiment using direct ground-truth instance feature injection. The results shows that direct injection of ground-truth instance features leads to inferior performance compared with ProtoDN. This suggests that directly using instance-level features may introduce noisy and less reliable appearance priors, potentially interfering with the reconstruction of denoising queries.

MethodAP$_{BEV}$ EasyAP$_{BEV}$ Mod.AP$_{BEV}$ HardAP$_{3D}$ EasyAP$_{3D}$ Mod.AP$_{3D}$ Hard
ground-truth feature injection40.2129.5026.9631.1223.5420.55
ProtoDN (Ours)42.5231.6228.1233.5624.5722.09
  1. Certainty weighting: Equation (8) uses the same visual-certainty score for both projected-box and depth perturbations, while Equation (13) gives less visual enhancement to instances with lower certainty. Could the authors explain the motivation behind these choices, particularly the latter, given the goal of improving detection for challenging instances?

We appreciate the reviewer’s question about the justifications behind these methodological design choices.

Shared Visual Certainty for Projected-box and Depth Perturbations. The projected box and depth of the same object instance both rely on the same visual representation. Factors such as occlusion, truncation, and insufficient appearance information may simultaneously affect the reliability of projected-box localization and depth estimation. Therefore, visual certainty serves as a shared cue of appearance reliability for both attributes. However, the perturbation strengths for the final projected box and depth remain different, because each attribute has its own geometric certainty, which is combined with the shared visual certainty.

Certainty-weighted Visual Prior Injection. For Eq. (13), although Top-k Soft Fusion can provide more stable appearance priors, these priors are not necessarily equally reliable for all instances. When an instance exhibits low similarity to the learned prototypes, the fused appearance priors may still fail to adequately represent that instance. Therefore, we adaptively adjust the injection strength based on visual certainty to prevent potentially mismatched appearance priors from interfering with the denoising process.

AuthorsOpenReview ↗

Response to Reviewer 3Wwp (Part 3)

  1. Performance and efficiency: Given the relatively small improvements over MonoDLGD, could the authors report results across multiple random seeds to demonstrate their consistency? Also, why is MonoDLGD not included in the computational efficiency comparison in Table 2?

Multi-seed Evaluation for Performance Consistency. We thank the reviewer for suggesting a multi-seed evaluation. To address this concern, we conducted three additional runs with different random seeds for both MonoDLGD and ProtoDN. The table below presents the mean and standard deviation of the experimental results. As shown, ProtoDN achieves higher mean AP3DAP_{3D} and APBEVAP_{BEV} across all difficulty levels than MonoDLGD, while exhibiting lower standard deviations. These results provide further evidence of the consistency of ProtoDN's performance improvements.

MethodAP$_{BEV}$ EasyAP$_{BEV}$ Mod.AP$_{BEV}$ HardAP$_{3D}$ EasyAP$_{3D}$ Mod.AP$_{3D}$ Hard
MonoDLGD41.05 ± 0.6730.30 ± 0.6327.38 ± 0.6132.77 ± 0.6724.25 ± 0.2721.13 ± 0.76
ProtoDN (Ours)41.92 ± 0.4931.43 ± 0.2327.92 ± 0.1633.25 ± 0.2724.56 ± 0.0721.71 ± 0.47

Computational Efficiency Comparison with MonoDLGD. We thank the reviewer for pointing out the omission of MonoDLGD from Table 2. We have updated the Table 2 in the revised manuscript (Lines 365–375) to include MonoDLGD.Additionally, in the original table, we mistakenly reported inference-time parameter counts instead of total parameter counts. We have corrected this and now report both metrics separately for clarity. As shown in the revised Table 2, ProtoDN improves 3D AP by 0.29 and BEV AP by 1.61 over MonoDLGD on the Moderate split, while maintaining the same FLOPs (69.31 G) and inference-time parameter count (40.31 M). The additional 0.07 M parameters are associated with training-specific modules and do not introduce extra inference overhead.

MethodAP$_{3D}$ Mod.AP$_{BEV}$ Mod.FLOPs ↓Total Params (M) ↓Inference Params (M) ↓
MonoDETR20.6126.9559.7237.6835.93
MonoDGP22.3428.2068.9944.1738.90
MonoCoP23.9831.2971.7742.5040.00
MonoCLUE24.1029.9172.7144.1740.71
MonoDLGD24.2830.0169.3143.2440.31
ProtoDN (Ours)24.5731.6269.3143.3140.31
  1. Hyperparameter sensitivity: Could the authors provide a sensitivity analysis for the warm-up period, ramp-up duration, and appearance-prior injection strength? It is currently unclear how sensitive the method is to these settings.

We thank the reviewer for suggesting an additional hyperparameter sensitivity analysis. To address this concern, we conducted additional experiments by varying the warm-up period, ramp-up duration, and appearance-prior injection strength individually, while keeping the other hyperparameters fixed at their default values (Warm-up period = 40, Ramp-up duration = 10, and Apearance-prior injection strength = 0.1). As shown in the table, the overall performance remains relatively stable across different hyperparameter configurations. Although different settings lead to some performance fluctuations, none results in substantial degradation in detection performance. These results suggest that ProtoDN is not overly sensitive to the selection of these hyperparameters and maintains robust detection performance.

ConfigurationAP$_{3D}$ Mod.ΔAP$_{BEV}$ Mod.Δ
Original24.57–31.62–
Warm-up period= 2024.38-0.1931.14-0.48
Warm-up period = 6024.46-0.1131.50-0.12
Ramp-up duration = 524.09-0.4831.28-0.34
Ramp-up duration = 2024.68+0.1131.32-0.30
Appearance-prior injection strength= 0.0524.26-0.3131.33-0.29
Appearance-prior injection strength = 0.224.66+0.0931.23-0.39

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and Responses to Reviewers

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their constructive comments and valuable suggestions. In this summary, we provide comprehensive responses to the concerns raised regarding the methodological design, robustness, and broader evaluation and generalization of our work.

Novelty and Design Justification. In response to Reviewer 3Wwp's concern about the novelty of ProtoDN, we clarified its differences from existing methods and highlighted its key contribution: introducing a unified appearance-aware denoising framework that leverages dynamic visual prototypes to jointly guide instance-level denoising difficulty estimation and enhance denoising queries with appearance priors, thereby addressing the underutilization of visual cues for challenging objects, an issue that remains underexplored in existing denoising methods. To substantiate our design choices (Reviewers 3Wwp and mDr7), we further explained the rationale behind visual-certainty estimation and prototype-guided visual enhancement. We also conducted controlled comparisons with a fixed-scaling control and a direct ground-truth feature injection to further validate the effectiveness of these design choices. Moreover, in response to Reviewer mDr7's concerns about prototype updates, we clarified how similarity-weighted aggregation and a high-momentum EMA effectively suppress the influence of unreliable features from occluded instances.

Performance Consistency and Robustness. To address concerns about performance consistency (Reviewers 3Wwp and t3vc), we conducted three-seed experiments for both the baseline and ProtoDN, reporting the mean and standard deviation. The results demonstrate the consistency of ProtoDN's improvements. We further plotted convergence curves based on these runs, demonstrating the stable convergence of ProtoDN during training, as requested by Reviewer t3vc. Furthermore, following Reviewer t3vc's suggestion, we conducted additional ablation studies on the Visual Enhancement and Dynamic Update modules across three random seeds without checkpoint selection, further confirming their contributions. To address Reviewer 3Wwp's concern about hyperparameter sensitivity, we also investigated different hyperparameter settings, with results demonstrating that ProtoDN maintains relatively stable performance across different configurations and achieves its best performance with the default settings.

Expanded Evaluation and Generalization. To address Reviewer t3vc's concerns about evaluation coverage, we supplemented the KITTI evaluation with results at an IoU threshold of 0.5 and additional multi-class results, providing a more comprehensive assessment of ProtoDN. We further conducted experiments on the Waymo Open Dataset and the nuScenes frontal validation set. ProtoDN achieves competitive performance on both datasets, demonstrating its applicability beyond KITTI. Moreover, the Waymo results show that ProtoDN achieves the highest AP3D and APH3D in the 30–50 m and 50 m+ ranges across all evaluated settings, addressing Reviewer t3vc's concern about long-range detection performance. We also conducted cross-dataset evaluation using KITTI-trained models on nuScenes. ProtoDN reduces the overall depth MAE from 3.65 m to 2.83 m compared with the baseline, providing further evidence of its cross-dataset generalization capability (Reviewer t3vc).

The reviewers have acknowledged the contributions and effectiveness of our proposed method. With the additional methodological justifications, robustness analyses, and expanded experimental evaluations, we believe we have thoroughly addressed all concerns regarding the method's reliability and performance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.