- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the insufficient robustness of voxel-based LiDAR 3D object detectors under data corruption scenarios such as adverse weather or sensor failures. This paper proposes VORL, a Voxel-wise Observation Reliability Learning framework for robust LiDAR 3D object detection. VORL estimates a continuous reliability score for each occupied voxel from its feature representation and lightweight observation cues, and introduces Reliability-aware Sparse Convolution (RaSC) to modulate source-voxel contributions before neighborhood aggregation.
Critical strengths
- This paper is well-written.
- The problem of robust perception for autonomous driving in bad weather is important.
Critical weaknesses
- A quantitative analysis of the accuracy of the predicted voxel reliability scores is needed to demonstrate that it is correlated with the robustness improvement.
- For the nuScenes-C experiments, a multimodal method such as MVP should be added to the comparison.
- The paper looks like a stacking of modules.
Questions for authors
- The chosen baseline is SQD. Considering that weather and noise can also affect the image modality, how to handle the production of pseudo labels for pseudo point clouds and predict the reliability of voxels
- Why the metric of SQD is lower than the original ACM MM published paper?
- Is it suitable to add an experiment on Waymo-C?
- What is the research problem and novelty for this work? Because 3D perception in autonomous driving is a very ripe topic.
9 comments
(4) Reliability learning for virtual points in SQDNet. In our SQDNet experiments, we focus on robustness against weather- and sensor-induced point cloud corruptions. Specifically, we first generate virtual points offline using clean LiDAR point clouds and RGB images. We then apply synthetic corruptions to both the original LiDAR points and the pre-generated virtual points, rather than leaving the virtual points unaffected. For reliability supervision, we use the corresponding clean points and corruption metadata to construct pseudo reliability targets for both point sources following Eqs. (8)-(10). These point-level targets are then aggregated into voxel-wise reliability targets. At inference, VORL directly predicts voxel reliability from the fused voxel features and observation descriptors, without requiring clean references or corruption metadata for reliability estimation.
To further investigate robustness when RGB images are also corrupted, we conduct additional zero-shot evaluations under three settings: (a) both LiDAR and RGB are corrupted, (b) only RGB images are corrupted while LiDAR remains clean, and (c) only LiDAR points are corrupted while RGB images remain clean. For each setting, virtual points are regenerated using the corresponding RGB images and LiDAR-derived sparse depth maps. We evaluate four weather corruptions at severity level 5 using the existing SQDNet and VORL checkpoints, without retraining or fine-tuning.
(a) LiDAR + RGB corrupted
| Method | Snow | Rain | Fog | Sunlight |
|---|---|---|---|---|
| SQDNet | 10.91 | 7.50 | 46.89 | 38.73 |
| +VORL | 45.20 | 39.80 | 64.51 | 78.03 |
(b) RGB Only Corrupted
| Method | Snow | Rain | Fog | Sunlight |
|---|---|---|---|---|
| SQDNet | 80.30 | 80.42 | 80.48 | 80.56 |
| +VORL | 80.42 | 80.62 | 80.56 | 80.57 |
(c) LiDAR Only Corrupted
| Method | Snow | Rain | Fog | Sunlight |
|---|---|---|---|---|
| SQDNet | 11.56 | 8.30 | 50.53 | 39.54 |
| +VORL | 45.95 | 40.47 | 66.89 | 78.45 |
These results show that VORL improves average detection AP when both LiDAR and RGB are corrupted (+30.88) and when only LiDAR is corrupted (+30.46), while maintaining comparable performance under RGB-only corruption. These results suggest that VORL remains effective against point-cloud degradation even when image corruption affects virtual point generation.
(5) Clarification of SQDNet Baseline Performance. We initially attempted to use the publicly available pretrained SQDNet checkpoint but encountered compatibility issues with the current officially released code. Therefore, we retrained SQDNet using the official implementation and used the reproduced checkpoint for our robustness experiments.
It is worth noting that the original SQDNet paper reports its 3D detection results on the official KITTI test set, whereas our experiments are evaluated on the KITTI validation split used to construct KITTI-C. This difference in evaluation splits may explain the apparent discrepancy in the reported performance. Moreover, both SQDNet and its VORL variant use the same checkpoint.
(6) Evaluation on Waymo-C. We add evaluation on Waymo-C using VoxelNeXt as the baseline detector.
| Method | Clean L1 | Snow L1 | Rain L1 | Fog L1 | Sunlight L1 | Crosstalk L1 | Gaussian L1 | Uniform L1 | Impulse L1 |
|---|---|---|---|---|---|---|---|---|---|
| VoxelNeXt | 74.37/71.27 | 58.27/55.45 | 65.30/62.37 | 58.94/55.72 | 39.30/37.20 | 53.61/51.04 | 20.82/18.34 | 49.08/45.62 | 66.07/61.78 |
| +VORL | 74.35/71.26 | 65.10/62.17 | 68.27/65.30 | 61.42/58.21 | 68.25/65.27 | 69.86/66.84 | 21.61/19.35 | 61.91/58.59 | 67.01/63.13 |
| Method | Clean L2 | Snow L2 | Rain L2 | Fog L2 | Sunlight L2 | Crosstalk L2 | Gaussian L2 | Uniform L2 | Impulse L2 |
|---|---|---|---|---|---|---|---|---|---|
| VoxelNeXt | 66.30/63.66 | 50.43/48.30 | 57.09/54.72 | 51.19/48.69 | 33.31/31.67 | 46.55/44.47 | 17.76/15.98 | 42.29/39.59 | 58.37/54.83 |
| +VORL | 66.31/63.63 | 56.86/54.47 | 59.92/57.46 | 53.39/50.88 | 60.15/57.64 | 61.82/59.25 | 18.46/16.83 | 54.11/51.36 | 59.19/55.95 |
(7) Research problem and novelty of VORL. Although LiDAR-based 3D object detection has been extensively studied, robustness against corrupted observations remains a significant challenge. Existing voxel-based detectors do not explicitly model voxel-wise observation reliability, allowing unreliable features to propagate through sparse convolutions and contaminate neighboring reliable representations.
The key novelty of VORL is to explicitly learn voxel-wise observation reliability and incorporate it into sparse feature propagation. Specifically, we introduce Reliability-aware Sparse Convolution to suppress unreliable source contributions before neighborhood aggregation, together with corruption-aware pseudo supervision that requires no additional 3D box annotations under corrupted conditions. Moreover, VORL is a general and lightweight framework that can be integrated into various voxel-based detectors with minimal computational and memory overhead. These properties highlight both the methodological value and practical applicability of VORL for robust LiDAR 3D detection.
Thanks. The core issues of Q1 and Q2 have not been resolved, and I insist on the original judgment.
(1) Reliability prediction accuracy and its impact on detection robuness. To quantitatively evaluate reliability prediction accuracy, we measure the Pearson correlation and mean absolute error (MAE) between predicted reliability scores q and pseudo targets q∗ on KITTI-C using SQDNet + VORL at severity level 5, as well as the corresponding detection AP gains.
The results show that the estimator achieves positive correlations with the pseudo reliability targets across all eight corruptions (Pearson: 0.657-0.898), as well as consistent detection improvements. Although prediction accuracy and AP gains are not directly proportional across corruption types, our additional oracle gating experiment provides further evidence of their connection to detection robustness. Replacing predicted scores with pseudo targets improves APcor from 67.06 to 70.93, compared with 60.28 for the baseline.
Furthermore, our feature discrepancy analysis shows that VORL effectively mitigates corruption-induced feature contamination, which can otherwise degrade detection performance. All voxels are categorized into three categorizes based on pseudo reliability: Reliable-Isolated (q∗≥0.8, without unreliable neighbors), Reliable-Exposed (q∗≥0.8, with unreliable neighbors), and Unreliable (q∗≤0.3). Feature discrepancies are computed over spatially matched voxels.
(2) Comparison with multimodal 3D detection method MVP. We evaluate VORL on MVP, a multimodal 3D detector that incorporates image-derived virtual points into LiDAR-based detection, using nuScenes-C at severity level 5.
[1] MVP. Multimodal Virtual Point 3D Detection, NeurIPS 2021.
(3) Generality and unified design of VORL. We would like to emphasize that VORL is designed as a general and lightweight robustness enhancement framework for existing voxel-based 3D detectors, rather than a simple stacking of independent modules.
First, VORL demonstrates strong generality, as it can be integrated into six different voxel-based detectors across KITTI-C and nuScenes-C. Experimental results show that VORL consistently improves corruption robustness with minimal architectural modifications.
Second, VORL introduces only marginal computational and memory overhead, while keeping the pretrained detector entirely frozen, without requiring retraining or fine-tuning of the detection backbone and head.
Third, VORL does not require additional 3D box annotations under corrupted conditions.
These advantages make VORL a practical and broadly applicable framework for enhancing the robustness of existing voxel-based detectors.