- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces a diffusion-based framework to solve LiDAR inverse problems i.e reconstruction of clean lidar data under real-world corruptions like adverse weather (e.g., fog, snow) and sensor flaws (e.g. missing beam, crosstalk). Diffusion posterior sampling is used to reconstruct clean data from corrupted samples. A surrogate correlative guidance using patch wise earth movers distance (EMD) is introduced along with standard L2 guidance. The authors use the method proposed in Robo3D (Kong et al) for different types of lidar degradation. The corruption operator (C) is basically a mapping from clean 3D points to corrupt 3D points for various adverse weather corruptions (missing beam, snow, fog). So the corruption is done in 3D and then the corrupted sample is projected into 2D (depth and reflectance) images which is then denoised using the proposed diffusion posterior sampling and then the inverse transform is applied to generate clean 3D point cloud. The method is very similar to the R2DM (Nakashima 2024) paper which is also used as the pretrained diffusion model in this work. The results for the R2DM method is not given in the paper though, which is strange.
Critical strengths
None
Critical weaknesses
-
The paper makes incremental contribution to the R2DM (Nakashima & Kurazume 2024) paper. The results from baseline R2DM is missing from the paper despite the fact that a pretrained R2DM model is used in this paper. I will like to see how the EMD addition improves the performance over R2DM.
-
The comparison with DPS and DPS+TV is not sufficient. Also, it is not clear if the authors train the DPS model on KITTI dataset or not ? Since R2DM is pretrained on KITTI dataset the comparison with DPS can only be valid if DPS is also trained on KITTI.
-
The ablation study in table 3 shows that there are other surrogate designs that perform better than EMD (atleast for depth and reflectance MAE). Also, its not clear why the point cloud and range error metrics (FPD and FRD) are high for SSIM when depth MAE is low ?
Questions for authors
Questions:
- Can you clarify if the naive DPS model (which is used for comparison) was trained on KITTI dataset ?
- In table 3 the ablation study show that there are other surrogate design that work better than EMD in some cases. Can you provide more discussion on this?
- The forward operator (A) relies heavily on exact, test-time oracle metadata, such as identical lidar ring drop IDs or precise synthetic noise maps. How will this framework perform in real-world scenarios where the exact degradation matrix and parameters are completely unknown?
Suggestion:
- Test on real-world adverse weather data (e.g. Canadian Adverse Driving Conditions dataset) and show how the method translates to real-world degradations.
5 comments
Official Comment for Reviewer 7azv (2/2)
Q3. Unknown operators.
We fully agree that solving LiDAR inverse problems when the exact degradation is unknown is an important direction, and we would like to clarify where our work stands on this. In the image domain the blind setting, where the operator is unknown and is recovered along with the signal, is already being taken on directly, as in BlindDPS. Such a method still evaluates the operator it estimates inside a likelihood, so the likelihood has to be right first, and for range-view LiDAR that is what was missing. We hope our work can be read as the groundwork that a blind treatment of LiDAR would build on.
We would also note what the operator is given in the meantime. Every entry it reads states which directions carry a return, how far the corruption displaced them, or by how much it attenuated or rescaled them, and none of them states what the clean scan held there. Knowing a realization does not undo it either, since the projection is not invertible and what the corruption removed stays in the null space of .
Section 5.5 already reports one case with an approximate operator, a fog-like corruption that suppresses points within 6 m, and we have extended it into a quantitative study. Here is estimated from the measurement alone, from the valid support of and a local-depth inlier set, without the corruption type or its realization. Both methods receive the same and the guidance weights were not re-tuned.
Table x1. Performance when the forward operator must be estimated from the measurement alone. . Lower is better for every metric.
| Corruption | Method | FPD ↓ | FRD ↓ | JSD ×10² ↓ | MMD ×10⁴ ↓ | MAE (m) ↓ |
|---|---|---|---|---|---|---|
| Snow | DPS | 162.2 | 531.5 | 2.394 | 0.376 | 0.519 |
| Snow | Ours | 120.3 | 514.5 | 2.240 | 0.355 | 0.510 |
| Crosstalk | DPS | 12.33 | 324.3 | 1.504 | 0.200 | 0.297 |
| Crosstalk | Ours | 8.20 | 312.4 | 1.438 | 0.185 | 0.290 |
| Fog | DPS | 23.03 | 308.9 | 4.135 | 0.235 | 0.675 |
| Fog | Ours | 21.10 | 300.8 | 4.105 | 0.229 | 0.666 |
Every one of the five metrics favours our method on all three corruptions, so the correlative guidance remains useful when the exact operator is not available.
In the revision we will put the study above into the main paper in place of Section 5.5, discuss BlindDPS in the related work and add the blind setting as future work. We thank the reviewer for this question.
S1. Real adverse weather.
We agree that evaluation on real adverse weather matters, and we have run our method on it. Following the suggestion we applied the same observation-derived to the Canadian Adverse Driving Conditions dataset, recorded in real snowfall, with no degradation parameters of any kind.
Neither the operator nor the prior is matched to this data. The operator is read off the measurement, and the prior is the same R2DM checkpoint used throughout the paper, trained on SemanticKITTI, so it was fit to a sensor with twice the beams, in a different city, and it has never seen a snowy scan. Nothing was retrained or fine-tuned.
Even so, the airborne returns that fill the near field of the measurement are removed, and the ground rings and the structures beside the road come back continuous. Real snowfall has no clean counterpart to score against, so we report this qualitatively. The reconstructions are now in Appendix G.3 of the revised supplementary material. We thank the reviewer for the suggestion.
Official Comment for Reviewer 7azv
Dear Reviewer 7azv,
We would like to thank you once again for your review and for your suggestions, which led us to run a study with the forward operator estimated from the measurement alone and to evaluate our method on real snowfall.
We would very much like to resolve the concerns you raised. Some of them, in particular the role of R2DM in our framework and the way the DPS baseline is set up, we believe our writing left unclear, and we have tried to clarify both in our response. We would be grateful for the chance to address anything that remains, and we would gladly run additional experiments while the discussion period is open.
Official Comment for Reviewer 7azv (1/2)
We thank the reviewer for the detailed reading of our setup. The three weaknesses point at places where our writing left the role of R2DM, the identity of the DPS baseline and the reading of Table 3 unclear, and we respond to each below.
W1. R2DM is our prior, not a baseline.
We believe there is a misunderstanding here, and we would like to clarify it. R2DM is not a method for LiDAR inverse problems, and our method is not a retrained or modified R2DM. R2DM is an unconditional generative model of clean LiDAR scans, and we use it as the prior with which we solve an inverse problem, that of recovering the clean scene behind a corrupted measurement y. R2DM itself never takes y, so it is not a baseline for that problem, and this is why no R2DM row appears in the paper.
Our paper supplies what R2DM lacks, namely the connection between a scene and a measurement of it, as differentiable forward operators for eight 3D corruptions and the correlative guidance built on the likelihood they define. The weights and the architecture of R2DM are untouched and we train nothing.
The comparison the reviewer asks for is already in Table 1 of the paper, where every diffusion row uses R2DM as its prior. The DPS row is R2DM with the standard ℓ2 guidance, and the Ours row is R2DM with that guidance and our correlative term, under one checkpoint, one forward operator and 128 sampling steps. The two rows differ in the EMD term alone, so reading them against each other is reading exactly what the EMD addition contributes, and Table 1 gives it for all eight corruptions.
W2, Q1. The DPS baseline.
We would like to clarify how the DPS baseline is set up. Nothing is trained here. DPS is a sampling procedure rather than a model, which takes a pretrained diffusion prior and adds a measurement-likelihood gradient at each reverse step, so there is no DPS model that could have been trained on any dataset. Every row of our comparison runs on one and the same diffusion model, the R2DM pretrained on SemanticKITTI, and on KITTI-360 on the R2DM pretrained on KITTI-360, as stated in Section 5.4. DPS and our method thus share the prior and its training data, and differ in the guidance alone.
W3, Q2. The surrogate ablation.
We are glad to discuss Table 3 of the paper in more detail, and we will add this discussion to the paper. Two alternatives each lead on a single metric, SSIM on depth MAE and histogram intersection on BEV JSD, and in both cases by a fraction of a percent, while the same two rows fall far behind on FPD. EMD leads on the remaining metrics and ties for the best reflectance MAE.
Why SSIM can hold a low depth MAE and a high FPD at the same time follows from what the two measure. Depth MAE averages over every pixel, and under Beam Missing about two thirds of the pixels lie on the observed support, where the measurement already fixes the value, so the average is dominated by the region where little is at stake. FPD and FRD compare feature distributions of the reconstructed point cloud and range image, so they respond to whether the recovered surfaces sit where surfaces can sit. SSIM matches local means, variances and covariance inside a window, and a patch that spreads a depth discontinuity over several pixels preserves those statistics while placing points between two surfaces that belong to neither. Such points lower the average absolute error and are what a point-distribution metric charges for.
Sorted-L1 shows the same effect from the other side, and it also explains our choice within the distribution-matching family. Comparing sorted depths in a patch is the same assignment-invariant comparison as EMD, but its cost grows without bound in the displacement, so a single gross outlier in a patch dominates the term. The binned soft histogram in the log-depth domain bounds that cost, and that is where the two rows separate on the distributional metrics.
Since the deliverable of this task is a point cloud that a downstream model consumes, we select the surrogate on the distributional metrics and treat per-pixel MAE as secondary, because a surrogate that pulls every pixel toward a local consensus lowers the average while flattening the geometry. We will add this to Section 5.4 and report depth MAE split over the observed and unobserved supports alongside the average. We thank the reviewer for raising this point.