- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
Noise to Noise approaches for point clouds assume fully aligned matched pairs or various other strong assumptions. This paper presents a step towards making this approach more directly useful for real scans. From a single noisy scan, this approach takes a patch, and forms two subsets. At a representation level, this paper factorizes latent space into a subset invariant structure stream and subset specific nuisance stream, and reconstructs the denoised coordinates from the structure stream alone. This is novel as noise to noise on point clouds until now has made strong assumptions that mostly require the scans to come from matched pairs, or have some ground truth. Key challenge is that the two pair sets have different locations, so the loss needs to not force them together. This follows previous work on factorizing noise and structure points, but does not have ground truth points, only noise. Also does not require per shape optimization.
Claims:
- characterize the coupling gap between subsets of a single noisy scan and show how sampling discrepencies introduce bias into set level consistency supervision
- Propose couplet paired-subset sampler whose pairs bound this gap, and a structure nuisance invertable factorization that learns shared geometry from noisy observations alone.
- Experiments.
The paper constructs coupled subsets from the same noisy patch. As both sample same underlying surface, should share stable geometric structure. Dual stream invertible architecture to factorize latent representation into structure and nuisance code. Uses a loop of geometric consistency to ensure the denoising process goes through latent consistency. Learn a denoising operator – map noisy to denoised patch. Steps: 1) construct clean X1 and X2 with reduced discrepancy of spatial coverage and density, 2) Dual stream backbone maps into structure and nuisance representations, 3) training with composite objective: geometry consistency in Euclid space, latent structure consistency, displacement regularization. Use EMD / Wasserstein distance with squared Euclidean cost to give coupling gap D_{EMD}(X1, X2) (X*1 is unknown clean).
Prop 1 coupling gap - D of empirical denoised outputs is less than or equal to sum of distances from Y to ideal for both of them.
Algorithm for coupled pair sampler: choose a pair within a voxel; otherwise get kNN for a point, and get the nearest available point.
Then disentangle extending Mao 20222 single stream with clean targets to the dual invertible stream without clean targets here. This explicitly routes stable geometric information and input-dependent nuisance variation through separate streams (no clean targets) by reconstructing point coordinates from the structure representation.
Training objectives. Impose consistency on denoised point coordinates and latent structure representation: Geometry loss aligns denoised subsets in Euclidean space and latent structure loss aligns structure codes in latent space. Loss for geometric consistence is the square of the EMD distance between estimated pointset 1 and estimated pointset 2. Charbonnier penalty on distance between estimated points and corresponding input points . Latent structure consistency. Encourage subsets to have similar structure codes in a permutation invariant manner – uses Sinkhorn distance. So, latent must be similar for each matched point, so enforcing spatial consistency of the latent space. The distance between the denoised points is minimised, which would map points to the same space on their own, but this is counter-balanced by The Charbonnier penalty that penalizes the denoised points going to far from their original point (assuming the match is successful).
Critical strengths
-
The paper addresses a clear issue in the noise-to-noise approach for point clouds.
-
The approach is interesting, and addresses it in a novel and convincing way.
-
The separating out the nuisance approach is also interesting.
-
The results that come from this are strong.
Critical weaknesses
Proposition 1 is not very clear. It's not given a lot of context to make clear why it is important in the paper, and the explanation supporting it is also a bit unclear.
The coupled paired-subset sampler seems a reasonable way to initialise matches, but under noise it is clear that it could get an incorrect pair assignment (it finds the closest, with noise the closest may not actually be correct). It's not clear if there is a way for the matches to recover from this using Sinkhorn, or if these are always binding. The paper says Sinkhorn distance, is this simply from the pre-assigned matches or is rematching possible?
Questions for authors
Is the coupled pair-subset sampler binding for all points or can they change? How would errors from this be addressed in the architecture.
Dear Reviewer SbUo,
We sincerely appreciate the time and effort you have devoted to reviewing our paper.
As we are past the midpoint of the discussion period and approaching the deadline, we would be grateful if you could let us know whether our responses have adequately addressed your concerns or if any further clarification would be helpful.
Thank you again for your time and consideration.
The Authors