- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes, SORD, a diffusion-based method to remove local OOD perturbation to recover clean data. SORD assumes the setting where the corruption is localized and we have access to this localized corruption support. It uses multiple inpaintings of the support as initial seeds of the estimated clean signals, and refines them iteratively. At each iteration, it first calculates the variance among the residuals between the estimated clean signals and the corruption, and denoises the residuals according to this variance. Then, it subtracts the residuals to update the estimated clean signals, and denoise the estimated signals. It stops when the variance converges. Experiments are performed for both the image and the audio modality, and consider tasks including additive corruption, watermarking and shadows. Results show that SORD generally outperforms or is comparable with prior works.
Critical strengths
- Generality: SORD does not assume corruption types and is generally applicable across them (as long as they are localized); experiments show applicability to diverse settings including additive, watermark, shadows. It also works across different modalities (images, audios).
- Performance: On the evaluated settings SORD generally outperforms DPS, the most directly applicable baseline, and often outperforms or is comparable to setting-specific baselines.
Critical weaknesses
- Efficiency: SORD involves hundreds of network passes per image (Appendix Table 4) and generally requires 32 inpainting examples for best performance (Appendix Table e), which is relatively expensive.
- Practical utility: performance is somewhat sensitive to accuracy of the corruption support (Appendix Table 5), which could be a concern in practice when perfect masks may not be available. Also the image experiments are performed mostly on distributions with moderate complexity, with traditional pixel-space diffusion models individually trained on the clean distribution; it remains a bit unclear how it performs under more complex data distributions and alternative model variants.
Questions for authors
See Weaknesses above -
- How would SORD scale when the clean signal distribution becomes more complex (e.g. ImageNet-1000), when it becomes more difficult for the diffusion models to accurately learn and represent the distribution?
- It would be helpful to have more ablation on the choice of diffusion models, e.g. how would SORD work in combination with general pretrained models not specific to the clean signals and/or with latent diffusion models.
- For imperfect corruption masks it would also be helpful to have some evaluations beyond simple transformations under more realistic settings, e.g. obtaining the masks from actual segmentation or anomaly detection models.