- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper uses a small number of clean images and a large number of images with unknown corruptions to learn the clean image distribution. Specifically, this paper aims to use an auxiliary corruption channel (Q_hat) to make the distribution of clean and degraded images nearly the same after Q_hat processing. Then, through the conditional diffusion model, combined with EM iteration to generate pseudo-clean data, and finally train to obtain the unconditional final generation model. This paper presents theoretical analysis of the method, and verifies the feasibility of the method on CIFAR-10, CelebA and ImageNet.
Critical strengths
- The idea is interesting and somewhat novel. Although the idea of reducing distributional differences by adding noise seems to have been explored in Ambient Omni, this paper extends it to more general auxiliary corruption channels, and combines distributional alignment, information preservation and conditional EM into a relatively complete framework.
- The paper provides some theoretical analysis, like establishing convergence guarantees under the ideal assumptions. It also analyzes the limitations of ambient omni, which makes the motivation clearer. The experiments also demonstrate improvements over the baselines on CIFAR-10 and CelebA.
Critical weaknesses
- This paper assumes that clean image and degraded image should have the same distribution after Q_hat. However, I haven’t found direct verification on how well the distributions are aligned under the selected Q_hat, such as random blur/masking. The influence of the distribution misalignment is also not clear. Thus, although there are some improvements in experiment, it is not clear whether these improvements come from the alignment strategy. There is still a gap between theoretical analysis and practical method.
- The selection of Q_hat is mainly empirical and lacks systematic analysis. The paper proposes that the selection of Q_hat should balance the distribution alignment and information preservation. However the paper mainly tests some relatively simple corruption types, and selects random blur/masking based on the FID. There is little direct evidence that these choices achieve a better trade-off between the above distributional alignment and information preservation. It is also unclear whether the selected auxiliary corruptions work well for other types of unknown corruptions, which might limit the generalizability of the paper.
Questions for authors
- Could you report some quantitative results to evaluate the distribution alignment under different auxiliary corruption channels, for example, by training a classifier? How does the alignment gap affect the final generation performance?
- Why does random gaussian blur outperform other Q_hat? Is it because of better alignment, information preservation or randomization? Could you also consider some other Q_hat, such as downsampling or combined corruptions?
- In ImageNet experiments, the final model is based on EDM2 pre-training and fine-tuned on it. Could you add a training-from-scratch comparison, or fine-tune EDM2 using the high quality subset under the same protocol, to verify the independent contribution of EMAC? Besides, could you report more generative metrics like FID or other diversity metrics?