- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies whether diffusion models can learn and exploit statistical structure in the noise used during training. The authors show that structured, non-IID noise can reduce denoising loss and influence generation, indicating that diffusion models may rely not only on data structure but also on regularities in the noise itself. Based on these observations, the paper argues that diffusion noise can itself provide learnable structure and may therefore serve as a controllable component of the generative process.
Critical strengths
None
Critical weaknesses
The most fundamental problem with this paper is that its main conclusion is not meaningful for generative modeling. The paper shows that, once the noise contains exploitable structure, the model can use that structure to predict the noise more accurately, but this is essentially expected and does not address the actual goal of a generative model, which is to learn the data distribution rather than the noise distribution. In standard diffusion models, noise prediction is meaningful only because, under the Gaussian corruption formulation, predicting is a particular parameterization of learning the score function of the corrupted data distribution and is therefore tied to modeling . Once additional predictable structure is introduced into the noise, the model can reduce the prediction loss by modeling the corruption process itself rather than by learning more about the data distribution, while learning such noise structure is itself irrelevant to the generative objective. In the extreme case of deterministic noise , the model could achieve zero prediction loss simply by always outputting , while learning nothing about . This example highlights that improved predictability of the noise can be completely decoupled from improved modeling of the data distribution. Structured noise is simply a less extreme version of the same issue, so demonstrating that such noise is "learnable" does not constitute a meaningful result for diffusion-based generative modeling.
Questions for authors
-
The paper shows that structured noise can make the noise-prediction problem easier, but it is unclear how this relates to the actual objective of generative modeling. Can the authors provide a principled argument, beyond improved noise-prediction loss, showing that exploiting noise structure leads to better modeling of rather than merely better modeling of the corruption process?
-
Once the noise distribution is changed from IID Gaussian noise to a structured non-IID distribution, the forward corruption process is also changed. What is the theoretically corresponding reverse process in this setting? In particular, is the standard DDIM sampling rule still theoretically justified?