- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
While a recent line of work has started to tackle the dichotomy between generalization and memorization in diffusion models, the authors of this work instead address the issue of hallucination. They define hallucination as the generation of samples that are locally coherent but globally incoherent (think of a hand with 6 fingers). The authors distinguish four distinct regimes: bad generation (both local and global inconsistencies), generalization (both locally and globally consistent), hallucination (locally consistent but globally inconsistent) and memorization (verbatim repetition of the training samples). The picture they propose is that when a model is trained, it starts in the bad generation regime and ends in the memorization regime (Bonnaire et al. 2025), while the window in between can be either generalization or hallucination. In this intermediate window of training times, both the train and test losses plateau. The authors claim that it is during this plateau that the model learns the global constraints, hence it goes from hallucination to generalization. More precisely, the authors define two times: , which corresponds to the beginning of the plateau and is also the time at which the model starts to learn the local rules, and , at which the model starts to learn the global rules. The window is the hallucination window.
To back their claims, the authors study the Random Hierarchy Model (RHM) introduced in Petrini et al. 2023 and studied in the context of diffusion in several papers (Sclocchi et al. 2024, Garnier-Brun et al. 2025, ). Created to model the hierarchical nature of real data, it has local and global rules at the same time. The loss itself can be decomposed into a term accounting for the global rules and another one accounting for the local rules.
When training a denoiser on the RHM, the local term decreases first, and the global term only decreases one or two orders of magnitude later in terms of gradient steps. This indicates that the model first learns to generate locally consistent samples before learning global consistency, hence generating hallucinated samples before samples faithful to the distribution.
In order to mitigate this, the authors propose the "Hierarchical Lag Regularizer". This regularizer adds a penalty to the loss that forces the global term to be smaller than the local term. The experiments done by the authors show that it reduces the hallucination window.
Critical strengths
- The authors study a simple synthetic model with a clear separation between hallucination and generalization.
- The authors propose and study a regularization that specifically mitigates hallucination. As far as I know this is the first paper studying doing this.
Critical weaknesses
- No experiments on real data and no applicability to real data. Overall, the paper is just analyzing one experiment. Although the ideas are interesting, it is more the beginning of an article than a complete article. I advise the authors to continue working on this project and add more content.
- For each set of parameters, the authors only did two runs and took only for the regularizer strength.
Questions for authors
- Why not study several values of and the effect of tuning ? This would be a valuable addition to this work.
- Why is there a wide white region in Fig. 5 and Fig. 6? Also, the plot is super hard to read and has too much information.
- In Fig. 1, the authors are plotting 2 quantities, but there is one curve with different colors. What are the authors plotting exactly? If it is two quantities, then it is misleading.
- Why only 2 seeds for numerical experiments? Would increasing the number of seeds over which the observables are averaged make the curves in the right panel of Fig. 3 less noisy?
- The regularizer the authors studied works only for the RHM. Can the authors deduce one to mitigate hallucinations for general data distributions?
- The authors say that and are separated by one or two orders of magnitude. What does this separation depend on: the data distribution, the choice of model architecture, the parameters of the RHM?
Dear Reviewers,
Thank you for your time in evaluating this paper. Since there is only one day left for this stage, could you please read the rebuttal and provide a response? This is important for the review process.
AC