- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies whether the class label and hallucination of a generated sample can be directly predicted or identified from its initial noise. It proposes to train a classifier on the noise-label pairs, where the noise is obtained through integrating data points back to noise space through deterministic ODE with the known label. Experiments are done on Gaussian mixtures and an MNIST dataset in its latent space to evaluate the prediction accuracy and hallucination fixing.
Critical strengths
This paper investigates an interesting problem: if we can predict the label of a generated sample directly from its initial noise by the invertible dynamics of a deterministic ODE.
Critical weaknesses
- Figure 1 is difficult to interpret. As the first figure to show the motivation of this paper, the caption does not clearly explain what the figure intends to convey.
- Section 3 introduces too many symbols without sufficient definitions. This includes flow map, endpoint map, \sigma(t), , the relationship between and . It is also not clear why is defined as . They should be explicitly defined at their first usages.
- From my understanding, the propositions are somewhat narrow. Proposition 1 requires every to satisfy the inequality, which is not necessarily true for cases. The proposition 4 seems stating the direction of gradient points towards the where the function changes fastest. It is also not very clear to me how to match eacb proposition conclusion to the insights in section 5.
- For the experiment in 6.2, it only reported prediction accuracy, but did not report the sampling quality with respect to mode coverage or the sampling distribution for the Gaussian mixtures. Same in the hallucination fixing in 6.3, it did not report the sampling quality of the generated image samples with concrete metrics, and generated samples' class distribution.
Questions for authors
- What's in Equation (2)?
- What's the difference between mode core and mode center?
- In the second insight from section 5, the paper gives a conclusion that we need to generate large number of samples for low-confidence region to characterize it. I do not see a direct connection here because low posterior confidence does not necessarily imply low sample density, so it does not imply that many samples are required.
- To construct the training dataset, it requires to integrate all labeled data points backward through ODE to the noise in noise space. This would incur a large amount of computational cost, and no training time, GPU resources and related information are reported in the paper.
3 comments
(Q5) How does a mode core differ from a mode centre?
The centre is a single point, the component mean. The core is the closed ball around it. The radius of this ball is defined in Prop. 2 covering at least of the mode responsibility. As a closed ball it is a Borel set, so the probability of landing in it is well defined.
(Q6) Does low confidence imply low density?
No. Low-confidence regions refer to areas near mode intersections where the pullback map is heavily distorted and small seed perturbations can flip the class outcome, not areas of low data density. We sample anchors densely there to resolve complex, easily mischaracterized decision boundaries, whereas sparse anchors suffice in stable, high-confidence regions.
Fig R3. illustrates this on a three-mode mixture (). In data space (left), we place anchors on two concentric circles around each mode: a high-confidence circle at and a low-confidence circle on the core edge at . Under DDIM inversion to seed space (right), high-confidence anchors remain in compact loops, while low-confidence anchors stretch along the decision boundaries and crowd at the three-way junction (red disc).
(Q7) What does it cost to build the training set?
Building the seed-classifier training set takes under minutes once per diffusion model, which is only of diffusion training time at . Because dataset creation stays roughly constant in while diffusion training scales with , this overhead is only noticeable on small models that train in about a minute.
We benchmarked both costs on an RTX A4500 GPU using Gaussian mixture modes, steps, and anchors:
- Diffusion training: We train for steps, which takes at , at , and at .
- Anchor pullback: Pulling the anchors back to seed space is timed under three settings: using the exact field for Gaussian mixtures (Tab. 1, Tab. S15), running backward DDIM through the learned network, or adding fixed-point corrections per step.
- Total relative cost: We report the relative cost for computing pullback relative to diffusion training in Tab. S16. We find that it takes about at and at .
References
[1] J. Song, C. Meng, S. Ermon. Denoising Diffusion Implicit Models. ICLR, 2021.
[2] T. Karras, M. Aittala, T. Aila, S. Laine. Elucidating the Design Space of Diffusion-Based Generative Models. NeurIPS, 2022.
[3] D. P. Kingma, T. Salimans, B. Poole, J. Ho. Variational Diffusion Models. NeurIPS, 2021.
[4] B. Wang, J. J. Vastola. The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and Its Applications. arXiv:2311.10892, 2023.
[5] M. H. Ashiq et al. Why DDIM Hallucinates More Than DDPM: A Theoretical Analysis of Reverse Dynamics. ICML, 2026.
I thank the authors for the detailed rebuttal and additional figures and tables. However, several concerns still remain:
-
Figure 1 is revised but it still remains difficult to interpret. The squares, dashed circles and two lines extended from the square corners are not explained.
-
For prop.1: the rebuttal states that "a trajectory that enters the core cannot leave it", but I could not find the corresponding statement that explicitly or implicitly conveys this idea.
-
In the experiment section, there are two definitions for hallucination in section 6.2 (line 420) and section 6.3 (line 445), respectively. Moreover, I do not see a direct response to the 4th weakness I raised.
The revision also adds many new results and sections to the appendix. Given the short discussion period, I could not verify these additions thoroughly. This might need another full round of review. I choose to maintain my score.
We thank the reviewer for the clarifying questions and holding our work to a high standard. We address below point by point the concerns the reviewer has raised.
We provide references to relevant figures in the manuscript in our response.
(Q1) How can Figure 1 be interpreted?
It shows how DDIM partitions the seed space of a four-mode Gaussian mixture (centres marked ×) and where each seed ends up. We rewrote the caption (Fig. 1) to make this explicit.
(Q2) How is Yt defined?
Yt=Xt/αˉt is a standard rescaling used in prior work [1,2,5]. This ensures the component means of the GMM are fixed {μi}i=1K with a changing variance σ(t). We provide a visualization highlighting the normalized time σ(t) and original time t in Fig. S23 of the revised appendix.
(Q3) Do the propositions hold widely and how do they link to the experiments?
We respectfully disagree with reviewer. In our response below, we explain the validity of each proposition and its utility in the experiments. We include figures as links for visual exposition.
In summary, Proposition 1-3 are required to characterize the critical radius ri(σ(t),δ) for each class, and when in the reverse process it is clearly definable (t<tij). In Section 5, we pullback anchors on a spheres smaller than critical radius r<R99 for class prediction. Proposition 4 characterizes the normal to the boundary at the beginning of the reverse process (t=T) on applying pullback on r=R99, which is used for hallucination mitigation.
(Q4) What is σa?
It is the noise scale at the start of generation (t=T), σa:=σ(T), where σ(t)2=σ02+(1−αˉt)/αˉt (Fig. S23, right).