acceptodds
Under review as a conference paper at ICLR 2027

Theory of Outcome Boundary Prediction in a Determinstic Diffusion ODE

Abstract

Are there patterns in the unstructured noise from which a diffusion samples? Surprisingly, deterministic ODE samplers impose structure in the noise space. When the data distribution consists of separable gaussian clusters, a deterministic ODE-based sampler partitions the sampling noise based on the class the noise belongs to. The boundaries that separate these clusters can be mathematically characterized to represent decision thresholds in the noise space where the score field transitions between distinct modes, causing nearby seeds to produce ambiguous or hallucinated outputs. We formalize this correspondence and introduce a method that traces this mapping, which we call pullback. Using pullback, we can predict hallucinations, or classes in any mixture-of-Gaussian setting.

Primary area:
generative models
Keywords:
Diffusion Models, Sampling Dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_83qeOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether the class label and hallucination of a generated sample can be directly predicted or identified from its initial noise. It proposes to train a classifier on the noise-label pairs, where the noise is obtained through integrating data points back to noise space through deterministic ODE with the known label. Experiments are done on Gaussian mixtures and an MNIST dataset in its latent space to evaluate the prediction accuracy and hallucination fixing.

Critical strengths

This paper investigates an interesting problem: if we can predict the label of a generated sample directly from its initial noise by the invertible dynamics of a deterministic ODE.

Critical weaknesses
  1. Figure 1 is difficult to interpret. As the first figure to show the motivation of this paper, the caption does not clearly explain what the figure intends to convey.
  2. Section 3 introduces too many symbols without sufficient definitions. This includes flow map, endpoint map, \sigma(t), π\pi, the relationship between σ\sigma and tt. It is also not clear why YtY_t is defined as Xt/αˉtX_t/\sqrt{\bar\alpha_t}. They should be explicitly defined at their first usages.
  3. From my understanding, the propositions are somewhat narrow. Proposition 1 requires every x∈Rdx\in\mathbb{R}^d to satisfy the inequality, which is not necessarily true for cases. The proposition 4 seems stating the direction of gradient points towards the where the function changes fastest. It is also not very clear to me how to match eacb proposition conclusion to the insights in section 5.
  4. For the experiment in 6.2, it only reported prediction accuracy, but did not report the sampling quality with respect to mode coverage or the sampling distribution for the Gaussian mixtures. Same in the hallucination fixing in 6.3, it did not report the sampling quality of the generated image samples with concrete metrics, and generated samples' class distribution.
Questions for authors
  1. What's σa\sigma_a in Equation (2)?
  2. What's the difference between mode core and mode center?
  3. In the second insight from section 5, the paper gives a conclusion that we need to generate large number of samples for low-confidence region to characterize it. I do not see a direct connection here because low posterior confidence does not necessarily imply low sample density, so it does not imply that many samples are required.
  4. To construct the training dataset, it requires to integrate all labeled data points backward through ODE to the noise in noise space. This would incur a large amount of computational cost, and no training time, GPU resources and related information are reported in the paper.
3 comments

We thank the reviewer for the clarifying questions and holding our work to a high standard. We address below point by point the concerns the reviewer has raised.

We provide references to relevant figures in the manuscript in our response.

(Q1) How can Figure 1 be interpreted?

It shows how DDIM partitions the seed space of a four-mode Gaussian mixture (centres marked ×\times) and where each seed ends up. We rewrote the caption (Fig. 1) to make this explicit.

(Q2) How is YtY_t defined?

Yt=Xt/αˉtY_t = X_t/\sqrt{\bar\alpha_t} is a standard rescaling used in prior work [1,2,5]. This ensures the component means of the GMM are fixed {μi}i=1K\{\mu_i\}_{i=1}^{K} with a changing variance σ(t)\sigma(t). We provide a visualization highlighting the normalized time σ(t)\sigma(t) and original time tt in Fig. S23 of the revised appendix.

(Q3) Do the propositions hold widely and how do they link to the experiments?

We respectfully disagree with reviewer. In our response below, we explain the validity of each proposition and its utility in the experiments. We include figures as links for visual exposition.

  • Prop. 1 does not require Gaussian classes. It only bounds how far the class score may deviate from that of the Gaussian reference gc=N(μc,σ02I)g_c=\mathcal N(\mu_c,\sigma_0^2\mathbf I): κc:=sup⁡y∥σ02∇log⁡fc(y)+(y−μc)∥,\kappa_c:=\sup_y\big\|\sigma_0^2\nabla\log f_c(y)+(y-\mu_c)\big\|, Whenever κc<r\kappa_c<r the class score points inward everywhere on the sphere, even for a non-Gaussian class, and a trajectory that enters the core cannot leave it. Fig. R1 shows this on the two-mode example where one of the classes is a Gaussian crescent f2f_2. For the Gaussian components of our mixtures κc=0\kappa_c=0, so the bound holds at every radius and every tt.
  • Prop. 2 characterizes the critical radius such that 1−δ1 - \delta of the mass can be concentrated at that specific mode. For pullback we use R99R_{99} as a practical proxy for the critical radius at the end of the reverse process (t=0t = 0): at δ=10−4\delta=10^{-4} the certified ball ri(σ0,δ)r_{i}(\sigma_{0}, \delta) already contains the R99R_{99} ball for every mixture in our experiments (App. RA.1).
  • Prop. 3. identifies the critical time tijt_{ij} at which single mode dominance concludes and gaussian approximation of the class becomes inaccurate. We show this in Fig.R2 by tracing a trajectory which enters the contact region.
  • Prop. 4 computes the normal to the boundary by mapping νa\nu_{a}, which is normal to the gaussian level set of r=R99r = R_{99} around the mode at t=0t=0. This proposition is central to hallucination mitigation in Sec 6.3.

In summary, Proposition 1-3 are required to characterize the critical radius ri(σ(t),δ)r_{i}(\sigma (t), \delta) for each class, and when in the reverse process it is clearly definable (t<tijt < t_{ij}). In Section 5, we pullback anchors on a spheres smaller than critical radius r<R99r < R_{99} for class prediction. Proposition 4 characterizes the normal to the boundary at the beginning of the reverse process (t=Tt = T) on applying pullback on r=R99r = R_{99}, which is used for hallucination mitigation.

(Q4) What is σa\sigma_a?

It is the noise scale at the start of generation (t=Tt=T), σa:=σ(T)\sigma_a:=\sigma(T), where σ(t)2=σ02+(1−αˉt)/αˉt\sigma(t)^2=\sigma_0^2+(1-\bar\alpha_t)/\bar\alpha_t (Fig. S23, right).

(Q5) How does a mode core differ from a mode centre?

The centre μi\mu_i is a single point, the component mean. The core Ci∙:=B(μi,ri)C^\bullet_i:=B(\mu_i,r_i) is the closed ball around it. The radius of this ball is defined in Prop. 2 covering at least 1−δ1-\delta of the mode responsibility. As a closed ball it is a Borel set, so the probability of landing in it is well defined.

(Q6) Does low confidence imply low density?

No. Low-confidence regions refer to areas near mode intersections where the pullback map is heavily distorted and small seed perturbations can flip the class outcome, not areas of low data density. We sample anchors densely there to resolve complex, easily mischaracterized decision boundaries, whereas sparse anchors suffice in stable, high-confidence regions.

Fig R3. illustrates this on a three-mode mixture (d=2d=2). In data space (left), we place anchors on two concentric circles around each mode: a high-confidence circle at r=σr=\sigma and a low-confidence circle on the core edge at r=R99r=R_{99}. Under DDIM inversion to seed space (right), high-confidence anchors remain in compact loops, while low-confidence anchors stretch along the decision boundaries and crowd at the three-way junction (red disc).

(Q7) What does it cost to build the training set?

Building the seed-classifier training set takes under 33 minutes once per diffusion model, which is only 0.17–1.4%0.17\text{--}1.4\% of diffusion training time at d=512d=512. Because dataset creation stays roughly constant in dd while diffusion training scales with dd, this overhead is only noticeable on small models that train in about a minute.

We benchmarked both costs on an RTX A4500 GPU using K=16K=16 Gaussian mixture modes, T=500T=500 steps, and 48,00048{,}000 anchors:

  1. Diffusion training: We train for 30,000 (1+d/16)(1+K/16)30{,}000\,(1+d/16)(1+K/16) steps, which takes 74 s74\text{\,s} at d=4d=4, 307 s307\text{\,s} at d=64d=64, and 3.2 h3.2\text{\,h} at d=512d=512.
  2. Anchor pullback: Pulling the 48,00048{,}000 anchors back to seed space is timed under three settings: using the exact field for Gaussian mixtures (Tab. 1, Tab. S15), running backward DDIM through the learned network, or adding 22 fixed-point corrections per step.
  3. Total relative cost: We report the relative cost for computing pullback relative to diffusion training in Tab. S16. We find that it takes about 19−39%19-39\% at d=4d = 4 and 0.17−1.4%0.17 - 1.4\% at d=512d = 512.

References

[1] J. Song, C. Meng, S. Ermon. Denoising Diffusion Implicit Models. ICLR, 2021.

[2] T. Karras, M. Aittala, T. Aila, S. Laine. Elucidating the Design Space of Diffusion-Based Generative Models. NeurIPS, 2022.

[3] D. P. Kingma, T. Salimans, B. Poole, J. Ho. Variational Diffusion Models. NeurIPS, 2021.

[4] B. Wang, J. J. Vastola. The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and Its Applications. arXiv:2311.10892, 2023.

[5] M. H. Ashiq et al. Why DDIM Hallucinates More Than DDPM: A Theoretical Analysis of Reverse Dynamics. ICML, 2026.

I thank the authors for the detailed rebuttal and additional figures and tables. However, several concerns still remain:

  • Figure 1 is revised but it still remains difficult to interpret. The squares, dashed circles and two lines extended from the square corners are not explained.

  • For prop.1: the rebuttal states that "a trajectory that enters the core cannot leave it", but I could not find the corresponding statement that explicitly or implicitly conveys this idea.

  • In the experiment section, there are two definitions for hallucination in section 6.2 (line 420) and section 6.3 (line 445), respectively. Moreover, I do not see a direct response to the 4th weakness I raised.

The revision also adds many new results and sections to the appendix. Given the short discussion period, I could not verify these additions thoroughly. This might need another full round of review. I choose to maintain my score.

Reviewer_J5NfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how deterministic ODE diffusion generators partition initial noise into sets leading to different mode zone outside those zone. For separated Gaussian mixtures with a common isotropic covariance, it derives a sufficient radius for single-component responsibility, analyzes contact between isolated reference spheres, and uses the generator Jacobian to obtain the pulled-back boundary normal and locally optimal perturbation direction. The algorithm samples from terminal cores and surrounding shells, adds noise, and trains predictors. Experiments include Gaussian mixtures up to 512 dimensions, several deterministic samplers, and MNIST dataset, with additional results on repairing samples outside the cores.

Critical strengths

The mathematical part is solid. The relationship between core zone and noise-space boundaries is mathematically explicit. It avoids treating entire Gaussian component supports as disjoint class regions and provides an analyzable object for seed control.

The experiments are clear and comprehensive, exposing both the capabilities and failure regimes of the predictors. These expeirmental results help identify conditions under which noise-space regions can be approximately recovered.

Critical weaknesses

GMM assumption is too strong. It limits the practical applicability of the theory and method. This theory may fail in text-to-image model, while restricting the method to class-labeled generative models would narrow its application.

Seed selection and repair address a narrow application objective and offer limited incremental value. For tasks that require only a valid sample, there is no intrinsic need to preserve a failed random seed; resampling or filtering generated samples are effective naive choices. Therefore, practical benefits requires a clearly defined use case for seed selection and repair, along with sufficiently frequent reuse of the proposed method.

Questions for authors

See weaknesses above.

2 comments

We thank the reviewer for the comments. We are glad that he finds the mathematical part solid and the experiments a comprehensive representation of the theory. Below, we address the reviewer's concerns about our work.

We provide references to relevant figures in the manuscript in our response.

(Q1) How does the theory extend to text-to-image models ?

Our theory extends seamlessly to text-to-image models. Classifier-free guidance [1] changes only one term of the theory, so the guided pullback has a closed form (Prop. 10, App. RD). We test the text-to-image diffusion on a latent text-to-image model trained on MNIST and conditioned on the CLIP [2]. The prompt used is "a handwritten digit kk". We execute the run on 20,00020{,}000 seeds per prompt with varying guidance weight Fig. S17. The images are embedded into a d=8d = 8 dimensional latent space.

We find that guided pullback predicts the outcome of almost every seed at all guidance weights ww, while a pullback that ignores the prompt fails once guidance is applied:

Guidance weight ww00.51235
Guided pullback (ours)97.492.698.999.899.698.1
Pullback ignoring the prompt≈97{\approx}9736.6≈10{\approx}10≈10{\approx}10≈10{\approx}10≈10{\approx}10

Values are the share (%) of seeds whose outcome is predicted. In Fig. S17 (a), a single seed that gives a 3 without guidance is steered to each prompted digit, and the guided pullback recovers the boundaries at every guidance weight.

(Q2) Why is Filtering and Resampling not an alternative to Seed Repair?

Our goal is to retain the details of generation in our output while fixing hallucinated results. In this vein, resampling the seed and filtering failed generations are weaker fallbacks than they appear. We provide two examples of this.

  • Cost of Resampling. Sampling a class that occurs with probability pp in the training distribution requires a number of runs that grows as 1/p1/p. For instance, a rare class appearing in 1%1\% of generations needs 100100 runs to produce a new sample. Once anchor points have been recovered via pullback, however, we can apply pullback once and execute it in an O(1)O(1) call to a Polar3 classifier.
  • Filtering during sampling is slow. IQ [3] and RODS [4] detect a hallucination along each trajectory and steer it, so the correction has to be repeated for every sample. Correcting every hallucinating seed takes IQ ≈12.6\approx 12.6 s on MNIST and ≈46.3\approx 46.3 s on F-MNIST, against ≈4.4\approx 4.4 s and ≈11.2\approx 11.2 s for the pulled-back normal translating to 2.9×2.9\times and 4.1×4.1\times faster correction per generation. Furthermore, IQ fixes only 27%27\%/47%47\% into the intended class against 84%84\%/89%89\% for ours (Tab. 2). RODS fixes at most 4%4\% on MNIST and F-MNIST (Tab. S5).

Seed selection and repair offers an efficient and reliable alternative to correcting a failed generation while retaining the desired specific contents of the image such as the class label.

(Q3) What are the practical use case of seed selection and repair?

Choosing the seed is an active area of research. For a large scale text-to-image model, reliable seeds can produce compositionally generalizable [5]. A well chosen single noise vector improves frozen diffusion robot policies on manipulation tasks without retraining [6].

Our focus application is on repairing a failed generation, i.e retaining the key details of a class while removing the hallucination. In this vein, IQ [3] and RODS [4] keep the seed fixed and steer the reverse trajectory, separately for every trajectory. Our pullback atlas is built once and reused to correct any number of hallucinated samples. Fig. 6 (more in Fig. S9) shows that class details are preserved while the sample is corrected. On the other hand, the pulled-back normal fixes the most hallucinations (Tab. 2, Tab. S5) with the smallest displacement (Tab. S4). The use case is stated in App. RB.1 of the revised paper.

References

[1] Ho & Salimans (2022). Classifier-free diffusion guidance. arXiv:2207.12598.

[2] Radford et al. (2021). Learning transferable visual models from natural language supervision. ICML.

[3] Sobieski et al. (2026). Local intrinsic dimension unveils hallucinations in diffusion models. arXiv:2605.05026.

[4] Tian et al. (2025). RODS: Robust optimization inspired diffusion sampling for detecting and reducing hallucination in generative models. NeurIPS 38.

[5] Li et al. (2024). All seeds are not equal: Enhancing compositional text-to-image generation with reliable random seeds. arXiv:2411.18810.

[6] Patil et al. (2026). You've got a golden ticket: Improving generative robot policies with a single noise vector. arXiv:2603.15757.

[7] Mokady et al. (2023). Null-text inversion for editing real images using guided diffusion models. CVPR.

Reviewer_Wm9gOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The work examines whether a deterministic diffusion sampler (DDIM or flow matching) yields an output predictable from its initial noise when data are a mixture of well-separated isotropic Gaussians. Its theory introduces dominance radii around each mode, a contact time for transported reference spheres, and the normal of the pulled-back boundary. Labeled anchors are mapped back to seed space, and a classifier is trained to predict either a seed's mode or a "hallucination". Experiments involve synthetic GMMs, a DDIM in an MNIST latent space, and a seed-correction procedure.

Critical strengths

The problem is practically motivated, and its formulation is unambiguous. GMM experiments are broad, with multiple samplers covered.

Critical weaknesses
  1. Coverage of the titled object. The analytical results leave the object named in the title outside their scope: dominance radii and contact times are derived for balls and transported reference spheres, not for the actual seed-space regions. Proposition 4 supplies only the normal at a point already known to be on the boundary, while the authors themselves observe that no closed form exists for the pulled-back cores. Predictions come from a classifier fitted to anchors; no error bound is given for it, and the shape of the outcome regions is not characterized. Consequently, the theory of outcome-boundary prediction does not encompass the predictor that is actually assessed.

  2. Experiments and stated results. The manuscript claims exact equality between the critical radius in Proposition 2 and R99R_{99}; that equality fails in general, yet R99R_{99} is what the experiments employ. Seed correction applies the boundary normal but takes a large step from off-boundary seeds, a regime not covered by a first-order result. No seed-space baselines are included, such as resampling the seed, shrinking it radially, or moving it in a random direction of the same norm.

  3. What the "hallucination" label captures. The quantity measured by the "hallucination" label remains ambiguous. Hallucinations are described as interpolations between modes, yet in the GMM experiments any sample outside all R99R_{99} balls is counted, and the data distribution itself places up to roughly 1% of its mass in that region. DDIM and flow matching generate about 1% of such samples; model errors are not separated from legitimate tail samples. Should many of these be tail samples, "correcting" them would discard valid samples. Given the rarity of these cases, overall accuracy in Fig. 4 carries little information about hallucination prediction.

  4. Thin real-data support. Evidence on real data is limited. In the MNIST case, the main text gives only the correction rate and omits classifier accuracy. Class labels are used to train the MNIST latent space, a choice that fits uneasily with the claim that no labels are required. Fashion-MNIST results, although announced, are absent.

Questions for authors
  1. For a seed lying off the boundary, what procedure yields the correction direction, and how does this correction perform relative to the simple baselines listed above?

  2. Is it possible to provide a bound either on the hallucination region's size or on the error of the anchor-based predictor?

  3. In the GMM setting, which fraction of the detected hallucinations would the exact sampler itself generate? Does correction preserve the mode proportions?

2 comments

We thank the reviewer for their detailed comments and are glad they found our formulation clear and unambiguous. In our response, we elaborate on our theory's applicability to image experiments and its efficacy in mitigating hallucinations. Specifically, while Pullback is rigorously developed on Gaussian mixtures, we find that it extends to general class-separated distributions, even when the underlying data is not strictly a Gaussian mixture.

(Q1) Do the transported spheres describe the actual seed regions?

Yes. For a well-separated GMM, the regions deform only along directions joining the mode centres, so the outcome depends on at most K−1K-1 coordinates (App. RC.3). Polar3 is exactly capable of classifying isolated modes and provides a bounded error for interacting ones (App. RC.6–RC.7). This characterization can be accurately made for t>tijt > t_{ij}, when several modes may interact. In our experiments, this translates to an accuracy of >80%>80\% up to d=512d=512 showing superior performance compared to kNN and w-kNN baselines.

(Q2) Can we provide an error bound for the Polar3 classifier?

We thank the reviewer for the suggestion. On careful analysis, we find that the Polar3 admits an error bound on the anchors transported using pullback. In particular, we find that it's VC dimension is atmost 2Wlog⁡2(128eK7)2W\log_2(128eK^7) [1] (App. RC.8), and held-out seeds give an assumption-free binomial bound [2] (App. RC.9).

(Q3) How does the critical radius relate to R99R_{99}?

They are not equal in general, but for any tolerance δ≥δ⋆\delta\ge\delta^\star the certified ball ri(σ,δ)r_{i} (\sigma, \delta) contains the R99R_{99} ball (App. RA.1). Since δ⋆<10−4\delta^\star<10^{-4} for d=2d=2–3232, labeling by R99R_{99} stays within the guarantees of the theory.

(Q4) Does the correction hold for large steps?

Yes, however very large steps can overshoot into a neighboring mode. We characterize this explicitly using the Lipschitz constant LL of the score function: a step of size t<ϵ<t+2ga/Lt<\epsilon<t+2g_a/L may be taken without overshooting (App. RA.2). We illustrate this on a three-mode GMM (Fig. R1), stepping a hallucinating seed near the junction along the normal na\boldsymbol n_a of its nearest anchor.

Near the junction the outcome changes abruptly over a very small distance in seed space, i.e. LL is high, so 2ga/L2g_a/L shrinks and even a moderate step crosses into a neighbouring mode. Large steps therefore succeed for hallucinations away from the junction and fail for those near it; for example, ϵ=0.22\epsilon=0.22 reaches the correct mode while ϵ=0.92\epsilon=0.92 overshoots into a neighbouring one. The same holds on image datasets: 94%94\%/80%80\% of hallucinations are fixed at ϵ=3\epsilon=3 but only 35%35\%/2%2\% at ϵ=8\epsilon=8 because an overshoot can switch the class. For instance, a sandal can be switched to the sneaker mode (Fig. S9, row 3). We provide more examples in Fig. R2 and Fig. R3.

(Q5) How is the correction direction found, and how does it compare with baselines?

One backward pass gives the seed direction that moves the sample fastest toward the nearest (intended) mode centre, and we step ϵ\epsilon along it. At ϵ=0.5\epsilon=0.5 this fixes 8484–91%91\% into the intended class on three datasets, against at most 75%75\% for radial shrink, 37%37\% for a random direction and about 10%10\% for resampling (Tab. S3, Tab. S6). It also beats popular baselines IQ [3] and RODS [4] (Tab. S5, Tab. 2, Fig. 6) on hallucination mitigation.

(Q6) What is the definition of hallucination used in the manuscript?

We count every sample outside all R99R_{99} balls, both interpolations and other off-mode samples such as tails. We introduce a correction in the manuscript (Sec. 6.1) which makes this explicit.

(Q7) Can model errors be told apart from tails samples from the gaussian mode?

A generated sample inside a ball belongs to that mode and outside every ball is what we call a "hallucination" (App. G, Eq. G.2). Our classifier is focused on learning class outcomes and the hallucination label is used for those samples that do not clearly belong to any specific class. Accordingly, we account for everything that is outside the mode r>R99r > R_{99} as a hallucination. We include this clarification in the updated manuscript (App. RB.5).

(Q8) Does Fig. 4 measure hallucination prediction?

No. Figure 4 presents the accuracy of pullback on outcome prediction and not hallucination prediction. Nonetheless, we report hallucination F1 across all settings (Fig. S26, S27). The hallucinated samples occupy only about 1%1\% of the sampling space at T=1000T=1000 steps (0.960.96–1.33%1.33\%, Tab. S12). A learnt Polar classifier recovers their decision boundaries better than KNN and w-KNN.

(Q9) Can the thickness of the hallucination region be computed?

Yes (App. RB.3). Thickness is measured by applying the pullback at an anchor on a mode core's edge and tracing along the resulting normal until entering another core, with the distance traveled expressed in units traveled space (Fig. S14). The thickness of the band, increases on reducing the number of sampling steps. At T=10T = 10 the median thickness is 0.0220.022 and at T=500T=500, the median thickness is 0.0070.007 (Tab. S9). and its mean stays below 0.080.08 for d=2d=2–512512, compared to a seed length of ≈d\approx\sqrt d (Tab. S10).

(Q10) How accurate is the MNIST classifier?

The quadratic classifier achieves 96.3%96.3\% accuracy on MNIST and 91.5%91.5\% on F-MNIST, when each of the classes are embedded on a sphere of radius R=4R=4. A CNN detector further confirms that each cluster accurately recovers its target class (Tab. S7).

(Q11) What is the performance on MNIST and F-MNIST?

Outcome prediction reaches 96.3%96.3\% (MNIST) and 91.5%91.5\% (F-MNIST) on a completely trained checkpoint (App. RB.2, Tab. S8). Correction is run on an earlier checkpoint, where the best classifier reaches 88%88\% and 76.8%76.8\% on MNIST and F-MNIST respectively. On applying pullback, one step along the pulled-back normal fixes 84%84\% and 89%89\% of hallucinations into the intended class, against at most 47%47\% for IQ (Tab. 2).

(Q12) Do the Fashion-MNIST results support the paper's main claim?

The latents form ten roughly Gaussian clusters (Fig. S12), and accuracy is 91.5%91.5\% for FashionMNIST and 96.3%96.3\% for MNIST. The learned sampler hallucinates on 1.43%1.43\% of seeds (Fig. S13). The mitigation strategy fixes 89%89\% of hallucinations versus 47%47\% for IQ (Tab. 2, Fig. S11).

(Q13) Does the exact sampler hallucinate too?

Gaussian Mixture The exact samplers hallucinate at nearly the same rate as learned ones across T∈{250,500,750,1000}T \in \{250, 500, 750, 1000\}. Exact DDIM on 1.32%1.32\%, 1.28%1.28\%, 1.27%1.27\% and 1.27%1.27\% of seeds and exact flow matching on 1.001.00–1.01%1.01\%, against 0.860.86–1.33%1.33\% and 0.760.76–0.96%0.96\% for the learned samplers (Tab. S12).

MNIST and F-MNIST: Evaluated under the same sampling schedule, the exact sampler yields 0.0%0.0\% hallucinations on both MNIST and F-MNIST, whereas the learned sampler hallucinates on 0.10%0.10\% and 1.43%1.43\% of seeds, respectively. The true probability flow reliably maps noise onto the data manifold, isolating neural network approximation error as the sole source of hallucinations on these datasets.

(Q14) Does correction preserve mode proportions?

Yes. We find that after a sufficiently small correction can reduce hallucinations in MNIST and Fasion-MNIST without affecting the mode proportions. We report the exact values in points (Tab. S13, S14) in the appendix of the paper.

References

[1] Goldberg & Jerrum (1995). Bounding the Vapnik–Chervonenkis dimension of concept classes parameterized by real numbers. Machine Learning 18.

[2] Clopper & Pearson (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26(4).

[3] Sobieski et al. (2026). Local intrinsic dimension unveils hallucinations in diffusion models. arXiv:2605.05026.

[4] Tian et al. (2025). RODS: Robust optimization inspired diffusion sampling for detecting and reducing hallucination in generative models. NeurIPS 38.

[5] Aithal et al. (2024). Understanding hallucinations in diffusion models through mode interpolation. NeurIPS 37.

[6] Ashiq et al. (2026). Why DDIM hallucinates more than DDPM: A theoretical analysis of reverse dynamics. ICML.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Changes in the Revised Manuscript

We thank all reviewers for their careful reading and constructive questions. Despite the initial low scores we believe that we address all the comments provided by the reviewers in this rebuttal. We would be grateful if the reviewers can consider submitting revisions. In our response below we summarize what has changed in the revision.

Anonymous repository. Figures and animations of the seed-space regions, the pullback and the seed correction are available at https://anonymous.4open.science/r/iclr2027_diffusion_ode_rebuttal-CE56/

Corrections to the main manuscript. We have made direct corrections to the main text; changed text is shown in red. The new experiments are now part of Sec. 6 (Experiments).

  • Sec. 1 (Introduction). The caption of Fig. 1 is updated to illustrate the example more clearly: on the left, the seed space partitioned by the mode each seed reaches (four mode basins and the hallucination lines in red); on the right, where the same seeds land after 200 deterministic DDIM steps. The paragraph that introduces the pullback now states how the seed-space regions are found: anchor points are chosen in data space and transported to the seed space, and for a Gaussian mixture they are placed on spheres around each mode.
  • Sec. 5 (The Pullback Algorithm). R99R_{99} is stated as a practical proxy for the critical radius rir_i of Prop. 2, replacing the earlier claim that the two are equal. At δ=10−4\delta=10^{-4} the certified ball ri(σ,δ)r_{i} (\sigma, \delta) contains the R99R_{99} ball (App. RA.1, Fig. S7).
  • Sec. 6.3 (Pullback on images: MNIST and Fashion-MNIST). A new setup paragraph for the image experiments: the latent space of an autoencoder (d=32d=32 for MNIST, d=64d=64 for Fashion-MNIST), a mixture fitted by kk-means without labels and a DDIM trained on the latents (full settings in App. RB.2).
  • Sec. 6.4 (Computational cost of pullback).\color{green}{\textbf{Sec. 6.4 (Computational cost of pullback).}} The time to pull back every anchor under DDIM and flow matching, under 90 seconds even at d=512d=512 with 480,000480{,}000 anchors (Tab. 1). Outcome prediction on images, with a newly trained autoencoder: 96.3%96.3\% on MNIST and 91.5%91.5\% on Fashion-MNIST. Correction: one step along the pulled-back normal fixes 84%84\% (MNIST) and 89%89\% (Fashion-MNIST) of hallucinating seeds into the intended class, against at most 47%47\% for IQ, and is 2.92.9 and 4.14.1 times faster (Tab. 2, Fig. 6).

Appendices for the rebuttal (Sec. RA–RD).\color{green}{\textbf{Appendices for the rebuttal (Sec. RA–RD).}} These are shown in red and come before the submitted appendices, which keep their letters A–I.

  • RA, from the propositions to the experiments: RA.1 relates the certified radius of Prop. 2 to R99R_{99} (Fig. S7); RA.2 bounds a finite correction step from an off-boundary seed.
  • RB, additional experiments: seed-space baselines for correction (RB.1); pullback on images, including Fashion-MNIST (RB.2); thickness (RB.3) and mass (RB.4) of the hallucination region; the exact sampler and mode proportions (RB.5); the computational cost of the pullback (RB.6).
  • RC, Polar3: A new decision rule, the span reduction of the endpoint map, exact representation of an isolated mode, a cubic approximation for interacting modes, its VC dimension, and a held-out bound on the error of the fitted predictor.
  • RD, pullback with classifier-free guidance: a closed form for the guided pullback (Prop. 10), and a text-to-image latent diffusion model on MNIST prompted with CLIP embeddings, where the guided pullback predicts 92.692.6–99.8%99.8\% of outcomes across guidance weights (Fig. S17).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.