acceptodds
Withdrawn from ICLR 2027

A Trail from Low-Precision Sampling: Diagnosing Super-Diffusive FP32–BF16 Drift in Diffusion Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Low-precision inference is now the default deployment path for diffusion image generators: BF16 is standard and FP8 is spreading fast. The rounding error this introduces is usually treated as small, zero-mean noise that averages out over the tens of denoising steps of a sampler. We show it does not average out. Running SDXL from an identical seed in FP32 and BF16, we find that the same-seed latent trajectories do not wander apart like independent noise: their per-step L2 divergence grows super-diffusively, with a log–log growth exponent of 1.04 (diffusive random-walk null = 0.5) and a locatable takeoff around step 25 of 50. This is a robust same-seed phenomenon (n=64 pairs), and we pin down its cause with an unusually thorough elimination: five controls that rule out the tempting statistical explanations one at a time. It is not a mean bias (sign-bias statistic |b| ≈ 0.012). It is not the error’s step-to-step directional correlation: although consecutive error vectors are nearly collinear (cos(e_t, e_t+1) = 0.971), that merely rides the trajectory’s own velocity autocorrelation (0.979), and a magnitude-matched but uncorrelated Gaussian perturbation injected afresh at every step reproduces the super-diffusion (exponent 1.01 ≈ 1.04). And it is not a fixed-magnitude dynamical amplification: the same continual injection at constant per-step magnitude is merely diffusive (0.43), while a one-time kick is damped (0.19). What remains is a growing-magnitude positive feedback: the per-step rounding error scales with the accumulating separation, so as the same-seed trajectories diverge the error grows and drives them further apart—faster than √t, independent of the error’s mean or correlation. Crucially, we do not stop at inferring this feedback but measure it directly: regressing the per-step divergence increment on the current divergence is decisively positive (Spearman 0.89, positive for all 64 of 64 prompts). We package these measurements as DriftProbe, a training-free diagnostic—usable on any UNet or DiT block—that reports the per-step sign bias, the same-seed divergence curve and its onset t*, and the injection controls that isolate the mechanism. We treat the drift as a lens, not a defect to correct, and lay out how t* could serve as an early signal for downstream generation defects. Results are measured on SDXL/BF16 on a single 48 GB GPU (n=64 same-seed pairs; n=16 per injection control); the FP8 arm, the FLUX backbone, and the defect-prediction evaluation are scoped as future work.

Primary area:
generative models
Keywords:
diffusion models, low precision, bfloat16, numerical error, sampling, SDXL, super-diffusion, inference, interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_KgKKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper shows that FP32 and BF16 diffusion trajectories from the same seed diverge super-diffusively over sampling steps. It investigates the source of this drift and argues that growing per-step numerical errors, rather than simple bias or error correlation, drive the divergence.

Critical strengths

Nothing

Critical weaknesses

This is an AI slop. The paper is extremely poorly written. The language is not for human understanding. In the appendix there are tables filled with TBD... The entire paper has zero image for visualization despite the studying on image generation models. This is not even close to the acceptance standard.

Questions for authors

N/A

Reviewer_kch2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies the effect of numerical precision (BF 16 vs FP 32) in diffusion model sampling.

The main experiment focuses on measuring the trajectory distance between ∣∣xt,32−xt,16∣∣||x_{t,32} - x_{t,16}|| given they start from the same initialization. The central finding is the distance does not like random error (which should scale as t\sqrt t, but instead scale roughly linearly as a function of time. The paper also shows the error can be propagated to influencing the final image outcome. The paper then proposes DriftProbe as a diagnosis tool.

Critical strengths

The problem is interesting, and potentially useful.

Critical weaknesses

The paper is not complete. There are many 'TBD' in the appendix. Also there seems to be authors comments left in the main text (e.g. lines 73-83, there are comments like "supported", "scoped as future work; specified but not yet run.".

The practical questions asked in the intro (e.g. six-fingered hands) is not discussed at all in the paper.

I felt the linear growth of error can potentially be explained by linear accumulation of the rounding error over time. I think the author can simple simulate a simple time series (e.g. Brownion motion) and then use BF16 vs FP 32 to record their distance over time to test this hypothesis.

Questions for authors

I recommend rejection as the paper is not a complete manuscript in its current form.

Reviewer_Lif3OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces an understanding of how errors build up during sampling and address how the gap between low precision and high precision samplers increases with diffusion steps. The authors argue that such a separation is identifable at a critical time in the diffusion process. To diagnose, their understanding they use latent divergence between the two precisions and propose 3 hypothesis to explain the mechanism of divergence.

Critical strengths

The paper identifies that large diffusion models like SDXL can suffer because of low precision sampling from trajectory divergence which can significantly affect generation quality.

Critical weaknesses
  1. The paper is fully written by AI and extremely hard to read throughout. Although the observation of low precision sampling may itself have some merit, the analysis is extremely narrow and is not supported by strong evidence, i.e the cause of the observed deviation is not clearly explained.

  2. The experiments are restricted to SDXL, which utilizes a U-Net based backbone and a DDIM style gaussian diffusion path. The authors do not conduct experiments on the newer variants of the Stable Diffusion (SD3.5) which are based on flow matching. The paper claims that the observation generalizes to these variants but no evidence is shown for this observation.

Questions for authors

Questions:

  1. How is Figure 1 affected on using SD3.5, which uses an MMDIT instead of U-Net?

  2. In the experiments conducted on U-NET, which block affects the growth of the sampler?

  3. In the Appendix in Table G, the defect precision and drift sumamries shows allow values as TBD. Could the authors provide the values?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.