acceptodds
Under review as a conference paper at ICLR 2027

Flicker-DDPM: Accelerating Denoising Diffusion Sampling via - Type Colored Noise

Abstract

Natural images' approximate power-law spectra motivate Flicker Diffusion, which couples -type spatially correlated Gaussian noise with a matching whitened objective. It retains the baseline U-Net and optimizer-update budget without additional training stages. On CIFAR-10, Flicker-DDIM achieves FID-50k 6.289 at /NFE20, versus 6.292 at /NFE100 for white DDIM: fivefold fewer network evaluations (NFE) at similar observed FID for these settings. A separate controlled test (both /NFE40) yields versus over three paired training seeds after identical 15-candidate sampler searches. Matched-schedule ancestral sampling reduces mean FID by approximately 66% on CelebA64 and 82% on LSUN Church64. Covariance-consistent stochastic differential equations and a perturbative path integral describe a leading affine response, its nonlinear corrections, and a bound on residual effects on spectral evolution. Held-out diagnostics and forward-to-reverse spectral predictions on a separate historical CIFAR model pair support a more nearly affine low-frequency response. Complementing the sampling gains, our analysis provides a theoretical framework for understanding partial spectral linearization.

Primary area:
generative models
Keywords:
diffusion models, colored noise, spectral analysis, stochastic differential equations, self-organized criticality

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hVthOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Flicker Diffusion, which replaces standard spatially independent Gaussian noise with spatially correlated Gaussian noise whose covariance decays as a power law with the Manhattan distance between pixels. The method uses the same covariance consistently in both the diffusion process and a corresponding whitened training objective. Its motivation is to better match the spatial and spectral correlations of natural images and thereby make the denoising dynamics across different frequency modes more synchronized. Empirically, Flicker Diffusion can achieve, in the low-NFE (e.g. 20) regime, sample quality comparable to white-noise diffusion using substantially (e.g. 5x) more network evaluations. This advantage appears to saturate as the sampling budget increases.

Critical strengths
  1. Simple and interesting design.
    The paper introduces a simple way to incorporate structured spatial correlations into diffusion training while keeping the model architecture unchanged. The same formulation can be used consistently during training and sampling, without requiring an additional distillation or post-training stage.

  2. Careful empirical validation.
    The experiments are relatively thorough, with multiple datasets, paired training seeds, matched training budgets, and controlled sampler selection. The paper also reports unfavorable settings, including cases where Flicker does not outperform the white-noise baseline, which makes the empirical conclusions more credible.

Critical weaknesses
  1. The practical advantage appears limited to the low-NFE regime.
    Flicker performs well with few sampling steps, but its performance saturates as NFE increases, while the white-noise baseline continues to improve and can outperform it at higher NFE. This limits the performance ceiling of the method, which is important because a strong high-NFE model can later serve as a better teacher for few-step distillation.

  2. The validation is limited to small pixel-space diffusion models.
    The experiments are mainly conducted on CIFAR-10 and 64×64 datasets with U-Nets. It remains unclear whether the same covariance design transfers to modern latent diffusion or DiT-based models, where the representation is higher-dimensional and has very different spatial and spectral statistics.

Questions for authors
  1. The main models are trained for 90k updates. Have the authors tried training both the white-noise and Flicker models for longer? It would be useful to know whether the observed performance ceiling remains with a larger training budget.
  2. Could the authors improve the presentation to make the experimental settings easier to follow? For example, T=150/NFE40 is a bit of unnatural.
  3. I acknowledge the effectiveness of the idea in short horizon, but the current evaluation setting feels somewhat dated, with most experiments conducted on small-scale pixel-space U-Nets and mainly evaluated by FID. Could the authors provide some results on a more modern setting, such as latent diffusion or DiT-based models at higher resolution, to show whether the method still transfers? (If computationally not applicable, I'd like to hear the discussion or attempts on it.)
  4. The paper evaluates DDIM and DPM-Solver++, but several stronger modern solvers are now available. Could the authors also compare Flicker with solvers such as UniPC and iPNDM [1] under the same NFE budget to see whether the low-NFE advantage still holds?

[1] UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models

Reviewer_L31LOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper combines spatially correlated Gaussian noise with a matching whitened training objective to improve diffusion sampling. It develops covariance-consistent samplers and a spectral analysis of the learned denoising response. Experiments show substantial improvements under selected short training paths, supported by multi-seed comparisons and extensive controls. However, the acceleration claim remains configuration-dependent, and the proposed spectral explanation is not yet clearly connected to the sampling gains.

Critical strengths
  1. Coherent covariance–objective design. The forward process, training loss, sampling prior, and reverse dynamics consistently account for the noise covariance.

  2. Experimental results are promising. At T=150 and NFE40, Flicker improves CIFAR-10 FID from 14.445 to 6.414 across three training seeds after equal-budget sampler searches.

  3. Extensive diagnostic evaluation. The appendix includes covariance–loss ablations, matched-terminal controls, alternative samplers, distribution-coverage metrics, and learned-field diagnostics.

Critical weaknesses
  1. The fivefold NFE reduction compares selected operating points rather than equally optimized quality–cost curves. Please optimize both methods at each NFE under matched search budgets before making a broader acceleration claim.

  2. The gains depend strongly on the training path. At T=500, tuned white-noise models outperform Flicker, while the matched-terminal benefit is smaller. Please add a matched-terminal DDIM comparison at the main low-NFE budget to clarify when Flicker is advantageous.

  3. The spectral analysis does not establish why generation improves. Flicker improves low-frequency affine fits but has worse global fit scores and larger split-step defects. Please connect the proposed mechanism to FID through targeted interventions, or present it strictly as a descriptive analysis.

  4. Practical speed and scalability remain untested. Covariance sampling and whitening add computation, particularly for the dense implementation. Please report end-to-end training and sampling time, peak memory, and higher-resolution results.

  5. Comparisons with related covariance designs are not fully matched. The EqualSNR-style controls change covariance, boundary conditions, and schedules together. Please isolate these factors and include a comparison following the original method’s training protocol.

  6. The whitening ablation also changes optimization scale. Please normalize or tune gradient scale across loss variants to distinguish the benefit of frequency weighting from a change in effective optimization strength.

  7. Clearly separate historical and primary results in the main figures and tables.

  8. Measuring the proposed residual bound on representative trained models would strengthen the connection between theory and diagnostics.

Questions for authors

See Weaknesses.

Reviewer_rLoHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Flicker Diffusion, which replaces the white Gaussian noise of DDPM/DDIM with spatially correlated noise whose covariance decays as a power law of pixel distance, together with a whitened training objective. The covariance exponent is guided by the power-law spectrum of natural images. The paper claims faster sampling, e.g., similar CIFAR-10 FID with 20 network evaluations as white-noise DDIM with 100, and presents a theoretical framework of "partial spectral linearization".

Critical strengths
  • Adapting the noise covariance to image statistics is a reasonable direction.
  • The paper is transparent about several negative results in the discussion and appendix.
Critical weaknesses
  • The claimed contributions are not solid

    1. The "data-motivated covariance–objective design" is not new: correlated-noise diffusion with a matched (whitened) objective is already covered by prior non-isotropic diffusion frameworks, as the paper itself acknowledges.
    2. The choice of η\eta is an ad hoc heuristic rather than genuinely data-driven: the rule is stated as a candidate heuristic, the value used is effectively selected by sweeping, and results are sensitive to it. 3.The theoretical framework is not evaluated (the residual bounds and corrections are not computed) and does not explain the reported gains.
  • The acceleration claim is not supported. The gains appear only at the short training horizon (T=150) or in cross-horizon comparisons, such as Flicker at T=150 with 20 NFE versus white noise at T=500 with 100 NFE. Under matched T=500 settings, white noise still gives the best results (Table 9), and stronger solvers reduce the claimed speed-up further. As a new diffusion training strategy, this does not demonstrate a genuine improvement over a properly configured white-noise baseline.

  • Presentation is not at the level of a top-tier venue. Many idiosyncratic terms are introduced without intuitive explanation; configuration labels and three-decimal numbers are interleaved in the abstract and main text; results come from many different protocols, making it unclear which comparisons are matched; key negative results appear only in the discussion and appendix.

Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.