acceptodds
Under review as a conference paper at ICLR 2027

When Are Diffusion Recovery Clocks Identifiable? A Spectral Measurement Protocol for Teacher-Forced Denoising

Abstract

During teacher-forced diffusion denoising, low-frequency structure typically becomes recoverable before high-frequency detail, but this ordering could reflect image statistics, latent decoder effects, or a generic model bias. Standard latent-versus-pixel comparisons cannot isolate these explanations because they change representation, objective, architecture, and data scale together. We show that, for power-law natural spectra measured in log-spaced bands, an affine data-spectrum clock is nearly indistinguishable from a monotone band-index clock. Guided by a Gaussian posterior-mean null, we develop a content-sensitive recovery protocol with calibration-strength tests and spectrum-breaking positive controls. On SD3, the data-spectrum clock predicts held-out recovery better than a restricted diagonal pullback proxy but not reliably better than band index on natural images; PixelDiT replicates the coarse-to-fine ordering. Under the prespecified population-average log-radial measurement, engineered non-power-law spectra separate the candidate clocks, although the result is not basis-invariant. We also show that high-frequency power can be draw-specific texture rather than recovered target content. These results recast coarse-to-fine recovery as an identifiability problem: a pattern supports a mechanism only when competing clocks are forced to disagree.

Primary area:
generative models
Keywords:
Coarse-to-fine recovery, Recovery clock, Spectral identifiability, Clock equivalence, Spectrum-breaking

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ffdFOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper study whether the diffusion 'coarse-to-fine' recovery really tells us what drives denoising. It tracks when different frequency bands in a model’s prediction line up with a known clean image. The main observation is that a spectrum-based clock becomes indistinguishable from a simple band-index clock under natural power-law spectra and log-spaced bands. To separate these explanations, the authors edit image spectra in SD3 and independently vary training and evaluation noise covariances in TinyDiT. They also check shuffled targets, random decoders, and PixelDiT. A fitted attenuation model predicts recovery shifts under unseen evaluation covariances, although its advantage depends on the measurement setup and intervention direction.

Critical strengths

The controlled covariance-intervention study is a major strength of this paper. The authors hold clean images and total expected noise power fixed, independently vary training and evaluation covariance, and fit attenuation on source endpoints before testing unseen evaluation conditions. This tests an explicit predictive hypothesis beyond descriptive coarse-to-fine correlations. It shows that the proposed protocol can produce useful, testable predictions about recovery behavior.

Critical weaknesses
  1. Contributions and novelty are both limited. No significant theoretical insights or empirical results are provided. Existing research already connects frequency-dependent recovery to image spectra and noise structure, and studies anisotropic corruption [1, 2, 3]. The affine-equivalence observation follows directly from power-law spectra and log-spaced bands. The paper adds a careful measurement protocol, but offers limited new understanding of diffusion.

  2. The results seem quite setup-dependent. The Fourier-clock advantage largely disappears with image-first averaging and reverses with different frequency bins. The training-dependent gain also loses to a shared gain on the orthogonal test.

[1] A Fourier Space Perspective on Diffusion Models [2] Show the Signal, Hide the Noise: Spectral Forcing for Pixel-Space Diffusion [3] Learning What Matters: Steering Diffusion via Spectrally Anisotropic Forward Noise

Questions for authors

Can the conclusions of this paper be used to build and train a better diffusion model?

Reviewer_D9zcOpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies what can be inferred from coarse-to-fine recovery during teacher-forced diffusion denoising. Its central observation is that a power-law data spectrum sampled in logarithmically spaced bands produces a negative-log-spectrum clock that is affine-equivalent to band index. It develops a target-alignment measurement protocol, tests engineered Fourier interventions in SD3, and independently varies training and evaluation noise covariance in TinyDiT. A source-fitted attenuation model improves prediction of centered recovery-time shifts at held-out evaluation levels and, in aggregate, along one orthogonal covariance direction. Decoder controls and PixelDiT experiments help delimit the conclusions.

Critical strengths
  • The paper distinguishes observing a coarse-to-fine recovery order from identifying its underlying mechanism. Comparing candidate clocks against a band-index baseline with the same affine calibration budget helps avoid attributing mechanistic significance to high correlation alone.
  • The covariance factorial independently varies training and evaluation noise covariance while holding clean images and total expected noise power fixed. This provides more informative evidence about the sources of recovery behavior than comparisons between latent- and pixel-space models that differ along several dimensions simultaneously. -The authors explicitly report the failed initial bilateral criterion, sensitivity to aggregation choices, and the shared-gain baseline’s superior performance on the orthogonal contrast. They also distinguish exploratory analyses from subsequent prospective prediction tests, making the strength and scope of the evidence easier to assess.
Critical weaknesses
  • The protocol averages per-image cosine curves, whereas the analytical comparator uses population-average spectral power. Because cosine depends nonlinearly on signal power, these quantities need not correspond under heterogeneous image spectra. This is particularly relevant to Table 11, where the identity-to-midbump advantage changes from −0.439 to −0.001 under image-first aggregation. A channel baseline incorporating per-image spectral powers would help explain this sensitivity.
  • The contribution of learned denoising beyond information already present in the corrupted input remains insufficiently quantified. Appendix E notes that positive scalar posterior-mean shrinkage preserves band cosine, meaning that the corrupted input can exhibit the same analytical recovery clock. The shuffled-target control establishes target specificity but does not quantify improvement over the input. Comparisons with the corrupted input and a simple linear denoiser, using both cosine and NMSE, would strengthen the interpretation.
  • The attenuation model has limited demonstrated generalization. The held-out interpolation conditions vary the amplitude of a single spectral direction, and all training covariance levels participate in calibration. On the orthogonal contrast, the simpler shared-gain model performs better than the training-dependent model, with RMSE 0.243 versus 0.269. Although these limitations are acknowledged, they constrain the broader significance of the proposed parameterization. Direct endpoint-interpolation baselines and additional covariance directions would clarify its incremental value.
Questions for authors
  1. Under what conditions does population-average spectral power adequately predict the average of per-image cosine curves? Could image-level power heterogeneity explain the strong aggregation sensitivity in Table 11, and do the available per-image measurements support or rule out this explanation?
  2. How do the measured recovery clocks compare with those of the corrupted input and a simple linear denoiser under the same measurement protocol? Do cosine and NMSE comparisons establish a contribution from learned denoising beyond the spectrum-dependent information already present in the input?
  3. On the original covariance path, does the training-dependent gate outperform direct interpolation of the measured endpoint clocks using the same source observations? Given the shared-gain model’s advantage on the orthogonal contrast, what incremental predictive value can be attributed specifically to the training-dependent parameterization?
Reviewer_XH7cOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates the frequency-dependent recovery dynamics of diffusion models and how to distinguish explanations based on image statistics, corruption covariance, and learned model behavior. The authors introduce a spectral measurement protocol and show that data-spectrum and frequency-index recovery clocks become indistinguishable under power-law image spectra. Through controlled Fourier and covariance interventions, they separate these hypotheses and develop a training-dependent attenuation model that predicts recovery-time shifts under unseen evaluation conditions.

Critical strengths
  1. This paper explores an identifiability problem that simpler spectral analyses often overlook. Observing coarse-to-fine recovery and observing that natural images have more low-frequency power do not establish that image statistics causally determine the recovery order. Their proposal to deliberately design interventions where the hypotheses disagree is scientifically meaningful.

  2. The paper is transparent about experimental limitations. For example, it explicitly states that the shared-gain comparator was introduced after observing the orthogonal covariance results. Likewise, the authors acknowledge that changing the recovery-time aggregation procedure or radial frequency coordinates can substantially weaken or reverse the Fourier-intervention conclusion.

Critical weaknesses
  1. The main analytical result is mathematically straightforward and close to trivial. The log of a power law evaluated at geometrically spaced frequencies is linear in the band index. Presenting this as the “central analytical finding” overstates its depth, even though its methodological consequence is reasonable.

  2. The main Fourier-intervention conclusion is highly dependent on measurement choices. The paper acknowledges that the spectral-clock advantage depends strongly on using population-first aggregation and logarithmically spaced radial frequency bands. Alternative aggregation and frequency discretization choices can weaken or reverse the observed preference. This raises a question about what exactly has been identified. If the preferred explanatory clock changes with reasonable choices of measurement procedure, then the experiments may identify a property of the chosen statistic rather than a robust characteristic of denoising dynamics.

Questions for authors
  1. Does the same attenuation relationship hold in larger pixel-space models or latent diffusion models such as SD3, beyond the TinyDiT setting?

  2. Do the clocks predict anything about free-running sample quality or improved schedules?

  3. Can the attenuation model predict a held-out training covariance level, which is the test that would support “training attenuates magnitude”?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.