acceptodds
Under review as a conference paper at ICLR 2027

Noise Schedule Design for Diffusion Models: An Optimal Control Perspective

12 views

Abstract

We develop a principled framework for analyzing and designing noise schedules in diffusion models. We show that one can recast this design problem as an optimal control problem, whose state is the Fisher information of the diffusion process, which evolves according to an ODE, and whose control input is the noise schedule. The objective of the optimal control problem is a functional involving the Fisher information, which is shown to be an upper bound on the Kullback-Leibler sampling error. By solving this optimal control problem, we obtain sufficient conditions on noise schedules under which sampling error is achievable, where is the data dimension and is the number of discretization steps. While existing theoretical works also prove that sampling error bounds are achievable, these results hold for specific noise schedules that differ from those used in practice. Under additional assumptions on the data distribution, we derive a family of noise schedules that recovers the functional forms of schedules widely used in practice. We show that these schedules achieve the error bound when their parameters are chosen according to our analysis. Our theory offers a principled approach to noise schedule design, yielding explicit functional forms and guidance for selecting their parameters.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
optimal control, diffusion models, fisher information, sampling error bounds

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GVQAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper asks which forward noise schedule minimises the KL error of the discretised reverse sampler. The authors combine Girsanov's theorem and a scalar ODE for the Fisher information. Together these give a KL upper bound, which reduces the problem to a scalar function of time. Proofs are provided in the appendix. The experiments compare an 8-parameter ACS instance against VP-linear, VP-sigmoid, VP-cosine and EDM schedules. Each schedule is tuned once with 32 Optuna trials on CIFAR-10 and ImageNet-64 at NFE 10 and 50. The tuned schedules are then transferred by SNR matching to pretrained EDM/EDM2 models on held-out datasets. In this setting ACS is reported to be competitive in FID-50k across the efficient sampling range of 10 to 50 NFE.

The paper provides three theorems

(i) An error bound for the sampler that depends explicitly on the chosen noise schedule. The bound is expressed through a single quantity that tracks how much structure remains in the noisy data over time (Thm. 1).

(ii) The schedule that minimises this bound (Thm. 2). Under this schedule, the remaining structure is removed at a steady, constant pace. It matches the best error rate currently known: the error grows with the data dimension and shrinks with the number of sampling steps. However, it cannot be used directly in practice, because constructing it requires knowledge of the data distribution that is not available.

(iii) For simple data distributions (well-behaved single-peaked distributions and mixtures of round, equally wide Gaussian clusters), a concrete family of schedules, called ACS, that can be written down explicitly. ACS achieves the same rate, provided problem-specific constants are ignored (Thm. 3). The family includes the shapes of several schedules already used in practice (VE-exponential, VP-sigmoid and VP-linear). It also prescribes how their settings should change with the number of sampling steps (Cor. 1).

Critical strengths

(i) The paper tracks the Fisher information, which is a single scalar that follows a simple ODE instead of optimising over evolving densities. The schedule design then becomes a one dimensional optimal control problem in theory with a clean closed-form solution (Thms. 1-2).

(ii) Prior sampling error guarantees rely on constructions rarely used in practice. Corollary 1 shows that VE-exponential, VP-sigmoid and VP-linear schedules reach the same rate if their parameters scale with the number of sampling steps. For example, the range of VP-linear should grow logarithmically with the step budget rather than stay fixed.

Critical weaknesses

(i) The paper does discuss related work, but I could not find a discussion of two closely related works in particular. [1] derive a KL bound that depends explicitly on a time-dependent noise schedule. [2] optimise the sampling time grid by minimising a Girsanov-based KL upper bound on the discretisation error. The paper itself claims that schedule design and grid design are equivalent (App. B.3), so these works address the same problem.

(ii) The headline guarantee may be weaker than stated, and the experiments do not test it. Theorem 3 reaches sampling error guarantees only “ignoring problem constants”, and its constant can grow with dimension. For Gaussian mixtures whose cluster means are far apart in high dimension, the constant grows linearly with d (Lemma 7). The paper's own bound then should become roughly d2/nd^2/n, which to my understanding is no better than the older bounds it aims to improve on. Furthermore, while showing clear trends, the experiments show no error bars, and can not fully isolate the components derived by theory. It is unclear if the empirical experiments are statistically significant.

[1] Strasman, S., Ocello, A., Boyer, C., Corff, S.L., Lemaire, V. (2024). An analysis of the noise schedule for score-based generative models. arXiv preprint arXiv:2402.04650.

[2] Sabour, A., Fidler, S., Kreis, K. (2024). Align Your Steps: Optimizing Sampling Schedules in Diffusion Models. arXiv preprint arXiv:2404.14507.

Questions for authors

(1) How does your work differ from [1,2]? Please add them to your related work discussion, if possible.

(2) How large is the constant in Theorem 3 for realistic data? For a Gaussian mixture whose clusters are far apart in high dimension, does the bound still scale like d/nd/n, or closer to d2/nd^2/n?

(3) Regarding Theorem 3, "where g0g_0 satisfies 0<g0≤CdJ∗log(1+n)0 < g_0 \leq C \frac{d}{J_*} log(1 + n)". This constraint on g0g_0 in Theorem 3 involves the Fisher information of the data distribution. How does this fit the claim that parameters can be chosen without knowing the Fisher information? How large must nn be in practice for the guarantee to apply?

(4) Which part of ACS is derived from the theory, and which is tuned freely? With 8 tuned parameters against 1-3 for the baselines, it is unclear whether the gains come from the theory derived step-dependence or from extra flexibility.

[1] Strasman, S., Ocello, A., Boyer, C., Corff, S.L., Lemaire, V. (2024). An analysis of the noise schedule for score-based generative models. arXiv preprint arXiv:2402.04650.

[2] Sabour, A., Fidler, S., Kreis, K. (2024). Align Your Steps: Optimizing Sampling Schedules in Diffusion Models. arXiv preprint arXiv:2404.14507.

Reviewer_ARD5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper formulates noise-schedule design for diffusion models as a Fisher-information-based optimal control problem and proves an O~(d/n)\widetilde{O}(d/n) upper bound on the sampling error under a uniform time schedule. Furthermore, it proposes an implementable noise-schedule design method under stronger assumptions on the data distribution.

Critical strengths
  • The paper proposes a novel framework based on Fisher information and optimal control for optimizing the upper bound of the sampling error.

  • The paper provides a practical noise-schedule design method together with concrete noise-schedule choices.

Critical weaknesses

The paper's positioning and core contribution remain unclear.

From a theoretical perspective, its O~(d/n)\widetilde{O}(d/n) bound under a uniform time-step discretization grid merely matches prior results obtained with non-uniform grids, while the significance of achieving this bound on a uniform grid is not sufficiently justified, particularly given that L168–169 and Appendix B.3 demonstrate how a non-uniform grid can be reformulated as a constant-step scheme through time reparameterization with a transformed noise schedule. From a practical perspective, the experiments in Section 5 provide insufficient evidence to substantiate the effectiveness of the proposed noise schedule.

Questions for authors
  1. Could the authors explain which theoretical result or practical capability in this paper cannot be obtained by applying the time reparameterization in Appendix B.3 to the prior nonuniform discretization grid constructions? Even if the two approaches are not equivalent, why is reproducing the same O~(d/n)\widetilde{O}(d/n) bound specifically on a uniform grid an important theoretical or practical objective?

  2. Could the authors provide broader experiments across additional pretrained model families and solvers, including direct comparisons with nonuniform time schedules introduced in existing works, and clarify why ACS shows limited improvements on ImageNet 512×512512\times512 at higher NFEs and FFHQ 64×6464\times64?

  3. Is Assumption 2 realistic for real-world data distributions? Even if such distributions can be approximated by the isotropic GMM in Assumption 2(ii), κ\kappa in Eqs. (63)-(64) enters quadratically as a distribution-dependent multiplicative factor in the final KL bound in Appendix E.4; could this factor become so large for the mixture required to approximate real images that the bound is uninformative at practical sampling budgets?

  4. What concrete theoretical or practical benefit does the affine coupling imposed by ACS in Eq. (15) provide for noise-schedule design, compared with the more general construction in Eq. (14)?

Reviewer_M4LoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

A KL-error bound is derived for a uniform-step exponential-integrator sampler in terms of the noise schedule, via an ODE for the Fisher information of the forward marginals. Minimizing this upper bound yields a schedule g⋆g^{\star} with O~(d/n)\tilde{O}(d/n) error. Under strong log-concavity or isotropic Gaussian mixtures, common schedules (VP-linear, VP-sigmoid, VE-exponential) also attain O~(d/n)\tilde{O}(d/n) once their parameters grow with nn. Experiments tune a new schedule family (ACS) and compare its FID against standard schedules on EDM/EDM2 checkpoints.

Critical strengths

Tracking the scalar Fisher information is an elegant device, and the main-text derivations are easy to follow. Whether practical schedules admit O~(d/n)\tilde{O}(d/n) guarantees is a pertinent question. Holding out datasets from tuning is a sensible choice.

Critical weaknesses
  1. Insufficient support for the central motivation. The introduction claims that practical schedules achieve O~(d/n)\tilde{O}(d/n) without knowledge of the Fisher information, whereas the best-known bounds for constant schedules depend linearly on J⋆\mathcal{J}_{\star}. For an absolute constant CC, however, Theorem 3 requires roughly log⁡(1+n)≳g0J⋆/(Cd)\log(1+n) \gtrsim g_{0}\mathcal{J}_{\star}/(Cd). Thus, in the fixed-g0g_{0} examples of Corollary 1, the guarantee takes effect only when nn is exponentially large in J⋆/d\mathcal{J}_{\star}/d, unless g0g_{0} is chosen from a bound on J⋆\mathcal{J}_{\star}. In that regime, the appendix's own bound for the constant VP schedule is also O~(d/n)\tilde{O}(d/n). The comparison involves two upper bounds and no lower bound.

  2. Contribution inflated by the optimal-control framing. The state equation is not closed, the solution collapses to a Cauchy-Schwarz argument, and the objective relies on an inequality that is exact only for isotropic Gaussians. Nonuniform-grid analyses (Benton et al., 2024; Conforti et al., 2025) already yield O~(d/n)\tilde{O}(d/n), and a time-varying schedule with uniform steps is equivalent to a nonuniform grid. What the new bounds add beyond this reparametrization therefore needs clarification. Two closely related works go uncited: Score-Optimal Diffusion Schedules (NeurIPS 2024) and Align Your Steps (ICML 2024). The O~(d/n)\tilde{O}(d/n) rates for practical schedules also omit problem constants, which grow polynomially with the mode separation for Gaussian mixtures.

  3. The experiments do not separate what the theory prescribes. No comparison is made between VP schedules with and without the nn-dependent rescaling prescribed by the theory. ACS carries more tuned parameters than the baselines and is the only schedule that varies with the step budget, so its gains cannot be attributed cleanly to the theory. Seed variance is absent, and several differences in Fig. 1 appear small. VP-Cosine fails on ImageNet-512, with FID in the hundreds and no diagnosed cause. Finally, no synthetic experiment tests the predicted scaling in nn.

Questions for authors
  1. With g0g_{0} held fixed, what size of nn relative to J⋆/d\mathcal{J}_{\star}/d is needed before the guarantee takes effect? In that regime, does the constant-schedule guarantee not have the same order?

  2. Relative to the nonuniform-grid results of Benton et al. and Conforti et al., what is gained by your bounds?

  3. Would it be possible to compare VP schedules with and without the nn-dependent rescaling under an equal tuning budget and to report seed variance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.