Optimizing Noise Schedules in Denoising Diffusion Probabilistic Models
Abstract
We study how to choose noise schedules in denoising diffusion probabilistic models by analyzing the error that can remain even with the best Gaussian reverse kernels. We derive an exact identity for the averaged one-step KL error and use it to obtain an explicit upper bound for every step after the first, assuming only a finite second moment of the data distribution. With the first-step noise and terminal signal level fixed, we minimize the resulting bound in closed form. The unique optimal schedule has geometrically growing noise-to-signal odds, or equivalently, linearly decreasing log-SNR. Across seven image datasets and class-conditional CIFAR-10, the resulting schedules improve the variational bound on negative log-likelihood compared with linear and cosine baselines, while also improving sample-quality metrics such as FID in many cases.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.