Reinforcement Noise Adaptation for Diffusion Models

Creating data from noise is generative modeling; its converse, creating noise from data, has been neglected yet could be unexpectedly non-trivial for diffusion models. Conventional diffusion models use a stepwise noising and denoising training mechanism, which can be viewed as a serial generation of training samples for different noise levels. Major diffusion models predefine the noising schedule, which treats the samples from the easy-to-learn and the hard-to-learn distributions equally. In this work, we investigate the adaptive noise schedule. We propose the Reinforcement Noise Adaptation (ReiNA), a framework that casts forward noise scheduling as a sequential decision-making problem. During the forward and backward learning process of the diffusion models, an adaptive policy observes the current learning state and adjusts the noise schedule, thereby reallocating learning capacity to the under-learned distributions. On the public datasets (CIFAR-10, CelebA-HQ, and LSUN Bedroom), ReiNA consistently improves FID, IS, Precision, and Recall compared with some SOTA methods. In addition, experiments on structured material simulation (StructSim) show the adaptive framework can be applied to sophisticated physical simulation tasks. The ablation studies and further analyses verify the contribution of each ReiNA component and reveal that ReiNA learns phase-aware and batch-adaptive noising strategies.