Adaptive Reuse of Residual Potentials in Low-Timestep Spiking Diffusion Models

Diffusion models generate high-quality samples through many denoising steps, requiring repeated forward passes of large denoising networks and leading to high inference cost. Spiking neural networks (SNNs) offer a promising way to reduce this cost through sparse spike-based computation, but few neuronal timesteps provide only a coarse approximation of activations, causing errors that can propagate to later denoising steps. We observe that residual potentials retain activation information not yet expressed through spikes. Because consecutive denoising steps operate on successive sample states, this information can remain useful across neighboring steps; however, changes in the sample state and diffusion-time conditioning make full inheritance unreliable. Based on this insight, we propose Residual-Adaptive Spiking Diffusion (RASD), which combines selective residual-potential inheritance with residual-informed threshold adaptation. The former uses a calibrated schedule to control how much residual potential is inherited across denoising steps, while the latter forms historical range observations from residual potentials and spike outputs and uses them to predict the current activation range for threshold adaptation.Experiments on DDPM, Improved-DDPM, and DiT show consistent improvements in low-timestep generation quality with only T=4 or T=8 neuronal timesteps, with approximately 65% energy savings.