Temporal-Difference Guidance for Diffusion Models

Learned diffusion guidance aims to sample from a tilted target distribution using gradient-based corrections derived from a noise-dependent value function. Standard regression-based guidance, however, trains this function through value prediction alone: small value error need not imply accurate gradients, creating a mismatch between the training objective and the quantity that drives sampling. We introduce temporal-difference (TD) guidance, a simple, easily implementable modification of existing guidance training that bootstraps across adjacent noise levels. By exploiting the temporal structure of the diffusion process, TD guidance addresses this value–gradient mismatch. We establish a finite-sample oracle inequality in a Sobolev norm that jointly controls value and spatial-gradient errors, balancing approximation, discretization, and statistical errors. Across image and molecular generation benchmarks, TD guidance achieves better quality–control and reward–validity trade-offs than direct regression at comparable conditioning strength.