Entropy-Based Bounds for Flow and Diffusion Models via Stochastic Thermodynamics

We derive a lower bound on the negative log-likelihood of diffusion and flow models from the entropy rates of their noising processes. Our main theoretical result is an exact decomposition of a plug-in surrogate of this quantity into data and noise entropies, a model-induced entropy rate integral, and a nonnegative score-error term. This yields a thermodynamically interpretable lower bound on the surrogate that complements variational upper bounds. The same decomposition gives a model-based upper bound on the differential entropy of the data, and both bounds are tight in the exact-score limit. We validate these results on Swiss roll data and finite Ising systems, where independent entropy estimates or analytic targets are available, and evaluate pretrained image models, including convolutional and transformer score-based diffusion models and a rectified flow, on CIFAR-10, ImageNet-32, FFHQ, CelebA-HQ, and LSUN. Improved likelihood is consistently accompanied by increased integrated entropy production, revealing an accuracy–dissipation relation within variance-preserving and variance-exploding model families. We also introduce the entropy reduction ratio, a likelihood-free measure of fit computed from the score alone. Together, these results establish entropy rates as both theoretical limits and empirical diagnostics for diffusion and flow models, linking likelihood, data entropy, and irreversibility within a unified stochastic thermodynamic framework.