Toward Generalization from Memorized Diffusion Models with Housekeeping-Current Injection
Abstract
We propose a sampling protocol that reduces copying in memorized diffusion models while retaining the large-scale structure of the data distribution. We describe memorization with a relative free-energy model, in which narrow local minima around individual training examples lie within broader energy wells. Using the Entropic Bound (EB) together with the continuity equation, we distinguish density-changing transport from housekeeping currents that preserve a prescribed reference density. This motivates housekeeping-current injection (HCI), which adds such a current during sampling to steer trajectories away from memorized paths. In the energy picture, HCI helps trajectories leave the narrow wells around training examples; the learned landscape itself is not changed, and the diffusion model is not retrained. We also add a simple transport correction to alleviate the concentration that can occur with HCI alone. A 2D ring illustrates the landscape and HCI sampling; experiments on an 8D Gaussian mixture model, CelebA-32 and CelebA-64 evaluate trained models that show strong memorization. Compared with recent sampling-based methods, HCI substantially reduces the memorization fraction at a smaller loss of sample quality.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.