Spectral Signatures of Memorization Basins in Diffusion Trajectories

Diffusion models are known to reproduce individual examples from their training data as a form of memorization with implications for generalization, privacy, and downstream use. Recent work demonstrated that memorized training samples reside inside contiguous regions of the data/latent space known as basins of memorization; once a denoising trajectory enters a basin, under finite perturbations it irreversibly reproduces a memorized example. Prior empirical work on such basins in trained models has primarily focused on classifier-free guidance based detection and mitigation strategies in text-to-image models, leaving basin entry in trained diffusion trajectories and the formation of such basins largely uncharacterized. In this paper, we present a geometric characterization of memorization basins for conditional, unconditional, and rectified-flow-based diffusion models. We observe a characteristic signature in the local spectrum of the function geometry that distinguishes basin entry: the emergence of a dominant singular mode followed by a sharp spectral recovery, both markedly stronger along memorized trajectories. This signature appears at two timescales: along denoising trajectories during inference, and across training. Crucially, these geometric signatures are actionable: regularizing them can suppress memorization during training, while during inference they can help identify and redirect trajectories falling into memorization basins. The resulting geometry-informed interventions remain effective under low-dimensional approximations, and geometry-informed steering extends to large-scale text-to-image diffusion models. Our results show that local spectral geometry provides a common basis for characterizing memorization and for suppressing, detecting, and steering away from it across training and generation.