Dynamical Memory Retrieval in Attractor Networks as a Denoising Diffusion Process

Successful memory retrieval requires the network state to fall into the "attractor basin" of the target memory, which presents a challenge in setting the basin’s size: a basin too broad leads to imprecise retrieval, while a basin too narrow fails to capture the network state. Here, we propose that the neural system can solve this problem by dynamically adjusting the basin’s size, starting wide and gradually narrowing. We use continuous attractor neural networks (CANNs) as the memory model, where the width of the neural activity bump shrinks over time, thereby narrowing the attractor basin. Intriguingly, we find that this dynamical memory retrieval process is equivalent to the reverse dynamics of denoising diffusion probabilistic models (DDPMs), where the decrease in noise variance effectively corresponds to the contraction of the basin’s size in CANNs. We carry out experiments demonstrating the advantages of this dynamical retrieval process; and for hierarchical memories, it gives rise to information retrieval from coarse to fine. We hope this study provides insight into dynamical memory retrieval in the neural system and its link to diffusion generative models in AI.