HACANN: Gated DeltaNet as the Native Dynamics of Continuous Attractor Neural Networks

Attention mechanisms like Gated DeltaNet drive modern large language models, yet whether such associative computations can natively emerge from biological neural circuits remains an open question. Here, we introduce Hierarchical Adaptive Continuous Attractor Neural Networks (HACANN), developing the hypothesis that continuous attractors serve as canonical building blocks for relational memory. In HACANN, coupled recurrent rate populations encode association components, enabling cues to retrieve predictions and steer error-driven corrections as new evidence arrives. Theoretically, we prove that the projected dynamics of these biologically plausible rate circuits yield a generalized gated Delta rule, exactly recovering the canonical Gated DeltaNet recurrence in a defined timing regime. Empirically, full spatial population simulations with over 4.4 million rate variables sustain both stable prediction and ongoing learning. On WikiText-2, HACANN achieves natural language prediction performance comparable to matched Gated DeltaNet models. Further semantic and visual experiments demonstrate selective revision of stored relations and activity-driven feedback calibration that restores representation learning. Together, our results provide a formal circuit foundation for modern linear attention and map out a pathway toward building large-scale language models entirely from neural population dynamics.