PSCAD: Parameter Space Continuous Attractor Dynamics in Continual Learning

Continual learning models acquire new knowledge while preserving previously learned information. Dynamic architecture methods allocate task-specific parameters to protect previously learned knowledge from direct overwriting, but leave the growing collection of task updates without explicit coordination. Inspired by the separation of human brain between knowledge acquisition during wakefulness and memory consolidation during sleep, we introduce Parameter-Space Continuous Attractor Dynamics (PSCAD), a parameter-consolidation mechanism for rehearsal-free continual learning that complements data-driven training process. After task-specific learning, the mechanism treats accumulated task parameters as dynamical states and refines them through CANN-inspired dynamics. The dynamics combine affinity-weighted interactions among task parameters, adaptation-based regulation, and associative retrieval from a learnable Hopfield memory. We establish sufficient conditions for Lyapunov energy dissipation and convergence to the equilibrium set of the continuous-time system with fixed memory. Experiments on ImageNet-R and ImageNet-A show consistent improvements in final average accuracy across all evaluated settings, together with gains in average anytime accuracy in most settings. Ablations further support the contribution of each consolidation component. The findings demonstrate the effectiveness of parameter-space consolidation through CANN-inspired dynamics in continual learning.