Geometry of Forgetting: Representation Flux in Continual Learning

Catastrophic forgetting remains a fundamental obstacle to continual learn- ing, where neural networks lose previously acquired knowledge while learn- ing new tasks. Existing methods primarily mitigate forgetting through pa- rameter regularization or experience replay, yet the representation-space dy- namics associated with forgetting remain less well understood. In this work, we investigate the evolution of latent representations during sequential learning and introduce representation flux, a geometric quantity that mea- sures sample-level representation displacement across training. We show that representation flux is strongly associated with catastrophic forgetting across multiple continual learning benchmarks, while temporal analyses provide evidence that elevated flux can precede subsequent performance degradation. Larger representation displacement is also associated with greater confidence degradation, and complementary geometric properties of representation transitions provide additional information about sample- level forgetting. Motivated by these observations, we propose FlowLess-R, a simple representation-space regularization method that constrains replay- sample representations relative to stored reference representations while al- lowing continued learning of new tasks. FlowLess-R is architecture-agnostic and can be integrated into existing replay-based continual learning methods by adding a representation-matching term to the training objective. Ex- periments on SplitMNIST, SplitFashionMNIST, SplitCIFAR10, and Split- TinyImageNet demonstrate improvements in final average accuracy and reductions in catastrophic forgetting when FlowLess-R is combined with ER, DER++, and ER-ACE. Together, our results identify representation flux as an informative geometric marker of forgetting and show that sta- bilizing latent representations provides a simple and effective strategy for mitigating catastrophic forgetting.