PAI: Plasticity-Anchored Initialization for Improved Stability-Plasticity Trade-offs in Continual Learning

Continual learning (CL) requires models to remain adaptable to nonstationary task streams while preserving previously acquired knowledge. Yet the standard practice of warm-starting each task from the preceding task solution can progressively impair plasticity: over long task sequences, we observe increasing spectral concentration and decreasing stable rank, accompanied by deteriorating adaptation to new tasks. Existing approaches often restore plasticity by resetting or modifying parameters, but such interventions risk disrupting learned representations and exacerbating forgetting. To address this challenge, we introduce Plasticity-Anchored Initialization (PAI), a simple task-boundary initialization strategy that exploits the learning trajectory itself to balance stability and plasticity. PAI maintains two complementary parameter states: a fast-memory anchor, given by the latest task solution, that preserves recent adaptation, and a slow-memory anchor, formed by averaging historical task endpoints, that aggregates information across the learning trajectory. Before each new task, PAI interpolates between these anchors, moderating task-specific drift while retaining learned structure and steering optimization toward a more plastic parameter region. Theoretically, we analyze how single-sample perturbations propagate across task boundaries and show that, under matched assumptions and learning rates, PAI admits a task-wise uniform-stability upper bound no larger than that of standard sequential SGD. Empirically, across class-incremental learning, all-data replay, and 5,000-task ImageNet sequences, PAI consistently improves both knowledge retention and late-stage adaptation across CNN and Transformer architectures. These results establish task-boundary initialization as a simple yet effective mechanism for sustaining plasticity in long-horizon continual learning.