CLIF: When Continual Learning Meets Influence-based Forgetting

Machine unlearning in continual learning (CL) addresses the growing need to remove parts of a CL model's accumulated knowledge in response to privacy regulations, biases, or noise. Influence functions (IFs) offer an effective, retraining-free route to unlearning in the standard single-task setting, but we show that simply applying IFs to CL models leads to common failure. We identify two core challenges (Cs) as the root causes: (C1) Non-stationarity. Sequential training drives the final CL model far from the stationary point of the joint full-history loss that IFs assume. This leaves a residual gradient that standard IFs ignore, biasing the unlearning target. (C2) Curvature unreliability. IFs rely on the inverse of Hessian to convert removal gradients into parameter updates. But in CL, Hessian is estimated from small, distributionally skewed available data, yielding ill-conditioned inversions that corrupt parameter updates. To overcome both challenges, we propose CLIF (Continual-Learning with Influence-based Forgetting), a novel framework—the first, to our knowledge, to rehabilitate IF-based unlearning for replay-based CL—via three complementary phases. Phase I addresses C1 by introducing a bias-corrected influence target that incorporates residual gradients overlooked by standard IFs. Phase II addresses C2 via a devised Continual Fisher Information preconditioner whose variance-form Fisher provides a full-rank, stably invertible alternative to standard Hessian inverse. Phase III further proposes a forget-subspace projection that confines updates to forget-relevant directions while removing retain-aligned components to reduce performance degradation. Extensive experiments across two unlearning scenarios show that CLIF matches the gold-standard retraining behaviour more closely than standard IFs and state-of-the-art baselines, delivering effective unlearning while retaining strong performance on remaining tasks.