Dissecting Catastrophic Forgetting for Understanding Class-Incremental Learning Mechanism

This paper explains catastrophic forgetting in class incremental learning (CIL) from a novel perspective of interactions (non-linear relationship) between different input variables. Specifically, we make the first attempt to explicitly identify and quantify which interactions w.r.t. previous classes that are forgotten and preserved over incremental steps, and reveal their distinct behaviors, so as to provide a more fine-grained explanation of catastrophic forgetting. Based on the forgotten interactions, we reveal a shared mechanism for the effectiveness of some classical CIL methods in mitigating catastrophic forgetting, i.e., these methods all reduce the forgetting of interactions w.r.t. previous classes, particularly those of low complexities, although these methods are originally designed based on different intuitions and observations. Intrigued by this, we further propose a simple-yet-efficient method with theoretical guarantees to investigate and verify that low-complexity interaction serves as an effective factor in resisting catastrophic forgetting. The code will be released if the paper is accepted.