acceptodds
Withdrawn from ICLR 2027

Dissecting Catastrophic Forgetting for Understanding Class-Incremental Learning Mechanism

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

This paper explains catastrophic forgetting in class incremental learning (CIL) from a novel perspective of interactions (non-linear relationship) between different input variables. Specifically, we make the first attempt to explicitly identify and quantify which interactions w.r.t. previous classes that are forgotten and preserved over incremental steps, and reveal their distinct behaviors, so as to provide a more fine-grained explanation of catastrophic forgetting. Based on the forgotten interactions, we reveal a shared mechanism for the effectiveness of some classical CIL methods in mitigating catastrophic forgetting, i.e., these methods all reduce the forgetting of interactions w.r.t. previous classes, particularly those of low complexities, although these methods are originally designed based on different intuitions and observations. Intrigued by this, we further propose a simple-yet-efficient method with theoretical guarantees to investigate and verify that low-complexity interaction serves as an effective factor in resisting catastrophic forgetting. The code will be released if the paper is accepted.

Primary area:
interpretability and explainable AI
Keywords:
Catastrophic Forgetting, Interaction, Explainability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QUgZOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Brief Summary of my Review:

The paper offers an understandable and potentially valuable interaction-based perspective on catastrophic forgetting. However, the methodological contributions and positioning relative to prior work need clearer presentation, including discussion of key related work in the main text. Stronger comparisons are needed to establish practical benefits beyond explainability. Missing code and incomplete reporting of runs and random seeds further limit reproducibility.

Paper Summary:

The paper addressed class-incremental learning (CIL) from a novel perspective of interactions. The authors propose a simple/efficient method with theoretical guarantees to investigate and verify that low-complexity interaction serves as an effective factor in resisting catastrophic forgetting. The main focus of the paper is how to identify what is forgotten in CIL.

Critical strengths

Strength:

  • The topic of class-incremental learning is an important research topic.
  • In general, the paper is well understandable.
  • The three questions in the Introduction introduce the problem well.
  • The paper established a fine-grained view of forgetting through preserved and forgotten interactions.
Critical weaknesses

Weaknesses:

  1. The abstract does not introduce the method well and the problem the method addresses generally. From the abstract the contributions of the paper become not clear.

  2. Section 2 “Preliminaries” introduces interactions which is mainly worked done by different papers from Ren et al. A Related Work Section is completely missing so it is unclear which work has already been done and how differs state-of-the-art from this work.

  3. Section 3 combines methodology, experimental setup, and evaluation in one Section. So it is not clear what is an actual methodological contribution.

  4. I did not find the number of runs/seeds trained for each dataset/method/backbone setup, which is important for reproducibility. As interactions changes may also vary between backbones/datasets, higher runs with different seeds would be required.

  5. The main weakness is (besides the missing related work) the lack of comparison to state-of-the-art techniques. Does the integration of interactions actually improve the CIL problem or is its main purpose a theoretical explainability.

  6. Only after reading the paper, I understood that the Related Work Section has been shifted to the appendix. Important related work should be discussed in the main part of the paper.

  7. Equation 7 assigns nonzero weights to orders below m_1n. Thus, the “high-order” intervention also includes low-order contributions.

Reproduciblity:

  • The code is not submitted as supplementary material. This does not allow reproducibility. In my eyes, this is a necessary point for accepting this paper.

Minor:

  • Citet and citep should be used properly to increase readability.
  • Using a footnote after using \mathbb{R} is misleading (see lines 126 on page 3).
Questions for authors

Questions:

  • Can Equation 3, 4, and 5 be used as differential loss functions to preserve information and mitigate catastrophic forgetting? -> The question in partly answer through Equation 8 and 9.
  • Are the absolute forgetting magnitudes normalized by initial strength?
  • Can the forgotten interactions be used in a uncertainty quantification setup, e.g., to reduce the model uncertainty?
Reviewer_M4dsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates catastrophic forgetting in class-incremental learning (CIL) through interactions between input variables encoded by DNNs. It quantifies the preservation and forgetting of old-class interactions across different orders, comparing standard incremental training with established CIL methods. It further encourages or suppresses interactions of selected orders during training to examine their role, particularly that of low-order interactions, in resisting catastrophic forgetting.

Critical strengths
  1. Broad empirical coverage. The study evaluates seven CIL methods from different methodological families, using ResNet-18/34, CIFAR-100 and ImageNet-100, and multiple B0/B50 class splits. Appendix D extends the analysis to audio-visual and CLIP-based CIL settings. This breadth is particularly relevant to the claim of a phenomenon shared across different approaches.

  2. A valuable explanatory research direction. Understanding what is lost during continual learning, beyond measuring declines in accuracy, is an important research question. Analyzing the preservation and loss of input interactions offers a potentially useful perspective on changes in the model’s prediction behavior.

  3. Interventions complement the observational analysis. Section 3.3 moves beyond observing interaction changes by intervening on the learning of both low- and high-order interactions and examining the resulting forgetting. Comparing these interventions is a meaningful step toward investigating the role of interaction order, although the controls discussed in the weaknesses are needed to strengthen the interpretation.

  4. Preliminary evidence of practical utility. Appendix F reports improvements in final and average accuracy when encouraging low-order interactions in both the baseline and several existing CIL methods. These results suggest that the analysis may inform useful training objectives. Additional controlled comparisons would help establish whether the gains are specifically attributable to the proposed interaction-based explanation.

Critical weaknesses

W1. The evidence linking interaction preservation to catastrophic forgetting needs strengthening.

A central claim of the paper is that interaction analysis explains catastrophic forgetting and reveals a shared mechanism underlying the effectiveness of different CIL methods. The paper pursues a reasonable approach: §3.2 identifies common patterns of interaction preservation, while §3.3 intervenes on the learning of different interaction orders and examines the resulting forgetting. Appendix G extends the penalization experiments to several CIL methods, and Appendix F provides complementary results from encouraging interactions. These experiments provide meaningful support beyond observational correlations.

My concern is that the current evidence does not sufficiently distinguish the proposed mechanism from alternative explanations. The observational comparisons do not establish whether interaction preservation explains the methods’ gains, while the interventions change more than interaction order alone. These limitations affect the connection between the two parts of the argument.

Observational evidence in §3.2. Reduced interaction forgetting is broadly consistent with the objectives of methods designed to retain previous knowledge through distillation, feature preservation, or replay. This reduction is not mathematically guaranteed, and its measurement can be informative. However, observing it across methods does not by itself establish why those methods work.

The comparisons also require accounting for interaction magnitude. Equation (5) measures each method’s forgetting relative to its own earlier model, so smaller absolute forgetting can reflect better retention, weaker initial interactions, or both. Across orders, larger absolute improvements need not indicate preferential protection: a uniform 30% reduction yields improvements of 0.06 and 0.015 when baseline forgetting is 0.20 and 0.05, respectively.

Please report initial per-order strengths and relative forgetting fractions,

\frac{I_{\mathrm{forget},D}^m} {I_{\mathrm{forget},D}^m+I_{\mathrm{preserve},D}^m},$$ alongside the absolute comparisons. By Eq. (3), the denominator equals the original interaction strength after the same aggregation. Here, the strengths are first aggregated using Eq. (4), and then divided; this is an initial-strength-weighted forgetting fraction, rather than an unweighted average of individual ratios. The relative reduction $\Delta I_{\mathrm{forget}}^m/I_{\mathrm{forget},\mathrm{base}}^m$ would additionally help assess whether the low-order emphasis survives scale correction. Negligible denominators should be handled explicitly. Further analysis could clarify what the interaction perspective explains beyond conventional performance measures—for example, comparisons of models with matched FM or old-class accuracy but different interaction profiles, or analyses connecting method-specific gains to relative preservation at particular orders. Such evidence would help distinguish a mechanism-related pattern from a general signature of improved retention and thereby alleviate my concern. **Intervention evidence in §3.3.** The finding that suppressing low-order interactions worsens forgetting is relevant to the proposed explanation. However, the interventions differ in both their achieved strength and their masking conditions. Equal loss coefficients do not ensure comparable suppression or disruption of task learning. Moreover, $L(0,0.3)$ constrains the output difference between an all-masked input and an input retaining approximately 30% of the variables, whereas $L(0.7,1.0)$ compares inputs retaining approximately 70% and 100%. Thus, the comparison changes the amount of visible evidence used by the auxiliary objective as well as the targeted interaction orders. Effects of masked-input regularization and optimization difficulty remain possible explanations. These concerns apply to both the penalization experiments and the encouragement experiments in Appendix F. Analyses of achieved suppression, intervention-strength sweeps, initial task performance, or suitable controls for masked-input regularization could strengthen the attribution to interaction order. These are possible directions, not a request to perform all of them. Convincing additional evidence for the proposed mechanism, or a substantially better-controlled version of §3.3, could address my main concern. If §3.2 remains primarily observational, its findings should be described as a shared empirical signature rather than independently establishing a shared mechanism. Establishing that low-order interactions affect forgetting and establishing that their preservation explains the gains of different CIL methods are related, but distinct, claims. **W2. Attribution, novelty, and the scope of the theoretical guarantees require clarification.** The construction in Eqs. (6)–(9) is closely related to the losses in Deng et al., *Discovering and Explaining the Representation Bottleneck of DNNs* (ICLR 2022): an output difference between nested masked samples, its decomposition into interaction orders, and entropy maximization to make this difference non-discriminative. Deng et al. also use the same $(0.7,1.0)$ setting for a low-order model and report that the loss mainly affects $[r_1n,r_2n]$, rather than $[0,r_2n]$, orders. This work is not cited. Please cite it and distinguish the inherited construction from the present modifications, including the interaction definition, removal of the $r_2/r_1$ scaling, and the entropy-based encouragement term in Appendix F. The contribution of these modifications should be explained explicitly, and statements presenting the loss as newly proposed should be calibrated accordingly. Relatedly, Theorem 3 establishes a decomposition of $\Delta f$; it does not establish that optimizing the entropy objective selectively suppresses interactions within $[m_1n,m_2n]$. This selectivity is reported empirically in Figure 5, with an analogous distinction between the decomposition’s range and the empirically affected range already reported by Deng et al. The description of the loss as having “theoretical guarantees” should therefore specify what is guaranteed—the decomposition—and distinguish it from the empirical effects of optimization and their interpretation in CIL.
Questions for authors

Seen in Critical Weaknesses

Reviewer_SNehOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies catastrophic forgetting, which is the main challenge in class-incremental learning. The authors analyze catastrophic forgetting from the perspective of interactions between input variables. By identifying and quantifying these interactions, they provide an explanation of catastrophic forgetting and propose a method with a theoretical guarantee to suppress it.

Critical strengths
  • This paper measures forgetting at the interaction level. Using interactions to explain catastrophic forgetting is novel to me.
  • The theoretical analysis is verified by the experimental results on real-world datasets.
Critical weaknesses
  • As a theoretical analysis, the authors claim that they provide an explanation of catastrophic forgetting, but the paper lacks a formal theorem showing whether forgotten interactions are a necessary or sufficient condition for forgetting. Moreover, the analysis seems only to reformulate forgetting at the interaction level, and it does not provide a way to suppress catastrophic forgetting. The claim that CIL methods forget fewer interactions also appears to be a reformulation of forgetting at the output level, and is therefore less informative.
  • The role of low-order interactions is not sufficiently supported by the experiments. Table 2 shows that suppressing low-order interactions increases forgetting more than suppressing high-order interactions. However, the accuracy of these models should also be reported to reflect their performance.
  • The definition of interactions is unclear, and understanding it requires clearer illustrations. The authors refer readers to other papers, but the explanation in this paper is insufficient. The preliminaries should explain the definition of interactions more clearly to improve readability.
Questions for authors

Does removing low-order interactions lead to a degradation in overall classification performance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.