- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Brief Summary of my Review:
The paper offers an understandable and potentially valuable interaction-based perspective on catastrophic forgetting. However, the methodological contributions and positioning relative to prior work need clearer presentation, including discussion of key related work in the main text. Stronger comparisons are needed to establish practical benefits beyond explainability. Missing code and incomplete reporting of runs and random seeds further limit reproducibility.
Paper Summary:
The paper addressed class-incremental learning (CIL) from a novel perspective of interactions. The authors propose a simple/efficient method with theoretical guarantees to investigate and verify that low-complexity interaction serves as an effective factor in resisting catastrophic forgetting. The main focus of the paper is how to identify what is forgotten in CIL.
Critical strengths
Strength:
- The topic of class-incremental learning is an important research topic.
- In general, the paper is well understandable.
- The three questions in the Introduction introduce the problem well.
- The paper established a fine-grained view of forgetting through preserved and forgotten interactions.
Critical weaknesses
Weaknesses:
-
The abstract does not introduce the method well and the problem the method addresses generally. From the abstract the contributions of the paper become not clear.
-
Section 2 “Preliminaries” introduces interactions which is mainly worked done by different papers from Ren et al. A Related Work Section is completely missing so it is unclear which work has already been done and how differs state-of-the-art from this work.
-
Section 3 combines methodology, experimental setup, and evaluation in one Section. So it is not clear what is an actual methodological contribution.
-
I did not find the number of runs/seeds trained for each dataset/method/backbone setup, which is important for reproducibility. As interactions changes may also vary between backbones/datasets, higher runs with different seeds would be required.
-
The main weakness is (besides the missing related work) the lack of comparison to state-of-the-art techniques. Does the integration of interactions actually improve the CIL problem or is its main purpose a theoretical explainability.
-
Only after reading the paper, I understood that the Related Work Section has been shifted to the appendix. Important related work should be discussed in the main part of the paper.
-
Equation 7 assigns nonzero weights to orders below m_1n. Thus, the “high-order” intervention also includes low-order contributions.
Reproduciblity:
- The code is not submitted as supplementary material. This does not allow reproducibility. In my eyes, this is a necessary point for accepting this paper.
Minor:
- Citet and citep should be used properly to increase readability.
- Using a footnote after using \mathbb{R} is misleading (see lines 126 on page 3).
Questions for authors
Questions:
- Can Equation 3, 4, and 5 be used as differential loss functions to preserve information and mitigate catastrophic forgetting? -> The question in partly answer through Equation 8 and 9.
- Are the absolute forgetting magnitudes normalized by initial strength?
- Can the forgotten interactions be used in a uncertainty quantification setup, e.g., to reduce the model uncertainty?