- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper argues that one-step curvature probes do not reliably predict terminal forgetting in continual learning. It proposes a theoretical analysis showing that the final old-task cost depends on retained fitting capacity, endpoint-direction curvature, and the null-space component selected by optimization.
Critical strengths
This paper provides a theoretical analysis of forgetting in continual learning and validates the theoretical predictions through experiments.
Critical weaknesses
-
I find this paper very hard to read. Large parts of the text read like writing generated by AI and the logic is often disjointed. AI can certainly help with theoretical proofs, but a reasonable approach would be for the authors to understand the material themselves and rewrite the AI-generated exposition into something readers can follow. Otherwise, the authors' contribution is close to nonexistent.
-
The experimental section is also confusing. The paper does not describe the experimental setup at all. For example, it does not say how many samples ER uses or what sampling strategy is adopted. It also does not say how many samples GPM uses to construct its subspace. In my own experiments, with the same number of samples, GPM forgets less than ER, which seems to contradict Table 3. This makes me question the credibility of the results.
Questions for authors
Please see Weaknesses.