- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper solves the catastrophic forgetting problem in continual learning from the perspective of stable and fragile learning. They argue that sequential learning can change all correct samples after new task parameters are introduced. They introduce acquisition stability (AS) to measure how correct predictions remain correct.
Critical strengths
- The idea is interesting and well motivated.
- The flow of the paper is understandable and logical.
Critical weaknesses
-
The structure of the paper can be improved with greater precision. For example: a. The notation table should appear before defining the notation, not at the end. Also, for ICLR with a 9-page cap, please consider removing the notation table and using space more efficiently. b. Lines 192 to 202 are not part of the method section. I do not understand why they are appearing there. They should be in the experiment section. c. Some acronyms, like Class-IL in line 269, appear without a full definition. Or CL and Continual learning have been used interchangeably, while you have already defined Continual Learning as CL. d. Fig. 2 should be more abstracted and informative. It reads like an AI-generated summary of the paper. The main point should be conceptually informative so a first reader, like me, can grasp the paper's idea, not a flowchart with text-based rectangles. e. Most of the paper reads as future-referencing; I see a lot of referring to upcoming sections and the appendix, which breaks storytelling consistency. f. Section 3.2 says methods addressing fragility (ii) “through parameter perturbations” include Bhat et al. 2022 and Hacohen & Tuytelaars 2025. Related Work describes Bhat as input-view consistency and Hacohen as learning-speed-based memory construction. Neither perturbs parameters, and Related Work itself contrasts AS-CL with weight perturbation (STAR, C-Flat).
-
The paper idea is interesting; however, I found some Algebraic errors, such as: a. lines 152- 153: the lowercase z notation is undefined on first appearance. b. The Sec. 3.2 coupling argument does not follow. ‖∇{W₁}m‖ = ‖g‖‖x‖ and ‖∇ₓm‖ = ‖W₁ᵀg‖ scale differently, so the orderings of examples can differ. Δ{t→s} moves all layers, not just W₁. Eq. 5 linearizes around a(x) − x, but a 10° rotation or 2-pixel shift is a large, non-gradient-aligned pixel change. The argument also assumes a linear first layer, while the model is a ResNet-18 with BatchNorm. This one breaks your logical motivation. Unfortunately, I couldn't find the code for the paper in your submission to see how you implemented this part. c. Alg. 1 and Eq. 8 normalize v(x) differently.
-
Some inconsistencies in your paper make me wonder if you have double-checked everything before submission or not. For example: a. Stage 2 is described two ways. The abstract and intro say replay weight goes to predictions that “remain fragile during later tasks” and that AS is “recomputed for the stored examples”. Sec. 3.5, Table 1 and Fig. 2 say q_x is computed once at θ_t and never updated. Recomputation at each replay only appears as an ablation (App. E, Table 8). How do you explain this inconsistency? b. Surprisingly, the base-epoch objective differs between text and algorithm. Sec. 3.5 says base epochs “use L_CL”. Alg. 1 line 1 says L_CL with the replay term replaced by L^AS_replay. It is never stated whether the stabilization epochs also use weighted replay. This matters for the Table 4a “Stage 1 only” row. I don't think this can be well justified in the rebuttal phase. c. Prop. 8 contradicts Alg. 1. The proof says that with λ_s = 0 “no view passes are computed”. Alg. 1 computes K = 32 views for Q̃ (lines 3–5) and for q_x (lines 10–12) unconditionally, so per-step cost does not reduce to the backbone’s. d. Some minro errors are: A(t,s) (accuracy matrix) and A_t (acquired set) share a symbol. T denotes both the number of tasks and the transformation family, and also the horizon in the Fig. 6 caption. C_t is used in Sec. 3.5 but is missing from Table 1.
-
I found some writing quality issues that prevent me from understanding the paper clearly. a. . Fragile, brittle, vulnerable, susceptible, unstable and low-AS are used interchangeably. Specifically, none of them are interchangeable. They have their own different meanings; be precise about this. b. Mixed spelling conventions. The text uses “stabilization” and “analyzed”, but Fig. 7 has “stabilisation budget”, Sec. 4.3 has “analyse”, and App. E.2 has “recognisable”. Choose US or UK spelling throughout, including figure text. c. Garbled Fig. 1 caption. d. The discussion paper is heavily dense and full of broken sentences; it needs to be written more clearly and consistently. I cannot point to each sentence one by one, but please read it carefully; it is fragmented, dense, and not easy to follow.
-
I expect to see a reproducibility statement at the end of the paper and the code of the paper attached as supplementary material. This can not be addressed during the rebuttal phase, and I cannot confirm that the numbers appear in the paper.
-
For a fairer comparison, please consider benchmarking also with the WSN and SNV papers, as those papers are based on mask freezing; it shows how freezing some part of the model can affect the AS introduced by you. Also, when comparing with related work, emphasize how your method differs from theirs and, in the buffer-free setting, how your method can perform.
WSN: Forget-free Continual Learning with Winning Subnetworks, Haeyong Kang, ICML 2022 SNV: Shapley Neuron Values for Continual Learning: Which Neurons Matter Most?, Vahedifar, ICML 2026
Questions for authors
Please refer to Weaknesses section.