Class-Incremental Learning (CIL) requires a model to acquire new classes while preserving a usable representation for earlier classes. Existing approaches usually treat this as either a capacity problem (making parameters available for new classes) or a representation problem (constraining feature drift), leaving the two decisions weakly coupled. We formulate the problem as a closed-loop stability–plasticity control problem and propose Closing the Stability–Plasticity Loop (CSPL). CSPL uses one utility-and-reference loop: a maturity-aware utility signal identifies channels whose capacity has become ineffective; function-preserving replacement recycles only those channels; and a shared spatial context memory provides a task-agnostic reference for the resulting features. The reference is enforced through the task-agnostic/task-specific branches and feature distillation, so that capacity renewal is performed without changing the function seen by the stabilizing pathway. Experiments on CIFAR100, Tiny-ImageNet, ImageNet100, and CUB200 show consistent improvements over strong baselines. Under the 50–10 protocol, CSPL improves Last accuracy over the strongest listed baseline by 2.16 percentage points on CIFAR100 and 3.51 points on ImageNet100.
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.Reviewer_Fg3MOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
The paper proposes CSPL, which combines utility-based channel replacement, a shared Spatial Context Memory (SCM), and task-agnostic/task-specific feature distillation for class-incremental learning.
Critical strengths
Jointly addressing loss of plasticity and catastrophic forgetting is an important problem.
Experiments cover CIFAR100, ImageNet100, TinyImageNet, and CUB200, including different initial class counts and increment sizes. This provides useful coverage of both general and fine-grained image classification.
CSPL improves final accuracy over the strongest listed baseline by 2.16 percentage points on CIFAR100 and 3.51 points on ImageNet100. The reported improvements generally extend to average incremental accuracy.
Critical weaknesses
The novelty relative to existing methods needs clearer isolation. -> Original continual backpropagation already includes activation-and-outgoing-weight utility, maturity protection, selective replacement, and zero initialization of outgoing weights. It also evaluates supervised continual learning, making the related-work characterization incomplete.
Although results are averaged over three runs, standard deviations, confidence intervals, and individual-run results are absent. This matters particularly for small improvements: final accuracy increases by only 0.12 percentage points on TinyImageNet 100–10 and 0.13 points on CUB200 over TagFex.
For the apparently corresponding CIFAR100 50–10 configuration with 2,000 rehearsal samples, main-paper Table 2 reports 72.28 Last / 77.71 Avg, whereas supplementary Table 3 reports 72.28 Last / 78.71 Avg. Several baseline values also differ. The authors should reconcile these numbers or specify the experimental differences responsible.
The source code is not attached. Exact reproduction of the reported experiments is therefore not possible from the supplied materials. Although pseudocode and some training settings are provided, important details remain unspecified.
I doulbe checked the references. The doi of the paper by Hayes et al. (2019) "Memory Efficient Experience Replay for Streaming Learning" leads to a different paper. The doi of the paper by Robins (1995) "Catastrophic Forgetting, Rehearsal and Pseudorehearsal" does not exist.
The AI statement is missing.
Minor:
Cite and citep should be used properly to increase readability.
As there was no response from the authors, the AI statement is still unresolved, and the weaknesses have not been addresses, I still recommend the rating 1 (clear rejection).
The paper proposes Closing the Stability–Plasticity Loop (CSPL) method. Unused channels are reinitialized (to improve plasticity) and they use a Spatial Context Memory (SCM) which constrain the representation after renewal. The methods are evaluated on several standard benchmarks and obtain competitive results.
Critical strengths
The reported results on ImageNet-100 are excellent.
Critical weaknesses
The overall presentation of the paper is not ready for publication in a top conference like ICLR.
references are in the wrong format
quality of bibliography: the authors should cite the most relevant works not just any work in the category; for example the section on DYNAMIC ARCHITECTURE does not discuss PNN or Dynamically Expandable Networks. For Class-Incremental Learning it cites the relatively unknown Fei 2023, etc.
Incomplete results. It is unclear why recent methods are not reported (like CL-LoRA, or E2-LoRA). The paper mentions several times TinyImageNet and Birds, but results are not included in the main paper (no reference to appendix is made) ! It is unclear what backbone is used: pretrained or from scratch ?
Questions for authors
The authors should address the weaknesses. I think the presentation of the paper should be improved before serious consideration by other conference. Here I highlight some problems. But the paper requires a total re-writing where the papers is well placed in the relevant state-of-the-art, and the explanation clarity is much improved.
the paper sets out to close the Stability–Plasticity Loop (CSPL), it seems to suggest that the reader should be aware of an open stability-plasticity loop; I think that needs better definition. For me, it remains unclear what the open stability-plasticity loop is which requires closing.
I seems that the Dohare et al. (2024) plays an important role in this paper. Would it not be good to discuss that paper in the introduction and then point out the problem addressed by the proposed method. Given that the paper proposes a new death neuron detection and reset method, the authors should also better review the follow-up papers of Dohare et al. (2024).
the method does not clearly state the limitations of the current methods, from which their method follows, resulting in a unclear novelty claim.
This paper proposes a new class incremental learning method based on closing plasticity and stability.
Critical strengths
They benchmarked against a sufficient number of baselines.
Restoring plasticity without destabilizing features is a relevant question for CIL, and the loss of plasticity is understudied in the exemplar-based CIL setting.
Critical weaknesses
This paper has a formatting issue:
a. It doesn't have the mandatory AI statement required for an ICLR submission.
b. Citations appearing in the paper body are not in ICLR format.
c. Notation tables appear after the notations are introduced.
d. The paper is not well written or well organized. In many places, the paper repeatedly relies on material that comes later: the Introduction describes Fig. 1, the TA/TS branches and SCM before they are defined, and Sec. 3.2 describes iCBP and SCM by forward reference to Secs. 3.3–3.4, with Fig. 2 placed before the components it depicts. Sentences are often abrupt or inconsistent: the Introduction's paragraph on contrastive learning is unrelated to the method; CL is expanded as "Continuous Learning"; each Related Work subsection is two or three sentences.
There are so many algebraic inconsistencies:
a. The reset is not function-preserving: Eq. (10) does not imply f(x;θt)=f(x;θt+1) (p. 6).
Zeroing W:,iout removes the old channel's contribution. For the next convolution, the pre-activation of output channel o changes by
Δzo(x)=−∑p,qWo,i,p,qfi(x)⋅+(p,q),
which is zero only if channel i was already contributing nothing. Using Eqs. (3)–(4) with β=0,
so the perturbation is bounded by a multiple of the utility: small for low-utility channels, but not zero.
b. The aging rule (Eq. 6) does the opposite of what it is introduced for.
Immediately after a reset, W:,iout=0, so ui=Fi⋅0=0 (Eq. 4). Since utilities are non-negative, Qqℓ(u)≥0 and Eq. (5) gives u~i=0. Eq. (6) then ages the new channel at the maximum possible rate α+α0.
c. The utility is ill-defined under BatchNorm (Eqs. 2–4).
If fi is measured before BN, E∣fi∣ is arbitrary up to the BN rescaling γi/σi, so utilities are not comparable across channels. If it is measured after BN (and ReLU), E∣fi∣ is essentially a function of (γi,βi) alone. Likewise, ∥wiout∥1 in Eq. (3) ignores the BN scaling of the next layer, so it is not the effective downstream weight.
d. SCM is an input-independent weight matrix with a redundant parameterization (Eqs. 11–15).
K and V are free parameters, not functions of the data, so
k: number of replaced channels (Eq. 8) and channel index in Fig. 2.
Table 1 defines ϕi, and Fig. 2 uses uk=ϕk∣wkout∣1, but Eq. (4) uses Fi; ϕi never appears in the text.
Fig. 2 combines the spatial context with ⊕, while Eq. (16) uses concatenation and the text stresses "concatenation rather than addition."
biin in Eq. (10) is never defined.
My biggest issue is the paper's overclaims. The central claim is that capacity renewal and the reference form one closed loop. In the equations, no signal from the reference reaches the replacement decision: ui, ai, M and k (Eqs. 4–8) depend only on a layer's own activations and outgoing weights, never on K, V, fta or Ldistill. The only link is that all parameters are trained under one summed loss (Eq. 18), which is true of any multi-term objective. There is no plant model, setpoint, error signal, feedback law, or stability/convergence statement, so the first contribution ("We formulate CIL capacity renewal and feature retention as a coupled stability–plasticity control problem") is not delivered. I don't think you can address this during the rebuttal phase because it harms your paper's story.
How are we supposed to verify numbers without including code in the submission?
The ablation contradicts the interaction claim (Table 3, CIFAR100 50–10). Single-component gains in Last accuracy over the 67.23 baseline are iCBP +1.92, SCM +2.26 and L1 +3.11. Every combination is sub-additive:
Components
Additive prediction
Observed
Interaction
SCM + iCBP
71.41
71.01
−0.40
L1 + iCBP
72.26
71.42
−0.84
L1 + SCM
72.60
71.68
−0.92
All three
74.52
72.28
−2.24
The paper is full of ambiguous terms:
"Mature channels"
"Contribute little to the network's performance"
"Maturity tracking"
"Persistently ineffective"
"Capacity has become ineffective," "capacity renewal," "representational diversity"
"Reference state," "admissible feature reference," "reference coordinate system," "coordinates in which old and new classes are represented"
"Global spatial stability," "spatial anchoring," "discontinuity in the feature space," "artificial jump," "the function seen by the stabilizing pathway".
Questions for authors
See weaknesses.
From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.
Rebuttal and final rating
As there was no response from the authors, the AI statement is still unresolved, and the weaknesses have not been addresses, I still recommend the rating 1 (clear rejection).