- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SPCL, a replay-based class-incremental learning method built on neural-collapse prototypes. ProNC rebuilds a global simplex ETF (a set of equally spaced class prototypes) at each task. SPCL instead freezes the old class prototypes, places new ones in their orthogonal complement, and adds distillation and margin losses to preserve old decisions. On Seq-CIFAR-100 and Seq-TinyImageNet, it reduces forgetting and slightly improves accuracy over ProNC.
Critical strengths
- Simple construction that removes prototype drift. Freezing old prototypes and placing new ones orthogonally removes drift in the class reference by design, with no extra trainable parameters. This directly addresses a clear limitation of ProNC.
Critical weaknesses
-
Limited novelty. Fixed, pre-allocated class prototypes have been explored in class-incremental learning [1], and orthogonal prototype placement in [2]. Response distillation, a margin loss on old classes and cosine feature distillation closely resemble LwF-style distillation and LUCIR [3].
-
Limited experimental scope. The experiments use only a ResNet-18 trained from scratch, on two small benchmarks (CIFAR100 and TinyImageNet).
References
[1] Class-incremental Learning with Pre-allocated Fixed Classifiers. ICPR 2020.
[2] OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning. CVPR 2024.
[3] Learning a Unified Classifier Incrementally via Rebalancing. CVPR 2019.
Questions for authors
- How does SPCL compare with LUCIR, or with a fixed orthogonal classifier trained with the same distillation losses, under the same protocol?