- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Matryoshka Continual Learning (MCL), an exemplar-free continual learning framework that adapts Matryoshka Representation Learning to mitigate catastrophic forgetting. The method jointly trains multiple nested feature prefixes using shared classifier weights. During sequential learning, a frozen model from the previous task serves as a teacher, and knowledge distillation is applied at each feature width to preserve old-class predictions. For Class-IL, the framework additionally incorporates local cross-entropy, cosine classification, weight alignment, and class-mean prototype fusion.
Critical strengths
The proposed method provides an exemplar-free solution without introducing task-specific network expansion or additional expert modules. Its fixed-capacity architecture and relatively straightforward implementation make it potentially practical for resource-constrained continual learning scenarios.
Critical weaknesses
-
The proposed MCL framework largely combines existing Matryoshka Representation Learning (MRL) with multi-width LwF, without introducing substantial new algorithmic mechanisms.
-
Multi-level and selective feature distillation have been extensively explored in continual learning (e.g., PODNet, AFC, and MDPCR). The paper fails to adequately distinguish its approach from these closely related methods.
-
The assumption that higher feature reach leads to greater stability is not sufficiently justified. Moreover, the paper lacks critical ablation studies and feature drift analyses to validate this claim.
Questions for authors
See above