- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the problem of old-class prototype mismatch caused by continual LoRA updates in exemplar-free class-incremental learning. ReLoRA-CL fixes the input subspace learned in the first task and updates the output factors in shallow layers, while adding new components orthogonal to historical directions in deeper layers. It also preserves knowledge of old classes through semantic drift compensation and classifier alignment. Its main contributions are an analysis of depth-dependent subspace reuse, corresponding adaptation strategies, and a conditional theoretical analysis of prototype transfer error.
Critical strengths
Modeling prototype compensation reliability has explanatory value. The conditional error bound distinguishes drift regularity, insufficient coverage of old classes by the current samples, and residual uncertainty, rather than simply assuming that limiting parameter updates can prevent forgetting.
Critical weaknesses
-
The experiments do not include two closely related works: CL-LoRA already uses a fixed random input projection in shallow layers and task-specific LoRA in deeper layers; MACIL already combines LoRA, mean-shift compensation, and classifier alignment. Therefore, the paper’s novelty should lie primarily in its analysis of depth-dependent reuse of learned subspaces and its connection to drift estimability, rather than in layer-wise division of labor or the compensation pipeline itself.
-
The central explanation that “drift is easier to estimate” has not been sufficiently validated. Proposition 2 compares only fixed reference updates and their projections, rather than the two solutions obtained through actual optimization. Although the authors explicitly acknowledge this limitation, the absolute prototype displacement estimation error in the experiments is also affected by drift magnitude and feature scale, and no controlled comparison of individual modules under consistent compensation and alignment conditions is clearly reported. It is therefore difficult at present to distinguish whether the gains come from more estimable drift, smaller drift, or the statistical maintenance itself.
3.The experimental performance is less than satisfactory. As far as I know, overall it should not be as effective as the LoDA method. If you could additionally explain the advantages of your method over LoDA, this would make your method more convincing.
CL-LoRA: CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning
MACIL: Navigating Semantic Drift in Task-Agnostic Class-Incremental Learning
LoDA: Task-Driven Subspace Decomposition for Knowledge Sharing and Isolation in LoRA-based Continual Learning
Questions for authors
My main concerns are all listed in the weaknesses. Overall, this paper is fairly solid. If the author addresses my concerns well, I will consider raising my rating.