- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DSDR to alleviate the stability–plasticity conflict in class-incremental learning. Its core idea is to exploit the feature-space geometry induced by the historical classifier to decompose temporal representation drift into decision-sensitive and decision-null components, and to regularize only the former, thereby reducing changes that may affect historical decisions while preserving greater representational flexibility for learning new knowledge.
Critical strengths
- The research question is clearly motivated, shifting the focus from the magnitude of representation drift to identifying which drift components actually affect historical classifiers. This perspective provides an insightful way to revisit stability in continual learning.
- The proposed method is conceptually simple and directly exploits the geometry induced by the historical classifier to decompose temporal representation drift, without introducing additional trainable modules.
- DSDR selectively regularizes classifier-sensitive drift while leaving complementary directions relatively unconstrained, providing a clear connection to the stability–plasticity trade-off in continual learning.
- As a training-time regularizer, DSDR can be readily integrated into existing PEFT-based continual learners without introducing additional inference-time parameters or computational overhead.
Critical weaknesses
- DSDR is largely built upon the established geometric properties of the row space and null space of the historical classifier. Its novelty therefore lies mainly in applying this geometry to representation drift regularization in CIL, while the core methodological contribution remains relatively limited.
- The definitions of “decision-sensitive” and “decision-null” are not sufficiently rigorous. The current derivation only guarantees that certain drift components do not change the historical logits, but this does not ensure that the final prediction over all observed classes remains unchanged under strict CIL. In particular, such drift may still affect new-class logits and old–new class competition.
Questions for authors
- DSDR estimates and constrains representation drift using only current-task samples, while its objective is to preserve historical-task decisions. How can drift measured on the current-task distribution reliably reflect and control representation changes on previous-task distributions?
- The random-subspace baseline already yields a substantial performance gain, while DSDR provides only a relatively small additional improvement. Moreover, this key ablation is conducted using a single seed. How robust is the advantage of the classifier-induced geometry across multiple random subspaces and random seeds?
- The correlation between decision-sensitive drift and prediction flips is relatively weak. What additional mechanistic evidence supports the claimed relationship between this drift component and changes in historical predictions?
- The weaker performance on CIFAR-100 is attributed to low image resolution, which is argued to produce more compressed representations and less informative classifier geometry. What direct evidence, such as feature rank or projected drift energy, supports this explanation?
- The performance advantage of DSDR over SplitLoRA decreases as the task sequence becomes longer, while the complementarity of SplitLoRA + DSDR is currently evaluated only on the 5-task ImageNet-R setting. How robust and generalizable is this complementarity across longer task sequences and additional datasets?
- Figure 1 uses different input images for the current and frozen models, whereas the temporal drift should be computed from the same input processed by both models. In addition, the figure does not clearly indicate that only P_D Δz is regularized. Why does the current figure differ from the formulation described in the method, and how should it be revised to accurately reflect the proposed procedure?