- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Paper proposes a new method for drift estimation based exemplar free continual learning. The method compensates past prototypes for backbone drift building upon a series of previous works on drift compensation. I think the method most similar to the proposed method is DPCR, which also has two projectors. The proposed method, in addition, proposes non-uniform distillation loss. The results are excellent for training from scratch (and on CUB from pretrained).
Critical strengths
Critical
- The paper combines a number of existing ideas of drift compensation and adds some new ones, mostly the non-uniform distillation regularization.
- The results of the paper are a notable improvement over existing methods, outperforming the most similar work DPCR with significant margin.
Non critical:
- the paper is well written and the presentation is clear (except for importance of mathematical proofs).
Critical weaknesses
Critical
- For me the main critical weakness is the clear definition of the novelty of the paper. I think the method section also requires citations to the most relevant papers, and the difference with the most relevant papers should be better explained. E.g. the projector comes from LDC, DPCR. The main difference with DPCR seems to be the drift-weighted distillation but in the ablation this does not really explain the large gap with DPCR (??).
- the purpose of the mathematical proofs is not clear to me (especially theorem 2) And I would almost prefer to move them to appendix all. The authors should motivate better their purpose
[a] Tan, Yuwen, et al. "Semantically-shifted incremental adapter-tuning is a continual vitransformer." CVPR 2024. [b] Jiarui Zhao,; Representation-Steered Incremental Adapter-Tuning for Class-Incremental Learning with Pre-Trained Models , CVPR 2026
Questions for authors
see weaknesses above
- some recent methods show that drift compensation can also be combined with LoRA update of large pretrained networks. It would be nice to see if the method also works in that case [a][b]. For example replacing SDC in [a] with the proposed method (I understand that this might be too much for rebuttal)
Scoring explanation: I am quite borderline on the paper, I could go up if the rebuttal addresses the raised weaknesses, but also down if I think they do not. I do not mind that the paper does not have results on large pretrained backbones currently required by mainy reviewers, I think both these research lines still have their merit.