- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies exemplar-free class-incremental learning based on pretrained ViTs. CAMP jointly uses CLS and patch distillation to constrain representation drift (FMC), compensates for old-class prototype drift with an Epanechnikov kernel (KSC), and then interpolates the old classifier weights toward the corrected prototypes (PACR). Standard classifier alignment (CA) is still performed afterward, so the method is not entirely free of retraining. The authors explain these designs in terms of gradient completeness, kernel estimation optimality, and geometric coupling, and validate them through performance comparisons and ablation studies across four datasets and six task configurations.
Critical strengths
1.Dividing ViT feature distillation into these four types of losses is intuitive.
2.The experimental results are consistently very good.
3.The coupling investigated in this paper should theoretically exist.
Critical weaknesses
-
The novelty claim regarding coupling seems somewhat overstated. Your paper argues that backbone learning and classifier-head recalibration should be coupled, but classifier-head calibration generally refers to retraining the classifier head. So the coupling you describe seems to be with prototype compensation, right? However, methods such as EFC++ already couple these components quite closely.
-
The second point in the Introduction says that classifier weights can be calibrated without retraining. Why, then, does Algorithm 1 require fine-tuning again?
-
Table 1 seems to summarize the meanings of the gradients for these four loss functions. I personally don’t think a table is necessary, since it doesn’t provide any additional information.
-
I find this paper difficult to understand. For example, the abstract says that the classifier induces a pullback metric on the backbone, but what exactly is a pullback metric? After introducing the pullback metric, the paper seems to simply describe how to initialize the classifier-head parameters. I find this quite strange. Section 3.3 says that an Epanechnikov kernel is used in place of a Gaussian kernel, but only gives a corresponding, rather cursory theoretical explanation. The 3/4 in Equation 14 also seems rather abrupt. The paper says that KSC handles forgetting on the backbone side, but KSC does not seem to be involved in the backbone loss function.
Questions for authors
The main issues have already been reflected in the weaknesses. In addition, I would suggest that the paper’s methodology lacks a strong logical structure. Many of its most important claims are conveyed through superficial textual descriptions, which diminishes the quality of the manuscript.
7 comments
W3.4: I find this paper difficult to understand. For example, the abstract says that the classifier induces a pullback metric on the backbone, but what exactly is a pullback metric? After introducing the pullback metric, the paper seems to simply describe how to initialize the classifier-head parameters. I find this quite strange. Section 3.3 says that an Epanechnikov kernel is used in place of a Gaussian kernel, but only gives a corresponding, rather cursory theoretical explanation. The 3/4 in Equation 14 also seems rather abrupt. The paper says that KSC handles forgetting on the backbone side, but KSC does not seem to be involved in the backbone loss function.
AW3.4: We thank the reviewer for the careful reading, the time spent on our manuscript, and the many constructive comments. We address the points in order.
(1) About pullback metric. A pullback metric is the metric that the classifier induces on the feature space. Each classifier weight is the gradient of its class score with respect to the feature, so it is a linear measurement of where the class lies in that space. The classifier weight can be tied to the prototype of its class, and we can correct the classifier by pulling its weights back toward the corrected prototypes. We have added this definition to the Introduction after the first mention of the pullback metric and highlighted it in blue.
(2) The metric correction is performed by initialization. Here is the classifier weight matrix, and its rows define this metric. The metric is correct precisely when each row is aligned with its class prototype. Initializing a row from the corrected prototype therefore corrects the metric itself, and CA then refines the boundaries from this starting point. We have added this explanation to Section 3.4 and highlighted it in blue as well.
(3) About the kernel. We conduct an experiment to compare the two kernels in practice. The Epanechnikov kernel is replaced by a Gaussian kernel while every other component is unchanged. Table R9 reports the results. Epanechnikov is better on last accuracy and forgetting in all four settings, and on average accuracy in three of four, consistent with the predicted AMISE constants (0.268 vs 0.282). The corresponding experiment has been added to Section 4.3.2.
Table R9. Kernel comparison in KSC (3 seeds, mean±std). Best in bold.
| Setting | Epanechnikov Acc | Forg | Gaussian Acc | Forg | ||
|---|---|---|---|---|---|---|
| IN-R (T=10) | 82.50±0.27 | 86.72±0.14 | 4.08±0.23 | 82.26±0.30 | 86.63±0.17 | 4.10±0.26 |
| IN-R (T=20) | 80.67±0.05 | 86.51±0.29 | 5.22±0.34 | 80.64±0.10 | 85.52±0.35 | 5.27±0.34 |
| IN-A (T=10) | 65.02±0.19 | 72.91±0.99 | 8.46±0.49 | 64.98±0.17 | 72.86±1.22 | 8.51±0.53 |
| IN-A (T=20) | 62.74±0.51 | 70.69±0.91 | 11.15±0.72 | 61.49±1.25 | 70.70±1.09 | 12.24±1.39 |
(4) The 3/4 in Eq. (14) is the normalization constant that makes the kernel integrate to one:
We have added this explanation to Section 3.3 of the paper and highlighted it in blue.
(5) About KSC and the backbone loss. KSC is applied to the stored prototypes after backbone training, which addresses the consequence of backbone drift but not the training objective. It estimates the residual prototype shift from the embedding shifts of current-task data and corrects the prototypes. We have rewritten the corresponding sentence in Section 3.4 and highlighted the change in blue.
Question
Q3.1: The main issues have already been reflected in the weaknesses. In addition, I would suggest that the paper's methodology lacks a strong logical structure. Many of its most important claims are conveyed through superficial textual descriptions, which diminishes the quality of the manuscript.
AQ3.1: We thank the reviewer for the careful reading of our manuscript. In response, we have added experiments for each of the three modules, so that the main claims rest on direct evidence, and we have revised the passages that could be confusing.
(i) FMC. We added two gradient-level checks on a trained checkpoint (ImageNet-A and ImageNet-R, T=10, final task, 128 test samples). The first measures how much the CLS and patch gradients overlap. For each sample we take the gradient of the two losses with respect to the last-block LoRA parameters and report the cosine similarity, the relative residual of each gradient projected onto the other's row space, and the fraction of each gradient's energy captured by the top-32 principal subspace of the other.
Table R10. CLS vs Patch gradient overlap.
| Dataset | cos(∇CLS, ∇Patch) | CLS outside Patch | Patch outside CLS | Patch in CLS top-32 | CLS in Patch top-32 |
|---|---|---|---|---|---|
| IN-A T=10 | 0.43 | 0.84 | 0.83 | 0.19 | 0.18 |
| IN-R T=10 | 0.47 | 0.79 | 0.80 | 0.20 | 0.21 |
The two gradients have a moderate cosine similarity, but about 80% of each lies outside the other's row space, and the leading 32 directions of one capture only about 20% of the other's energy. CLS and Patch therefore constrain largely different directions: they are nearly complementary, not redundant.
The second check compares Norm against CLS and Sim against Patch, and measures each loss's gradient magnitude relative to the combined CLS+Patch gradient.
Table R11. Norm and Sim against the CLS+Patch pair.
| Dataset | cos(Norm, CLS) | Norm in CLS top-32 | cos(Sim, Patch) | Sim in Patch top-32 | ‖∇Norm‖/‖∇(CLS+Patch)‖ | ‖∇Sim‖/‖∇(CLS+Patch)‖ |
|---|---|---|---|---|---|---|
| IN-A T=10 | 0.36 | 0.37 | 0.35 | 0.30 | 0.42% | 0.44% |
| IN-R T=10 | 0.38 | 0.39 | 0.37 | 0.31 | 0.42% | 0.36% |
Norm overlaps CLS and Sim overlaps Patch, and the component orthogonal to the CLS+Patch gradient carries only about of its energy. Norm and Sim are therefore redundant. Both checks are now in Appendix B.6.
(ii) KSC. We added a direct comparison between the Epanechnikov and Gaussian kernels, keeping every other component fixed, on four settings with three seeds. Epanechnikov is better on last accuracy and forgetting in all four settings, and on average accuracy in three of four, consistent with the theory. The results are in Table R9 and in Section 4.3.2.
(iii) PACR. We compared PACR with a standard EMA of the classifier weights under the same protocol.
Table R12. PACR versus standard EMA (3 seeds, mean±std). Best in bold, second-best underlined.
| Setting | w/o PACR and EMA () Acc | Forg | EMA () Acc | Forg | PACR () Acc | Forg | |||
|---|---|---|---|---|---|---|---|---|---|
| IN-R (T=10) | |||||||||
| IN-R (T=20) | |||||||||
| IN-A (T=10) | |||||||||
| IN-A (T=20) |
PACR attains the best result in almost every case. EMA is worse than the no-mechanism baseline on both ImageNet-A settings across all three metrics, because it averages the model's own previous weights, which have already drifted. The comparison is now in Section 4.3.4.
Besides, we have also revised several passages that could be confusing as mentioned in AW3.4, and repeated the main comparison with a self-supervised iBOT backbone to show the generality of CAMP, reported in Appendix F.3.
Weaknesses
AW3.1: We thank the reviewer for pointing this out, and we will explain the coupling more precisely. Prototype compensation (i.e., KSC) is one part of our coupling. In the pipeline of CAMP, FMC provides a stable feature manifold by distillation. KSC and PACR provide a corrected anchor and a corrected classifier, where KSC estimates corrected prototypes on that stable manifold, and PACR uses the corrected prototypes to set the starting point of each old-class classifier row before CA refines it. Together they complete the coupling, each supplies what the next one needs. FMC keeps the manifold fixed, KSC makes the anchor correct, and PACR makes the anchor reach the classifier. The ablation confirms this dependency, with KSC+PACR without FMC performing worse than KSC alone.
EFC++ differs in three respects. First and most importantly, it has no re-anchoring step. EFC++ updates prototypes and retrains the classifier, but the corrected prototype is not used to set the classifier's starting point, so the two processes are coupled only through the shared representation rather than through an explicit dependency at the classifier level. Second, the setting is different. EFC++ trains a ResNet from scratch and keeps updating it, whereas CAMP adapts a pretrained ViT with LoRA, so the drift we correct and the cost of correcting it are different. Third, the regularization differs because of the difference of backbone. EFC++ applies a feature-drift regularizer to a from-scratch ResNet, whereas FMC is a gradient-subspace analysis of which feature-level constraints are independent for a ViT, and the correction and anchoring stages are built on the stable manifold it produces. This distinction is now stated in Related Work and highlighted in blue.
AW3.2: PACR itself requires no retraining: it is a single re-anchoring that only sets the starting point of the classifier, while the retraining in Algorithm 1 is the CA step. We have replaced "without re-training" with "before CA training" in the Introduction and highlighted in blue, which states this more precisely.
AW3.3: We included the table to summarize the four losses and make their respective contributions easier to see at a glance. Following the reviewer's suggestion, we have moved it to Appendix B.