acceptodds
Under review as a conference paper at ICLR 2027

CAMP: Coupled Alignment of Manifold and Prototypes for Exemplar-Free Class-Incremental Learning

Abstract

Exemplar-free class-incremental learning (EFCIL) requires sequentially learning new classes without storing past data, a setting where catastrophic forgetting stems from the progressive drift of feature representations across tasks. Existing methods treat feature drift and classifier misalignment as separate concerns, applying knowledge distillation to constrain the backbone and post-hoc compensation to correct prototypes, without a unified understanding of how these two processes interact. We show that they are in fact two sides of the same geometric problem: the linear classifier defines a pullback metric on the feature manifold, so manifold stability is a prerequisite for valid metric recalibration, and the two must be addressed as a single coupled system rather than as independent stages. We propose CAMP (oupled lignment of anifold and rototypes), a framework where these two processes are interlocked as a causal pipeline rather than stacked as independent modules. CAMP consists of eature anifold onstraint (FMC), which provides a gradient-complete manifold constraint during training, ernel hift ompensation (KSC), which corrects residual drift via a non-parametric minimax-optimal estimator, and rototype-nchored lassifier ecalibration (PACR), which re-anchors drifted classifier weights to corrected prototypes, together forming a single causal chain. Experiments on multiple benchmarks demonstrate state-of-the-art performance with superior retention, showing that these two problems are best solved as a coupled system rather than in isolation.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual learning, Class-Incremental Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mDhnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies exemplar-free class-incremental learning based on pretrained ViTs. CAMP jointly uses CLS and patch distillation to constrain representation drift (FMC), compensates for old-class prototype drift with an Epanechnikov kernel (KSC), and then interpolates the old classifier weights toward the corrected prototypes (PACR). Standard classifier alignment (CA) is still performed afterward, so the method is not entirely free of retraining. The authors explain these designs in terms of gradient completeness, kernel estimation optimality, and geometric coupling, and validate them through performance comparisons and ablation studies across four datasets and six task configurations.

Critical strengths

1.Dividing ViT feature distillation into these four types of losses is intuitive.

2.The experimental results are consistently very good.

3.The coupling investigated in this paper should theoretically exist.

Critical weaknesses
  1. The novelty claim regarding coupling seems somewhat overstated. Your paper argues that backbone learning and classifier-head recalibration should be coupled, but classifier-head calibration generally refers to retraining the classifier head. So the coupling you describe seems to be with prototype compensation, right? However, methods such as EFC++ already couple these components quite closely.

  2. The second point in the Introduction says that classifier weights can be calibrated without retraining. Why, then, does Algorithm 1 require fine-tuning again?

  3. Table 1 seems to summarize the meanings of the gradients for these four loss functions. I personally don’t think a table is necessary, since it doesn’t provide any additional information.

  4. I find this paper difficult to understand. For example, the abstract says that the classifier induces a pullback metric on the backbone, but what exactly is a pullback metric? After introducing the pullback metric, the paper seems to simply describe how to initialize the classifier-head parameters. I find this quite strange. Section 3.3 says that an Epanechnikov kernel is used in place of a Gaussian kernel, but only gives a corresponding, rather cursory theoretical explanation. The 3/4 in Equation 14 also seems rather abrupt. The paper says that KSC handles forgetting on the backbone side, but KSC does not seem to be involved in the backbone loss function.

Questions for authors

The main issues have already been reflected in the weaknesses. In addition, I would suggest that the paper’s methodology lacks a strong logical structure. Many of its most important claims are conveyed through superficial textual descriptions, which diminishes the quality of the manuscript.

7 comments

Weaknesses

W3.1: The novelty claim regarding coupling seems somewhat overstated. Your paper argues that backbone learning and classifier-head recalibration should be coupled, but classifier-head calibration generally refers to retraining the classifier head. So the coupling you describe seems to be with prototype compensation, right? However, methods such as EFC++ already couple these components quite closely.

AW3.1: We thank the reviewer for pointing this out, and we will explain the coupling more precisely. Prototype compensation (i.e., KSC) is one part of our coupling. In the pipeline of CAMP, FMC provides a stable feature manifold by distillation. KSC and PACR provide a corrected anchor and a corrected classifier, where KSC estimates corrected prototypes on that stable manifold, and PACR uses the corrected prototypes to set the starting point of each old-class classifier row before CA refines it. Together they complete the coupling, each supplies what the next one needs. FMC keeps the manifold fixed, KSC makes the anchor correct, and PACR makes the anchor reach the classifier. The ablation confirms this dependency, with KSC+PACR without FMC performing worse than KSC alone.

EFC++ differs in three respects. First and most importantly, it has no re-anchoring step. EFC++ updates prototypes and retrains the classifier, but the corrected prototype is not used to set the classifier's starting point, so the two processes are coupled only through the shared representation rather than through an explicit dependency at the classifier level. Second, the setting is different. EFC++ trains a ResNet from scratch and keeps updating it, whereas CAMP adapts a pretrained ViT with LoRA, so the drift we correct and the cost of correcting it are different. Third, the regularization differs because of the difference of backbone. EFC++ applies a feature-drift regularizer to a from-scratch ResNet, whereas FMC is a gradient-subspace analysis of which feature-level constraints are independent for a ViT, and the correction and anchoring stages are built on the stable manifold it produces. This distinction is now stated in Related Work and highlighted in blue.

W3.2: The second point in the Introduction says that classifier weights can be calibrated without retraining. Why, then, does Algorithm 1 require fine-tuning again?

AW3.2: PACR itself requires no retraining: it is a single re-anchoring that only sets the starting point of the classifier, while the retraining in Algorithm 1 is the CA step. We have replaced "without re-training" with "before CA training" in the Introduction and highlighted in blue, which states this more precisely.

W3.3: Table 1 seems to summarize the meanings of the gradients for these four loss functions. I personally don't think a table is necessary, since it doesn't provide any additional information.

AW3.3: We included the table to summarize the four losses and make their respective contributions easier to see at a glance. Following the reviewer's suggestion, we have moved it to Appendix B.

W3.4: I find this paper difficult to understand. For example, the abstract says that the classifier induces a pullback metric on the backbone, but what exactly is a pullback metric? After introducing the pullback metric, the paper seems to simply describe how to initialize the classifier-head parameters. I find this quite strange. Section 3.3 says that an Epanechnikov kernel is used in place of a Gaussian kernel, but only gives a corresponding, rather cursory theoretical explanation. The 3/4 in Equation 14 also seems rather abrupt. The paper says that KSC handles forgetting on the backbone side, but KSC does not seem to be involved in the backbone loss function.

AW3.4: We thank the reviewer for the careful reading, the time spent on our manuscript, and the many constructive comments. We address the points in order.

(1) About pullback metric. A pullback metric is the metric that the classifier induces on the feature space. Each classifier weight is the gradient of its class score with respect to the feature, so it is a linear measurement of where the class lies in that space. The classifier weight can be tied to the prototype of its class, and we can correct the classifier by pulling its weights back toward the corrected prototypes. We have added this definition to the Introduction after the first mention of the pullback metric and highlighted it in blue.

(2) The metric correction is performed by initialization. Here WW is the classifier weight matrix, and its rows define this metric. The metric is correct precisely when each row is aligned with its class prototype. Initializing a row from the corrected prototype therefore corrects the metric itself, and CA then refines the boundaries from this starting point. We have added this explanation to Section 3.4 and highlighted it in blue as well.

(3) About the kernel. We conduct an experiment to compare the two kernels in practice. The Epanechnikov kernel is replaced by a Gaussian kernel while every other component is unchanged. Table R9 reports the results. Epanechnikov is better on last accuracy and forgetting in all four settings, and on average accuracy in three of four, consistent with the predicted AMISE constants (0.268 vs 0.282). The corresponding experiment has been added to Section 4.3.2.

Table R9. Kernel comparison in KSC (3 seeds, mean±std). Best in bold.

SettingEpanechnikov AccAˉ\bar AForgGaussian AccAˉ\bar AForg
IN-R (T=10)82.50±0.2786.72±0.144.08±0.2382.26±0.3086.63±0.174.10±0.26
IN-R (T=20)80.67±0.0586.51±0.295.22±0.3480.64±0.1085.52±0.355.27±0.34
IN-A (T=10)65.02±0.1972.91±0.998.46±0.4964.98±0.1772.86±1.228.51±0.53
IN-A (T=20)62.74±0.5170.69±0.9111.15±0.7261.49±1.2570.70±1.0912.24±1.39

(4) The 3/4 in Eq. (14) is the normalization constant that makes the kernel integrate to one:

∫−11(1−u2) du=43,34⋅43=1.\int_{-1}^{1}(1-u^2)\,du=\frac{4}{3},\qquad \frac{3}{4}\cdot\frac{4}{3}=1.

We have added this explanation to Section 3.3 of the paper and highlighted it in blue.

(5) About KSC and the backbone loss. KSC is applied to the stored prototypes after backbone training, which addresses the consequence of backbone drift but not the training objective. It estimates the residual prototype shift from the embedding shifts of current-task data and corrects the prototypes. We have rewritten the corresponding sentence in Section 3.4 and highlighted the change in blue.

Question

Q3.1: The main issues have already been reflected in the weaknesses. In addition, I would suggest that the paper's methodology lacks a strong logical structure. Many of its most important claims are conveyed through superficial textual descriptions, which diminishes the quality of the manuscript.

AQ3.1: We thank the reviewer for the careful reading of our manuscript. In response, we have added experiments for each of the three modules, so that the main claims rest on direct evidence, and we have revised the passages that could be confusing.

(i) FMC. We added two gradient-level checks on a trained checkpoint (ImageNet-A and ImageNet-R, T=10, final task, 128 test samples). The first measures how much the CLS and patch gradients overlap. For each sample we take the gradient of the two losses with respect to the last-block LoRA parameters and report the cosine similarity, the relative residual of each gradient projected onto the other's row space, and the fraction of each gradient's energy captured by the top-32 principal subspace of the other.

Table R10. CLS vs Patch gradient overlap.

Datasetcos(∇CLS, ∇Patch)CLS outside PatchPatch outside CLSPatch in CLS top-32CLS in Patch top-32
IN-A T=100.430.840.830.190.18
IN-R T=100.470.790.800.200.21

The two gradients have a moderate cosine similarity, but about 80% of each lies outside the other's row space, and the leading 32 directions of one capture only about 20% of the other's energy. CLS and Patch therefore constrain largely different directions: they are nearly complementary, not redundant.

The second check compares Norm against CLS and Sim against Patch, and measures each loss's gradient magnitude relative to the combined CLS+Patch gradient.

Table R11. Norm and Sim against the CLS+Patch pair.

Datasetcos(Norm, CLS)Norm in CLS top-32cos(Sim, Patch)Sim in Patch top-32‖∇Norm‖/‖∇(CLS+Patch)‖‖∇Sim‖/‖∇(CLS+Patch)‖
IN-A T=100.360.370.350.300.42%0.44%
IN-R T=100.380.390.370.310.42%0.36%

Norm overlaps CLS and Sim overlaps Patch, and the component orthogonal to the CLS+Patch gradient carries only about 10−510^{-5} of its energy. Norm and Sim are therefore redundant. Both checks are now in Appendix B.6.

(ii) KSC. We added a direct comparison between the Epanechnikov and Gaussian kernels, keeping every other component fixed, on four settings with three seeds. Epanechnikov is better on last accuracy and forgetting in all four settings, and on average accuracy in three of four, consistent with the theory. The results are in Table R9 and in Section 4.3.2.

(iii) PACR. We compared PACR with a standard EMA of the classifier weights under the same protocol.

Table R12. PACR versus standard EMA (3 seeds, mean±std). Best in bold, second-best underlined.

Settingw/o PACR and EMA (β=0\beta=0) AccAˉ\bar AForgEMA (M=0.3M=0.3) AccAˉ\bar AForgPACR (β=0.3\beta=0.3) AccAˉ\bar AForg
IN-R (T=10)82.11±0.1882.11±0.1886.84±0.30\textbf{86.84±0.30}6.06±0.096.06±0.0982.27±0.22‾\underline{82.27±0.22}86.72±0.20‾\underline{86.72±0.20}5.20±0.24‾\underline{5.20±0.24}82.50±0.27\textbf{82.50±0.27}86.72±0.14‾\underline{86.72±0.14}4.08±0.23\textbf{4.08±0.23}
IN-R (T=20)80.12±0.1980.12±0.1985.36±0.40‾\underline{85.36±0.40}6.97±0.456.97±0.4580.20±0.02‾\underline{80.20±0.02}85.34±0.4385.34±0.436.89±0.45‾\underline{6.89±0.45}80.67±0.05\textbf{80.67±0.05}86.51±0.29\textbf{86.51±0.29}5.22±0.34\textbf{5.22±0.34}
IN-A (T=10)64.19±0.51‾\underline{64.19±0.51}72.45±1.05‾\underline{72.45±1.05}11.17±0.43‾\underline{11.17±0.43}63.81±0.3063.81±0.3072.23±1.2472.23±1.2412.18±0.2612.18±0.2665.02±0.19\textbf{65.02±0.19}72.91±0.99\textbf{72.91±0.99}8.46±0.49\textbf{8.46±0.49}
IN-A (T=20)59.82±0.20‾\underline{59.82±0.20}69.57±1.19‾\underline{69.57±1.19}14.38±0.43‾\underline{14.38±0.43}59.38±0.4659.38±0.4669.31±1.0669.31±1.0615.46±0.3815.46±0.3862.74±0.51\textbf{62.74±0.51}70.69±0.91\textbf{70.69±0.91}11.15±0.72\textbf{11.15±0.72}

PACR attains the best result in almost every case. EMA is worse than the no-mechanism baseline on both ImageNet-A settings across all three metrics, because it averages the model's own previous weights, which have already drifted. The comparison is now in Section 4.3.4.

Besides, we have also revised several passages that could be confusing as mentioned in AW3.4, and repeated the main comparison with a self-supervised iBOT backbone to show the generality of CAMP, reported in Appendix F.3.

Reviewer_mUCaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper studies exemplar-free class-incremental learning on top of a pre-trained ViT-B/16 adapted with per-task LoRA. Its central claim is that feature drift in the backbone and misalignment of the classifier are two aspects of one geometric problem. Based on this view, the authors propose CAMP, a three-stage pipeline: (1) FMC distils the CLS token and the patch tokens from the previous model. (2) KSC corrects the drift of old-class prototypes without training, using a kernel-weighted average of the feature shifts observed on current-task data. (3) PACR pulls each old-class classifier weight part of the way toward its corrected prototype. Experiments on ImageNet-R, ImageNet-A, CUB-200 and CIFAR-100 (10 and 20 tasks) compare CAMP against prompt-based, adapter-based and recent drift-compensation methods.

Critical strengths
  1. CAMP beats the strongest baseline (MACIL) on last and average accuracy in all six settings, with 3-seed std. The largest gains are on the long, hard ImageNet-A T=20 sequence. Code is provided.
  2. All 8 component combinations are reported on three datasets with std. This makes it possible to see where the gains actually come from.
Critical weaknesses
  1. FMC gradient analysis (Sec. 3.2, App. B) analysis builds on c=∑iwipic = \sum_i w_i p_i with wiw_i from the final attention layer. That is not how a ViT computes its output CLS token: it also passes through the last MLP, residual connections, and the final LayerNorm, and it is not an attention-weighted sum of the final-layer patch tokens.
  2. The empirical contribution is small. FMC alone already matches MACIL (82.04 / 90.27) on IN-R and CUB. FMC itself is CLS + patch-token distillation, a combination of known objectives widely used in CIL methods. The remaining gain from the two components the paper presents as novel, KSC + PACR, is only +0.42 Acc on IN-R, and +0.45 on CUB, roughly 1–2 std. Moreover, the full model does not have the best Aˉ\bar A on IN-R.
  3. Only one backbone (ViT-B/16, IN-21K supervised) is used. There are no self-supervised or differently pre-trained backbones.
  4. The abstract and conclusion claim "superior retention", but CAMP does not have the lowest forgetting in four of six settings. Examples: CODA-Prompt 3.77 vs 8.46 on IN-A T=10; DualPrompt 4.49 vs 5.22 on IN-R T=20.
Questions for authors

Can the FMC analysis be made rigorous for the actual ViT output (with MLP, residuals and LayerNorm)

8 comments

Weaknesses

W2.1: FMC gradient analysis (Sec. 3.2, App. B) builds on c=∑iwipic = \sum_i w_i p_i with wiw_i from the final attention layer. That is not how a ViT computes its output CLS token: it also passes through the last MLP, residual connections, and the final LayerNorm, and it is not an attention-weighted sum of the final-layer patch tokens.

AW2.1: We thank the reviewer for pointing this out, and we have rewritten these analyses. The changes are as follows.

(1) The chain rule (Section 3.2.1 and Appendix B, Eq. 21). We merge the final attention, the residual connections, the MLP and the final LayerNorm into a single readout AθA_\theta that produces the CLS token. The residual connections make the CLS state zz an input of the readout in its own right, and θa\theta_a denotes the parameters of these operations. With Ji=∂Aθ/∂piJ_i = \partial A_\theta/\partial p_i and Jr=∂Aθ/∂rJ_r = \partial A_\theta/\partial r, where r=(z,θa)r=(z,\theta_a), the gradient of the CLS token is exactly

∂c∂θ=∑iJi ∂pi∂θ+Jr ∂r∂θ.\frac{\partial c}{\partial\theta} = \sum_i J_i\ \frac{\partial p_i}{\partial\theta} + J_r\ \frac{\partial r}{\partial\theta}.

Compared with the original equation, the only difference is that the scalar weight wiw_i becomes the Jacobian JiJ_i, and the attention-weight term ∑ipi ∂wi/∂θ\sum_i p_i\ \partial w_i/\partial\theta becomes Jr ∂r/∂θJ_r\ \partial r/\partial\theta. Eq. (21) then becomes

∂LCLS∂θ=∑i[2Ji⊤(c−c(t−1))]⊤∂pi∂θ+2(c−c(t−1))⊤Jr ∂r∂θ,\frac{\partial \mathcal{L}_{\mathrm{CLS}}}{\partial\theta} = \sum_i \big[2J_i^\top (c-c^{(t-1)})\big]^\top \frac{\partial p_i}{\partial\theta} + 2(c-c^{(t-1)})^\top J_r\ \frac{\partial r}{\partial\theta},

where the first term is the gradient through the patch states and the second is the gradient through the readout.

(2) The gradient of the norm loss (Appendix B, Eq. 24). The substitution now uses the same expansion, giving

∂LNorm∂θ=2(∥c∥−∥c(t−1)∥)∥c∥ c⊤[∑iJi ∂pi∂θ+Jr ∂r∂θ],\frac{\partial \mathcal{L}_{\mathrm{Norm}}}{\partial\theta} = \frac{2(\|c\|-\|c^{(t-1)}\|)}{\|c\|}\ c^\top\Big[\sum_i J_i\ \frac{\partial p_i}{\partial\theta} + J_r\ \frac{\partial r}{\partial\theta}\Big],

so the two losses still share the same Jacobian of cc and differ only in the feature-space direction that weights it, and the inclusion ∇θLNorm⊂∇θLCLS\nabla_\theta \mathcal{L}_{\mathrm{Norm}} \subset \nabla_\theta \mathcal{L}_{\mathrm{CLS}} is unchanged.

(3) The patch-to-prototype transfer (Appendix C, Eq. 29). The bound is now obtained from the same decomposition, written in the form of Eq. (29):

∥μc(t)−μc(t−1)∥≤∥Aθt(r(t),p(t))−Aθt(r(t),p(t−1))‾∥+∥Aθt(r(t),p(t−1))−Aθt−1(r(t−1),p(t−1))‾∥,\big\|\mu_c^{(t)}-\mu_c^{(t-1)}\big\| \le \big\|\overline{A_{\theta_t}(r^{(t)},p^{(t)})-A_{\theta_t}(r^{(t)},p^{(t-1)})}\big\| + \big\|\overline{A_{\theta_t}(r^{(t)},p^{(t-1)})-A_{\theta_{t-1}}(r^{(t-1)},p^{(t-1)})}\big\|,

where ⋅‾\overline{\cdot} is the class mean. The first term is the patch path, which the patch term of FMC controls, and the second is the readout path, which only the CLS term constrains independently. The second half of the second term is exactly c(t−1)c^{(t-1)}. The patch term still governs the within-class coherence of the drift field that KSC averages.

None of these rewrites changes our conclusions. We have made the changes in the paper and highlighted them in blue.

W2.2: The empirical contribution is small. FMC alone already matches MACIL (82.04 / 90.27) on IN-R and CUB. FMC itself is CLS + patch-token distillation, a combination of known objectives widely used in CIL methods. The remaining gain from the two components the paper presents as novel, KSC + PACR, is only +0.42 Acc on IN-R, and +0.45 on CUB, roughly 1–2 std. Moreover, the full model does not have the best Aˉ\bar{A} on IN-R.

AW2.2: We thank the reviewer for the careful reading and the comments.

Retention is the main indicator of stability when accuracy is already high, and the contribution of the two components on retention is substantial. Moving from FMC alone to the full pipeline lowers forgetting by 2.01 points on ImageNet-R (6.09 → 4.08), 3.38 points on ImageNet-A (11.84 → 8.46), and 0.75 points on CUB (4.48 → 3.73), which correspond to relative reductions of 33%, 29% and 17%. On ImageNet-A, where drift is strongest, the gain is not confined to forgetting when last accuracy rises by +1.25 at T=10.

We also do the same comparison on the harder sequences (T=20) on ImageNet-R and ImageNet-A. As shown on Table R13, the full pipeline attains lower forgetting and a larger accuracy gain than FMC alone. On ImageNet-R it gains +0.95 in last accuracy and lowers forgetting by 2.41. On ImageNet-A it gains +2.92 and lowers forgetting by 3.35.

Table R13. FMC alone versus the full pipeline on T=20 (Last / Avg / Forg, mean±std). Δ = full − FMC only.

SettingFMC onlyFMC+KSC+PACRΔ LastΔ Forg
IN-R (T=20)79.72±0.21 / 84.86±0.41 / 7.63±0.5780.67±0.05 / 86.51±0.29 / 5.22±0.34+0.95−2.41
IN-A (T=20)59.82±0.26 / 69.80±0.86 / 14.50±0.4062.74±0.51 / 70.69±0.91 / 11.15±0.72+2.92−3.35

Besides, the novelty of FMC is the analysis that selects the distillation targets, not the targets themselves. The gradient-subspace decomposition shows that CLS and patch constraints are complementary and jointly complete, while norm and similarity constraints are strict subsets. FMC therefore provides a choice of distillation targets that is supported by both the theory and the experiments, as we detail in Appendix B.6 and under Q2.1.

We have corrected the bold in Table 3, where the highest Aˉ\bar A on ImageNet-R is 86.84 (FMC+KSC), and added the T=20 comparison to Appendix F.

W2.3: Only one backbone (ViT-B/16, IN-21K supervised) is used. There are no self-supervised or differently pre-trained backbones.

AW2.3: We have added a self-supervised backbone (ViT-B/16, iBOT) and run six methods, CAMP, MACIL, SLCA, SLDC, RanPAC and FeTrIL, on all six settings (T=10 on four datasets, T=20 on ImageNet-R and ImageNet-A, 3 seeds each).

Table R7. Comparison on the iBOT backbone: ImageNet-R and CUB-200 (mean±std over 3 seeds). Best in bold, second-best underlined.

MethodIN-R (T=10) AccAˉ\bar AForgIN-R (T=20) AccAˉ\bar AForgCUB (T=10) AccAˉ\bar AForg
SLCA73.33±0.9073.33±0.9078.52±1.1078.52±1.1012.73±1.0012.73±1.0070.42±1.7070.42±1.7075.93±0.9075.93±0.9015.41±2.1015.41±2.1076.66±0.3776.66±0.3783.60±0.8383.60±0.8315.90±0.7915.90±0.79
SLDC71.44±0.1971.44±0.1977.96±1.0377.96±1.038.53±0.428.53±0.4265.92±0.1665.92±0.1674.64±0.8774.64±0.8713.33±0.4913.33±0.4978.85±0.1878.85±0.1883.94±0.5983.94±0.596.10±0.766.10±0.76
RanPAC73.98±0.0573.98±0.0579.79±0.5379.79±0.535.15±0.31‾\underline{5.15±0.31}73.56±0.1773.56±0.1779.99±0.6679.99±0.666.09±0.46\textbf{6.09±0.46}81.96±0.1081.96±0.1087.47±0.7387.47±0.735.70±0.51‾\underline{5.70±0.51}
FeTrIL55.67±0.7455.67±0.7463.15±1.3163.15±1.3119.85±1.2419.85±1.2454.62±0.5654.62±0.5662.47±1.3962.47±1.3922.45±1.4222.45±1.4269.69±1.0669.69±1.0677.67±0.9977.67±0.9912.36±0.9812.36±0.98
MACIL77.99±0.23‾\underline{77.99±0.23}82.99±1.00‾\underline{82.99±1.00}6.61±0.256.61±0.2574.63±0.37‾\underline{74.63±0.37}80.84±1.00‾\underline{80.84±1.00}8.86±0.648.86±0.6482.70±0.83‾\underline{82.70±0.83}88.20±0.93‾\underline{88.20±0.93}7.58±1.267.58±1.26
CAMP (ours)78.75±0.03\textbf{78.75±0.03}83.37±0.90\textbf{83.37±0.90}4.68±0.20\textbf{4.68±0.20}76.85±0.37\textbf{76.85±0.37}82.23±0.75\textbf{82.23±0.75}6.16±0.65‾\underline{6.16±0.65}84.45±0.26\textbf{84.45±0.26}89.08±0.91\textbf{89.08±0.91}5.39±0.22\textbf{5.39±0.22}

Table R8. Comparison on the iBOT backbone: ImageNet-A and CIFAR-100 (mean±std over 3 seeds). Best in bold, second-best underlined.

MethodIN-A (T=10) AccAˉ\bar AForgIN-A (T=20) AccAˉ\bar AForgCIFAR-100 (T=10) AccAˉ\bar AForg
SLCA40.18±1.9840.18±1.9849.52±2.6549.52±2.6522.12±3.8222.12±3.8240.51±1.9040.51±1.9050.10±2.3050.10±2.3021.98±1.2021.98±1.2082.71±0.7082.71±0.7088.65±0.8288.65±0.8213.07±0.8613.07±0.86
SLDC43.21±1.5643.21±1.5654.01±1.0154.01±1.0110.17±1.42‾\underline{10.17±1.42}34.96±1.6234.96±1.6248.42±0.8548.42±0.8515.81±2.3115.81±2.3183.06±0.5383.06±0.5389.79±0.4389.79±0.4310.32±0.6110.32±0.61
RanPAC47.95±0.6447.95±0.6458.42±0.9058.42±0.9010.92±1.2310.92±1.2337.88±0.9937.88±0.9948.68±0.9748.68±0.9712.28±0.57\textbf{12.28±0.57}85.75±0.6485.75±0.6490.85±0.2790.85±0.274.93±0.60‾\underline{4.93±0.60}
FeTrIL30.37±1.0030.37±1.0040.83±0.7640.83±0.7615.34±2.4815.34±2.4830.17±2.0030.17±2.0040.96±0.5540.96±0.5517.08±2.3017.08±2.3066.28±0.1566.28±0.1574.77±0.1174.77±0.1125.37±0.2825.37±0.28
MACIL51.11±0.53‾\underline{51.11±0.53}59.81±1.33‾\underline{59.81±1.33}10.81±0.3710.81±0.3743.69±0.10‾\underline{43.69±0.10}53.92±0.46‾\underline{53.92±0.46}16.11±0.5416.11±0.5486.69±0.32‾\underline{86.69±0.32}91.34±0.32‾\underline{91.34±0.32}5.56±0.595.56±0.59
CAMP (ours)53.83±0.40\textbf{53.83±0.40}62.28±1.26\textbf{62.28±1.26}9.00±0.49\textbf{9.00±0.49}49.04±0.69\textbf{49.04±0.69}58.86±1.04\textbf{58.86±1.04}14.30±0.73‾\underline{14.30±0.73}87.13±0.21\textbf{87.13±0.21}91.82±0.31\textbf{91.82±0.31}4.65±0.21\textbf{4.65±0.21}

On the iBOT backbone, CAMP attains the best last and average accuracy in all six settings. Against MACIL, the strongest competitor on this backbone, the last-accuracy margins are +2.72 on ImageNet-A (T=10), +0.76 on ImageNet-R (T=10), +1.75 on CUB-200, +0.44 on CIFAR-100, +5.35 on ImageNet-A (T=20) and +2.22 on ImageNet-R (T=20). CAMP has the lowest forgetting in four of the six settings. In the other two RanPAC is lower, by 0.07 on ImageNet-R (T=20) and 2.02 on ImageNet-A (T=20), but there CAMP's last accuracy is higher by 3.29 and 11.16 points. The conclusions of Section 4.2 therefore carry over to a self-supervised backbone.

We have added these tables to Appendix F.3 and highlighted the changes in blue.

W2.4: The abstract and conclusion claim "superior retention", but CAMP does not have the lowest forgetting in four of six settings. Examples: CODA-Prompt 3.77 vs 8.46 on IN-A T=10; DualPrompt 4.49 vs 5.22 on IN-R T=20.

AW2.4: We thank the reviewer for pointing this out. We have replaced it with "strong retention" in the abstract and in the contributions.

Our intent was to claim strong retention among methods of comparable accuracy. CODA-Prompt reaches forgetting 3.77 on ImageNet-A (T=10) at last accuracy 47.84 against CAMP's 65.02, and DualPrompt reaches 4.49 on ImageNet-R (T=20) at 62.60 against 80.67. A method that has learned less has less to forget, so these numbers reflect weak plasticity rather than stronger retention.

We have made these changes in the paper and highlighted them in blue.

Reviewer_a5KjOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates Exemplar-Free Class-Incremental Learning built upon pre-trained ViT and LoRA. The authors argue that existing approaches treat backbone feature drift and classifier misalignment as disjoint problems, whereas they are geometrically coupled via the pullback metric induced by the linear classifier on the feature manifold. To address this, the authors propose CAMP, a three-stage causal pipeline.

Critical strengths
  1. Clear motivation. Unifying backbone regularization and decision boundary adaptation into an interconnected causal pipeline is well-motivated.
  2. Superior performance. CAMP achieves the highest final accuracy and average accuracy across all tested benchmarks and sequence lengths.
Critical weaknesses
  1. Overstating of theoretical foundations. The paper repeatedly invokes advanced geometric terminology, such as "pullback metric structure", "differential covectors on Riemannian manifolds", and "minimax-optimal causal pipelines". They are all fundamentally conventional heuristic procedures.
  2. Missing PEFT-CIL Baselines. The paper compares against L2P, DualPrompt, CODA-Prompt, EASE, SLCA, MACIL, and SLDC. However, several strong recent PEFT continual learning methods that utilize pre-trained ViTs are absent, such as RanPAC, and FeTrIL. RanPAC in particular sets very strong baselines without replay.
Questions for authors
  1. Empirical Validation of Kernel Choice. A head-to-head comparison should be provided to show final accuracy and forgetting when replacing the Epanechnikov kernel in KSC with a Gaussian kernel and an Epanechnikov kernel across ImageNet-R and ImageNet-A.
  2. Mechanism of PACR vs. Standard EMA: How does PACR differ mathematically and functionally from simple EMA or moving anchor initialization? If we remove the "pullback metric" narrative, what is the fundamental algorithmic novelty over standard prototype-based boundary adjustment?
5 comments

Weaknesses

W1.1: Overstating of theoretical foundations. The paper repeatedly invokes advanced geometric terminology, such as "pullback metric structure", "differential covectors on Riemannian manifolds", and "minimax-optimal causal pipelines". They are all fundamentally conventional heuristic procedures.

AW1.1: We thank the reviewer. We agree that some wording was stronger than what our analysis rigorously supports, and we have made minimal wording revisions: (i) the "pullback metric structure" is now described as the metric that the linear classifier induces on the feature space, and we state that we use this metric rather than claim it as a structural property. (ii) the "differential covector" terminology is removed and replaced by saying that a weight wcw_c is the gradient of the class-cc score with respect to the feature and that WW⊤WW^\top is the metric they jointly define. (iii) "causal pipeline" is replaced by "coupled pipeline", since we claim only that the three stages are coupled in the order FMC→KSC→PACR, which is what the ablation measures.

We have changed the corresponding wording in the paper and highlighted the changes in blue. The algorithm, experiments and results are unchanged, the technical content is unaffected, and the contributions we claim remain unchanged.

W1.2: Missing PEFT-CIL baselines. The paper compares against L2P, DualPrompt, CODA-Prompt, EASE, SLCA, MACIL, and SLDC. However, several strong recent PEFT continual learning methods that utilize pre-trained ViTs are absent, such as RanPAC, and FeTrIL. RanPAC in particular sets very strong baselines without replay.

AW1.2: We have run both methods under the same protocol as all other experiments (pre-trained ViT-B/16, exemplar-free, three seeds), and added them as two new rows to each of the two state-of-the-art tables. RanPAC keeps its official adapter-based backbone and ridge-regression classifier. FeTrIL keeps the ViT frozen, as its design assumes. Tables R1 and R2 below compare CAMP with these two methods and with MACIL, a strong baseline, and mirror Tables 1 and 2 of the paper.

Table R1. Comparison with PEFT-CIL baselines on ImageNet-R and CUB-200 (mean±std over 3 seeds). Best in bold, second-best underlined.

MethodIN-R (T=10) AccAˉ\bar AForgIN-R (T=20) AccAˉ\bar AForgCUB (T=10) AccAˉ\bar AForg
MACIL82.04±0.09‾\underline{82.04±0.09}86.40±0.33‾\underline{86.40±0.33}5.74±0.465.74±0.4679.53±0.13‾\underline{79.53±0.13}84.80±0.19‾\underline{84.80±0.19}7.11±0.307.11±0.3090.27±0.40‾\underline{90.27±0.40}93.53±0.27‾\underline{93.53±0.27}4.47±0.664.47±0.66
RanPAC77.66±0.1477.66±0.1482.94±0.2382.94±0.234.66±0.39‾\underline{4.66±0.39}77.03±0.4577.03±0.4582.52±0.1082.52±0.105.01±0.60\textbf{5.01±0.60}90.13±0.3090.13±0.3093.44±0.2293.44±0.223.59±0.34\textbf{3.59±0.34}
FeTrIL63.62±0.1663.62±0.1669.33±0.9769.33±0.9715.55±0.3215.55±0.3262.72±0.2762.72±0.2769.15±0.8969.15±0.8917.80±0.4417.80±0.4487.60±0.2087.60±0.2091.86±0.9591.86±0.956.84±0.326.84±0.32
CAMP (ours)82.50±0.27\textbf{82.50±0.27}86.72±0.14\textbf{86.72±0.14}4.08±0.23\textbf{4.08±0.23}80.67±0.05\textbf{80.67±0.05}86.51±0.29\textbf{86.51±0.29}5.22±0.34‾\underline{5.22±0.34}90.71±0.06\textbf{90.71±0.06}93.82±0.17\textbf{93.82±0.17}3.73±0.41‾\underline{3.73±0.41}

Table R2. Comparison with PEFT-CIL baselines on ImageNet-A and CIFAR-100 (mean±std over 3 seeds). Best in bold, second-best underlined.

MethodIN-A (T=10) AccAˉ\bar AForgIN-A (T=20) AccAˉ\bar AForgCIFAR-100 (T=10) AccAˉ\bar AForg
MACIL63.31±0.31‾\underline{63.31±0.31}71.58±0.9571.58±0.9510.58±0.5510.58±0.5559.23±0.57‾\underline{59.23±0.57}68.59±0.86‾\underline{68.59±0.86}14.48±0.5814.48±0.5891.74±0.14‾\underline{91.74±0.14}94.76±0.16‾\underline{94.76±0.16}3.07±0.25‾\underline{3.07±0.25}
RanPAC62.76±0.4562.76±0.4571.62±1.06‾\underline{71.62±1.06}9.25±0.64‾\underline{9.25±0.64}59.05±1.3159.05±1.3168.27±3.0568.27±3.059.78±0.17\textbf{9.78±0.17}90.75±0.3490.75±0.3494.22±0.1894.22±0.183.17±0.143.17±0.14
FeTrIL51.61±0.2751.61±0.2760.11±0.2560.11±0.2513.47±0.8513.47±0.8551.57±0.4851.57±0.4860.24±0.5260.24±0.5214.49±0.9514.49±0.9582.77±0.2382.77±0.2388.09±0.0288.09±0.0213.56±0.1213.56±0.12
CAMP (ours)65.02±0.19\textbf{65.02±0.19}72.91±0.99\textbf{72.91±0.99}8.46±0.49\textbf{8.46±0.49}62.74±0.51\textbf{62.74±0.51}70.69±0.91\textbf{70.69±0.91}11.15±0.72‾\underline{11.15±0.72}92.43±0.15\textbf{92.43±0.15}94.99±0.13\textbf{94.99±0.13}2.45±0.15\textbf{2.45±0.15}

As shown in Tables R1 and R2, CAMP achieves the highest Last and average accuracy in all six settings. Against RanPAC the margins in Last accuracy are +4.84 (IN-R, T=10), +3.64 (IN-R, T=20), +2.26 (IN-A, T=10), +3.69 (IN-A, T=20), +0.58 (CUB) and +1.68 (CIFAR-100). Against FeTrIL they are 3.1 to 21.2 points. On forgetting, CAMP is the lowest in three of the six settings. RanPAC is lower in the other three, but there CAMP's accuracy is higher, and substantially so on ImageNet-A (T=20) (+3.69) and ImageNet-R (T=20) (+3.64). Forgetting measures how much a model has lost, so a model with much lower accuracy has less to lose. We therefore treat forgetting as a secondary indicator: CAMP attains a relatively low forgetting while keeping high accuracy, which is the behaviour we aim for.

We have added these results to the paper and discussed the two methods in Section 4.2, with the changes highlighted in blue.

Questions

Q1.1: Empirical Validation of Kernel Choice. A head-to-head comparison should be provided to show final accuracy and forgetting when replacing the Epanechnikov kernel in KSC with a Gaussian kernel and an Epanechnikov kernel across ImageNet-R and ImageNet-A.

AQ1.1: We replaced the Epanechnikov kernel in KSC with a Gaussian kernel, left every other component unchanged, and reran four settings. Table R3 reports the results as mean±std over three seeds, the same protocol as every other experiment in the paper.

Table R3. Kernel comparison in KSC (3 seeds, mean±std). Better values in bold.

SettingEpanechnikov (ours) AccAˉ\bar AForgGaussian AccAˉ\bar AForg
IN-R (T=10)82.50±0.2786.72±0.144.08±0.2382.26±0.3086.63±0.174.10±0.26
IN-R (T=20)80.67±0.0586.51±0.295.22±0.3480.64±0.1085.52±0.355.27±0.34
IN-A (T=10)65.02±0.1972.91±0.998.46±0.4964.98±0.1772.86±1.228.51±0.53
IN-A (T=20)62.74±0.5170.69±0.9111.15±0.7261.49±1.2570.70±1.0912.24±1.39

The two kernels give nearly the same result. In Last accuracy the difference is at most 0.24 points on three of the four settings, and 1.25 points on ImageNet-A (T=20), where the Gaussian variant also has the larger seed variance. Aˉ\bar A differs by at most 0.99 points, and forgetting by at most 0.05 points except on ImageNet-A (T=20), where it differs by 1.09. This matches the analysis in Appendix D, which predicts a modest but systematic advantage for Epanechnikov (AMISE constant 0.268 vs. 0.282). Since all values are means over three seeds, the small margins are a consistent effect rather than noise.

We have added this comparison as a new ablation in Section 4.3, together with the table, and highlighted the changes in blue.

Q1.2: Mechanism of PACR vs. Standard EMA. How does PACR differ mathematically and functionally from simple EMA or moving anchor initialization? If we remove the "pullback metric" narrative, what is the fundamental algorithmic novelty over standard prototype-based boundary adjustment?

AQ1.2: A standard EMA[1] averages the weights that the model itself produces, and it must therefore be applied after every update throughout continual training, since its purpose is to smooth the noise in those weights. PACR instead initializes the new classifier from the KSC-corrected prototypes, which is legitimate because a weight wcw_c is the gradient of the class-cc score with respect to the feature, a linear measurement on the feature space, and therefore another form of the prototype. Since a classifier weight and a prototype are both linear measurements on the same feature space, PACR links the two. In effect, an EMA repeatedly pulls the model back toward old knowledge that has already drifted and may no longer be correct, whereas PACR only fixes a starting point and leaves the classifier free to learn the new classes.

We also ran experiments comparing the two. EMA is applied after every epoch as w←(1−M) w+M wCAw\leftarrow(1-M)\,w+M\,w_{\mathrm{CA}} with M=β=0.3M=\beta=0.3, the same value as the PACR coefficient. We compare EMA, PACR, and a variant with neither mechanism (β=0\beta=0).

Table R4. PACR versus standard EMA (3 seeds, mean±std). Best in bold, second-best underlined.

Settingw/o PACR and EMA (β=0\beta=0) AccAˉ\bar AForgEMA (M=0.3M=0.3) AccAˉ\bar AForgPACR (β=0.3\beta=0.3) AccAˉ\bar AForg
IN-R (T=10)82.11±0.1882.11±0.1886.84±0.30\textbf{86.84±0.30}6.06±0.096.06±0.0982.27±0.22‾\underline{82.27±0.22}86.72±0.20‾\underline{86.72±0.20}5.20±0.24‾\underline{5.20±0.24}82.50±0.27\textbf{82.50±0.27}86.72±0.14‾\underline{86.72±0.14}4.08±0.23\textbf{4.08±0.23}
IN-R (T=20)80.12±0.1980.12±0.1985.36±0.40‾\underline{85.36±0.40}6.97±0.456.97±0.4580.20±0.02‾\underline{80.20±0.02}85.34±0.4385.34±0.436.89±0.45‾\underline{6.89±0.45}80.67±0.05\textbf{80.67±0.05}86.51±0.29\textbf{86.51±0.29}5.22±0.34\textbf{5.22±0.34}
IN-A (T=10)64.19±0.51‾\underline{64.19±0.51}72.45±1.05‾\underline{72.45±1.05}11.17±0.43‾\underline{11.17±0.43}63.81±0.3063.81±0.3072.23±1.2472.23±1.2412.18±0.2612.18±0.2665.02±0.19\textbf{65.02±0.19}72.91±0.99\textbf{72.91±0.99}8.46±0.49\textbf{8.46±0.49}
IN-A (T=20)59.82±0.20‾\underline{59.82±0.20}69.57±1.19‾\underline{69.57±1.19}14.38±0.43‾\underline{14.38±0.43}59.38±0.4659.38±0.4669.31±1.0669.31±1.0615.46±0.3815.46±0.3862.74±0.51\textbf{62.74±0.51}70.69±0.91\textbf{70.69±0.91}11.15±0.72\textbf{11.15±0.72}

PACR attains the best result in almost every case. EMA, by contrast, is worse than the β=0\beta=0 baseline on both ImageNet-A settings across all three metrics, because it averages the model's own previous weights, which have already drifted with learning and are therefore not a reliable target. Revisiting old knowledge at every update can also restrict the learning of new classes. With neither mechanism, the model at least remains free to learn.

We have added this comparison to Section 4.3, with the changes highlighted in blue.

[1] Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. NeurIPS'17.

Comments on the paper

1 comment

We cordially thank all reviewers for their thorough and constructive feedback. We are encouraged that reviewers found our motivation "clear" and performance "superior" (a5Kj), that CAMP "beats the strongest baseline (MACIL)" (mUCa), and that our experimental results are "consistently very good" (mDhn).

The main concerns relate to three points: (1) the clarity and precision of the theoretical presentation, (2) the more in-depth experiments for each module, and (3) the generality of the evaluation, including comparisons with stronger recent methods such as RanPAC and a self-supervised backbone. Importantly, these are not fundamental critiques of our method's design but rather requests for further clarification and additional analysis. We have addressed each point with new experiments and detailed explanations, and we believe all concerns are fully resolved.

We respectfully request a re-evaluation in light of our clarified methodology and the core contributions of our work, which remain both novel and well-supported.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.