- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work studies exemplar-free class-incremental learning (CIL) methods with strong backbones (ViT-B, DINOv2 and v3) where the classifier is a closed-form solution (SimpleCIL, SLDA, S-QDA, FeCAM, RanPACK) or adjusted on frozen features during CIL (MiN, EASE). The authors evaluated methods on six commonly used dataset, and two with a bigger shift (OpenEarthSensing, MedMNIST). The main claim is that strong feature extractors, i.e., Dino-v3 allows state-of-the-art solution with simple closed-form classifiers, and that the performance gap comes from forgetting rather than learning. Moreover, such composable solutions give exact federated merging, class deletation and order-invariant solutions.
Critical strengths
- The authors tackle an important problem of transferring results from strong feature extractors, but older ones (e.g. ViT), to up-to-date backbones like Dino-v3, where simple methods yield good results and the main problem is not necessarily "catastrophic forgetting."
- The extensive evaluation results with many interesting results in the Appendix.
Critical weaknesses
- "On DINOv3 they outperform MiN on most benchmarks" -- but looking at the table A5, where all the results are reported, seems like a fixed version of MiN with per-cell hyper-pramaeters selection is still the best. Unfortunately, not all the experiments are presented with this version. This leaves the reader of this work in confusion. See also, for example, Table A9, which should support the claim "forgetting, not learning" is a problem. However, here still only a single result is with MiN repaired + hp per-cell selection, and it's the best with the biggest forgetting... . It would be nice to present the complete results and recheck all the claims, make it clear for the reader.
- In connection to the first point, looking at the results, the work would be stronger if it shows the results for few other adaptation methods, e.g. SLCA, MOS, E2-LoRA, etc.
- Federated learning setting is very limited; see works that introduce training-free methods like Fed3R, FedNCM to check the common setting for the evaluation.
- Presentation - there are too many MiN variants that the reader needs to follow, which one is used in what experiment? This could be simplified, and just use the best version everywhere to not cause confusion.
Questions for authors
- How do published RanPAC (with first-session adaptation) perform on DINOv3-B/L/H, including OES and MedMNIST?
- It's stated in the limitation that 3 out of 5 backbones are Dino-v3. Have you experimented with other non-DINO SOTA backbones?
Reviewer q7TQ Rebuttal
We thank the reviewer for concrete, actionable requests. Changes are in purple in the revised PDF, which now reports full per-benchmark results for every method. Numbers below are eight-benchmark means.
Weaknesses
W1 and W4. MiN variants and claims. The "Repaired MiN" column was the per-cell best of six configurations chosen on the reported seeds, and MiN was scored by task rather than exact class. We now report one MiN everywhere: plain validation-tuned MiN, scored by exact class. It trails the best closed-form head by 0.8–1.1 pp on DINOv3-B/L, and collapses on some DINOv3-H seeds (65.8). The repaired MiN (L2-normalised features, γ = 1) appears only in the brittleness discussion (Table A13).
W2. More adaptation baselines. We added MOS (AAAI 2025), run with the official code under our protocol on all 40 cells. It trails the best closed-form head by 1.1–1.9 pp on every backbone. Freezing its adapters costs 1.4–2.6 pp, most of it on OES, so most of its accuracy comes from its classifier. SLCA and E2-LoRA are listed as a limitation.
W3. Federated protocol. We now follow Fed3R and FedNCM: 100 clients, Dirichlet label skew with α ∈ {0, 0.01, 0.1, 0.5, 1000}, heads merged in one round. In float64 the merged SLDA and S-QDA heads equal the centrally trained heads at every skew level (at most 0.006 pp apart). On CIFAR-100 at α = 0 with ViT-B/16, S-QDA reaches 89.5%, level with Fed3R-RF (89.4%) and above Fed3R (84.5%) and FedNCM (77.4%). RanPAC's statistics also merge exactly, but its ridge penalty has to be set without data. With our data-free rule the merged head stays within 1.0 pp of central on CIFAR-100 and 0.6 pp on MedMNIST and loses up to 3.6 pp on ImageNet-R, but it falls to chance on CUB-200 for α ≤ 0.5; we state this as a limitation.
Questions
Q1. Published RanPAC with first-session adaptation. We ran it on all five backbones and eight benchmarks. It is the best method at every DINO backbone (84.9–89.9) and on OES and MedMNIST at every backbone, with its gains over frozen RanPAC concentrated on shifted domains. It adapts once and is a closed-form head from task 2 on, so we read this as support for our argument and treat our frozen RanPAC as its no-adaptation ablation.
Q2. Non-DINO backbones. We have not experimented with other backbone families. We chose these backbones because their architectures are very similar, which lets us show the effect of changing pretraining quality as well as parameter count.