- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Feature Distribution Calibration (FDC) for class-incremental learners sharing an adapted encoder. Task Subspace Filtering, Residual Likelihood Calibration, and Prototype Affinity Calibration use stored feature statistics to calibrate task-head competition without additional gradient-based optimization.
Critical strengths
The routing problem is clearly motivated, and comparisons with and without FDC reuse the same checkpoints and test examples.
Critical weaknesses
- RLC’s Gaussian approximation needs stronger validation beyond standardization. Comparing final-encoder features with historical statistics also raises calibration concerns.
- TSF uses centered PCA but filters uncentered features with unchanged biases. Please justify this choice against mean-preserving residual attenuation.
- FDC is evaluated only at final checkpoints under a fixed class order. Please report intermediate-stage performance across multiple class orders.
- Improve figure readability.
Questions for authors
- Do the selective-configuration gains persist when components are chosen on independent validation data rather than the reported evaluation results?
- Can the authors quantify residual-distribution drift and its impact on routing accuracy for earlier tasks?
- Does FDC outperform direct prototype- or residual-based routing under matched checkpoints, statistics, and inference budgets, given its roughly 1.9× dual-view latency?
7 comments
Author Response to Reviewer D3jP (Part 2)
Q3: Does FDC outperform direct prototype- or residual-based routing under matched checkpoints, statistics, and inference budgets, given its roughly 1.9× dual-view latency?
A5: We first tried simple logit and temperature calibration controls under the same three-order protocol:
| Method | Mean final ACC ± SD (%) |
|---|---|
| Maximum logit | 70.541 ± 0.682 |
| Training-statistics logit z-score | 70.124 ± 0.826 |
| Maximum softmax, T=1 | 64.449 ± 0.253 |
| Maximum softmax, T=2 | 67.550 ± 0.154 |
| Maximum softmax, T=4 | 69.561 ± 0.227 |
| FDC ET | 75.739 |
FDC ET exceeds these simple controls by at least 5.198 pp on average.
We then directly compare with subspace residuals, likelihood-based scoring, and prototype similarity, using ViM-style scoring [1], ConjNorm [2], prototype classification [3], shared-covariance Mahalanobis [4], Mahalanobis++ [5], and FeCAM [6].
| ET scoring method | Standalone ACC (%) | With FDC (%) | Gain over standalone (pp) |
|---|---|---|---|
| ViM-style residual scoring [1] | 73.871 | 75.907 | +2.036 |
| ConjNorm [2] | 69.783 | 75.681 | +5.898 |
| Prototype similarity [3] | 76.547 | 77.032 | +0.485 |
| Shared-covariance Mahalanobis [4] | 76.370 | 77.038 | +0.668 |
| Mahalanobis++ [5] | 75.757 | 76.871 | +1.114 |
| FeCAM, PILOT-ViT scoring [6] | 75.597 | 76.767 | +1.170 |
| FDC alone | 75.739 | — | — |
These are scoring adaptations on the same Cumulative-LoRA checkpoints. For their fusion with FDC, we add the scoring signal to the final FDC class scores and take a global argmax, using a common coefficient of 1. Our point is that although some standalone geometric classifiers are already strong, adding FDC further improves them.
In other words, FDC provides task-conditioned calibration that is complementary to the evaluated post-hoc scoring adaptations. Its distinction lies in how statistical evidence is used: TSF filters head responses according to task-specific feature support, while RLC and PAC interpret residuals and prototype affinities relative to task-specific training statistics.
We further compared FDC with all six scoring adaptations on InfLoRA, InfLoRA-CA, EWC-LoRA, SD-LoRA, Split-LoRA, LoRA-Sub, RSIAT, and LoDA, across five datasets and three class orders (120 evaluations).Each method was evaluated independently using the same learner checkpoints and a single encoder view. The table reports gains over the original learner and the number of evaluations in which accuracy decreased:
| Method | Mean gain over original learner (pp) | Degraded evaluations |
|---|---|---|
| FDC ET | +3.060 | 12/120 |
| ViM-style ET | −2.150 | 56/120 |
| ConjNorm ET | −1.832 | 68/120 |
| Prototype ET | +2.705 | 32/120 |
| Shared Mahalanobis ET | +2.474 | 28/120 |
| Mahalanobis++ ET | +2.725 | 32/120 |
| FeCAM ET | +1.746 | 35/120 |
Our fusion results are consistent with FDC's complementary role. Across 120 matched learner–dataset–order evaluations, adding FDC improves the overall mean accuracy of all six evaluated scoring adaptations. For Prototype, Shared Mahalanobis, Mahalanobis++, and FeCAM scoring, fusion exceeds both the standalone scorer and standalone FDC, with gains of 0.606–0.802 percentage points over FDC. These results support empirical complementarity rather than statistical independence or universal improvement for every learner. Since the current fusion combines complete score functions, it does not by itself isolate task-level calibration from the contribution of the retained classifier.
We also agree that the extra view has a substantial cost. ET provides most of the improvement, while dual requires a second encoder view.
| Configuration | Mean final ACC (%) | Gain over baseline (pp) | Encoder views |
|---|---|---|---|
| Cumulative baseline | 70.541 | — | 1 |
| FDC ET | 75.739 | +5.198 | 1 |
| FDC dual | 76.817 | +6.276 | 2 |
ET retains 82.8% of the dual version's gain; dual adds 1.078 pp over ET.
For convenience, we summarize the inference-time measurements already reported in Appendix C.2, Table 7 of the submitted paper:
| Dataset | Baseline (ms/batch) | FDC ET (ms/batch) | FDC dual (ms/batch) |
|---|---|---|---|
| CIFAR-100 | 170.5 | 173.3 (+1.7%) | 323.1 (+89.5%) |
| CUB-200 | 171.3 | 174.1 (+1.6%) | 324.2 (+89.3%) |
| ImageNet-R | 170.5 | 171.7 (+0.7%) | 320.1 (+87.7%) |
| ImageNet-A | 171.0 | 172.5 (+0.8%) | 321.5 (+88.0%) |
| OmniBenchmark | 169.9 | 171.3 (+0.8%) | 319.2 (+87.9%) |
References
[1] ViM: Out-of-Distribution With Virtual-Logit Matching. CVPR 2022.
[2] ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection. ICLR 2024.
[3] iCaRL: Incremental Classifier and Representation Learning. CVPR 2017.
[4] A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. NeurIPS 2018.
[5] Mahalanobis++: Improving OOD Detection via Feature Normalization. ICML 2025.
[6] FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning*. NeurIPS 2023. (We use the pretrained-ViT scoring implementation in PILOT)
Author Response to Reviewer D3jP (Part 3)
Q2: Can the authors quantify residual-distribution drift and its impact on routing accuracy for earlier tasks?
A6: To quantify residual drift directly, we measured residuals on a fixed set of training samples from earlier tasks: the mean residual rises from 0.2505 at task arrival to 0.2943 under the final encoder with the old PCA, and falls to 0.2512 after rebuilding PCA.
We also isolated RLC refresh while keeping PAC and the other components fixed:
| RLC-only oracle refresh | ET ACC change (pp) | ET routing accuracy change (pp) |
|---|---|---|
| Distribution statistics only; original PCA and τ | +0.179 | +0.186 |
| PCA and distribution statistics; original τ | +0.451 | +0.474 |
| PCA, distribution statistics, and τ | +0.580 | +0.629 |
W4: Improve figure readability.
A7: We will improve figure readability.
Author Response to Reviewer D3jP (Part 1)
We sincerely thank Reviewer D3jP for the constructive comments and for recognizing the clear motivation of the routing problem and the controlled comparisons using the same checkpoints and test examples. We have directly tested the Gaussian approximation in RLC and the suggested mean-preserving variant of TSF. We have also added experiments using class orders 1993/1994/1995, focusing on the cost-efficient ET single-view version. We provide pointwise responses below.
A1: we agree that standardization alone does not establish Gaussianity. We therefore directly tested more flexible distributions, including skew-normal and Beta, with parameters fitted from training residuals.
These replacements bring little average improvement, even when the calibration scale is refitted. This supports the practical adequacy of the Gaussian approximation in these experiments. We address the separate concern about historical statistics through the residual-drift and routing analysis in Q2 below.
A2: We tested the mean-preserving variant of TSF, keeping the attenuation strength and calibration scales unchanged. This comparison examines whether preserving the task mean improves performance under the same attenuation and calibration settings.
Mean-preserving attenuation reduces the mean accuracy on all five datasets, by 0.414 pp overall and most strongly on CUB-200. These results support retaining the current TSF formulation in the evaluated settings. We conjecture that attenuating the task-dependent mean component also helps balance cross-task competition; this accuracy comparison alone does not establish the mechanism.
A3: We will use full-component FDC as the main result in the paper and provide leave-one-dataset-out component selection as a supplementary result. For selection, we choose the TSF/PAC/RLC configuration with the largest average gain on the other four datasets, then evaluate it on the held-out dataset. Each of the five datasets is held out in turn for every learner and class order, giving 8 learners × 5 held-out datasets × 3 orders = 120 evaluation pairs per view.
Held-out selection gives a slightly larger average gain, while the proportion of improved pairs remains similar.
A4: We now report cumulative AIA at every task, rather than only the endpoint. AIA is the average of the accuracies measured after each task learned so far, with each evaluation covering all classes seen at that stage. We divide each sequence into five equal intervals (four tasks per interval for 20-task sequences; two for 10-task sequences). Each entry averages the per-task cumulative AIA gains within that interval.
These results show that FDC's gains persist during incremental learning, and the interval-averaged AIA gains increase as more tasks are learned on all five datasets.
Feature drift does occur, and refreshing PAC/RLC statistics using historical training images under the current encoder further improves performance overall. Nevertheless, FDC remains effective with historical statistics despite this drift. Refreshing requires revisiting old training data; approximating it without retaining old data is a direction for future work.