acceptodds
Under review as a conference paper at ICLR 2027

Unlocking Frozen Potential: Dynamic Analytical Prototypes for Few-Shot Class-Incremental Learning

Abstract

Few-Shot Class Incremental Learning challenges models to continuously learn novel categories from very few samples while mitigating the stability-plasticity dilemma. Recent methods increasingly leverage the robust generalization of Pre-Trained Models (PTMs) and stable prototype-based classifiers. We identify a critical limitation: Rich Features but Constrained Potential. Although PTMs provide rich generic representations, their potential is constrained by the frozen backbone's insufficient adaptation to novel concepts, the statistical bias of prototypes derived from very few samples and the absence of global discriminative optimization in static prototypes. Consequently, static prototype-based methods often struggle with feature entanglement and unreliable prototypes. To unlock the frozen potential, we propose Dynamic Analytical Prototypes that refine the post-backbone discriminative space. The first module Prior-Guided Distribution Calibration effectively improves the discriminative capability of representations by leveraging the base-class priors. Second, to achieve a global optimal decision, we derive a Global Knowledge-Aware Solver. This recursive solution updates class-balanced weights and theoretically mitigates the risk of catastrophic forgetting. Extensive experiments demonstrate consistent improvements over representative feature- and classifier-level optimization methods, validating its balance between stability and plasticity.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Few-Shot Class Incremental Learning, Continual Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_HLy6OpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Dynamic Analytical Prototypes (DAP) for few-shot class-incremental learning with a backbone adapted on the base session and frozen thereafter. It combines base-statistics-guided calibration of novel-class distributions, synthetic feature sampling, a fixed Gaussian-RBF anchor mapping, and a class-balanced ridge classifier updated recursively.

Critical strengths
  1. The paper is well written and well structured.

  2. The shared-feature comparisons across three datasets and three backbone families provide useful evidence that the combined classifier improves on the evaluated alternatives.

  3. Incremental updates require no backbone optimization or historical raw images, and adding DAP improves both SEC-Prompt and CASP in the reported experiments.

Critical weaknesses
  1. The additional contribution over prior work needs clearer discussion. PGDC appears closely related to the calibration of Goswami et al. (2024)[1]. The kernel mapping and recursive updates also build on ideas in GKEAL[2]. The class-balanced objective warrants comparison with AIR (Fang et al., 2024)[3]. Although several related methods are cited and evaluated, the methodological differences remain unclear. The ablations also do not isolate the benefit of class weighting within the same solver. These gaps make it difficult to assess the extent of the advance.

  2. It is unclear whether the gains come from better retention, better learning of new classes, or both. The main comparisons report aggregate accuracy. The performance-drop metric compares different class sets and does not isolate forgetting. Table 12 examines calibration, but does not provide this comparison for the full method. Separate old- and new-class results across sessions would be helpful. Without this analysis, the claimed improvement in the stability–plasticity trade-off is difficult to assess.

[1] Dipam Goswami et al. Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformers. CVPR Workshops, 2024.

[2] Huiping Zhuang et al. GKEAL: Gaussian Kernel Embedded Analytic Learning for Few-Shot Class Incremental Task. CVPR, 2023.

[3] Di Fang et al. AIR: Analytic Imbalance Rectifier for Continual Learning. arXiv:2408.10349v1, 2024.

Questions for authors

Please see Critical Weaknesses 1 and 2. In particular:

  1. What is the methodological advance over [1]–[3], and what evidence isolates the benefit of class weighting?

  2. How does DAP compare with the strongest baselines on old- and new-class accuracy across sessions?

7 comments
AuthorsOpenReview ↗

Reviewer HLy6 | Q1 (Part 1/3): Comparison with prior methods and system mechanism

We greatly appreciate your expert feedback and your insightful concerns. Your concerns and questions are responded as follows:

Concern: The reviewer requests a clearer account of the methodological advance beyond prior statistical calibration, kernel recursion, and analytic class balancing, as well as a matched experiment that isolates the benefit of class weighting.

Response: We propose globally coordinated dynamic analytical prototypes (DAP) to address a central difficulty of few-shot class-incremental learning: as new classes appear, the competition among all observed categories changes, so their decision representations need to be coordinated again. DAP achieves this by placing retained historical class statistics and prior-calibrated novel-class statistics in a shared, class-normalized analytical objective. Consequently, each decision prototype wc(t)=At−1μcΨw_c^{(t)}=A_t^{-1}\mu_c^{\Psi} is shaped by the accumulated second-order evidence of all observed classes, and the shared operator is updated when the label set expands—without revisiting historical images. We distinguish this design from the works named by the reviewer and then measure the respective effects of calibration and weighting.

This coordination follows directly from the analytical solution. On fixed RBF features, wc(t)=At−1μcΨw_c^{(t)}=A_t^{-1}\mu_c^{\Psi} with At=∑jSjΨ+λIA_t=\sum_jS_j^{\Psi}+\lambda I. Here the statistical prototype μcΨ\mu_c^{\Psi} retains evidence about class cc, while the common inverse operator accounts for the second moments of all observed classes. Therefore, the arrival of a new class can change historical decision directions even when their stored class statistics remain unchanged.

Comparison with the closest methods. Goswami et al.'s C-RanPAC has already combined calibrated class statistics with analytical classification. GKEAL establishes Gaussian-RBF recursive learning with augmented-feature balancing, AIR studies inverse-class-frequency weighting for analytical classifiers, and ACIL provides the recursive least-squares foundation. DAP builds on these results by using prior-calibrated novel and retained historical statistics together in a fixed base-anchor RBF space, then optimizing their decision prototypes under a class-averaged objective. In other words, the individual mathematical tools are established; the question DAP tackles is how their joint use can keep old and new decisions coordinated as class competition evolves.

Why this combination matters. PGDC determines the first- and second-order evidence available for an incoming class. Because it uses synthetic features, however, the number of fitted rows can be much larger for a novel class than for a base class—approximately 505 versus 30 in the recorded CUB setting. GKAS therefore normalizes each class's total coefficient weight instead of letting row count determine its importance. The shared At−1A_t^{-1} then uses these class moments to update the decisions of all observed categories. The three steps address successive parts of the same problem: estimating scarce-class evidence, controlling its influence, and coordinating global decisions.

To see whether those choices are useful rather than merely conceptually connected, we use the same frozen feature cache and RBF Ridge family in the six-cell comparison (Table 1). We then vary the synthetic budget (Table 2) and transfer the mean and covariance separately (Table 3). Finally, the original multi-benchmark ablation (Table 4) checks the additional calibration effect when KM+GKAS is already present.

AuthorsOpenReview ↗

Reviewer HLy6 | Q1 (Part 2/3): Matched calibration and class-weighting controls

To quantify the statistical-construction and weighting choices discussed in Part 1, we report matched ablations below.

Table 1. Six-cell matched RBF analytical controls

Experimental protocol. CUB-200/ViT fixed cache, same RBF solver; A = real supports; B = support-only Gaussian; C = prior-calibrated Gaussian. B/C retain five supports and add 500 synthetic features per novel class. U = uniform row weights; W = inverse class-size weights

Novel-class fitted dataU: Unweighted AVG (%)W: Class-normalized AVG (%)
A · Five supports only82.201 ± 0.00083.534 ± 0.000
B · Support Gaussian +50082.183 ± 0.00284.300 ± 0.026
C · Prior-calibrated Gaussian +50085.006 ± 0.05685.895 ± 0.012

Result interpretation. Under an equal 500-feature synthesis budget and the same class-normalized solver, C-W achieves 85.895% AVG and B-W 84.300%, a +1.595 pp difference attributable to their statistical input. Holding the calibrated rows fixed and changing only the weighting rule, C-W exceeds C-U (85.006%) by +0.889 pp. These comparisons do not make either calibration or class weighting a new algebraic invention; instead, they show why both are useful in the proposed class-expansion setting. PGDC supplies informative moments, while the class-normalized solver determines how those moments influence the shared all-class decision operator.

Table 2. Synthetic-feature count at matched budget

Synthetic / classB-W AVG (%)C-W AVG (%)C-W − B-W (pp)
083.534 ± 0.00083.534 ± 0.0000.000
5084.223 ± 0.04785.521 ± 0.059+1.298
10084.258 ± 0.07985.706 ± 0.044+1.448
20084.289 ± 0.02985.810 ± 0.039+1.521
50084.300 ± 0.02685.895 ± 0.012+1.595

Result interpretation. The same conclusion holds when the amount of synthetic data changes. Both methods begin at 83.534% with no synthetic samples, yet C-W leads B-W by +1.298 to +1.595 pp across equal budgets of 50–500 draws per class. Since both receive the same number of rows, the advantage is linked to the calibrated distribution rather than the sheer volume of augmentation. This helps explain why the statistical construction in DAP matters before the global decision update: the shared solver can only coordinate the evidence it receives.

Table 3. Prior mean and covariance transfer

Same-budget variantAVG (%)
B-W · No base prior84.300 ± 0.026
Mean transfer only84.618 ± 0.026
Covariance transfer only84.576 ± 0.017
C-W · Full calibrated transfer85.895 ± 0.012

Result interpretation. The moment-transfer study shows what that evidence contains. Mean-only and covariance-only transfer reach 84.618% and 84.576%, respectively, compared with 84.300% without base priors; joint transfer raises AVG to 85.895%. Both the class center and its dispersion therefore contribute. In the analytical solution, these statistics play complementary roles: the mean supplies class-specific evidence, while second moments enter the operator shared by historical and novel decisions. This is how distribution calibration contributes to DAP's global prototype formulation, rather than remaining a standalone correction to a class center.

AuthorsOpenReview ↗

Reviewer HLy6 | Q1 (Part 3/3): Cross-benchmark calibration evidence

The original multi-benchmark ablation complements these matched CUB-200 controls by testing PGDC within the full analytical pipeline.

Table 4. Original Table 3: calibration added to KM + GKAS

Experimental protocol. The source manuscript's ViT-B-IN21K ablation on CIFAR-100, CUB-200 and ImageNet-R; AVG is mean session-wise overall accuracy

BenchmarkKM + GKAS AVG (%)PGDC + KM + GKAS AVG (%)Gain
CIFAR-10089.6190.61+1.00
CUB-20085.5385.89+0.36
ImageNet-R80.0880.75+0.67

Result interpretation. The original multi-benchmark results provide a broader check. Adding PGDC to KM+GKAS increases AVG by +1.00 pp on CIFAR-100, +0.36 pp on CUB-200, and +0.67 pp on ImageNet-R. These are incremental gains on top of an already effective analytical head, so their size should be read in that context. Together with the matched controls, they show that calibrated statistics provide a repeatable contribution across benchmarks and that the contribution can be isolated under a common representation. The resulting evidence supports the role of statistical calibration within DAP's global decision coordination.

Conclusion. Relative to the cited analytical and calibration methods, DAP combines retained historical and calibrated novel-class statistics in a fixed-anchor, class-normalized objective to re-coordinate all decision prototypes. The matched and multi-benchmark comparisons support the value of its calibration and weighting choices.

Reviewer_7ybqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Dynamic Analytical Prototypes (DAP) for few-shot class-incremental learning with a frozen feature extractor. DAP has three components: prior-guided distribution calibration, which adjusts novel-class statistics using similarities to base classes; an RBF feature map based on fixed base-session anchors; and a recursively updated, class-weighted ridge-regression classifier. The paper evaluates the method on CIFAR100, CUB200, and ImageNet-R with several backbones, and reports gains over a number of frozen-feature baselines, as well as improvements when DAP is combined with two existing FSCIL frameworks.

Critical strengths
  1. DAP has the best average accuracy in all nine dataset/backbone settings in Table 1, with shared frozen features within each backbone group.

  2. Table 3 tests all component combinations, and Table 2 shows a clear time and memory benefit from recursive updates.

Critical weaknesses
  1. The novelty seems relatively limited. The approach combines distribution calibration inspired by prior work, an RBF mapping, and recursive analytical classification. The related-work section discusses these connections, but it remains unclear which part of the method constitutes the principal technical contribution and how much of the gain comes from a new algorithmic insight versus combining known components. In addition, I see the comparison in Section 2.2, but what does DAP add beyond Goswami et al.'s calibration and GKEAL's RBF mapping and recursive classifier? Is the main new piece the class-balanced weighting, or is there another change I'm missing?

  2. Some theoretical claims appear stronger than the analysis supports. The discussion of high-dimensional kernel mappings and anchor-feature concentration does not, as presented, establish that the mapped class distributions are linearly separable or that the method guarantees stability for anchors sampled from the base-feature distribution.

  3. With the fixed mixing weights used in the experiments, Eq. (3) pulls every new-class mean toward base classes. What happens when none of those classes is a good match? Can DAP tell when it should skip calibration and keep the original estimate?

  4. Table 3 replaces GKAS with NCM, changing both the classifier and the class weighting at once. How much of the gain comes from class balancing? Could you add unweighted ridge regression with the same calibrated samples and RBF features to isolate that effect?

Questions for authors

Please refer to the Critical Weaknesses.

6 comments
AuthorsOpenReview ↗

Reviewer 7ybq | W1 (Part 1/2): Overall methodological contribution and objective

We greatly appreciate your expert feedback and your insightful concerns. Your concerns and questions are responded as follows:

Concern: The reviewer asks what DAP contributes beyond Goswami et al.'s calibration and GKEAL's RBF recursive classifier, and whether class balancing alone explains the improvement.

Response: We propose globally coordinated dynamic analytical prototypes (DAP) because every new class changes the competition faced by classes learned earlier. The challenge is therefore not simply to estimate another class center, but to keep the decisions of all observed classes coordinated as the label space grows. DAP combines retained historical statistics with prior-calibrated novel-class statistics in a fixed base-anchor RBF space, then updates all decision prototypes through a shared class-normalized analytical objective. The resulting wc(t)=At−1μcΨw_c^{(t)}=A_t^{-1}\mu_c^{\Psi} depends on all observed classes through At−1A_t^{-1}, while historical raw images need not be revisited. We next distinguish this design from existing methods and measure its statistical and weighting contributions.

Relation to prior work. C-RanPAC has already combined calibrated synthetic features with analytical classification, and GKEAL has established Gaussian-RBF recursive learning with augmented-feature balancing. DAP builds on both lines of work but places prior-calibrated novel moments and retained historical moments in one class-normalized RBF objective. This matters when classes arrive sequentially: changing the common operator updates the decision prototypes of old and new classes together, instead of leaving their global competition to separate classwise estimates. The following matched experiments identify how much calibration and weighting contribute to this formulation.

The objective makes this coordination explicit. For fixed RBF features zi=Ψ(xi)z_i=\Psi(x_i), assigning each row weight 1/Nc1/N_c gives

\sum_{i\in c}\\|z_i^\top W-e_c^\top\\|_2^2+\lambda\\|W\\|_F^2.}$$ Solving the normal equations yields the unique optimum $W_t=A_t^{-1}[\mu_c^{\Psi}]_c$, where $A_t=\sum_cS_c^{\Psi}+\lambda I\succ0$. The class mean $\mu_c^{\Psi}$ supplies class-specific evidence, but the shared $A_t^{-1}$ reflects the second moments of every observed class. This explains why historical decision prototypes change after a new class arrives even when their statistical means stay fixed: it is the **joint discrimination**, not the historical evidence itself, that is being updated. PGDC and GKAS play different but connected roles in this process. PGDC first uses five real supports and base priors to construct a calibrated distribution, from which synthetic features are drawn to estimate the class moments. Yet generating more rows should not automatically give that class more influence in the global objective. GKAS addresses this by making each class's total data-loss coefficient weight one. The resulting moments then enter $A_t^{-1}$, which coordinates all class decisions. In this sense, **statistical estimation and global discrimination are linked without equating sample quantity with class importance**. **Evidence.** To test these two roles separately, we use the same frozen ViT feature cache and RBF Ridge family in the six-cell study (**Table 1**). At equal synthesis budget, the prior-calibrated C-W variant exceeds B-W by **+1.595 pp**; with the calibrated rows held fixed, class-normalized C-W exceeds C-U by **+0.889 pp**. The budget sweep and moment-transfer ablations in **Tables 2–3** further examine where the calibration benefit comes from. Because the comparisons hold the representation or fitted inputs constant, they probe the statistical and optimization choices without changing the classifier family.
AuthorsOpenReview ↗

Reviewer 7ybq | W1 (Part 2/2): Matched calibration and class-weighting controls

The following experiments quantify calibration and class weighting under the same frozen CUB-200/ViT representation.

Table 1. Six-cell matched RBF analytical controls

Experimental protocol. CUB-200/ViT fixed cache, same RBF solver; A = real supports; B = support-only Gaussian; C = prior-calibrated Gaussian. B/C retain five supports and add 500 synthetic features per novel class. U = uniform row weights; W = inverse class-size weights

Novel-class fitted dataU: Unweighted AVG (%)W: Class-normalized AVG (%)
A · Five supports only82.201 ± 0.00083.534 ± 0.000
B · Support Gaussian +50082.183 ± 0.00284.300 ± 0.026
C · Prior-calibrated Gaussian +50085.006 ± 0.05685.895 ± 0.012

Result interpretation. The two comparisons answer different parts of the novelty concern. With 500 synthetic rows in both variants, C-W reaches 85.895% and B-W 84.300% (+1.595 pp), showing that using calibrated class statistics matters even when the solver is already class-normalized. On the same calibrated rows, C-W improves over C-U from 85.006% to 85.895% (+0.889 pp), so the gain is not explained by calibration alone. Even without weighting, C-U exceeds B-U (82.183%) by +2.823 pp at the same budget. These results place PGDC and GKAS in a common decision-prototype framework while showing that each changes a different part of the learning process.

Table 2. Synthetic-feature count at matched budget

Synthetic / classB-W AVG (%)C-W AVG (%)C-W − B-W (pp)
083.534 ± 0.00083.534 ± 0.0000.000
5084.223 ± 0.04785.521 ± 0.059+1.298
10084.258 ± 0.07985.706 ± 0.044+1.448
20084.289 ± 0.02985.810 ± 0.039+1.521
50084.300 ± 0.02685.895 ± 0.012+1.595

Result interpretation. The budget sweep provides a second check on calibration. Without synthetic draws, B-W and C-W are identical at 83.534%. Once draws are added, C-W stays ahead by +1.298 to +1.595 pp for matched budgets from 50 to 500 per class. Since the number of fitted rows is the same, the advantage points to the information carried by the calibrated surrogate. That information is then summarized in class moments and enters the shared operator, rather than acting only as a separate data-augmentation step.

Table 3. Prior mean and covariance transfer

Same-budget variantAVG (%)
B-W · No base prior84.300 ± 0.026
Mean transfer only84.618 ± 0.026
Covariance transfer only84.576 ± 0.017
C-W · Full calibrated transfer85.895 ± 0.012

Result interpretation. We next ask which statistical quantities make the difference. Relative to B-W at 84.300%, mean-only transfer reaches 84.618% and covariance-only transfer 84.576%; combining both yields 85.895%. The joint improvement suggests that a novel class benefits from more than a corrected center: its spread matters as well. Within DAP, this is useful because the estimated mean enters the class's decision prototype, while second moments affect the shared global metric. C-RanPAC also combines calibration with analytical classification; DAP's distinct choices are the fixed-anchor RBF representation and the class-normalized objective through which those moments are used.

Conclusion. The controls show benefits from both prior-calibrated statistics and class-normalized fitting, rather than from class weighting alone. DAP uses those two choices to continually re-coordinate historical and novel decision prototypes as class competition changes.

AuthorsOpenReview ↗

Reviewer 7ybq | W2 (Part 1/2): Kernel concentration, separability, and score stability

Concern: The reviewer questions whether high-dimensional RBF mapping and concentration alone establish strict class separation or stable decisions for base-distributed anchors.

Response: The useful property of the RBF anchors is that their induced similarities approach a reference geometry at rate O(K−1/2)O(K^{-1/2}). This gives a stable basis for representing decisions, but does not by itself make the classes linearly separable. We show that finite-set separability follows when the reference Gram matrix has sufficient diagonal-dominance margin, and that a reference prediction is preserved when its score margin exceeds the kernel-induced perturbation. These are the additional conditions requested in the review. The CUB-200 anchor study below then examines whether the sampling variability decreases as the number of anchors grows.

Kernel concentration and finite-set separability. Consider a fixed set of nn inputs, including the training examples and queries under analysis. Let α1,…,αK\alpha_1,\ldots,\alpha_K be independent base-distributed anchors and r(x)=Ψ(x)/Kr(x)=\Psi(x)/\sqrt K the normalized RBF feature. Stack the features into R=[r(xi)⊤]i=1nR=[r(x_i)^\top]_{i=1}^n. We first compare the finite-anchor similarities with their expectation under the anchor distribution:

k_P(x_i,x_j)=\mathbb E_{\alpha}[\kappa(x_i,\alpha)\kappa(x_j,\alpha)].$$ Since each RBF product lies in $[0,1]$, Hoeffding's inequality applies to every pair. A union bound over the $n^2$ pairs then gives, with probability at least $1-\delta$, $$\boxed{\max_{i,j}\left|\widehat k_K(x_i,x_j)-k_P(x_i,x_j)\right|\le \sqrt{\frac{\log(2n^2/\delta)}{2K}}=:\epsilon_K.}$$ Concentration controls the entrywise approximation error. To infer separability, we additionally require every reference row to satisfy $k_P(x_i,x_i)-\sum_{j\ne i}|k_P(x_i,x_j)| > n\epsilon_K$. Under this condition, the finite Gram matrix $RR^\top=[\widehat k_K(x_i,x_j)]_{ij}$ is strictly diagonally dominant and positive definite. Hence $R$ has full row rank (with $n\le K$), and for any finite-set labels $L$, the coefficients $B_{\mathrm{int}}=R^\top(RR^\top)^{-1}L$ satisfy $RB_{\mathrm{int}}=L$. This establishes **finite-set linear separability under the explicit reference margin**, rather than deriving it from Cauchy–Schwarz alone. **Prediction stability.** Finite-set interpolation does not yet characterize the regularized classifier used by DAP. For that purpose, let $\Omega=\operatorname{diag}(\omega_i) > 0$, $\bar\lambda=\lambda/K$, $a=\bar\lambda/\omega_{\max}$, and $Y$ be the training label matrix. The finite-anchor and reference dual Ridge scores are $$s_K(x)=k_K(x,X)^\top(K_K+\bar\lambda\Omega^{-1})^{-1}Y,\qquad s_P(x)=k_P(x,X)^\top(K_P+\bar\lambda\Omega^{-1})^{-1}Y.$$ Here $K_K$ and $K_P$ are the training Gram matrices, while $k_K(x,X)$ and $k_P(x,X)$ compare the query with the training inputs. The same concentration event bounds their differences, allowing us to propagate the kernel approximation error to the classifier scores. Indeed, the concentration event gives $\\|K_K-K_P\\|_2\le n\epsilon_K$ and $\\|k_K(x,X)-k_P(x,X)\\|_2\le\sqrt n\\,\epsilon_K$, while both regularized inverses have norm at most $1/a$. Applying the resolvent identity therefore yields $$\boxed{\\|s_K(x)-s_P(x)\\|_2\le\sqrt n\\,\epsilon_K\\|Y\\|_2 \left(\frac1a+\frac{n}{a^2}\right)=:\eta_K.}$$
Reviewer_qqraOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Dynamic Analytical Prototypes (DAP), a framework for Few-Shot Class-Incremental Learning (FSCIL) that aims to improve the discriminative capability of frozen pretrained representations. The authors identify two limitations of conventional prototype-based methods: unreliable class statistics estimated from scarce novel-class samples and the lack of global discriminative optimization in static prototypes. To address these issues, DAP introduces two main components. First, Prior-Guided Distribution Calibration (PGDC) leverages base-class statistics to calibrate novel-class feature distributions, generates synthetic features, and employs Gaussian-RBF kernel mapping to construct a shared discriminative space. Second, the Global Knowledge-Aware Solver (GKAS) performs class-balanced analytical optimization and recursively updates classifier weights without revisiting historical raw data. The authors evaluate DAP on CIFAR-100, CUB-200, and ImageNet-R using multiple pretrained backbones, including ResNet-18, ViT-B-IN21K, and CLIP-ViT-B. Experimental results demonstrate competitive improvements over representative prototype-based and analytical learning methods. Additional ablation studies and efficiency comparisons support the effectiveness of the proposed components and the computational advantages of recursive updates.

Critical strengths

The paper provides reasonably comprehensive experiments on CIFAR-100, CUB-200, and ImageNet-R using ResNet-18, ViT-B-IN21K, and CLIP-ViT-B backbones. DAP consistently achieves competitive or superior average accuracy compared with representative classifier-level baselines. For example, it improves CUB-200 average accuracy from 84.37% to 85.89% under ViT-B-IN21K and CIFAR-100 average accuracy from 70.28% to 72.92% under ResNet-18. The additional improvements obtained when integrating DAP with SEC-Prompt and CASP further support its practical compatibility with existing FSCIL frameworks. These results constitute meaningful empirical evidence for the proposed approach.The recursive analytical solver avoids revisiting historical raw images during incremental classifier updates. Table 2 reports identical average accuracy for recursive and batch solutions while reducing update time from 530.21 ms to 92.63 ms and memory consumption from 1229.77 MB to 207.61 MB in the reported comparison. Furthermore, the component ablations across three benchmarks reveal useful interactions between distribution calibration, kernel mapping, and analytical optimization. These findings strengthen the practical motivation for the framework.

Critical weaknesses

My main concern is the theoretical analysis. In Proposition 3.2 and Appendix B.3, the authors use the Cauchy–Schwarz inequality to establish a bound on the correlation between kernel representations and subsequently claim strict linear separability and diagonal dominance of the Gram matrix. However, the stated correlation inequality does not imply class-wise linear separability or diagonal dominance. Similarly, the concentration argument only establishes concentration around an input-dependent expected energy and does not guarantee that all samples lie on a common hyperspherical shell. More importantly, the paper repeatedly claims that the recursive analytical solver theoretically eliminates catastrophic forgetting. The equivalence between recursive weighted ridge regression and batch optimization establishes the optimality of the solution for the specified least-squares objective, but it does not guarantee preservation of previous-class predictions or accuracy when new classes are introduced. The whitening interpretation in Proposition 3.4 also does not establish that the transformed class prototypes have identity covariance. These unsupported theoretical conclusions directly affect the claimed technical contribution and substantially weaken the soundness of the paper. The proposed framework combines several mechanisms closely related to existing approaches: base-class statistical transfer for distribution calibration, Gaussian-RBF feature mapping, and recursive analytical ridge regression. These ideas have been investigated in related methods such as C-FeCAM, GKEAL, ACIL, and RanPAC. Although their integration is practically meaningful, the paper does not sufficiently distinguish its core algorithmic novelty from these established techniques. Moreover, Table 3 shows that adding distribution calibration to the KM+GKAS combination improves average accuracy by only 0.36 percentage points on CUB-200 and 0.67 points on ImageNet-R. Some improvements over the strongest baselines are also relatively modest, while the main comparison tables do not report standard deviations or confidence intervals. Stronger matched comparisons against conventional class-balanced kernel ridge regression and controlled synthetic-feature augmentation are needed to establish whether the proposed design provides a substantial improvement beyond existing analytical learning methods.

Questions for authors

Q1. Validity of the theoretical guarantees. Could the authors clarify how Proposition 3.2 establishes strict linear separability from the Cauchy–Schwarz correlation inequality? In particular, what additional assumptions guarantee diagonal dominance of the Gram matrix? Furthermore, how does the equivalence between recursive and batch ridge regression imply the claimed elimination of catastrophic forgetting, given that the classifier weights and competition among classes can change after each incremental session? A corrected theoretical analysis, together with explicit old-class retention measurements, would substantially influence my assessment of the paper's soundness.

Q2. Novelty and controlled comparisons. Could the authors more precisely distinguish the proposed GKAS from conventional recursive weighted ridge regression and existing analytical methods such as ACIL, GKEAL, and RanPAC? In particular, it would be helpful to compare DAP against a class-balanced Gaussian-RBF analytical classifier under identical frozen representations, and to isolate the effect of prior-guided calibration from that of increasing the number of synthetic training features. Such comparisons would clarify whether the performance gains arise from the proposed methodological contributions or from combining established techniques.

Q3. Statistical reliability and reproducibility. The paper states that its main results are averaged over three random seeds, whereas Appendix D describes a fixed random seed of 1993. Could the authors clarify the experimental protocol and report the per-seed results or standard deviations for the main comparisons? Additionally, could they specify the synthetic sample count (N_{\mathrm{syn}}), the ridge regularization coefficient (\lambda), and the memory required to retain kernel anchors and base-class statistics? These details are important for assessing the robustness and practical efficiency of the method.

9 comments
AuthorsOpenReview ↗

Reviewer qqra | Q1 (Part 1/4): Kernel concentration and finite-set separability

We greatly appreciate your expert feedback and your insightful concerns. Your concerns and questions are responded as follows:

Concern: Proposition 3.2 should justify its kernel-separability and Gram-matrix claims; Proposition 3.3 should distinguish recursive–batch equivalence from actual forgetting; and Proposition 3.4 should clarify the whitening interpretation. Explicit historical-class retention results are requested.

Response: We will clarify the three theoretical claims in the order raised by the reviewer. For Proposition 3.2, the Cauchy–Schwarz inequality alone does not establish class separability. We instead derive an O(K−1/2)O(K^{-1/2}) concentration bound for finite-anchor similarities and show that a sufficient reference Gram-row margin implies diagonal dominance and finite-set separability; a separate score-margin condition ensures prediction stability. For Proposition 3.3, recursive GKAS attains the same weighted-ridge optimum as batch recomputation, while explicit score conditions describe when old predictions survive new-class competition. This distinction is supported empirically: accuracy on the original base cohort drops by only about 0.34 pp after ten increments. For Proposition 3.4, the shared inverse operator defines a regularized global second-moment metric with spectrum σi/(σi+λ)\sigma_i/(\sigma_i+\lambda), rather than an identity covariance of class centers. These three results respectively clarify the representation geometry, the evolution of decisions, and their common discriminative metric.

(1) Proposition 3.2: kernel separability and anchor geometry. The question is whether a bound on correlations is sufficient to separate classes. It is not: such a bound does not by itself ensure that the Gram matrix is diagonally dominant. We therefore analyze the finite-anchor Gram matrix and identify the additional reference-geometry condition under which the desired result follows. Specifically, its entries concentrate around a reference matrix at rate O(K−1/2)O(K^{-1/2}); if the reference rows have sufficient dominance margins, the finite Gram matrix is positive definite and the mapped finite set is linearly separable. Prediction stability of the regularized classifier then follows under a further score-margin condition. To establish these statements, take a fixed set {x1,…,xn}\{x_1,\ldots,x_n\} covering the training inputs and the queries under consideration, and independent base-distributed anchors α1,…,αK\alpha_1,\ldots,\alpha_K. For Gaussian-RBF similarity 0<κ≤10<\kappa\le1, define

k_P(x_i,x_j)=\mathbb E_{\alpha}[\kappa(x_i,\alpha)\kappa(x_j,\alpha)].$$ Each product in the empirical kernel lies in $[0,1]$. Applying Hoeffding's inequality to one pair, then taking a union bound over all $n^2$ pairs, gives the following uniform bound with probability at least $1-\delta$: $$\boxed{\max_{i,j}|\widehat k_K(x_i,x_j)-k_P(x_i,x_j)|\le \sqrt{\frac{\log(2n^2/\delta)}{2K}}=: \epsilon_K.}$$ Hence, increasing $K$ reduces random fluctuations around the reference geometry. However, concentration alone does not make different samples separable. For that stronger conclusion, the reference Gram matrix must already possess enough diagonal-dominance margin to absorb the finite-anchor error. A sufficient condition for every row is $$\boxed{k_P(x_i,x_i)-\sum_{j\ne i}|k_P(x_i,x_j)|>n\epsilon_K\quad (\forall i).}$$ To see what this condition implies, let $R=[r(x_i)^\top]_{i=1}^n$ contain the normalized features $r(x_i)=\Psi(x_i)/\sqrt K$. Under the concentration event and the row-margin condition above, $RR^\top$ is symmetric, strictly diagonally dominant, and therefore positive definite. It follows that $R$ has full row rank (so this sufficient condition concerns a finite set with $n\le K$). Indeed, for any label matrix $L$, the interpolation coefficients $B_{\mathrm{int}}=R^\top(RR^\top)^{-1}L$ satisfy $RB_{\mathrm{int}}=L$. This proves finite-set linear separability **under the stated assumptions**, rather than from Cauchy–Schwarz alone. Separately, GKAS has a unique weighted-ridge optimum because, for positive weights and $\lambda>0$, $v^\top(\Phi^\top\Omega\Phi+\lambda I)v=\|\Omega^{1/2}\Phi v\|_2^2+\lambda\|v\|_2^2>0$ whenever $v\ne0$.
AuthorsOpenReview ↗

Reviewer qqra | Q1 (Part 2/4): Ridge-score stability and anchor evidence

Following the finite-set Gram-matrix conditions in Part 1, we examine the stability of the regularized classifier and the anchor experiment.

Full row rank concerns the possibility of separating a finite set; the regularized classifier still needs to preserve its own prediction scores. We therefore propagate the same kernel perturbation through the ridge solution. For normalized features r(x)=Ψ(x)/Kr(x)=\Psi(x)/\sqrt K, regularization λˉ=λ/K\bar\lambda=\lambda/K, sample weights Ω=diag⁡(ωi)\Omega=\operatorname{diag}(\omega_i), and a=λˉ/ωmax⁡a=\bar\lambda/\omega_{\max}, the finite-anchor and reference scores are

\qquad s_P(x)=k_P(x,X)^\top(K_P+\bar\lambda\Omega^{-1})^{-1}Y.$$ Because both regularized inverse matrices are bounded in norm by $1/a$, the resolvent identity converts the kernel-entry error into a bound on the score difference: $$\boxed{\|s_K(x)-s_P(x)\|_2\le \sqrt n\,\epsilon_K\|Y\|_2\left(\frac1a+\frac{n}{a^2}\right)=:\eta_K.}$$ A reference prediction is therefore retained whenever its correct-class score margin exceeds $2\eta_K$. This establishes the link requested by the reviewer: anchor concentration controls geometric fluctuations, while an explicit margin controls their effect on classification. To examine the first part empirically, we vary the number of anchors on the same 400 CUB-200 examples in **Table 1**. The 500- and 1,000-anchor subsets are drawn without replacement from a finite pool; at 3,000 anchors the entire pool is used, whereas the proposition above analyzes independent reference draws. **Table 1. Anchor-reference geometry, CUB-200** *Experimental protocol.* 400 fixed test examples; three subset draws for $K=500,1000$; full base-anchor pool at $K=3000$ | Anchors $K$ | Examples | Across-draw self-energy SD | Between-example energy SD | Draw regime | | ---: | ---: | ---: | ---: | :--- | | 500 | 400 | 0.001025 | 0.010621 | Three subset draws | | 1000 | 400 | 0.000642 | 0.010534 | Three subset draws | | 3000 | 400 | 0.000000 | 0.010518 | Full pool | **Result interpretation.** Moving from **500 to 1,000 anchors** reduces the across-draw SD of normalized self-energy from **0.001025 to 0.000642**, a **37.4% reduction**. When all **3,000** anchors are used, the SD becomes **zero** because changing their order leaves the induced kernel geometry unchanged. The between-example energy SD nevertheless stays near **0.0105**, as expected when reference self-energy depends on the input. Thus the experiment supports the concentration-based account of anchor consistency without assuming that all examples share one energy level. For DAP, this consistency matters because historical and new decision prototypes must be updated in the same reference geometry as the class set grows.
AuthorsOpenReview ↗

Reviewer qqra | Q1 (Part 3/4): Recursive analytical updates and retention conditions

We next address Proposition 3.3: how the recursive update changes historical decision prototypes and under what conditions old predictions are retained.

(2) Proposition 3.3: recursive equivalence and historical predictions. The next concern is whether solving the accumulated objective recursively also preserves historical predictions. We establish the two statements separately. First, GKAS exactly recovers the weighted-ridge batch optimum and updates the historical decision prototypes without revisiting past images. Second, a previously correct prediction is retained when the old-class score changes and the competing new-class scores satisfy explicit margin conditions. The fixed-base experiment then measures how much historical accuracy is preserved in practice. To derive the update, let Gt=∑cScΨG_t=\sum_cS_c^{\Psi} and At=Gt+λIA_t=G_t+\lambda I. Incoming data contribute Ht+1=Φt+1⊤Ωt+1Φt+1⪰0H_{t+1}=\Phi_{t+1}^{\top}\Omega_{t+1}\Phi_{t+1}\succeq0 and Mt+1=Φt+1⊤Ωt+1Yt+1M_{t+1}=\Phi_{t+1}^{\top}\Omega_{t+1}Y_{t+1}. Keeping previous fitted rows and weights unchanged, we obtain

A_{t+1}=A_t+H_{t+1}.}$$ The identity follows by writing the historical cross-statistics as $A_tW_t=(A_{t+1}-H_{t+1})W_t$ and substituting them into the new batch normal equation. Woodbury then updates the inverse from the previous state and the incoming session alone. In exact arithmetic, the resulting classifier is identical to batch weighted ridge, although no historical raw images are needed: $$A_{t+1}^{-1}=A_t^{-1}-A_t^{-1}\Phi_{t+1}^{\top} \bigl(\Omega_{t+1}^{-1}+\Phi_{t+1}A_t^{-1}\Phi_{t+1}^{\top}\bigr)^{-1} \Phi_{t+1}A_t^{-1}.$$ The important consequence is that **historical statistical prototypes do not have to move for historical decisions to adapt**. Once the new session modifies the shared second-moment operator, each old decision prototype changes according to $$\boxed{\Delta w_c=w_c^{(t+1)}-w_c^{(t)}=-A_{t+1}^{-1}H_{t+1}w_c^{(t)},\qquad \\|\Delta w_c\\|_2\le\frac{\\|H_{t+1}\\|_2}{\lambda}\\|w_c^{(t)}\\|_2.}$$ The frozen representation and retained class statistics therefore provide continuity across sessions, while the shared inverse operator adjusts the decision boundaries to the expanded label set. That explains how the same analytical update accommodates plasticity without discarding historical evidence. It does not, however, make the old labels immune to competition. We address that remaining question through the prediction margin. For an old-class example $x$ with true label $c$, compare its previous score against every other historical label, and denote the margin by $m_t(x)=s_c^{(t)}(x)-\max_{j\in\mathcal C_t\setminus\\{c\\}}s_j^{(t)}(x)$. The analytical update yields the following upper bound on the changes of old-class scores: $$\eta_t(x)=\frac{\\|\Psi(x)\\|_2\\|H_{t+1}\\|_2}{\lambda} \max_{j\in\mathcal C_t}\\|w_j^{(t)}\\|_2.$$ The old prediction remains correct at session $t+1$ if $m_t(x) > 2\eta_t(x)$ and $s_c^{(t)}(x)-\eta_t(x) > \max_{j\in\mathcal C_{t+1}\setminus\mathcal C_t}s_j^{(t+1)}(x)$. The first inequality preserves the ranking among old labels; the second ensures that no incoming label overtakes the correct class. We then examine these effects on a fixed group of base examples rather than inferring retention from the solver identity alone.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.