acceptodds
Under review as a conference paper at ICLR 2027

Robust Few-Shot Class-Incremental Learning via Adversarial Two-Player Game Optimization

Abstract

Few-Shot Class-Incremental Learning (FSCIL) demands that a model continuously acquire new categories from scarce labeled data while retaining prior knowledge. Recent LoRA-based methods dynamically construct lightweight adaptation modules upon Pre-Trained Models (PTMs), achieving impressive performance with minimal additional parameters. However, most existing approaches largely neglect adversarial robustness, leaving the adapted models vulnerable to adversarial threats in real-world deployment scenarios. In this work, we simultaneously addresses catastrophic forgetting, data scarcity, and adversarial fragility in FSCIL by introducing a new LoRA-based optimization framework that dynamically creates a dedicated LoRA expert per session, controlled by learnable layer-wise gates that adaptively balance base and incremental representations. To robustify the model under extreme data scarcity, we introduce a two-player adversarial game between a co-evolving attacker expert and a defender expert. The attacker continuously sharpens its adversarial generation capability via transferable perturbations, while the defender improves robustness by resisting both transferred and self-generated adversarial examples. We further derive that this minimax formulation admits a Nash Equilibrium and converges at a linear rate under mild smoothness conditions. To prevent representational collapse under few-shot constraints, we impose a margin-based Centered Kernel Alignment (CKA) regularization that explicitly penalizes redundancy between the base and incremental experts, steering each new expert toward a complementary feature subspace and sustaining meaningful adversarial co-evolution.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Few-Shot Class-Incremental Learning (FSCIL), Adversarial Robustness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ecV2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses adversarial robustness in Few-Shot Class-Incremental Learning under severe data scarcity. It proposes FSCIL-AT, combining session-wise LoRA experts, adversarial two-player game optimization, and CKA regularization. Experiments on four benchmarks evaluate clean and robust accuracy under multiple attacks, with ablations and few-shot robustness analyses.

Critical strengths

The paper investigates an important and relatively underexplored problem by introducing adversarial robustness into the FSCIL setting.

Critical weaknesses

The experimental evidence is insufficient to fully support the paper’s main claims. The paper claims to address catastrophic forgetting, data scarcity, and adversarial fragility, which are major challenges in FSCIL and adversarial robustness. These broad claims require more extensive and direct experimental evidence.

Questions for authors
  1. The overall story, method, and experiments are not yet fully aligned. The paper spans FSCIL and adversarial robustness and claims to address catastrophic forgetting, data scarcity, and adversarial fragility. However, the current experimental results mainly focus on adversarial robustness and do not sufficiently support all of these claims. The authors should reconsider the core problem and main focus of the method, and ensure that the motivation, method design, and experiments form a clear and consistent logical chain.
  2. The experimental evidence is unbalanced with respect to the three challenges claimed in the abstract. While adversarial fragility and data scarcity are explicitly studied, the claim of mitigating catastrophic forgetting is only indirectly supported by session-averaged clean accuracy and performance-drop curves. More direct continual-learning metrics, such as old/new-class accuracy would be necessary to substantiate this claim.
  3. Since the central motivation is robustness under extreme data scarcity, the robustness analysis should be more systematic. Fig. 1 varies the number of shots on CUB-200, which is useful, but this evidence is limited to one benchmark and does not examine other important FSCIL factors, such as the number of incremental classes, session length, or class-growth rate. A broader robustness analysis across different scarcity would better support the claimed generality of the method.
  4. The theoretical analysis appears to be inconsistent with the actual optimization of the proposed method. In the theoretical analysis, the attacker and defender are formulated using a shared minimax objective, while in the actual training process, the defender is optimized with a joint loss containing the clean loss, transferred adversarial loss, self-generated adversarial loss, and CKA regularization. Therefore, the optimization analyzed in the theory is not fully identical to the implemented training objective. In addition, the strong convex-concavity assumption required for the claimed linear convergence is not verified for the deep LoRA model with adversarial inner optimization. The authors should reconsider whether the current theoretical analysis can rigorously support the proposed method.
  5. The problem positioning appears broader than the actual technical contribution. The abstract frames the work as simultaneously addressing catastrophic forgetting, data scarcity, and adversarial fragility, whereas the central technical novelty and most experimental evidence focus on adversarial robustness with data scarcity. The FSCIL mechanisms for retaining previous knowledge are comparatively conventional and are not independently validated to the same extent. The paper would benefit from a more focused problem statement clarifying whether its primary goal is robust FSCIL or, more specifically, robust adaptation under extreme few-shot supervision.
  6. Although the paper provides module-level ablations for LWG, ATPGO, and MBCKAR, the contribution of the individual terms within ATPGO remains insufficiently isolated. In particular, the defender combines clean, transferred-adversarial, self-adversarial, and CKA losses. Controlled comparisons such as self-only, transfer-only, self+transfer, and static-versus-co-evolving attacker variants would better establish whether the proposed co-evolution mechanism itself, rather than simply additional adversarial supervision, is responsible for the improvement.
  7. The continual robustness and scalability of the method are not sufficiently evaluated. The paper mainly reports robust performance after the final session, while it remains unclear how the robustness of old and new classes evolves as more sessions are introduced. Moreover, since a new LoRA expert is created for each session and ATPGO and MBCKAR are repeatedly performed, both computational and storage costs may grow with the number of sessions. The authors should evaluate the robustness evolution across sessions and analyze the training, inference, and storage costs under longer continual-learning sequences.
  8. The main paper is not sufficiently self-contained. Key experiments and experimental settings supporting the claims on catastrophic forgetting, data scarcity, and adversarial fragility are not fully presented in the main text, with important details deferred to the appendix. The authors should include the essential evidence and settings in the main paper, since reviewers are not required to read the supplementary material.
9 comments
AuthorsOpenReview ↗

Response 1/3 to Reviewer ecV2

We sincerely thank the reviewer for the constructive feedback. We have strengthened the experimental evidence and refined the relevant content accordingly, as detailed below.

Q1: Alignment of Motivation, Method, and Experimental Evidence

A1: We thank the reviewer for the valuable suggestion to strengthen the paper’s overall logical consistency. We will adjust the problem statement in the abstract and introduction to emphasize robust adaptation under extreme few-shot class-incremental supervision in the revised version. ATPGO addresses adversarial fragility under limited samples and sparse per-session supervision, while LWG and MBCKAR support knowledge retention and feature complementarity. Building on the reviewer’s subsequent comments, we further provide comprehensive experimental evidence supporting our method, including old/new-class accuracy, forgetting rate, and multifactor analyses covering shots, base-class size, session length, and class-growth rate. These experiments and analyses are also included in the Appendix to strengthen the paper’s overall coherence.

Q2: Direct Evaluation of Continual Forgetting

A2: We thank the reviewer for this valuable comment. Tab. R1 report per-session old / new-class accuracy and forgetting rate. old / new accuracy denotes mean class accuracy over previously learned / current classes, while forgetting rate measures the relative drop from each old class’s historical best accuracy.

Tab. R1: Old/new-class accuracy (%) and forgetting rate (%) on CUB-200.

MethodMetricS0S1S2S3S4S5S6S7S8S9S10Average
FSCIL-AdvOld--87.8687.1886.1585.7784.5784.2483.6882.8682.4882.0684.69
New88.3689.2375.1873.5991.3866.8384.4184.3575.5587.1592.3982.01
FR--0.140.702.101.943.733.483.764.664.504.462.95
CL-LoRAOld--83.9282.4280.9979.0677.4476.0874.4472.9671.7070.2476.92
New87.3742.6961.1829.7963.5032.4855.8672.8484.5474.6190.8260.83
FR--2.742.802.773.423.643.513.763.793.523.533.35
KeepLoRAOld--79.2178.1175.5772.5772.1870.1569.2668.2067.3567.3071.99
New80.0364.7154.5242.7566.4843.6259.4059.3056.5070.3974.2359.19
FR--3.271.962.693.112.283.403.673.983.774.253.24

FSCIL-Adv achieves the strongest overall results, indicating better retention of previously learned classes while maintaining effective adaptation to new classes.

Q3: Robustness Generalization across FSCIL Factors

A3: We thank the reviewer for this valuable suggestion. We additionally report the same few-shot analysis on CIFAR-100 in Tab. R2, including rival exposure / robust generalization gap / held-out test robustness. ATPGO consistently achieves broader rival exposure, a smaller gap, and higher held-out robustness.

Tab. R2: Few-shot adversarial robustness and rival exposure on CIFAR-100.

KNaive Self-ATDual Naive ATATPGO
11.44/22.60/12.301.40/21.15/12.381.57/20.58/13.43
33.22/19.96/14.704.38/17.00/16.595.64/14.86/19.03
54.35/16.27/24.116.38/14.30/25.028.09/13.90/30.15
107.54/10.56/35.449.62/9.08/37.5811.57/7.83/40.78
2012.70/7.50/49.1314.26/5.68/52.3018.15/4.42/53.36

We further extend the analysis to three FSCIL factors in Tabs. R3–R5.

Tab. R3: Base-class size and session length on CUB200

MethodB50-W10-K5-S16B100-W10-K5-S11
FSCIL-AT72.82/20.0681.41/28.21
Naive-AT70.40/14.3879.36/22.46
KeepLoRA63.42/7.8271.08/20.22

Tab. R4: Data scarcity by shots per class on CUB200

MethodK1K3K5K10
FSCIL-AT67.83/20.1877.91/24.9381.41/28.2184.81/36.57
Naive-AT68.11/18.6975.49/20.5179.36/22.4683.83/28.19
KeepLoRA65.88/7.2968.51/8.3171.08/20.2271.39/18.78

Tab. R5: Class-growth rate and session length on CUB200

MethodW5-S21W10-S11W20-S6
FSCIL-AT65.87/22.0881.41/28.2177.06/24.61
Naive-AT66.29/17.5879.36/22.4676.48/23.09
KeepLoRA56.89/16.4371.08/20.2267.08/19.40

Here, BB, WW, KK, and SS denote base classes, new classes per session, shots per class, and total sessions, respectively. Each entry reports Clean / Rob-Avg (%), where Rob-Avg averages five attacks. The original protocol is B100-W10-K5-S11. Tab. R3 varies BB, Tab. R4 varies KK, and Tab. R5 fixes the final class count and K=5K=5 while varying WW, thereby changing session length and the growth rate W/BW/B. FSCIL-AT achieves the highest Rob-Avg in every setting and the strongest clean accuracy in nearly all configurations, demonstrating robustness to diverse data-scarcity and class-growth conditions.

AuthorsOpenReview ↗

Response 2/3 to Reviewer ecV2

Q4: Problem Scope and Contribution Focus

A4: We thank the reviewer for encouraging this clarification. We clarify the issue below.

(1) Unified Regularized Minimax Objective.

In Eq.(14), uu denotes the attacker-defender coupling term rather than the complete defender objective. Specifically, define

$R_t(\Theta_t)=\mathbb{E}_{(\mathbf{x},y)\sim\mathcal{D}_t}\left[\mathcal{L}_{\mathrm{CE}}(\mathcal{E}_t,\mathbf{x},y)+ \lambda_2\mathcal{L}_{\mathrm{CE}}(\mathcal{E}_t,\mathbf{x}^{\mathrm{self}},y)\right]+\lambda_{\mathrm{CKA}}\mathcal{L}_{\mathrm{CKA}},$ where the CKA term is omitted for the base session, and uˉt(Θ~t,Θt)=λ1u(Θ~t,Θt)+Rt(Θt)\bar{u}_t(\widetilde{\Theta}_t,\Theta_t)=\lambda_1 u(\widetilde{\Theta}_t,\Theta_t)+R_t(\Theta_t). Because RtR_t is independent of Θ~t\widetilde{\Theta}_t, $\nabla_{\widetilde{\Theta}_t}\bar{u}_t= \lambda_1\nabla_{\widetilde{\Theta}_t}u$ , $\nabla_{\Theta_t}\bar{u}_t=\lambda_1\nabla_{\Theta_t}u+\nabla_{\Theta_t}R_t= \nabla_{\Theta_t}\mathcal{L}_{\mathrm{def}}$. For λ1>0\lambda_1>0, the scaling of the attacker gradient is absorbed into its step size and does not change the ascent direction. Thus, the implemented two-stage procedure corresponds to gradient ascent/descent on the augmented minimax payoff uˉt\bar{u}_t. The clean, self-generated, and CKA terms are defender-side regularizers within the same game, rather than objectives external to the theoretical formulation. Eq.(14) isolates the coupled transfer term, whereas Eq.(13) specifies the complete regularized implementation.

(2) Conditional Linear Convergence.

The linear-convergence proposition characterizes the effective ATPGO game operator after replacing the coupling payoff uu with the augmented payoff uˉt\bar{u}_t, under local smoothness and strong convexity-concavity of uˉt\bar{u}_t. Under these conditions, the corresponding operator is strongly monotone and Lipschitz continuous, yielding a linear contraction toward the local Nash equilibrium. The theorem concerns the actual alternating ATPGO updates in the LoRA parameter space and does not require the original deep-network loss to be globally convex. The frozen backbone, low-dimensional LoRA adaptation, and local treatment of the adversarial generation map define the effective game analyzed by the proposition. Hence, deep-model nonlinearity does not invalidate the result; the stated guarantee is a local structural characterization rather than an unconditional global-convergence claim. This conditional treatment is consistent with established minimax analyses under explicit structural assumptions[1][2].

Q5: Problem Scope and Contribution Focus

A5: We thank the reviewer for this valuable suggestion. In the revised manuscript, we adjusted the motivation and problem statement to clarify that our primary goal is robust adaptation under extreme few-shot continual supervision. Our method addresses limited samples and sparse class spaces in each session by balancing clean accuracy, adversarial robustness, and continual knowledge retention.

Q6: ATPGO Mechanism-Level Validation

A6: We thank the reviewer for this helpful suggestion. We conduct ablations anlysis, results in Tab.R6 and R7. ''Clean + S/T'' denotes clean training with self-generated / transferred supervision, ''Clean + T + S'' combines both, and ''Frozen'' versus ''Co-evolving'' indicates whether the attacker is fixed or jointly updated with the defender.

Tab. R6: Ablations of ATPGO components and attacker co-evolution in CIFAR100.

SettingAttackerClean Avg.ℓ∞-PGDℓ2-PGDFGSMBIMCW
Clean + SCo-evolving79.757.0233.7346.765.6136.50
Clean + TCo-evolving81.076.8133.5849.684.4036.85
Clean + T + SFrozen attacker80.069.8035.7250.067.4337.84
Clean + T + SCo-evolving81.9611.7144.1152.128.9438.65

Tab. R7: Ablations of ATPGO components and attacker co-evolution in CUB200.

SettingAttackerClean Avg.ℓ∞-PGDℓ2-PGDFGSMBIMCW
Clean + SCo-evolving76.831.1247.8530.611.1043.02
Clean + TCo-evolving76.421.4243.9129.931.4441.64
Clean + T + SFrozen attacker78.911.7349.5631.331.6246.34
Clean + T + SCo-evolving81.412.4252.8232.352.9450.54

The co-evolving model performs best on both datasets, showing gains beyond additional adversarial supervision.

[1] Finding Second-Order Stationary Points in Nonconvex-Strongly-Concave Minimax Optimization, NeurIPS 2022

[2] Efficient Algorithms for Smooth Minimax Optimization, NeurIPS 2019

AuthorsOpenReview ↗

Response 3/3 to Reviewer ecV2

Q7: Continual Robustness and Computational Scalability

A7: We thank the reviewer for this suggestion and evaluate session-wise robustness and computational costs below.

(1) Session-wise Robustness.

Tab. R8 reports old/new robust accuracy (%) averaged over FGSM, ℓ2\ell_2-PGD, and CW, with entries ordered as old / new. ℓ∞\ell_\infty-PGD and BIM are omitted due to uniformly low accuracies and limited inter-method discrimination. FSCIL-Adv achieves the strongest overall robustness.

Tab. R8: Session-wise adversarial robustness on CUB-200 (%).

SESSIONFSCIL-AdvCL-LoRAKeepLoRAARMCL
S0— / 60.89— / 58.76— / 51.44— / 32.77
S154.68 / 56.2357.98 / 35.0150.73 / 22.8929.38 / 16.16
S253.96 / 26.2955.07 / 16.5348.27 / 11.6521.05 / 8.94
S351.40 / 19.9450.76 / 16.0744.65 / 12.5025.92 / 8.33
S448.24 / 45.3948.04 / 14.8741.53 / 14.6124.18 / 20.00
S546.91 / 29.5344.63 / 20.4738.55 / 9.9422.45 / 14.62
S644.92 / 36.3642.63 / 23.9436.31 / 16.3620.83 / 10.91
S743.93 / 34.8441.27 / 35.3434.09 / 13.0315.35 / 7.02
S842.64 / 23.6940.66 / 20.1132.30 / 8.5416.22 / 9.92
S941.44 / 57.1039.09 / 41.5330.13 / 24.0414.10 / 24.86
S1042.00 / 49.8639.00 / 44.7229.51 / 16.2617.20 / 28.73
Avg47.01 / 37.9245.91 / 26.8638.61 / 14.9820.67 / 14.95

(2) Computational and Storage Costs.

We compare FSCIL-Adv with native robust baselines using PGD-1010 for AIR/DSEF and PGD-2020 for ARMCL. In Tab. R9, WW denotes the number of new classes per session and SS the total number of sessions. Each cell reports training time (min) / clean-forward latency (ms/image) / model tensor footprint (MiB) for W5-S21, W10-S11, and W20-S6. Overall, our method remains within the average cost range.

Tab. R9: Cost comparison on CUB-200.

METHODW5-S21W10-S11W20-S6
FSCIL-Adv77.38 / 44.30 / 434.6966.61 / 28.06 / 383.8361.48 / 14.43 / 358.40
AIR-PGD1048.61 / 35.74 / 398.1240.78 / 17.26 / 364.7333.45 / 6.84 / 347.66
DSEF-PGD1092.94 / 46.88 / 410.3682.17 / 28.33 / 371.9272.80 / 17.21 / 350.88
ARMCL-PGD2083.47 / 47.15 / 417.4867.92 / 28.60 / 378.1464.61 / 17.38 / 353.72

Q8: Self-Contained Experimental Evidence

A8: We thank the reviewer for this valuable structural suggestion. In the revised manuscript, we moved the essential experimental settings from the Appendix C to the main text, including class splits and shots, backbone and training schedules, attack budgets and iterations. We also incorporated evaluations under a broader range of attack scenarios, analyses of forgetting rates, comparisons across different few-shot settings, and attack-wise component ablations from Appendix F, providing more comprehensive empirical support for our core claims.

Reviewer_piqaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the problem of adversarial robustness in few-shot class-incremental learning. The algorithm is developed form the LoRA-based framework where a dedicated LoRA expert per session, controlled by learnable layer-wise gates. In addition, a two-player adversarial game between the attacker expert and defender expert is proposed.

Critical strengths
  1. this paper addresses an uncharted problem, namely adversarial robustness in the few-shot class-incremental learning problem.
  2. the presentation of this paper is easy to follow.
  3. numerical result confirms the advantage of the proposed approach.
Critical weaknesses
  1. details of experiments are not elaborated. Implementation details are missing from the main paper.

  2. The proposed approach is only tested in 1 dataset, CUB-200. Although the proposed method is superior to other works, it is interesting to see whether this is generalizable to other datasets as well.

Questions for authors

please address the two comments above.

2 comments
AuthorsOpenReview ↗

Response 1/2 to Reviewer piqa

We sincerely thank the reviewer for recognizing our research motivation and module design and for providing valuable feedback, we address the comments below.

Q1: Experimental Setup and Implementation Details

A1: We thank the reviewer for this valuable comment. Due to space constraints, detailed experimental and implementation settings are provided in Appendix C in the original manuscript. We have also added the hardware specifications there. summarized as follows:

(1) Model and Training Settings. All images are resized to 224×224224\times224. We use a frozen ViT-B/16 pretrained on ImageNet-21K and fine-tuned on ImageNet-1K, with rank-1616 LoRA on Q / K / V / O projections, gates initialized to 0.50.5, and CKA applied to blocks 66-1111. SGD uses a learning rate of 0.030.03, momentum 0.90.9, weight decay 10−410^{-4}, batch size 6464, and cosine decay. Base/incremental sessions use 3030 / 1010 epochs, including 1515 / 33 clean warm-up epochs.

(2) Adversarial Settings. FGSM training uses ϵ=8/255\epsilon=8/255 and step size 10/25510/255, with adversarial updates every three base-session mini-batches and every incremental mini-batch. For evaluation, FGSM, ℓ∞\ell_\infty-PGD, and BIM use ϵ=8/255\epsilon=8/255, iterative attacks use 1010 steps with step size 2/2552/255. ℓ2\ell_2-PGD uses ϵ=128/255\epsilon=128/255, 1010 steps, and step size 15/25515/255, while CW uses c=1c=1, κ=0\kappa=0, 100100 iterations, and learning rate 0.010.01.

(3) Hardware Configuration. All experiments were conducted on an NVIDIA A100 80GB PCIe GPU. We have included this hardware specification in Appendix C of the revised manuscript to improve reproducibility.

AuthorsOpenReview ↗

Response 2/2 to Reviewer piqa

Q2: Generalization Across Datasets

A2: We thank the reviewer for the comment. Our method has been evaluated on four FSCIL benchmarks, as summarized in Tables 2 and 3 of Section 4.1 of the original manuscript. Detailed results on CIFAR-100, mini-ImageNet, and ImageNet-R are provided in Appendix E, Tables 4-6 and presented below in Tabs. R1–R3. All results are reported as mean Top-1 accuracy (%), including clean accuracy and attack-specific accuracy over three independent runs.

Tab. R1: Clean and adversarial accuracy on CIFAR-100 (%).

MethodTrainCleanFGSMℓ∞\ell_\infty-PGDℓ2\ell_2-PGDBIMCWAvg-All
LwFStd70.1123.382.2922.131.550.0019.91
Adv69.6825.324.9923.996.230.0021.70
RanPACStd72.4322.390.0224.820.016.3621.01
Adv72.4231.860.8127.160.759.4323.74
CL-LoRAStd84.8622.630.0631.330.0812.6025.26
Adv83.7526.390.3438.110.2327.0529.31
KeepLoRAStd80.6235.273.3836.364.0412.5328.70
Adv80.0944.878.1137.385.7823.0933.22
TEENStd79.2314.611.8619.590.680.0019.33
Adv79.2314.611.8819.590.680.0019.33
BiMCStd72.9715.710.004.030.003.6716.06
Adv73.0616.640.004.570.004.2016.41
ConCMStd72.4917.530.5018.740.263.2518.80
Adv71.2516.860.4618.760.253.0718.44
AIRNative35.7814.927.0913.447.160.0013.07
DSEFNative26.567.611.345.422.460.5907.33
ARMCLNative60.5018.986.2817.547.450.0018.46
FSCIL-ATATPGO81.8353.8711.0241.689.9036.3939.12

Tab. R2: Clean and adversarial accuracy on mini-ImageNet (%).

MethodTrainCleanFGSMℓ∞\ell_\infty-PGDℓ2\ell_2-PGDBIMCWAvg-All
LwFStd91.8552.5219.5651.4217.670.0638.85
Adv86.7853.317.8555.086.020.1134.86
RanPACStd76.1340.480.3847.580.2123.3331.35
Adv76.1349.932.4550.272.2135.0036.00
CL-LoRAStd94.9954.211.9066.121.5751.9245.12
Adv94.8665.322.2572.941.6470.8651.31
KeepLoRAStd91.2955.449.3360.588.0336.1643.47
Adv90.5665.4515.5869.9514.0853.0351.44
TEENStd97.9764.1717.2753.6717.010.0241.69
Adv97.9764.1717.2853.6717.010.0241.69
BiMCStd90.9136.380.0132.350.0026.2330.98
Adv90.9537.980.0234.240.0027.1631.73
ConCMStd93.3551.064.6454.953.2419.1737.74
Adv92.9349.535.5653.403.6717.7537.14
AIRNative62.9926.0716.8127.5717.270.0125.12
DSEFNative21.116.881.895.082.261.4906.45
ARMCLNative92.0747.6114.7357.1814.090.1737.64
FSCIL-ATATPGO93.8071.7624.2973.4622.3271.1359.46

Tab. R3: Clean and adversarial accuracy on ImageNet-R (%).

MethodTrainCleanFGSMℓ∞\ell_\infty-PGDℓ2\ell_2-PGDBIMCWAvg-All
LwFStd54.3412.434.0213.523.690.0814.68
Adv55.5111.350.4614.310.370.1513.69
RanPACStd58.829.480.2529.570.2222.6820.17
Adv58.7221.003.4032.413.2427.5324.38
CL-LoRAStd79.4515.570.7041.850.5638.2229.39
Adv79.0334.633.4346.963.4143.7135.20
KeepLoRAStd72.8223.833.2035.733.2026.8527.61
Adv72.6634.624.7042.666.2836.9732.98
TEENStd61.1410.520.6620.660.540.3815.65
Adv61.1410.520.6920.660.540.3815.66
BiMCStd74.9216.970.0418.810.0218.9021.61
Adv75.5717.460.0319.440.0119.1321.94
ConCMStd59.379.880.1221.530.1411.3817.07
Adv59.269.860.1021.370.0911.3917.01
AIRNative24.164.652.654.942.770.0206.53
DSEFNative11.230.810.850.810.810.7802.55
ARMCLNative46.047.252.1511.571.970.1311.52
FSCIL-ATATPGO75.9237.216.9349.046.7349.7137.59

Overall, FSCIL-AT consistently achieves the best Avg-All across all four benchmarks, demonstrating strong cross-dataset generalization and competitive performance.

Reviewer_He4UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies adversarial robustness in few-shot class incremental learning (FSCIL). The authors identify a critical but overlooked vulnerability: state-of-the-art FSCIL models can catastrophically collapse under adversarial perturbations, even though they perform well on clean inputs. To address this, they propose FSCIL AT, a framework built on a frozen pre-trained Vision Transformer with session-wise LoRA experts. The method integrates three main components. Experiments are conducted on CIFAR 100, mini ImageNet, CUB 200, and ImageNet R under standard FSCIL protocols. Results show that FSCIL AT achieves the best Avg All on all four benchmarks while remaining competitive on clean accuracy, with each component contributing to the final performance.

Critical strengths
  • The paper identifies and formalizes a genuine and underexplored problem, namely adversarial robustness in few-shot class incremental learning, and characterizes two concrete failure modes of naive adversarial training under data scarcity that motivate ATPGO.
  • The paper integrates its components into a coherent pipeline and validates it broadly. It combines frozen ViT features, session-wise LoRA experts with learnable gating, two-player adversarial optimization with a cloned attacker, and margin-based CKA regularization, with ablations indicating that each component contributes.
Critical weaknesses
  • All attacks are white-box, fixed epsilon, and generated against the training-time cosine classifier, while test-time inference uses the non-parametric prototype classifier. The prototype rule is never attacked directly, and there is no adaptive, black-box, or transfer evaluation. Because ATPGO is itself adversarial training, the defender may overfit to its training-time attacker, so the reported robustness may not reflect the deployed rule.
  • Some baselines use different backbones, and non-robust methods receive a shared FGSM Adv variant while native robust methods keep their original objectives.
  • The methodological novelty is limited, and the specific new mechanism is not clearly articulated. The components are largely existing, including frozen ViT, session-wise LoRA, layer-wise gating, adversarial training with a cloned attacker, and CKA regularization.
  • The gating design lacks justification and ablation. All LoRA injection positions within a block share a single gate coefficient, but the paper does not explain why sharing is preferable to per-position gates, nor does it ablate this choice.
Questions for authors

Please see the Weaknesses for details.

2 comments
AuthorsOpenReview ↗

Response 1/2 to Reviewer He4U

We thank the reviewer for the positive assessment and address the concerns below.

Q1: Robustness Generalization under Diverse Threat Models

A1: We thank the reviewer for emphasizing robustness generalization. We add evaluations under richer attack mechanisms, as summarized below.

(1) Robustness under Varying ϵ\epsilon.

We evaluate FSCIL-AT against competitive conventional FSCIL baselines, KeepLoRA and CL-LoRA, and the native adversarial-training baseline ARMCL. Results are shown in Tab. R1.

Tab. R1: Robust accuracy (%) under varying ϵ\epsilon.

Attack (ϵ\epsilon)MethodCIFAR-100CUB-200
FGSM (4/8/16)KeepLoRA43.41/42.74/43.1730.55/26.76/21.39
CL-LoRA34.48/30.24/27.5232.47/27.30/17.13
ARMCL25.48/19.48/15.5320.61/19.83/19.30
FSCIL-AT59.50/52.12/45.4735.35/32.35/22.33
ℓ∞\ell_\infty-PGD (4/8/16)KeepLoRA13.93/8.42/6.283.78/2.09/0.47
CL-LoRA0.81/0.21/0.032.14/0.03/0.00
ARMCL6.99/6.73/5.812.29/1.65/1.35
FSCIL-AT13.94/11.71/7.474.71/2.42/2.04
BIM (4/8/16)KeepLoRA8.98/5.60/4.393.82/2.40/0.32
CL-LoRA0.81/0.25/0.041.57/0.03/0.00
ARMCL6.77/6.45/5.392.13/1.63/1.25
FSCIL-AT12.27/8.94/6.254.61/2.94/2.06
ℓ2\ell_2-PGD (64/128/256)KeepLoRA47.91/35.20/25.3657.12/43.69/38.89
CL-LoRA51.92/41.60/32.1056.64/50.19/27.91
ARMCL26.41/14.01/12.4926.50/20.59/17.84
FSCIL-AT61.96/44.11/32.8870.31/52.82/41.62

FSCIL-AT maintains the strongest robust accuracy across most attacks and budgets, showing that its advantage is not tied to a single ϵ\epsilon.

(2) Consistency between Training and Inference Classifiers.

We thank the reviewer for identifying this ambiguity. The local cosine classifier is used only during the clean warm-up stage. After warm-up, we compute class prototypes from clean features and fix them as the prototype head. ATPGO adversarial example generation and inference during testing then use the same prototype-based prediction rule, while only the expert and gate parameters are updated. We clarify this protocol in Sections 3.2 and Appendix C of the revised manuscript.

(3) Adaptive, Black-box, and Transfer Attack Evaluation.

Adaptive Attack. The attack evaluation reported in our manuscript employs the prototype-based classifier rather than the discarded cosine proxy, making the attacks aware of the prototype decision rule and therefore adaptive.

Black-box Attack. We conduct two representative black-box evaluations: Square Attack[1] with ϵℓ∞=8/255\epsilon_{\ell_\infty}=8/255 and Q=50Q=50, and HSJA[2] with ϵℓ2=128/255\epsilon_{\ell_2}=128/255 and Q=20Q=20. Results are reported in Tab. R2.

Tab. R2: Black-box attack evaluation (%) on CUB-200.

AttackMethodRobustSeenUnseen
SquareKeepLoRA46.9064.0529.99
CL-LoRA36.6443.6929.11
ARMCL26.5628.7723.90
FSCIL-AT48.3066.0630.54
HSJAKeepLoRA68.2480.3156.17
CL-LoRA64.8174.3255.24
ARMCL53.4154.7751.90
FSCIL-AT72.0983.5260.83

Transfer Attack. We use a unified ViT-B/16 surrogate to generate transfer attacks. In addition to the five attacks used in the paper, we include three representative transfer-oriented attacks: MI-FGSM[3] with ϵ=8/255\epsilon=8/255, α=2/255\alpha=2/255, 1010 steps, and μ=1.0\mu=1.0; DI-MI-FGSM[4] with the same settings and input-diversity probability p=0.5p=0.5; and TI-FGSM[5] with the same settings and a 15×1515\times15 Gaussian kernel with σ=3.0\sigma=3.0. Generated examples are directly evaluated on all target methods, results are shown in Tab. R3.

Tab. R3: Transfer-attack accuracy (%) on CIFAR-100.

AttackKeepLoRACL-LoRAARMCLFSCIL-AT
FGSM24.8426.6823.8556.37
ℓ∞\ell_\infty-PGD0.9647.126.3156.09
ℓ2\ell_2-PGD67.0570.5844.8468.61
BIM0.7543.475.7555.99
CW69.8478.1147.8477.97
MI-FGSM0.5535.154.7555.09
DI-MI-FGSM1.2035.225.3651.85
TI-FGSM3.3311.6810.7029.59

Although ATPGO uses white-box adversarial training, its competitive performance under the added black-box and transfer attacks shows that co-evolving self and transferred adversarial supervision broadens robustness beyond the training-time attacker rather than merely fitting to it.

[1] Square Attack: A Query-Efficient Black-Box Adversarial Attack via Random Search, ECCV 2020.

[2] HopSkipJumpAttack: A Query-Efficient Decision-Based Attack, IEEE S&P 2020.

[3] Boosting Adversarial Attacks with Momentum, CVPR 2018.

[4] Improving Transferability of Adversarial Examples With Input Diversity, CVPR 2019.

[5] Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks, CVPR 2019.

AuthorsOpenReview ↗

Response 2/2 to Reviewer He4U

Q2: Backbone and Training-Protocol Consistency across Baselines

A2: We sincerely thank the reviewer for consistent comparisons. In the manuscript’s comparison tables (Tables 1, 2, and 4–6), all non-native-robust baselines use the same ViT-B/16 backbone and shared FGSM-Adv protocol. BiMC retains its frozen CLIP ViT-B/16 encoder because its training-free design is intrinsically CLIP-based. AIR, DSEF, and ARMCL are additionally evaluated with their native PGD-1010/PGD-2020 protocols. Results are reported in Tab. R4, these baselines achieve varying degrees of performance improvement under their native protocols, while FSCIL-AT maintains the best overall performance and leads on most accuracy metrics.

Tab. R4: Native-protocol evaluation (%) on CUB-200.

MethodCleanFGSMℓ∞\ell_\infty-PGDℓ2\ell_2-PGDBIMCWAvg-All
AIR-PGD$10$45.3710.242.3412.182.070.1312.06
DSEF-PGD$10$24.288.171.184.131.071.626.74
ARMCL-PGD$20$53.4623.283.1726.242.930.4718.26
FSCIL-AT81.4132.352.4252.822.9450.5437.08

Q3: Methodological Novelty and Mechanism Design

A3: We thank the reviewer for raising this important question. Our key novelty is addressing a specific FSCIL bottleneck rather than simply assembling existing components. Under extreme sample scarcity and limited per-session class diversity, self-generated attacks repeatedly explore a narrow set of rival-class directions. ATPGO addresses this limitation through a session-wise attacker–defender game, in which an independently updated attacker discovers transferable vulnerabilities while the defender learns against both transferred and self-generated examples, thereby expanding adversarial-direction coverage. To the best of our knowledge, this is the first robust FSCIL formulation to use attacker–defender co-evolution for this purpose. Theoretical analysis further supports its effectiveness.

Q4: Methodological Novelty and Mechanism Design

A4: We thank the reviewer for prompting clarification of the gating strategy. Shared block gating provides a low-variance inductive bias for few-shot adaptation by coordinating all LoRA positions within each Transformer block, while avoiding the extra degrees of freedom and overfitting risk of per-position gates under extreme data scarcity. We compare fixed, per-position, and shared block gates in Tab. R5. Shared block gating achieves the best clean and robust accuracy.

Tab. R5: Gating-strategy ablation on CUB-200 (%).

MetricFixed Gate (0.5)Per-Position GatesShared Block Gate (Ours)
Clean79.8880.7281.41
FGSM29.6431.6932.35
ℓ∞\ell_\infty-PGD1.611.762.42
ℓ2\ell_2-PGD51.6051.5652.82
BIM1.721.722.94
CW49.5349.7850.54

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all reviewers for their time, thoughtful evaluation, and constructive suggestions. Across reviews, the reviewers recognized robust FSCIL as an important underexplored problem and appreciated our diagnosis of scarcity-induced failure modes, coherent design combining session-wise LoRA with adversarial game optimization, and supporting empirical results.

This paper focuses on robust adaptation under extreme few-shot class-incremental supervision. We identify that extreme sample scarcity and limited per-session class diversity cause self-generated attacks to repeatedly explore a narrow set of rival-class directions, limiting adversarial coverage. To address this issue, we propose FSCIL-Adv, which introduces ATPGO, an attacker–defender co-evolution process that expands adversarial-direction coverage beyond vulnerabilities exposed by scarce session data. Session-wise LoRA experts with shared block gating enable parameter-efficient continual adaptation, while MBCKAR promotes complementary representations and supports knowledge retention. Theoretical analysis and evaluations across four benchmarks further validate this design.

The reviewers’ insights provided valuable guidance for improving our paper. We addressed each concern through systematic supplementary experiments and in-depth analyses, summarized below:

1.Comprehensive Robustness Evaluation. We extended evaluations to varying perturbation budgets and diverse attack scenarios, and clarified the consistency between adversarial training and inference decision rules. [Reviewers He4U]

2.Continual Learning and Few-Shot Validation. We added old/new-class accuracy, forgetting rate, and session-wise robustness, along with systematic analyses across shots, base-class size, session length, and class-growth rate, and further evaluated training, inference, and storage costs under longer incremental sequences. [Reviewer ecV2]

3.Methodological and Theoretical Validation. We conducted ablations of ATPGO's loss components and attacker co-evolution, compared alternative gating strategies, and clarified the regularized minimax formulation and conditional convergence guarantee. [Reviewers He4U, ecV2]

4.Fair Baseline Comparisons. We clarified backbone and training-protocol consistency and evaluated native robust baselines under their original settings. [Reviewers He4U]

5.Manuscript Clarification. We clarified experimental details and results on other FSCIL benchmarks, further refined the problem statement, and restructured the manuscript to incorporate essential experimental analyses and key findings into the main text. [Reviewers piqa, ecV2]

We have addressed all constructive comments in our detailed responses to each reviewer and commit to incorporating the corresponding supplements and revisions in the next version. Overall, FSCIL-Adv provides an effective solution for robust few-shot class-incremental learning, demonstrating strong performance across diverse adversarial threats and incremental settings, while offering a new perspective on overcoming scarcity-induced limitations in adversarial coverage. We sincerely thank all reviewers again for their valuable feedback and hope our responses adequately address their concerns.

Best regards,

Authors of Paper

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.