Robust Few-Shot Class-Incremental Learning via Adversarial Two-Player Game Optimization

Few-Shot Class-Incremental Learning (FSCIL) demands that a model continuously acquire new categories from scarce labeled data while retaining prior knowledge. Recent LoRA-based methods dynamically construct lightweight adaptation modules upon Pre-Trained Models (PTMs), achieving impressive performance with minimal additional parameters. However, most existing approaches largely neglect adversarial robustness, leaving the adapted models vulnerable to adversarial threats in real-world deployment scenarios. In this work, we simultaneously addresses catastrophic forgetting, data scarcity, and adversarial fragility in FSCIL by introducing a new LoRA-based optimization framework that dynamically creates a dedicated LoRA expert per session, controlled by learnable layer-wise gates that adaptively balance base and incremental representations. To robustify the model under extreme data scarcity, we introduce a two-player adversarial game between a co-evolving attacker expert and a defender expert. The attacker continuously sharpens its adversarial generation capability via transferable perturbations, while the defender improves robustness by resisting both transferred and self-generated adversarial examples. We further derive that this minimax formulation admits a Nash Equilibrium and converges at a linear rate under mild smoothness conditions. To prevent representational collapse under few-shot constraints, we impose a margin-based Centered Kernel Alignment (CKA) regularization that explicitly penalizes redundancy between the base and incremental experts, steering each new expert toward a complementary feature subspace and sustaining meaningful adversarial co-evolution.