Reliable Adversarial Continual Learning via Robust Expert Collaboration

Adversarially robust continual learning must acquire new tasks while preserving previously learned knowledge and adversarial robustness. These objectives can conflict because adversarial updates to a shared representation may amplify forgetting, induce representation drift, and yield class-imbalanced robust solutions. We propose Robust Expert Collaboration (REC), a replay-free framework that separates task-level robust expert construction from guarded backbone integration. REC first applies Diversified Multi-Expert Learning (DMEL), which jointly optimizes temporary task experts with feature-diversity and gradient-diversity regularization, together with image- and feature-space adversarial objectives, before fusing them into a single task expert. Robust Anchoring and Stability Cooperation (RASC) then adapts the shared backbone using clean and adversarial supervision, clean–adversarial consistency, a frozen teacher feature anchor, and historical decision distillation. A Class-wise Robust Constraint (CRC) selects checkpoints on a held-out gating split using clean accuracy, adversarial accuracy, per-class adversarial recall, teacher feature similarity, and historical decision agreement; when no candidate satisfies the predefined criteria, REC restores the pre-integration checkpoint. Experimental results demonstrate that REC achieves a favorable robustness–retention trade-off while maintaining competitive clean accuracy.