- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LREERC, which adds task-specific LoRA experts to a frozen ViT and reuses selected historical experts through residual composition. Historical experts are retrieved using cosine similarity between gradient fingerprints, while an orthogonality loss encourages complementary expert parameters. Experiments on four image datasets report strong performance against the evaluated attack sequence.
Critical strengths
The framework is parameter-efficient. The paper reports memory, parameter, and training-time costs. The empirical comparison covers four datasets and a broad collection of continual-learning and defense baselines. The paper evaluates a complete predictor without assuming a test-time attack label.
Critical weaknesses
-
The main robustness results focus primarily on attacks already adopted for training. Tab. 1 lists AutoAttack as the fifth training task, achieving 84.20% final accuracy on CIFAR-10. In contrast, Tab. 9 demonstrates 0.00% accuracy for genuinely unseen AutoAttack after Clean–FGSM–PGD–CW training, for both 4/255 and 8/255 perturbation radii. This heavily weakens the generalization claim for unseen attacks. Additionally, the final predictor applies class-wise maximum over several configurations (Eq. 38), but the work does not report results for attack unions, smooth-max surrogate attacks, or stricter multi-restart assessments.
-
The key gradient-fingerprint component has not been validated in isolation. Tab. 2 merges residual composition, fingerprint selection and orthogonality into the LREERC setup, with missing comparisons against random selection, the newest single expert, and an experiment without the orthogonality term. The fingerprints are gradients computed after training; their norms and stability across sampled subsets are relevant characteristics but go unreported. The theoretical transfer bound depends on unregulated discrepancy and curvature terms, and consequently cannot confirm cosine similarity as a trustworthy selector in real use.
Questions for authors
-
Does the advantage remain under a per-example union of PGD and AutoAttack failures, attacks targeting each retained configuration separately, or a smooth approximation to the class-wise maximum in Eq. (38)?
-
Can the authors provide isolated ablations for fingerprint selection and orthogonality, including random/all/latest-expert selection, and report fingerprint norms and selection stability across subsets and seeds? Please clarify how each incoming adversarial training set is generated: against which checkpoint, whether it is regenerated during adaptation.
-
Which statement about the gate is correct, Appendix A.4 or Appendix B.3.1?