Data-free Adversarial Training: Achieving Robustness without Data and Robust Teacher

Deep neural networks are vulnerable to adversarial perturbations, while the most empirically reliable defenses (i.e., adversarial training and robustness distillation) typically require access to real training data and, in the latter case, an additional robust teacher. These extravagant assumptions hinder the deployment of existing defenses when sensitive data and robust teacher are unavailable, while only a naturally trained pretrained model can be accessed. To achieve adversarial robustness when only a naturally pre-trained model is available, we propose Data-Free Adversarial Training (DFAT), a robust post-training framework that requires neither real data nor a robust teacher. We theoretically show that a locally accurate student surrogate can reveal attack directions aligned with the natural teacher's local sensitivity, and that robust risk on real data can be controlled by optimizing teacher quality, robust anchoring error, and the discrepancy between synthetic and real distributions. Based on this, DFAT first performs data-free adversarial training to construct a natural student surrogate and learn a surrogate data distribution. It then uses the frozen natural teacher as a clean semantic oracle, and performs self-adversarial training on the student by maximizing and minimizing teacher-student output discrepancies over synthetic local neighborhoods. We further identify two additional challenges amplified in the data-free scenario, namely the fairness challenge and the generalization challenge, and propose to mitigate them from the perspectives of entropy reweighting and probability sparsity, respectively. To further mitigate the performance loss caused by surrogate distribution shift, we incorporate a test-time adaptation (TTA) module to further explore the self-adaptation potential of DFAT. Experiments and analyses demonstrate that DFAT achieves effective and reliable adversarial robustness in the practical setting with neither real data nor robust teacher.