- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
To address adversarial vulnerability in deep neural networks and to overcome the limitations of adversarial training, such as high computational overhead and the natural-robust accuracy trade-off, the authors propose a lightweight plug-and-play framework termed RIDE. RIDE uses a combination of detection and purification to identify and purify adversarial examples. Specifically, this method utilizes information from the low-variance subspace to distinguish adversarial examples from natural ones. The identified adversarial examples are purified using a denoising autoencoder. According to the authors, the experimental results show that RIDE substantially improves robust accuracy while maintaining comparable natural accuracy.
Critical strengths
- The paper is well-organized and decently motivated.
- The authors validated their method using various datasets and under various attacks.
Critical weaknesses
-
The paper contains a few misleading or strong claims. For example, in the abstract, the authors claim that adversarial training is trained for specific attacks. This claim is misleading. In the introduction section, second paragraph, the authors claim that adversarial training causes gradient obfuscation. This is another misleading claim. These statements suggest that the discussion of prior adversarial robustness literature requires clarification and more accurate characterization.
-
In Theorem 1, the authors prove that natural samples are concentrated around their mean in low-variance directions. However, the theoretical proof does not explicitly prove that adversarial examples leave this subspace.
-
Prior works [1, 2] have shown that many purification defenses can appear substantially more robust than they actually are when evaluated using insufficiently adaptive attacks. Adversarial purification defenses are notorious for causing gradient masking. A close look at the results provided by the authors in Tables 1 and 2 suggests that their method could also be exhibiting gradient masking. In Tables 1, 2, and 3, the model achieves better performance under some stronger attacks than under weaker attacks such as PGD and C&W, which is unusual. Their model also shows better performance under Square Attack, a black-box attack, than under white-box attacks such as C&W and DeepFool. These results are consistent with a symptom of gradient masking.
References
- Croce, F., Gowal, S., Brunner, T., Shelhamer, E., Hein, M. and Cemgil, T., 2022, June. Evaluating the adversarial robustness of adaptive test-time defenses. In International Conference on Machine Learning (pp. 4421-4435). PMLR.
- Athalye, A., Carlini, N. and Wagner, D., 2018, July. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning (pp. 274-283). PMLR.
Questions for authors
- Can the authors provide additional evaluations under multiple perturbation budgets, e.g., , ,, , , and using PGD, C&W, and AutoAttack? Evaluating robustness across different perturbation strengths would help clarify how RIDE behaves as attack severity increases.
Suggestion
The authors should consider improving the introduction. The readability of the paper could also be improved by using the appropriate citation style. Citations that support a statement should be placed in parentheses using \citep{}. When the authors of a cited work are mentioned directly within a sentence, \citet{} or \cite{} should be used.