acceptodds
Under review as a conference paper at ICLR 2027

RIDE: A Lightweight Plug-and-Play Framework for Enhancing Adversarial Robustness

Abstract

Deep neural networks (DNNs) remain vulnerable to adversarial attacks based on deliberately crafted perturbations. Previous arts target specific adversarial attacks (e.g., adversarial training) or impose substantial computational overhead on inference time (e.g., adversarial purification). Moreover, both of them significantly compromise the accuracy on clean examples, which dominate real-world tasks, thereby marginalizing their practical applications. To address these challenges, we propose a lightweight plug-and-play framework, namely RIDE, to protect DNNs in inference time. By combining detection and purification, RIDE precisely identifies and purifies adversarial examples, enhancing model robustness while maintaining comparable accuracy on clean examples. In particular, RIDE leverages information from the low-variance subspace (a subspace that exhibits severe selection bias) to precisely detect adversarial examples, which have been demonstrated to diverge from clean examples in this subspace. Moreover, the samples identified as adversarial are purified through an adversarial filter, which uses a denoising autoencoder to optimize the input toward regions of higher training data coverage, driving adversarial examples back to the clean data manifold. Finally, RIDE applies inference-time data augmentation to reduce noise in the predictions, further enhancing the robustness of this framework. Extensive experiments demonstrate that RIDE increases robust accuracy at most by 50.97% while maintaining comparable accuracy on clean examples. Moreover, RIDE uses only 0.87% of the inference time compared to that of classical inference-time defense methods.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Adversarial Robustness, Inference-time defense, Adversarial detection, Adversarial purification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fuSGOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

To address adversarial vulnerability in deep neural networks and to overcome the limitations of adversarial training, such as high computational overhead and the natural-robust accuracy trade-off, the authors propose a lightweight plug-and-play framework termed RIDE. RIDE uses a combination of detection and purification to identify and purify adversarial examples. Specifically, this method utilizes information from the low-variance subspace to distinguish adversarial examples from natural ones. The identified adversarial examples are purified using a denoising autoencoder. According to the authors, the experimental results show that RIDE substantially improves robust accuracy while maintaining comparable natural accuracy.

Critical strengths
  1. The paper is well-organized and decently motivated.
  2. The authors validated their method using various datasets and under various attacks.
Critical weaknesses
  1. The paper contains a few misleading or strong claims. For example, in the abstract, the authors claim that adversarial training is trained for specific attacks. This claim is misleading. In the introduction section, second paragraph, the authors claim that adversarial training causes gradient obfuscation. This is another misleading claim. These statements suggest that the discussion of prior adversarial robustness literature requires clarification and more accurate characterization.

  2. In Theorem 1, the authors prove that natural samples are concentrated around their mean in low-variance directions. However, the theoretical proof does not explicitly prove that adversarial examples leave this subspace.

  3. Prior works [1, 2] have shown that many purification defenses can appear substantially more robust than they actually are when evaluated using insufficiently adaptive attacks. Adversarial purification defenses are notorious for causing gradient masking. A close look at the results provided by the authors in Tables 1 and 2 suggests that their method could also be exhibiting gradient masking. In Tables 1, 2, and 3, the model achieves better performance under some stronger attacks than under weaker attacks such as PGD and C&W, which is unusual. Their model also shows better performance under Square Attack, a black-box attack, than under white-box attacks such as C&W and DeepFool. These results are consistent with a symptom of gradient masking.

References

  1. Croce, F., Gowal, S., Brunner, T., Shelhamer, E., Hein, M. and Cemgil, T., 2022, June. Evaluating the adversarial robustness of adaptive test-time defenses. In International Conference on Machine Learning (pp. 4421-4435). PMLR.
  2. Athalye, A., Carlini, N. and Wagner, D., 2018, July. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning (pp. 274-283). PMLR.
Questions for authors
  1. Can the authors provide additional evaluations under multiple perturbation budgets, e.g., 4/2554/255, 16/25516/255,64/25564/255, 128/255128/255, 255/255255/255, and using PGD, C&W, and AutoAttack? Evaluating robustness across different perturbation strengths would help clarify how RIDE behaves as attack severity increases.

Suggestion

The authors should consider improving the introduction. The readability of the paper could also be improved by using the appropriate citation style. Citations that support a statement should be placed in parentheses using \citep{}. When the authors of a cited work are mentioned directly within a sentence, \citet{} or \cite{} should be used.

Reviewer_p2gmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a Lightweight Plug-and-Play Framework for Enhancing Adversarial Robustness, named RIDE (Residual-aware Inference-time Defender). By combining detection and purification, RIDE precisely identifies and purifies adversarial examples, enhancing model robustness while maintaining comparable accuracy on clean examples. In particular, RIDE leverages information from the low-variance subspace (a subspace that exhibits severe selection bias) to precisely detect adversarial examples, which have been demonstrated to diverge from clean examples in this subspace. Moreover, the samples identified as adversarial are purified through an adversarial filter, which uses a denoising autoencoder to optimize the input toward regions of higher training data coverage, driving adversarial examples back to the clean data manifold. Finally, RIDE applies inference-time data augmentation to reduce noise in the predictions, further enhancing the robustness of this framework. Extensive experiments demonstrate that RIDE increases robust accuracy and maintains comparable standard accuracy.

Critical strengths
  1. The manuscript is clearly organized and easy to understand, with well-presented figures.
  2. The research problem is practically relevant. The paper focuses on improving adversarial robustness at inference time without modifying the target classifier, while seeking to preserve accuracy on clean samples.
  3. The proposed RIDE framework is computationally efficient, with relatively low reported inference overhead and resource requirements.
  4. The experiments cover a wide range of adversarial attacks and evaluation settings, providing a broad assessment of the proposed framework.
  5. The paper includes both theoretical and empirical analyses to support the design and effectiveness of the proposed method.
Critical weaknesses
  1. The evaluation protocol may overestimate the effectiveness of the proposed defense. The main experiments use a mixed test set containing 90% clean samples and 10% adversarial samples. Consequently, the reported accuracy is largely dominated by performance on clean samples, potentially obscuring the model's vulnerability to adversarial attacks. What motivates this evaluation setting instead of the conventional practice of reporting clean and robust accuracy separately? Although the paper includes results on fully adversarial samples for CIFAR-10, I suggest using standard robust accuracy as a primary evaluation metric and reporting it consistently across all datasets.
  2. The evaluation against adaptive attacks requires further investigation. In particular, it remains unclear whether the detection and purification components can be effectively attacked jointly. As shown in Table 4, RIDE performs noticeably worse than DiffPure under BPDA+EOT, which raises concerns about its robustness against adaptive adversaries. More rigorous end-to-end adaptive attack evaluations would help establish the reliability of the defense.
  3. The inference-time measurements need further clarification. RIDE involves multiple processing stages, including adversarial detection, iterative purification, and prediction enhancement. Does the inference time reported in Table 5 account for the complete pipeline, including all these operations? A more detailed breakdown of the computational costs would make the efficiency claims more convincing.
  4. The comparison with existing inference-time defenses is not sufficiently comprehensive. The primary competing method is DiffPure, which was published in 2022. Given the considerable progress in adversarial purification and inference-time defense in recent years, comparisons with more recent and competitive approaches are necessary to better establish the advantages of RIDE.
Questions for authors

See Weakness

Reviewer_aMx8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RIDE, a lightweight plug-and-play framework for improving adversarial robustness without retraining the target classifier. RIDE consists of three components: (1) a residual-aware detector that identifies adversarial examples based on their deviations in a low-variance subspace; (2) an adversarial filter that employs a denoising autoencoder (DAE) to purify detected adversarial examples by minimizing reconstruction error; and (3) a prediction enhancement module that aggregates predictions over randomly transformed inputs.

The authors provide theoretical analyses to motivate the detection and purification mechanisms. Experiments on four datasets against various adversarial attacks demonstrate improvements in accuracy and inference efficiency compared with existing defense methods.

Critical strengths

S1. Practically important problem. Balancing adversarial robustness, clean accuracy, and inference efficiency is an important challenge. Many purification-based defenses require substantial computational resources, while adversarial training may reduce clean accuracy. The proposed selective-purification strategy is motivated by a relevant practical trade-off.

S2. A reasonable detection-guided purification design. Rather than purifying every input, RIDE selectively processes samples identified as suspicious. This can reduce unnecessary modifications to clean samples and potentially lower average inference latency. Such a strategy is particularly attractive in deployment settings where the adversarial-example prevalence is low.

S3. Evaluation across multiple attacks and datasets. The experiments include FGSM, BIM, PGD, C&W, DeepFool, Square Attack, AutoAttack, Jitter, GAA, and MUMODIG. This provides useful coverage across several attack families. The inclusion of PGD+EOT and BPDA+EOT is also positive, as it demonstrates awareness of the need for adaptive evaluation of randomized and non-smooth defenses.

S4. Promising inference efficiency. Table 5 reports an inference time of 1.4664 ms/sample for RIDE, compared with 168.7433 ms/sample for DiffPure, together with substantially lower additional storage requirements.If these measurements are reproducible under a comprehensive and fair end-to-end protocol, the efficiency advantage could be practically valuable.

Critical weaknesses

W1. Insufficient adaptive attack evaluation. Although the authors evaluate RIDE using PGD+EOT and BPDA+EOT, it remains unclear whether these attacks effectively account for the entire defense pipeline, including detection, iterative purification, and randomized prediction enhancement. Without more detailed attack configurations and stronger adaptive evaluations, potential gradient masking cannot be ruled out.

W2. Gaps in theoretical justification. Theorem 1 establishes a concentration bound for clean data in the low-variance subspace but does not guarantee that adversarial examples necessarily exhibit larger deviations. Furthermore, Theorem 2 analyzes iterative DAE application, whereas the actual purification minimizes reconstruction error. The connection between the theoretical results and the proposed methods requires further justification.

W3. Potentially misleading robustness evaluation. The main experiments use a mixture of 90% clean and 10% adversarial samples. Consequently, the reported accuracy is heavily influenced by clean accuracy and does not directly represent adversarial robust accuracy. In particular, the claimed maximum improvement of 50.97% may overstate the actual robustness gain.

W4. Limited methodological novelty. RIDE primarily combines existing techniques, including PCA-based detection, DAE-based purification, and randomized prediction enhancement. Although the residual-subspace detector is potentially useful, its advantages over conventional PCA-based anomaly detection methods have not been sufficiently demonstrated, making the independent methodological contribution unclear.

Questions for authors

Q1. Although PGD+EOT and BPDA+EOT are evaluated, it is unclear whether these attacks optimize against the entire RIDE pipeline, including the detector, the 100-step purification process, and randomized prediction enhancement. Could the authors provide detailed attack configurations, including the number of iterations, EOT samples, random restarts, and gradient approximation strategies?

Q2. In the main experiments, are adversarial examples generated against the original classifier or the complete defended model? If attacks are generated against the undefended classifier, could the authors clarify how the evaluation accounts for an adaptive adversary with full knowledge of RIDE?

Q3. The main experiments use a mixture of 90% clean and 10% adversarial samples. Since the resulting accuracy is heavily influenced by clean accuracy, could the authors separately report robust accuracy on fully adversarial inputs? Additionally, how is the claimed maximum improvement of 50.97% calculated, and to what extent does it reflect genuine robustness improvement?

Q4. Theorem 1 establishes a concentration bound for clean data in the low-variance subspace, but it does not directly demonstrate that adversarial examples necessarily exhibit larger deviations in this subspace. Could the authors provide further theoretical or empirical evidence supporting this connection?

Q5. In Theorem 2, does (p(x)) represent the clean-data distribution or the noise-corrupted distribution? Furthermore, the theorem considers iterative DAE application, whereas Eq. (4) minimizes the reconstruction error. Could the authors explain why the theoretical guarantee remains applicable to the actual purification procedure?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.