acceptodds
Under review as a conference paper at ICLR 2027

Expected Adversarial Risk: A Data-Centric View of Adversarial Purification

Abstract

Adversarial purification has emerged as a promising defense strategy to mitigate adversarial attacks. Despite its effectiveness, the underlying mechanism remains unclear, making researchers resort to empirical algorithms rather than principled ones. In this paper, we propose a probabilistic, data-centric view of adversarial purification through the lens of Expected Adversarial Risk (EAR), which measures the probability that an example is assigned a wrong label across randomly sampled empirical datasets. We provide theoretical analysis for the effectiveness of adversarial purification, and attribute its success to alignment of purification directions with negative gradients of EAR. We further show that such alignment may not hold when severe class overlap or imbalance exist locally. Accordingly, we construct an EAR-inspired objective for purification, and demonstrate its effectiveness through experiments on synthetic and image datasets.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
adversarial purification, adversarial robustness, expected adversarial risk

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LKbGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Expected Adversarial Risk (EAR), a data-centric measure of adversarial vulnerability defined as the probability that a classifier trained on a randomly sampled dataset disagrees with the population-optimal label at a fixed input. By instantiating EAR with a fixed-radius nearest-neighbor classifier, the authors derive Gaussian approximations relating local classification instability to class-mass margins, sample size, and covariance structure. They further analyze when marginal-density and class-conditional-density gradients align with the negative EAR gradient, providing an explanation for the effectiveness and limitations of density-guided adversarial purification. Motivated by this analysis, they propose a simplified class-density-margin objective for purification. Experiments on synthetic Gaussian mixtures and CIFAR-10 report improvements over density-guided purification baselines.

Critical strengths

S1. The paper provides an interesting data-centric perspective on adversarial vulnerability. By defining EAR through training-set randomness, the work highlights finite-sample decision instability as a potential source of adversarial vulnerability. Its analysis of local class margins and effective sample size offers a useful perspective complementary to conventional classifier- and attack-centric explanations.

S2. The connection between local class geometry and purification directions is conceptually valuable. The distinction between marginal-density guidance and class-conditional guidance helps explain why increasing data density may improve classification stability in some regions but become problematic under class overlap or imbalance. This perspective provides useful motivation for investigating class-aware purification objectives.

Critical weaknesses

W1. The theoretical guarantees do not sufficiently establish the claimed relationship between EAR and effective adversarial purification. The main approximation is derived for a fixed-radius nearest-neighbor classifier, and the Berry–Esseen bound controls probability approximation errors rather than errors in the corresponding spatial gradients. Moreover, the sufficient alignment condition in Eq. (8) is obtained by neglecting cross-class gradient terms in Appendix C.5 under a qualitative dominance assumption, without a quantitative bound ensuring that these terms cannot reverse the alignment. Dominance of the margin-gradient term in norm also does not, by itself, guarantee alignment of the complete gradient with an arbitrary density-guidance direction. Finally, the relationship between EAR and conventional attack success rate in Appendix B relies on a strong, unvalidated assumption that adversarial events are nested according to EAR values. These limitations leave the claimed theoretical explanation of practical purification insufficiently established.

W2. The experimental evidence is insufficient to establish reliable adversarial robustness or isolate the proposed method's contribution. The CIFAR-10 evaluation uses only 512 images and non-adaptive APGD-CE examples generated against the undefended classifier. The reported 91.99% accuracy therefore does not establish robustness against attacks targeting the complete purification pipeline. Importantly, adaptive evaluation of purification defenses has been studied by Athalye et al. (ICML 2018) and Lee and Kim (ICCV 2023), neither of which is cited. The latter specifically analyzes the reliability of gradient-based attacks against diffusion purification. Furthermore, the implemented EAR-inspired objective reduces to a simple combination of class-conditional and marginal density terms. Its incremental benefit over existing class-aware purification methods, including the already cited COUP (Zhang et al., ECAI 2024), remains insufficiently isolated. Stronger adaptive evaluations and controlled comparisons are necessary to substantiate the practical contribution.

  1. Athalye, A., Carlini, N., and Wagner, D. (2018). Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. International Conference on Machine Learning (ICML), pp. 274–283.
  2. Lee, M., and Kim, D. (2023). Robust Evaluation of Diffusion-Based Adversarial Purification. IEEE/CVF International Conference on Computer Vision (ICCV).
  3. Croce, F., and Hein, M. (2020). Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse Parameter-Free Attacks. International Conference on Machine Learning (ICML).
Questions for authors

Q1. In Appendix C.5, how can the neglected cross-class terms be quantitatively bounded to ensure that Eq. (8) is a sufficient alignment condition? Can the authors establish gradient approximation error bounds, or otherwise justify transferring the Gaussian EAR analysis to actual neural-network purification?

Q2. Appendix B assumes that, for any fixed classifier, vulnerability at a lower-EAR point implies vulnerability at every higher-EAR point within the perturbation set. What theoretical or empirical evidence supports this assumption? Can the authors evaluate the relationship between EAR and actual attack success rates across independently trained neural classifiers?

Q3. Can the authors evaluate the complete CIFAR-10 purification pipeline using adaptive attacks that account for the purification steps, including appropriate gradient approximation or differentiation through the defense? Results on the full test set and across multiple random seeds would materially affect my assessment.

Q4. The final objective can be written as SEAR(x)=KK−1p^k^⋆(x)−1K−1p^(x)S_{\mathrm{EAR}}(x)=\frac{K}{K-1}\hat p_{\hat k^\star}(x)-\frac{1}{K-1}\hat p(x), where p^k^⋆\hat p_{\hat k^\star} is the estimated joint density of the selected class, p^\hat p is the estimated marginal density, and KK is the number of classes. What advantage does this formulation offer over simpler class-density-margin guidance or existing class-aware purification approaches under matched computational budgets?

5 comments

W1 / Q1. Gradient approximation and marginal-density alignment

Gaussian gradient agreement. The full Gaussian EAR gradient closely matched the independently estimated finite-sample EAR gradient on the numerically resolved CIFAR-10 queries: the median angle was approximately 0∘0^\circ, the 90th percentile was 9.15∘9.15^\circ, and no angle exceeded 90∘90^\circ. The Berry–Esseen bound controls probability error, rather than gradient error, so we evaluated gradient agreement directly. Under the feature-space setup of Appendix A.5, we randomly sampled 1,000 queries from 5,398 satisfying nP(Br(x))>20nP(B_r(x))>20 and Gaussian EAR >8.93×10−158>8.93\times10^{-158}, without an upper EAR cutoff. The KDE neighborhood probabilities define a multinomial model for the local class counts among n=40,000n=40{,}000 reference samples. Its EAR is the probability that any competing class receives at least as many local votes as the fixed reference class. We estimated the gradient of this probability using a multinomial gradient identity with conditional integration to reduce variance. Four independent MC estimates assessed sampling stability. Of the 1,000 pairs, 772 were resolved; the remaining 228 were excluded using numerical-degeneracy and estimation-stability criteria, independently of MC–Gaussian agreement. These results support the Gaussian gradient approximation on the resolved queries of this fitted feature-space neighborhood model.

Margin-term gradient agreement. Additional experiments on synthetic GMMs and on CIFAR-10 show that omitting the covariance term T2T_2 has little effect on the direction of the Gaussian EAR gradient for most evaluated queries. The median and 90th percentile of the angle between T1T_1 and T1+T2T_1+T_2 were approximately 0∘0^\circ and 5.51∘5.51^\circ on CIFAR-10, and 2.33∘2.33^\circ and 9.73∘9.73^\circ on the synthetic GMMs. This direct directional agreement supports using T1T_1 as an approximation to the full Gaussian EAR gradient in the evaluated models. On CIFAR-10, 999 of 1,000 comparisons were numerically resolved, and no angle exceeded 90∘90^\circ. Across 6,682 retained GMM queries, ∥T1∥>∥T2∥\lVert T_1\rVert>\lVert T_2\rVert held in 99.87% of queries.

The GMM experiment follows the paper’s synthetic construction, with five configurations of 2D, five-class GMMs and four components per class. Within each seed, configurations share component means and within-class mixture weights. Class priors are uniform or (0.40,0.25,0.17,0.11,0.07)(0.40,0.25,0.17,0.11,0.07); component covariances are scaled as shown below, with means unchanged. We draw 1,024 queries per configuration for each of four independent seeds, totaling 20,480 queries. With n=9,000n=9{,}000, r=0.12r=0.12, and continuity correction c=0.5c=0.5, we retain all 6,682 queries satisfying nP(Br(x))>20nP(B_r(x))>20 and EAR >0.01>0.01, without filtering by gradient norms or angles.

Class priorsCovariance scale∥T1∥>∥T2∥\lVert T_1\rVert>\lVert T_2\rVert (count/total)Median angle90th-percentile angle
Balanced1.00×1.00\times1,300 / 1,3002.26∘±0.11∘2.26^\circ\pm0.11^\circ8.38∘±0.85∘8.38^\circ\pm0.85^\circ
Imbalanced1.00×1.00\times1,143 / 1,1502.62∘±0.38∘2.62^\circ\pm0.38^\circ10.95∘±2.33∘10.95^\circ\pm2.33^\circ
Balanced1.60×1.60\times1,705 / 1,7052.03∘±0.13∘2.03^\circ\pm0.13^\circ8.21∘±1.40∘8.21^\circ\pm1.40^\circ
Imbalanced1.60×1.60\times1,486 / 1,4882.33∘±0.56∘2.33^\circ\pm0.56^\circ11.10∘±3.49∘11.10^\circ\pm3.49^\circ
Balanced0.65×0.65\times1,039 / 1,0392.75∘±0.29∘2.75^\circ\pm0.29^\circ9.71∘±0.59∘9.71^\circ\pm0.59^\circ

The table reports the mean ± sample standard deviation of within-seed angle statistics; counts are pooled across seeds. The overall GMM median of 2.33∘2.33^\circ and 90th percentile of 9.73∘9.73^\circ are calculated from pooled queries.

W1 / Q1. Gradient approximation and marginal-density alignment (continued)

Cross-class terms retained. We have clarified the alignment condition in Theorem 2 and revised its derivation in Appendix C.5 to explicitly retain the cross-class terms. This revision affects only the presentation of Theorem 2 and does not alter our core conclusions. Both revisions are highlighted in blue in the revised manuscript. Eq. (116) now serves as an interpretable approximation to the complete margin-term expression.

Cross-class contribution on CIFAR-10. We measured the magnitude of the aggregate cross-class contribution relative to the magnitude of the complete alignment inner product retaining all cross-class terms. This ratio was below 1% on 836 of 838 images (99.76%), with a maximum of 7.30%. Omitting the cross-class terms preserved every alignment sign. We evaluated this approximation on images that passed the prespecified Gaussian EAR probability-agreement check in the Appendix A.5 feature-space setup.

To quantify the effect of Eq. (116), we decomposed the marginal-density alignment inner product within the T1T_1 approximation. Let ui=∇z[πiqi(z)]u_i=\nabla_z[\pi_i q_i(z)] and let w~j\widetilde w_j denote the Gaussian EAR margin sensitivities normalized to sum to one. The complete inner product is I=⟨∑i=1Kui, uk⋆−∑j≠k⋆w~juj⟩=A+CI=\langle\sum_{i=1}^{K}u_i,\ u_{k^\star}-\sum_{j\ne k^\star}\widetilde w_j u_j\rangle=A+C, where A=∥uk⋆∥2−∑j≠k⋆w~j∥uj∥2A=\|u_{k^\star}\|^2-\sum_{j\ne k^\star}\widetilde w_j\|u_j\|^2 is the term retained in Eq. (116), and C=∑j≠k⋆w~j⟨∑i∉{k⋆,j}ui, uk⋆−uj⟩C=\sum_{j\ne k^\star}\widetilde w_j\langle\sum_{i\notin\{k^\star,j\}}u_i,\ u_{k^\star}-u_j\rangle is the cross-class contribution. For each competitor jj, the corresponding summand in CC measures how the density gradients of the remaining classes project onto the margin direction uk⋆−uju_{k^\star}-u_j. The relative change reported above is ∣C∣/∣I∣|C|/|I|. When ∣C∣/∣I∣|C|/|I| is small, the positive-alignment criterion I>0I>0 can be approximated by A>0A>0 using the simplified expression in Eq. (116).

W1/Q2 Relationship between ASR and EAR

Empirical support for shared vulnerabilities. In five GMM settings, given a misclassified point, another randomly selected point with no lower EAR was also misclassified at an average rate of 94.42%, consistent with the tendency described by Assumption 1. The two sides of Eq. (41) differed by 0.0192–0.0570 percentage points. These results support shared vulnerability structure across independently trained classifiers and approximate empirical agreement between ASR and EAR.

We trained 100 MLPs per setting, each on an independently sampled dataset of 9,000 examples, and independently sampled 10,000 ordinary test inputs. The five settings vary class balance and covariance scale. All classifiers use a 2–256–256–128–5 SiLU architecture, with initialization and validation data fixed to isolate training-set resampling. Each input has a 33×33 perturbation grid within a normalized-coordinate infinity-norm radius of 0.014. Valid perturbations retain the center’s known GMM Bayes label.

To evaluate Assumption 1, we identify correctly classified and misclassified grid points by comparing each classifier’s predictions with the known Bayes labels. We retain only classifier–input pairs containing both types of points; this selection does not require a correctly classified center. For each tested classifier, EAR at a point is estimated as the fraction of the other 99 classifiers that misclassify that same point.

We randomly select a misclassified point, then another valid point with no lower estimated EAR, and check whether the tested classifier also misclassifies the second point. We attempt this comparison 100 times per selected classifier–input pair. If no eligible second point exists, that comparison is omitted. We calculate the proportion of completed comparisons satisfying the implication for each pair, then average these proportions equally. We also exhaustively check for strict EAR reversals: a misclassified point paired with a correctly classified point having strictly higher estimated EAR. “No strict EAR reversal” reports the percentage of selected classifier–input pairs with no such pair anywhere on the valid grid. A pair with equal estimated EAR is not counted as a strict reversal, even if one point is misclassified and the other is correctly classified.

For Eq. (41), both sides use all 10,000 test inputs and all 100 classifiers, without the above filtering. The LHS is the mean ASR across classifiers. For the RHS, we estimate each point’s EAR using the same classifiers, maximize over valid perturbations for each input, and average over inputs.

GMM settingSampled satisfaction of Assumption 1No strict EAR reversalLHSRHSAbsolute difference (pp)
Balanced baseline94.7548%40.0585%10.6121%10.5929%0.0192
Imbalanced baseline94.5437%38.6493%10.2794%10.2403%0.0391
Balanced, wider covariance94.2023%36.4448%11.5383%11.4958%0.0425
Imbalanced, wider covariance93.9791%36.7746%10.6238%10.5668%0.0570
Balanced, narrower covariance94.6253%38.8349%9.7342%9.6891%0.0451

The equally weighted LHS and RHS means are 10.5576% and 10.5170%, respectively, differing by 0.0406 percentage points. Here, ASR uses the fixed-denominator neighborhood-error definition in Eq. (41), including already-misclassified centers.

Although Assumption 1 does not hold universally, its implication holds for most sampled point pairs in these experiments. If a classifier misclassifies one perturbation, it usually also misclassifies another perturbation that causes errors in at least as many independently trained classifiers. This suggests that independently trained classifiers tend to share vulnerable regions. Together with the close agreement in Eq. (41), these findings provide empirical support for using EAR to characterize ASR in the evaluated settings, even when exact nesting is relaxed.

Reviewer_2UqsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Expected Adversarial Risk (EAR, which characterizes the probability that a classifier's prediction deviates from the Bayes-optimal decision due to randomness in finite training data sampling.

Using a fixed-radius nearest-neighbor classifier and a high-dimensional Gaussian approximation, the authors analyze how EAR relates to local class probability masses, class margins, and effective sample size. They further investigate the alignment between the EAR gradient and the gradients of the marginal density and maximum class-conditional density, providing a theoretical explanation for why conventional density-guided adversarial purification works and under what conditions it may fail.

Building on these analyses, the authors propose a simplified objective based on the average class-density margin and demonstrate its effectiveness on synthetic datasets and CIFAR-10, achieving better performance than two density-guided baselines.

Critical strengths

S1. A Meaningful Research Question.

This paper aims to explain why probability density maximization can improve adversarial robustness and under what conditions this relationship may break down. This provides a more insightful explanation than simply proposing another generative model-based purification strategy.

S2. Solid Theoretical Analysis.

Theorem 1 establishes a connection between finite-sample-induced local classification instability and class margins, sample size, and covariance structure. Theorem 2 further investigates the relationship between density gradients and the EAR descent direction. These analyses provide useful insights into the mechanisms underlying existing adversarial purification methods.

S3. Promising Results on Both Synthetic and Real-World Datasets.

On synthetic datasets, the EAR-inspired objective achieves lower conditional attack success rates than both marginal-density and maximum class-conditional density guidance. Under transfer attacks on CIFAR-10, the proposed method achieves 91.99% accuracy on adversarial examples while maintaining 94.53% clean accuracy.

Critical weaknesses
  1. The motivation for the final purification objective is not sufficiently convincing.

My main concern is that the EAR-inspired objective in Eq. (11) is essentially an average class-density margin. From the perspective of certified robustness, maximizing the margin between the top two classes is already a natural choice. It is therefore unclear why the EAR framework is needed to motivate such an objective. Moreover, the second-largest class can change during iterative purification, meaning that optimizing the Top-2 margin can already account for different competing classes over multiple updates. I think the authors should at least compare their method with a simple Top-2 margin or soft-min margin objective under the same purification setup. Without these baselines, it is difficult to determine whether the improvement comes from the proposed EAR framework or simply from using a class-aware margin objective instead of maximizing probability density.

  1. There is a gap between the theoretical analysis and the final objective.

The derived EAR gradient depends on the margins between the dominant class and different competing classes, with different weights assigned according to their contributions to the risk. However, the final objective assigns equal weights to all competing classes and removes the covariance-related terms. In fact, Eq. (11) can be rewritten as ((Kp_{k^\star}(x)-p(x))/(K-1)), which means that it does not explicitly distinguish the strongest competing class from the others. This seems somewhat inconsistent with the earlier theoretical analysis, since EAR is concerned with the probability that a competing class overtakes the dominant one. Although Appendix D.2 includes ablation experiments on different levels of approximation, these results do not fully explain what unique benefits of the EAR formulation are retained by the final average-margin objective compared with a standard margin-based objective.

  1. Some theoretical claims require more rigorous justification.

In the proof of Theorem 2 (Appendix C.5), the authors omit the cross-class terms when moving from Eq. (115) to Eq. (116), based on the local class-conditional dominance assumption. However, this assumption is described mainly in qualitative terms, and the paper does not provide an explicit bound on the omitted terms. Since Theorem 2 claims to establish sufficient conditions for gradient alignment, these terms may matter. Even if they are relatively small, they could still change the sign of the inner product when the remaining terms are close to zero. I think the authors should provide more precise conditions under which these cross-class terms can be safely ignored, or clarify that the resulting conditions are approximate rather than rigorous sufficient conditions.

  1. The robustness evaluation is still limited.

The results on CIFAR-10 are promising, but the evaluation uses only 512 images and mainly considers transfer attacks, without adaptive attacks against the full purification pipeline. Although the synthetic experiments include adaptive attacks, they are performed against a distilled approximation of the purifier rather than the original iterative purification process. Given that the robustness of adversarial purification methods can be sensitive to the attack setup, the current experiments are not sufficient to support strong claims about robustness. An adaptive attack evaluation against the full purification pipeline on real images would make the experimental results more convincing.

Questions for authors

Q1. The final EAR-inspired objective in Eq. (11) is essentially an average class-density margin. Have the authors compared it with directly maximizing the Top-2 margin or a soft-min margin under the same purification setup? These seem like natural baselines from the perspective of certified robustness. Such a comparison would help clarify whether the improvement comes from the EAR framework or simply from optimizing class margins.

Q2. The theoretical EAR gradient assigns different weights to competing classes based on their contributions to the risk, but the final objective uses uniform weights. What justifies this choice? In particular, is there any theoretical or empirical evidence that averaging over all competing classes is preferable to focusing on the strongest competitors? The ablation in Appendix D.2 examines different approximations, but does not seem to directly address this question.

3 comments

Potential misunderstanding to clarify. We would like to first clarify that our main goal is not about proposing a novel algorithm such as Eq. (11). Instead, we propose a data-centric view (EAR) to explain the effectiveness and insufficiency of density-guided purification. Eq. (11) is a secondary contribution that illustrates how EAR can be used in practice to improve purification (without sacrificing too much efficiency). We agree that certified robustness motivates a similar top-2 margin-based form. However, certified robustness does not by itself establish a connection to adversarial purification. Our EAR analysis addresses this question by examining the alignment between density-guidance directions and EAR descent. Translating this analysis into practical guidance requires estimating densities, covariances, and sensitivity terms, which is challenging on real images. Eq. (11) therefore serves as a simplified practical instantiation that retains the competing-class margin structure identified by EAR while avoiding heavy computation of estimating covariance and sensitivity terms.

W1 / Q1. Comparison with dynamic Top-2

Eq. (11) outperforms dynamic Top-2. On CIFAR-10, this holds under both transfer and adaptive evaluation. The Top-2 baseline recomputes the strongest non-target competing class at every purification step in both evaluations. Eq. (11) improves adversarial accuracy by 3.51 percentage points under APGD-CE transfer and by 10.23 and 26.37 points on DiffHammer’s [2] average and worst accuracy metrics, respectively.

EvaluationAccuracy (%)Dynamic Top-2Eq. (11)
APGD-CE transferClean92.5894.53
APGD-CE transferAdversarial88.4891.99
Adaptive DiffHammerClean average / worst92.44 / 79.6995.63 / 90.23
Adaptive DiffHammerAdversarial average / worst69.22 / 31.0579.45 / 57.42

W2 / Q2. What Eq. (11) retains from EAR

New explanatory examples. We have revised Appendix D.2 of the manuscript, adding Figures 8 and 9 as two intuitive examples that directly illustrate what Eq. (11) retains from EAR: the correction provided by marginal-density subtraction and the simultaneous consideration of competitors beyond the runner-up.

Marginal-density subtraction accounts for competing-class density changes. The reviewer’s rewriting, (Kpk⋆−p)/(K−1)(Kp_{k^\star}-p)/(K-1), expresses the dominant-class density minus the average competing-class density. Figure 8 shows why this matters even under equal priors: a lower-density competitor can have a steeper local slope, so M(x)M(x) ascent can strengthen it faster than the dominant class. Subtracting the marginal-density gradient corrects this component while largely preserving the component that widens the margin. In this example, M(x)M(x) ascent locally increases Gaussian EAR, whereas Eq. (11) decreases it.

Runner-up switching does not generally recover simultaneous consideration of all competitors. We agree that dynamic Top-2 includes different competitors over successive updates. However, Figure 9 shows that switching can select opposing margin gradients and keep the trajectory near the switching surface, interrupting EAR reduction. Eq. (11) continues to include a competitor’s contribution after it drops to third place, allowing a small decrease in one margin for a substantially larger improvement in another. This reduces EAR in the illustrated mixture. Thus, although Eq. (11) simplifies EAR’s risk-sensitive weights and covariance terms, it retains contributions from all competing-class margins.

W3. Cross-class terms in Theorem 2

Cross-class terms retained. We have revised Theorem 2 (Eq. (8)) and its derivation in Appendix C.5 of the manuscript to retain all cross-class terms, while presenting Eq. (116) only as a simpler, more interpretable approximation. This revision affects only the presentation of Theorem 2 and does not alter our core conclusions. These changes are highlighted in blue in the revised manuscript. We agree that a small omitted contribution can still change the alignment sign when the retained contribution is close to zero. The revised condition therefore retains these contributions explicitly, and the theorem’s wording clarifies that the resulting alignment conditions under T1T_1 dominance are approximate rather than rigorous sufficient conditions for the full EAR gradient.

Cross-class contribution on CIFAR-10. Under the probability estimation setup in Appendix A.5, the magnitude of the aggregate cross-class contribution CC was below 1% of the magnitude of the complete alignment inner product retaining all cross-class terms on 836 of 838 evaluated images (99.76%). We measured both the contribution’s relative magnitude and its effect on the alignment sign.

Specifically, writing ui=∇z[πiqi(z)]u_i=\nabla_z[\pi_iq_i(z)], we computed I=⟨∑iui, uk⋆−∑j≠k⋆w~juj⟩=A+CI=\langle\sum_i u_i,\,u_{k^\star}-\sum_{j\ne k^\star}\widetilde w_j u_j\rangle=A+C, where A=∥uk⋆∥2−∑j≠k⋆w~j∥uj∥2A=\|u_{k^\star}\|^2-\sum_{j\ne k^\star}\widetilde w_j\|u_j\|^2 and C=∑j≠k⋆w~j⟨∑i∉{k⋆,j}ui, uk⋆−uj⟩C=\sum_{j\ne k^\star}\widetilde w_j\langle\sum_{i\notin\{k^\star,j\}}u_i,\,u_{k^\star}-u_j\rangle. Thus, CC is exactly the cross-class contribution omitted between Eqs. (115) and (116). The weights w~j\widetilde w_j are the normalized Gaussian EAR margin sensitivities. We evaluated E=∣C∣/∣I∣E=|C|/|I| and compared the signs of II and AA.

Following the setup in Appendix A.5, we evaluated 1,000 randomly sampled CIFAR-10 test images. Of these, 838 passed a prespecified empirical Gaussian-EAR reliability criterion: agreement with local-count Monte Carlo within 0.01, accounting for Monte Carlo and integration uncertainty. This screening did not depend on cross-class magnitude or alignment sign.

The median, 95th percentile, and maximum of EE were 1.03×10−5431.03\times10^{-543}, 8.38×10−188.38\times10^{-18}, and 7.30%7.30\%, respectively. The contribution was below 1%1\% at 836/838 points and below 5%5\% at 837/838 points. The complete inner product was positive at 837 points and negative at one point; omitting CC preserved every alignment sign. No inner products were numerically indistinguishable from zero under a scale-relative tolerance. We additionally verified I=A+CI=A+C, with a maximum scale-normalized discrepancy of 5.83×10−165.83\times10^{-16}, and checked all 10,000 class-density gradients against finite differences, obtaining a maximum relative error of 2.79×10−82.79\times10^{-8}. Extremely small contributions were recorded in the log domain to avoid underflow. These results support the simplified interpretation in the evaluated CIFAR-10 feature-density setting. The revised condition in Theorem 2 also accommodates configurations with substantial cross-class contributions by retaining them explicitly.

W4. Adaptive evaluation of the full iterative purifier

Adaptive attacks on the full pipeline. We have added adaptive attack evaluations against the full iterative purification pipeline on CIFAR-10 using DiffHammer and PGD+EOT; EAR achieved the highest adversarial accuracy among the compared purification objectives in both evaluations. We thank the reviewer for highlighting the need for this evaluation. To clarify the sample size, evaluating on 512 CIFAR-10 test images is also a practice adopted by existing purification methods, including COUP[3], DiffPure[4], and RDC[5].

DiffHammer [2]. EAR achieved 79.45% Avg.Rob and 57.42% Wor.Rob. We used an L∞L_\infty perturbation budget of 8/2558/255 and a WRN-70-16 downstream classifier.

Following DiffHammer, we use NN-evaluation with N=10N=10. Average robustness is Avg.Rob=1−1SN∑j=1S∑i=1NAi(j)\mathrm{Avg.Rob}=1-\frac{1}{SN}\sum_{j=1}^{S}\sum_{i=1}^{N}A_i^{(j)}, and worst-case robustness is Wor.Rob=1−1S∑j=1Smax⁡1≤i≤NAi(j)\mathrm{Wor.Rob}=1-\frac{1}{S}\sum_{j=1}^{S}\max_{1\le i\le N}A_i^{(j)}. Here, Ai(j)A_i^{(j)} indicates whether sample jj was misclassified at the ii-th evaluation, and S=512S=512 is the number of evaluated CIFAR-10 test images. To assess whether purification degrades classification performance on clean images, we additionally report average clean accuracy (Avg.clean) and worst-case clean accuracy (Wor.clean). Let P(x;ξi)P(x;\xi_i) denote the purified output under seed ξi\xi_i, and ff the downstream classifier. For each clean image xjx_j with label yjy_j, define Bi(j)=1 ⁣[f(P(xj;ξi))≠yj]B_i^{(j)}=\mathbf{1}\!\left[f(P(x_j;\xi_i))\neq y_j\right]. Average clean accuracy is Avg.clean=1−1SN∑j=1S∑i=1NBi(j)\mathrm{Avg.clean}=1-\frac{1}{SN}\sum_{j=1}^{S}\sum_{i=1}^{N}B_i^{(j)}, and worst-case clean accuracy is Wor.clean=1−1S∑j=1Smax⁡1≤i≤NBi(j)\mathrm{Wor.clean}=1-\frac{1}{S}\sum_{j=1}^{S}\max_{1\le i\le N}B_i^{(j)}.

PurificationAvg.cleanWor.cleanAvg.RobWor.Rob
p(x)p(x)87.30%72.85%28.42%15.23%
M(x)M(x)90.18%76.56%71.82%42.19%
EAR95.63%90.23%79.45%57.42%

We used three APGD restarts with CW, CE, and DLR losses, respectively, each for up to 50 iterations. Each attack candidate was evaluated under 10 distinct purification seeds. For each image, average robustness is one minus the largest misclassification fraction across evaluated candidates, with each fraction computed over the 10 seeds. Worst-case robustness counts an image as non-robust if any evaluated candidate causes misclassification under any of those seeds. These evaluations assess sensitivity to purification randomness throughout the attack history and are distinct from independent repetitions of the complete attack.

PGD+EOT [1]. EAR also achieved the highest adversarial accuracy when differentiating through the full purification process.

PurificationClean accuracyRobust accuracy
p(x)p(x)84.18%42.77%
M(x)M(x)89.84%28.13%
EAR95.12%72.85%

We used one 200-step PGD run initialized at the clean input, with a step size of 0.0070.007, differentiating through all five Adam purification updates and their optimizer-state dependencies without BPDA. Due to rebuttal time constraints, we used EOT =5=5 rather than 2020.

[1] Lee and Kim. Robust Evaluation of Diffusion-Based Adversarial Purification. ICCV, 2023.

[2] DiffHammer: Rethinking the Robustness of Diffusion-Based Adversarial Purification. NeurIPS, 2024.

[3] Mingkun Zhang, Jianing Li, Wei Chen, Jiafeng Guo, and Xueqi Cheng. Classifier Guidance Enhances Diffusion-Based Adversarial Purification by Preserving Predictive Information. ECAI, 2024.

[4] Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Anima Anandkumar. Diffusion Models for Adversarial Purification. ICML, 2022.

[5] Huanran Chen, Yinpeng Dong, Zhengyi Wang, Xiao Yang, Chengqi Duan, Hang Su, and Jun Zhu. Robust Classification via a Single Diffusion Model. ICML, 2024.

Reviewer_eZX1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates adversarial purification from a data-centric perspective. The authors propose Expected Adversarial Risk (EAR), which measures the probability that a classifier trained on a randomly sampled dataset disagrees with the Bayes-optimal label. Under a fixed-radius nearest-neighbor model, the authors derive a Gaussian approximation for EAR and analyze the conditions under which density-guided purification methods can effectively reduce it. Based on this analysis, the authors propose a class-margin-based objective function for purification. Experimental results on both synthetic data and the CIFAR-10 dataset demonstrate that this approach outperforms traditional density-guided purification schemes.

Critical strengths
  • The proposed data-centric perspective is interesting. It provides a different explanation for adversarial purification from existing analyses based on generative models.
  • The theoretical analysis provides insights into when density-guided purification works and when it may fail, particularly under class overlap and imbalance.
Critical weaknesses
  • The relationship between EAR and conventional adversarial risk is unclear. EAR measures the probability that a classifier trained on a random dataset disagrees with the Bayes-optimal classifier at a fixed input. This is essentially a form of finite-sample classification error. For instance, a classifier that perfectly aligns with the Bayes-optimal classifier across all input points would have an EAR of zero; however, this does not necessarily imply robustness against adversarial perturbations.

  • The theoretical analysis relies on a simplified classifier model and several additional assumptions. While the definition of EAR is applicable to general model families, the core theoretical results are derived specifically for a "fixed-radius nearest-neighbor classifier." Furthermore, the subsequent gradient analysis depends on conditions such as "local class-conditional dominance" and the "dominance of the margin term." It remains uncertain whether these conditions hold for neural networks.

  • The authors claim that there are currently no reliable adaptive attack methods for diffusion-based purification. This claim overlooks some existing studies [1] [2]. It would be better to evaluate the proposd method against stronger adaptive attacks before drawing conclusions about its robustness.

[1] Lee, Minjong, and Dongwoo Kim. "Robust evaluation of diffusion-based adversarial purification." 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023.

[2] Wang, Kaibo, et al. "Diffhammer: Rethinking the robustness of diffusion-based adversarial purification." Advances in Neural Information Processing Systems 37 (2024): 89535-89562.

Questions for authors
  1. Can the authors establish a more rigorous relationship between EAR and conventional adversarial risk? In particular, does reducing EAR necessarily improve robustness against small adversarial perturbations?
  2. Does optimizing the surrogate objective in Equation (11) necessarily reduce EAR?
  3. Have the authors tried adaptive attacks against the actual diffusion-based purifier?
3 comments

W1/Q1 Relationship between EAR and adversarial risk

EAR concerns training-set randomness. It averages disagreement with the Bayes label over repeated draws of the training set, with the classifier family and learning rule held fixed. A single fitted classifier agreeing with the Bayes classifier does not establish zero EAR, because EAR measures disagreement at a fixed input across training-set draws, and under the fixed-radius voting rule it remains nonzero at decision-boundary points with positive local probability mass from a competing class. EAR is therefore a pointwise probability over training datasets, whereas ASR aggregates neighborhood attack events for a fitted classifier.

EAR–ASR relationship. We have discussed EAR-ASR relationship in Appendix B (Proposition 3), i.e., under some assumptions, the expectation of ASR (over randomly sampled training set) equals the expectation of neighborhood maximum EAR (over test inputs). For further discussion of Assumption 1, please see our response to Reviewer LKbG (W1/Q2).

W2 · Reference model and gradient assumptions

Reference model. Fixed-radius nearest neighbors make EAR analytically tractable, and the resulting spatial risk patterns also appear in independently trained MLPs. To quantify EAR, we must specify how a classifier is obtained from each sampled training set. We use fixed-radius nearest neighbors as a simple nonparametric reference model because its predictions depend on local class counts. Appendix A.6 compares analytical EAR with empirical MLP EAR across five 2D, three-class GMM settings, using 800 independently drawn training sets per setting. Both measures place high-EAR regions near decision boundaries.

Margin-term dominance. Additional experiments on synthetic GMMs and on CIFAR-10 show that omitting the covariance term T2T_2 has little effect on the direction of the Gaussian EAR gradient for most evaluated queries. The median and 90th-percentile angles between T1T_1 and the full Gaussian EAR gradient T1+T2T_1+T_2 were approximately 0∘0^\circ and 5.51∘5.51^\circ in the CIFAR-10 feature space, and 2.33∘2.33^\circ and 9.73∘9.73^\circ on the synthetic GMMs. This directional agreement supports using T1T_1 to approximate the full Gaussian EAR gradient in the evaluated settings. On CIFAR-10, we used the feature-space density estimator from Appendix A.5. Of 1,000 sampled queries, 999 yielded numerically reliable angle estimates, all at or below 90∘90^\circ. Across the 6,682 retained GMM queries, ∥T1∥>∥T2∥\lVert T_1\rVert>\lVert T_2\rVert held on 99.87% of queries.

The GMM experiment follows the paper’s synthetic construction. We evaluate five configurations of 2D, five-class GMMs with four components per class. Within each seed, the configurations share component means and within-class mixture weights. Class priors are uniform or (0.40,0.25,0.17,0.11,0.07)(0.40,0.25,0.17,0.11,0.07), and each component’s baseline covariance is multiplied by the factor shown below. We draw 1,024 queries per configuration for each of four independent seeds, totaling 20,480 queries. With n=9,000n=9{,}000, r=0.12r=0.12, and continuity correction c=0.5c=0.5, the inclusion criteria are nP(X∈Br(x))>20nP(X\in B_r(x))>20 and EAR >0.01>0.01. We retain all 6,682 queries meeting these criteria.

Class priorsGMM component covariance multiplier∥T1∥>∥T2∥\lVert T_1\rVert>\lVert T_2\rVert (count/total)Median angle90th-percentile angle
Balanced1.00×1.00\times1,300 / 1,3002.26∘±0.11∘2.26^\circ\pm0.11^\circ8.38∘±0.85∘8.38^\circ\pm0.85^\circ
Imbalanced1.00×1.00\times1,143 / 1,1502.62∘±0.38∘2.62^\circ\pm0.38^\circ10.95∘±2.33∘10.95^\circ\pm2.33^\circ
Balanced1.60×1.60\times1,705 / 1,7052.03∘±0.13∘2.03^\circ\pm0.13^\circ8.21∘±1.40∘8.21^\circ\pm1.40^\circ
Imbalanced1.60×1.60\times1,486 / 1,4882.33∘±0.56∘2.33^\circ\pm0.56^\circ11.10∘±3.49∘11.10^\circ\pm3.49^\circ
Balanced0.65×0.65\times1,039 / 1,0392.75∘±0.29∘2.75^\circ\pm0.29^\circ9.71∘±0.59∘9.71^\circ\pm0.59^\circ

The table reports the mean ± sample standard deviation of within-seed angle statistics. Counts and the overall GMM angles reported above are computed from pooled queries.

W2 · Reference model and gradient assumptions (continued)

Cross-class terms retained. We revised Theorem 2 and its derivation in Appendix C.5 to retain the cross-class terms explicitly. This revision clarifies the presentation of Theorem 2 and preserves our main conclusions. Both manuscript revisions are highlighted in blue. Eq. (116) presents the simplified margin-term expression. On CIFAR-10, we measured the magnitude of the aggregate cross-class contribution relative to the magnitude of the complete alignment inner product retaining all cross-class terms. We evaluated 838 images that passed the prespecified Gaussian EAR probability-agreement check in the Appendix A.5 feature space. This ratio was below 1% on 836 images (99.76%), with a maximum of 7.30%. Omitting the cross-class terms preserved the alignment sign on all 838 images.

Within the T1T_1 approximation, let ui=∇z[πiqi(z)]u_i=\nabla_z[\pi_iq_i(z)] and let w~j\widetilde w_j denote the Gaussian EAR margin sensitivities normalized to sum to one. The complete marginal-density alignment inner product is I=⟨∑i=1Kui, uk⋆−∑j≠k⋆w~juj⟩=A+CI=\langle\sum_{i=1}^{K}u_i,\,u_{k^\star}-\sum_{j\ne k^\star}\widetilde w_j u_j\rangle=A+C. Here A=∥uk⋆∥2−∑j≠k⋆w~j∥uj∥2A=\lVert u_{k^\star}\rVert^2-\sum_{j\ne k^\star}\widetilde w_j\lVert u_j\rVert^2 is the term retained in Eq. (116), and C=∑j≠k⋆w~j⟨∑i∉{k⋆,j}ui, uk⋆−uj⟩C=\sum_{j\ne k^\star}\widetilde w_j\langle\sum_{i\notin\{k^\star,j\}}u_i,\,u_{k^\star}-u_j\rangle is the aggregate cross-class contribution. For each competitor jj, its summand in CC measures how the remaining classes’ density gradients project onto the margin direction uk⋆−uju_{k^\star}-u_j. We report the relative change as ∣C∣/∣I∣|C|/|I|. When ∣C∣/∣I∣|C|/|I| is small, the positive-alignment criterion I>0I>0 can be approximated by A>0A>0 using the simplified expression in Eq. (116).

Q2 · Does optimizing Eq. (11) reduce EAR?

Observed EAR reduction. Our main synthetic experiments show that Eq. (11)-guided purification reduces mean EAR across all three settings (Sec. 5.1, Table 1). Under transfer attack, mean EAR decreased from 46.82% to 31.08% with the iterative purifier and from 46.82% to 32.14% with the distilled purifier. Under adaptive attack, it decreased from 46.45% to 31.64% with the distilled purifier.

From Gaussian EAR to Eq. (11). Appendix D.2 and Table 4 trace the simplifications from the full Gaussian EAR objective to Eq. (11) and compare their empirical EAR reductions. On real images, local covariances and Gaussian sensitivity weights are difficult to estimate accurately. Eq. (11) retains the average class-density margin and provides a tractable EAR-motivated guidance direction.

W3/Q3 · Adaptive evaluation of the full CIFAR-10 purification pipeline

Adaptive evaluation of the full pipeline. We evaluated the full five-update diffusion-based purifier and its downstream classifier under two suggested attacks, and EAR achieved the highest robust accuracy among the compared objectives under both.

DiffHammer [2]. EAR achieved 79.45% Avg.Rob and 57.42% Wor.Rob. For DiffHammer, we used an L∞L_\infty perturbation budget of 8/2558/255 and a WRN-70-16 downstream classifier.

Following DiffHammer, we use NN-evaluation with N=10N=10. Average robustness is Avg.Rob=1−1SN∑j=1S∑i=1NAi(j)\mathrm{Avg.Rob}=1-\frac{1}{SN}\sum_{j=1}^{S}\sum_{i=1}^{N}A_i^{(j)}, and worst-case robustness is Wor.Rob=1−1S∑j=1Smax⁡1≤i≤NAi(j)\mathrm{Wor.Rob}=1-\frac{1}{S}\sum_{j=1}^{S}\max_{1\leq i\leq N}A_i^{(j)}. Here, Ai(j)A_i^{(j)} indicates whether sample jj was misclassified at the ii-th evaluation, and S=512S=512 is the number of evaluated CIFAR-10 test images. To assess whether purification degrades classification performance on clean images, we additionally report average clean accuracy (Avg.clean) and worst-case clean accuracy (Wor.clean). Let P(x;ξi)P(x;\xi_i) denote the purified output under seed ξi\xi_i, and ff the downstream classifier. For each clean image xjx_j with label yjy_j, define Bi(j)=1 ⁣[f(P(xj;ξi))≠yj]B_i^{(j)}=\mathbf{1}\!\left[f(P(x_j;\xi_i))\neq y_j\right]. Average clean accuracy is Avg.clean=1−1SN∑j=1S∑i=1NBi(j)\mathrm{Avg.clean}=1-\frac{1}{SN}\sum_{j=1}^{S}\sum_{i=1}^{N}B_i^{(j)}, and worst-case clean accuracy is Wor.clean=1−1S∑j=1Smax⁡1≤i≤NBi(j)\mathrm{Wor.clean}=1-\frac{1}{S}\sum_{j=1}^{S}\max_{1\leq i\leq N}B_i^{(j)}.

We used three APGD restarts with CW, CE, and DLR losses, respectively, each for up to 50 iterations. Each attack candidate was evaluated under 10 distinct purification seeds. For each image, average robustness is one minus the largest misclassification fraction across evaluated candidates, with each fraction computed over the 10 seeds. Worst-case robustness counts an image as non-robust if any evaluated candidate causes misclassification under any of those seeds. These evaluations assess sensitivity to purification randomness throughout the attack history and are distinct from independent repetitions of the complete attack.

PurificationAvg.cleanWor.cleanAvg.RobWor.Rob
p(x)p(x)87.30%72.85%28.42%15.23%
M(x)M(x)90.18%76.56%71.82%42.19%
EAR95.63%90.23%79.45%57.42%

PGD+EOT [1]. EAR achieved 72.85% robust accuracy when the attack differentiated through all five Adam purification updates, including their optimizer-state dependencies. We ran 200 PGD steps from the clean input with a step size of 0.0070.007. We use EOT=5 instead of EOT=20, since the latter requires more time than the rebuttal phase can provide.

PurificationClean accuracyRobust accuracy
p(x)p(x)84.18%42.77%
M(x)M(x)89.84%28.13%
EAR95.12%72.85%

[1] Lee and Kim. Robust Evaluation of Diffusion-Based Adversarial Purification. ICCV, 2023.
[2] Wang et al. DiffHammer: Rethinking the Robustness of Diffusion-Based Adversarial Purification. NeurIPS, 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.