- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Expected Adversarial Risk (EAR), a data-centric measure of adversarial vulnerability defined as the probability that a classifier trained on a randomly sampled dataset disagrees with the population-optimal label at a fixed input. By instantiating EAR with a fixed-radius nearest-neighbor classifier, the authors derive Gaussian approximations relating local classification instability to class-mass margins, sample size, and covariance structure. They further analyze when marginal-density and class-conditional-density gradients align with the negative EAR gradient, providing an explanation for the effectiveness and limitations of density-guided adversarial purification. Motivated by this analysis, they propose a simplified class-density-margin objective for purification. Experiments on synthetic Gaussian mixtures and CIFAR-10 report improvements over density-guided purification baselines.
Critical strengths
S1. The paper provides an interesting data-centric perspective on adversarial vulnerability. By defining EAR through training-set randomness, the work highlights finite-sample decision instability as a potential source of adversarial vulnerability. Its analysis of local class margins and effective sample size offers a useful perspective complementary to conventional classifier- and attack-centric explanations.
S2. The connection between local class geometry and purification directions is conceptually valuable. The distinction between marginal-density guidance and class-conditional guidance helps explain why increasing data density may improve classification stability in some regions but become problematic under class overlap or imbalance. This perspective provides useful motivation for investigating class-aware purification objectives.
Critical weaknesses
W1. The theoretical guarantees do not sufficiently establish the claimed relationship between EAR and effective adversarial purification. The main approximation is derived for a fixed-radius nearest-neighbor classifier, and the Berry–Esseen bound controls probability approximation errors rather than errors in the corresponding spatial gradients. Moreover, the sufficient alignment condition in Eq. (8) is obtained by neglecting cross-class gradient terms in Appendix C.5 under a qualitative dominance assumption, without a quantitative bound ensuring that these terms cannot reverse the alignment. Dominance of the margin-gradient term in norm also does not, by itself, guarantee alignment of the complete gradient with an arbitrary density-guidance direction. Finally, the relationship between EAR and conventional attack success rate in Appendix B relies on a strong, unvalidated assumption that adversarial events are nested according to EAR values. These limitations leave the claimed theoretical explanation of practical purification insufficiently established.
W2. The experimental evidence is insufficient to establish reliable adversarial robustness or isolate the proposed method's contribution. The CIFAR-10 evaluation uses only 512 images and non-adaptive APGD-CE examples generated against the undefended classifier. The reported 91.99% accuracy therefore does not establish robustness against attacks targeting the complete purification pipeline. Importantly, adaptive evaluation of purification defenses has been studied by Athalye et al. (ICML 2018) and Lee and Kim (ICCV 2023), neither of which is cited. The latter specifically analyzes the reliability of gradient-based attacks against diffusion purification. Furthermore, the implemented EAR-inspired objective reduces to a simple combination of class-conditional and marginal density terms. Its incremental benefit over existing class-aware purification methods, including the already cited COUP (Zhang et al., ECAI 2024), remains insufficiently isolated. Stronger adaptive evaluations and controlled comparisons are necessary to substantiate the practical contribution.
- Athalye, A., Carlini, N., and Wagner, D. (2018). Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. International Conference on Machine Learning (ICML), pp. 274–283.
- Lee, M., and Kim, D. (2023). Robust Evaluation of Diffusion-Based Adversarial Purification. IEEE/CVF International Conference on Computer Vision (ICCV).
- Croce, F., and Hein, M. (2020). Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse Parameter-Free Attacks. International Conference on Machine Learning (ICML).
Questions for authors
Q1. In Appendix C.5, how can the neglected cross-class terms be quantitatively bounded to ensure that Eq. (8) is a sufficient alignment condition? Can the authors establish gradient approximation error bounds, or otherwise justify transferring the Gaussian EAR analysis to actual neural-network purification?
Q2. Appendix B assumes that, for any fixed classifier, vulnerability at a lower-EAR point implies vulnerability at every higher-EAR point within the perturbation set. What theoretical or empirical evidence supports this assumption? Can the authors evaluate the relationship between EAR and actual attack success rates across independently trained neural classifiers?
Q3. Can the authors evaluate the complete CIFAR-10 purification pipeline using adaptive attacks that account for the purification steps, including appropriate gradient approximation or differentiation through the defense? Results on the full test set and across multiple random seeds would materially affect my assessment.
Q4. The final objective can be written as , where is the estimated joint density of the selected class, is the estimated marginal density, and is the number of classes. What advantage does this formulation offer over simpler class-density-margin guidance or existing class-aware purification approaches under matched computational budgets?
5 comments
W1 / Q1. Gradient approximation and marginal-density alignment (continued)
Cross-class terms retained. We have clarified the alignment condition in Theorem 2 and revised its derivation in Appendix C.5 to explicitly retain the cross-class terms. This revision affects only the presentation of Theorem 2 and does not alter our core conclusions. Both revisions are highlighted in blue in the revised manuscript. Eq. (116) now serves as an interpretable approximation to the complete margin-term expression.
Cross-class contribution on CIFAR-10. We measured the magnitude of the aggregate cross-class contribution relative to the magnitude of the complete alignment inner product retaining all cross-class terms. This ratio was below 1% on 836 of 838 images (99.76%), with a maximum of 7.30%. Omitting the cross-class terms preserved every alignment sign. We evaluated this approximation on images that passed the prespecified Gaussian EAR probability-agreement check in the Appendix A.5 feature-space setup.
To quantify the effect of Eq. (116), we decomposed the marginal-density alignment inner product within the approximation. Let and let denote the Gaussian EAR margin sensitivities normalized to sum to one. The complete inner product is , where is the term retained in Eq. (116), and is the cross-class contribution. For each competitor , the corresponding summand in measures how the density gradients of the remaining classes project onto the margin direction . The relative change reported above is . When is small, the positive-alignment criterion can be approximated by using the simplified expression in Eq. (116).
W1/Q2 Relationship between ASR and EAR
Empirical support for shared vulnerabilities. In five GMM settings, given a misclassified point, another randomly selected point with no lower EAR was also misclassified at an average rate of 94.42%, consistent with the tendency described by Assumption 1. The two sides of Eq. (41) differed by 0.0192–0.0570 percentage points. These results support shared vulnerability structure across independently trained classifiers and approximate empirical agreement between ASR and EAR.
We trained 100 MLPs per setting, each on an independently sampled dataset of 9,000 examples, and independently sampled 10,000 ordinary test inputs. The five settings vary class balance and covariance scale. All classifiers use a 2–256–256–128–5 SiLU architecture, with initialization and validation data fixed to isolate training-set resampling. Each input has a 33×33 perturbation grid within a normalized-coordinate infinity-norm radius of 0.014. Valid perturbations retain the center’s known GMM Bayes label.
To evaluate Assumption 1, we identify correctly classified and misclassified grid points by comparing each classifier’s predictions with the known Bayes labels. We retain only classifier–input pairs containing both types of points; this selection does not require a correctly classified center. For each tested classifier, EAR at a point is estimated as the fraction of the other 99 classifiers that misclassify that same point.
We randomly select a misclassified point, then another valid point with no lower estimated EAR, and check whether the tested classifier also misclassifies the second point. We attempt this comparison 100 times per selected classifier–input pair. If no eligible second point exists, that comparison is omitted. We calculate the proportion of completed comparisons satisfying the implication for each pair, then average these proportions equally. We also exhaustively check for strict EAR reversals: a misclassified point paired with a correctly classified point having strictly higher estimated EAR. “No strict EAR reversal” reports the percentage of selected classifier–input pairs with no such pair anywhere on the valid grid. A pair with equal estimated EAR is not counted as a strict reversal, even if one point is misclassified and the other is correctly classified.
For Eq. (41), both sides use all 10,000 test inputs and all 100 classifiers, without the above filtering. The LHS is the mean ASR across classifiers. For the RHS, we estimate each point’s EAR using the same classifiers, maximize over valid perturbations for each input, and average over inputs.
| GMM setting | Sampled satisfaction of Assumption 1 | No strict EAR reversal | LHS | RHS | Absolute difference (pp) |
|---|---|---|---|---|---|
| Balanced baseline | 94.7548% | 40.0585% | 10.6121% | 10.5929% | 0.0192 |
| Imbalanced baseline | 94.5437% | 38.6493% | 10.2794% | 10.2403% | 0.0391 |
| Balanced, wider covariance | 94.2023% | 36.4448% | 11.5383% | 11.4958% | 0.0425 |
| Imbalanced, wider covariance | 93.9791% | 36.7746% | 10.6238% | 10.5668% | 0.0570 |
| Balanced, narrower covariance | 94.6253% | 38.8349% | 9.7342% | 9.6891% | 0.0451 |
The equally weighted LHS and RHS means are 10.5576% and 10.5170%, respectively, differing by 0.0406 percentage points. Here, ASR uses the fixed-denominator neighborhood-error definition in Eq. (41), including already-misclassified centers.
Although Assumption 1 does not hold universally, its implication holds for most sampled point pairs in these experiments. If a classifier misclassifies one perturbation, it usually also misclassifies another perturbation that causes errors in at least as many independently trained classifiers. This suggests that independently trained classifiers tend to share vulnerable regions. Together with the close agreement in Eq. (41), these findings provide empirical support for using EAR to characterize ASR in the evaluated settings, even when exact nesting is relaxed.
W1 / Q1. Gradient approximation and marginal-density alignment
Gaussian gradient agreement. The full Gaussian EAR gradient closely matched the independently estimated finite-sample EAR gradient on the numerically resolved CIFAR-10 queries: the median angle was approximately 0∘, the 90th percentile was 9.15∘, and no angle exceeded 90∘. The Berry–Esseen bound controls probability error, rather than gradient error, so we evaluated gradient agreement directly. Under the feature-space setup of Appendix A.5, we randomly sampled 1,000 queries from 5,398 satisfying nP(Br(x))>20 and Gaussian EAR >8.93×10−158, without an upper EAR cutoff. The KDE neighborhood probabilities define a multinomial model for the local class counts among n=40,000 reference samples. Its EAR is the probability that any competing class receives at least as many local votes as the fixed reference class. We estimated the gradient of this probability using a multinomial gradient identity with conditional integration to reduce variance. Four independent MC estimates assessed sampling stability. Of the 1,000 pairs, 772 were resolved; the remaining 228 were excluded using numerical-degeneracy and estimation-stability criteria, independently of MC–Gaussian agreement. These results support the Gaussian gradient approximation on the resolved queries of this fitted feature-space neighborhood model.
Margin-term gradient agreement. Additional experiments on synthetic GMMs and on CIFAR-10 show that omitting the covariance term T2 has little effect on the direction of the Gaussian EAR gradient for most evaluated queries. The median and 90th percentile of the angle between T1 and T1+T2 were approximately 0∘ and 5.51∘ on CIFAR-10, and 2.33∘ and 9.73∘ on the synthetic GMMs. This direct directional agreement supports using T1 as an approximation to the full Gaussian EAR gradient in the evaluated models. On CIFAR-10, 999 of 1,000 comparisons were numerically resolved, and no angle exceeded 90∘. Across 6,682 retained GMM queries, ∥T1∥>∥T2∥ held in 99.87% of queries.
The GMM experiment follows the paper’s synthetic construction, with five configurations of 2D, five-class GMMs and four components per class. Within each seed, configurations share component means and within-class mixture weights. Class priors are uniform or (0.40,0.25,0.17,0.11,0.07); component covariances are scaled as shown below, with means unchanged. We draw 1,024 queries per configuration for each of four independent seeds, totaling 20,480 queries. With n=9,000, r=0.12, and continuity correction c=0.5, we retain all 6,682 queries satisfying nP(Br(x))>20 and EAR >0.01, without filtering by gradient norms or angles.
The table reports the mean ± sample standard deviation of within-seed angle statistics; counts are pooled across seeds. The overall GMM median of 2.33∘ and 90th percentile of 9.73∘ are calculated from pooled queries.