- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates adversarial robustness at the intermediate-neuron level. The authors propose Neuron Sensitivity Attribution (NSA), which applies DeepLIFT to paired clean and adversarial inputs and scores each neuron by subtracting its attribution to the adversarial-class logit from its attribution to the ground-truth-class logit. These class-wise scores are then used in Sensitivity-Aware Neuron Recalibration (SANR), a weighted feature-alignment regularizer incorporated into adversarial training. On CIFAR-10, neuron-masking and Grad-CAM analyses are conducted with ResNet-18 and VGG-16. Integrating SANR into AT, TRADES, and DKL generally produces modest improvements in adversarial robustness.
Critical strengths
- The paper targets a relevant gap in activation-change-based neuron sensitivity analysis. Measuring how changes in intermediate neurons relate to the true-versus-adversarial class decision offers a useful class-conditional perspective on adversarial vulnerability, beyond activation magnitude alone [2].
- SANR is a straightforward regularization component compatible with multiple adversarial training objectives. Table 1 shows improvements on both tested architectures when it is added to AT, TRADES, and DKL, including under PGD-100 and AutoAttack rather than only a single attack.
- The authors examine a meaningful limitation of hard masking. Section 4 and the class-wise visualizations show that removing neurons identified as sensitive can help certain attacked inputs but also substantially reduce clean accuracy, which motivates a softer training-time intervention.
Critical weaknesses
- The novelty of NSA relative to existing attribution methods is insufficiently established. Section 3.1 and Equation (4) compute the difference between two DeepLIFT attributions of the same clean/adversarial input pair [1]. This score measures contributions to the change in the ground-truth-minus-adversarial logit margin. The paper does not establish what substantive information its dual-target construction contributes beyond attributing that margin directly, or how it advances existing class-aware neuron-importance approaches [3]. A matched comparison with direct margin attribution would clarify the methodological contribution.
- The contribution of NSA-derived weights to the reported robustness gains is not convincingly isolated. In Appendix Table 5, standard feature alignment (lambda = 0) already achieves 49.95% AutoAttack accuracy on ResNet-18 with DKL, compared with 50.19% using the proposed weighting (lambda = 1). This is a 0.24-point incremental improvement in that setting. Random or class-shuffled weights and simpler neuron scores should be compared under the identical recalibration loss to determine whether the specific NSA scores matter. The overall improvements in Table 1 are modest, and training evaluations are limited to CIFAR-10 and two CNN architectures.
- The neuron-masking experiments do not establish the claimed causal robustness mechanism. Section 4 uses ground-truth-class masks; Appendix A.3 states that adversarial examples for the visual analyses were generated against the original, unmasked network. Therefore, restored accuracy after masking does not demonstrate robustness to attacks adapted to the masked model or to a label-blind deployment setting. These experiments support NSA as a diagnostic ranking, but the claim that its identified neurons are the primary causal targets of attacks is stronger than the evidence warrants. Grad-CAM realignment alone does not bridge this gap.
- The sensitivity definition and attribution-to-training procedure need clarification. Section 3.1.2 defines the adversarial target as argmax f(x_adv). If an attacked input remains correctly classified, this target equals the ground-truth class and Equation (4) becomes zero for every neuron, despite potentially meaningful activation changes. It is unclear how such examples are handled. Furthermore, Appendix Algorithm 1 assumes precomputed class-wise masks without making sufficiently clear the source checkpoint, whether masks are refreshed as model parameters change, or the additional attribution/training cost. These choices are material to reproducibility and the plug-and-play claim.
[1] Shrikumar, A., Greenside, P., and Kundaje, A. “Learning Important Features Through Propagating Activation Differences.” International Conference on Machine Learning (ICML), 2017.
[2] Zhang, C., Liu, A., Liu, X., et al. “Interpreting and Improving Adversarial Robustness of Deep Neural Networks with Neuron Sensitivity.” IEEE Transactions on Image Processing, 30:1291–1304, 2021.
[3] Yan, H., Zhang, J., Niu, G., et al. “CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection.” International Conference on Machine Learning (ICML), 2021.
Questions for authors
- How are correctly classified adversarial inputs handled when the ground-truth and adversarial attribution targets coincide? From which model checkpoint are the class-wise masks computed, and are they updated during training?
- Under the same SANR objective, how do direct logit-margin attribution, random/class-shuffled weights, and activation-change-based sensitivity weights compare with NSA-derived weights?
- Are the Section 4 masking results intended solely as an interpretability diagnostic? Were attacks regenerated after masking, and what additional computational cost does NSA/SANR introduce?
2 comments
Thank you for your detailed response and clarifications. I appreciate that several concerns regarding the masking experiments and implementation details have been addressed. However, my main concerns about the novelty of NSA relative to direct margin attribution and the lack of controlled experiments isolating the contribution of NSA-derived weights remain unresolved. Therefore, I will maintain my original rating.
Response to Reviewer 8PMR
Dear Reviewer,
We sincerely thank you for your careful review, constructive critique, and insightful questions. Below, we address your concerns point by point.
Response to Weakness 1: Novelty of NSA vs. CIFS [3] Methods like CIFS operate on neuron importance (clean prediction contributions). However, an "important" neuron is not inherently "vulnerable." A strong feature's massive activation pushes it into the saturated regime of non-linear functions (Sigmoid/Tanh/Softmax). In CIFS, this neuron receives a high score because it anchors forward prediction. Yet, from an adversarial sensitivity perspective, its saturated state yields a near-zero local derivative, meaning a minute ϵ-perturbation barely alters probability. Thus, important neurons are not equivalent to vulnerable ones. Furthermore, while direct margin attribution is possible, it obscures internal network dynamics. Our dual-target construction backpropagates directly from the output layer via two distinct causal paths, separately quantifying contributions to suppressing ground-truth confidence and promoting adversarial confidence, isolating hyper-sensitive neurons that direct margin differences cannot reveal.
Response to Weakness 2: Isolation of NSA Contribution and Scope We acknowledge that evaluating random or class-shuffled weight baselines under identical losses would rigorously isolate NSA utility. Due to the tight rebuttal window and GPU limits for retraining, we could not complete these control experiments in time. However, SANR is a proof-of-concept application to validate our analytical framework rather than chase SOTA benchmarks. The modest 0.24-point improvement over the baseline reflects an inherent characteristic of neuron-level adversarial defenses: standard AT already imposes an implicit penalty on vulnerable paths. Our method superimposes explicit, category-specific guidance, resulting in incremental refinement. Although ImageNet training is precluded by resource constraints during rebuttal, the NSA theoretical framework is inherently architecture-agnostic, and we will incorporate larger-scale evaluations in future work.
Response to Weakness 3: Causal Mechanism and Masking Evaluation We agree our evaluation—using adversarial examples generated against the unmasked network—does not constitute an evaluation against adaptive attacks tailored to the masked model. However, our objective is Diagnostic Ranking Validation: masking top-NSA-ranked sensitive neurons is diagnostic rather than claiming an impenetrable adaptive defense. Showing that masking restores accuracy demonstrates that these targeted neurons are primary pathways exploited by the adversary. Grad-CAM realignment further provides strong diagnostic evidence that disabling high-NSA neurons mitigates specific adversarial vulnerability. We have revised the manuscript to tone down over-emphatic causal claims.
Response to Weakness 4: Adversarial Success, Checkpoints, and Cost 1. Adversarial Success: Masks are computed using successfully attacked adversarial examples (where prediction diverges from ground truth). Rare instances where an attacked input remains correctly classified are naturally filtered out. 2. Checkpoints & Static Masks: Masks are extracted from the checkpoint achieving highest clean accuracy before adversarial training. They are not updated dynamically; they are pre-computed exactly once offline and remain static throughout Algorithm 1. 3. Computational Cost: Calculating masks requires two DeepLIFT passes per sample. While intensive, this is a one-time offline cost. By decoupling mask generation from training, Algorithm 1 incurs zero additional DeepLIFT overhead (taking a few hours on a single GPU for CIFAR-10). Changes: Incorporated into Appendix A.4.
Responses to Author Questions:
Q1: How are correctly classified adversarial inputs handled? Checkpoints and mask updates? Response: As detailed in Weakness 4, correctly classified inputs are filtered out, we use the checkpoint with highest clean accuracy, and masks remain completely static.
Q2: How do baseline weights compare with NSA? Response: Please refer to our discussions in Weakness 1 and 2 regarding the necessity of dual-target attribution over baseline approaches.
Q3: Are Section 4 masking results solely an interpretability diagnostic? Attacks regenerated? Cost? Response: Yes, Section 4 results are strictly an interpretability diagnostic rather than an adaptive defense, and attacks were not regenerated post-masking. Mask computation is a one-time offline process (taking a few hours on one GPU for CIFAR-10), introducing zero additional runtime overhead during training.