- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces ViTViz to study how ViT attention changes under adversarial attacks. By comparing Attention Rollout and GAE, the authors show that different ways of reading attention can lead to very different, and sometimes opposite, conclusions about the relationship between attack success and attention displacement.
Critical strengths
The experiments are relatively systematic, covering multiple ViT models, attacks, and data domains. The authors also use bootstrap and several control settings involving perturbation budget, anchor, and normalization to validate the main observations. In particular, Rollout and GAE show a significant directional reversal on GTSRB, indicating that the choice of attention reading can indeed affect the conclusion.
Critical weaknesses
1.The main contribution is mostly a combination of existing Rollout, GAE, standard redistribution metrics, and a visualization tool. There is no new attack or attention method. Since Rollout is class-agnostic while GAE uses class gradients, some disagreement is not very surprising. To me, this is closer to an engineering/measurement study than a strong methodological contribution.
2.Rollout and GAE differ in several ways at the same time, including class anchor, gradient weighting, aggregation, and normalization. So it is hard to tell what actually causes the disagreement. The current controls only explain part of it.
3.Personally, PGD and MIM are too successful in some settings, leaving very few unflipped samples. This makes the successful-vs-failed comparison less convincing in those cells. For example, comparing 990 flipped samples with only 10 unflipped samples is clearly imbalanced. The GTSRB experiment also changes the dataset, checkpoint, and class space together, so I do not think the effect can be attributed only to domain shift.
4.The experiments show that different attention readings can give different conclusions, but they do not really explain why. They also do not establish a clear mechanism between attention redistribution and attack success. The paper’s own variance analysis even shows that image identity matters much more than attack identity.
5.The ablation is still not complete. I would like to see a more direct factor-by-factor analysis of the differences between Rollout and GAE. Also, only comparing two attention operators is not enough to support a broad claim about attention reading in general.
6.The attack set is also somewhat outdated. Recent ViT-specific attacks such as ATT (NeurIPS 2024) and FPR (CVPR 2025) are not included. It is unclear whether the same conclusions still hold for stronger and more recent attacks.
Questions for authors
See weaknesses.
2 comments
Good
We thank Reviewer JT71 for the thoughtful review and for highlighting the systematic experiments across models, attacks, and data domains.
W1 (Contribution and class information). We benchmark displacement by model, attack, operator, and outcome and test which findings carry across readings. With a random anchor, GAE and LeGrad retain 8 and 6 of 22 same-direction separations, and GMAR keeps all 15 of its original separations. Using the predicted class moves GAE’s map by 64%–89% of the mean attack displacement and LeGrad’s by 1.7–2.2 times. Random anchoring changes GMAR’s displacement by a median of 1.1% per image. Aggregation drives its eight-cell gap from rollout.
Changes in the revision. Table 11 reports random anchors, Appendix I predicted-class anchors, and Appendix P GMAR’s decomposition.
W2 (Separating factors). We varied operator choices on the same images. Across-layer gradient weighting has the largest Shapley share for all three measures. On raw attention, GAE’s composition gives rollout up to a constant. Shares are in cells that separate, for the original five attacks.
Changes in the revision. Section 5 and Table 18 (Appendix P) give the decomposition. Table 11 controls for the anchor.
W3 (Small samples and GTSRB). The 20-unflipped-image floor excludes splits such as 990/10. We replaced the percentile bootstrap with a Welch-statistic permutation test. With 20 unflipped images, false separation fell from 8.5% to 5% in simulation. Simulated power was 38%, so we report each cell’s resolution. The original rollout and GAE verdicts are unchanged. We also tested five more GTSRB checkpoints. Rollout and GAE agree in 20 of 26 estimable cells, including 11 of 16 under FGSM, PGD, MIM, and TGR. The PGD reversal did not recur in the three with estimable PGD results. Dataset and class space still change together in the ImageNet–GTSRB comparison.
Changes in the revision. Section 5 and Appendix N describe the test. Section 6.3 and Appendix K report the checkpoints.
W4 (Mechanism and image identity). We localized the contrast to late layers and patched their attention. Restoring the clean pattern recovers 12%–39% of the lost margin and undoes 2%–14% of flips. Restoring the full attention output recovers 75%–94%. In the reverse test, another clean image’s pattern lowers the true-class probability at least as much as the attacked pattern in five of six cells. Late attention plays a partial role. Most flips survive restoration.
With the original five attacks under GAE, image identity explains 42%–63% of displacement variance and attack identity 3%–24%. Across rollout and GAE, the image–operator interaction explains 17% and the attack 7%. This motivates comparing flipped and unflipped images within each model and attack and using the run’s own reference group.
Changes in the revision. Appendices Q and R add the depth analysis and patching. Section 6 and Appendix B give the variance estimates.
W5 (Four operators). We added LeGrad and GMAR and decomposed three operator gaps. Rollout, GAE, LeGrad, and GMAR separate 11, 29, 35, and 22 of 35 estimable cells. On the original five attacks, no pair agrees in more than 16 of 24 cells. Across-layer gradient weighting, logit choice, and aggregation drive different pairs. We recommend comparing multiple readings and reporting where their conclusions agree.
Changes in the revision. Table 3 and Appendix J extend the comparison. Appendix P gives the three decompositions.
W6 (Recent attacks). We added FPR and ATT, two recent ViT attacks designed for transfer. In our white-box benchmark, rollout and GAE agree in zero of six FPR cells and three of five estimable ATT cells. The probe averages attack success rate (ASR) over four models, with 200 images per cell. SSIM is relative to the clean image. ATT is near its plateau at 4/255, while FPR gains 6.6 points at 8/255.
Changes in the revision. Section 5, Tables 3 and 5, and Appendices A, B, G, and Q cover FPR and ATT. Appendix M qualifies the common budget.
We hope these responses address the reviewer's concerns. If so, we would be grateful if the reviewer would consider updating the score. We are happy to provide further clarification.