ViTViz: Analyzing Attention Redistribution Under Adversarial Attacks in Vision Transformers

Vision Transformers (ViTs) have achieved state-of-the-art performance in image classification, yet they remain vulnerable to adversarial attacks. Since attention is a fundamental mechanism underlying these models, understanding how it behaves under adversarial perturbations is crucial. However, to the best of our knowledge, the literature lacks tools for systematically exploring attention under adversarial regimes. To address this gap, we introduce ViTViz, an open-source tool for exploring attention under adversarial attacks. ViTViz enables researchers to independently investigate key dimensions of attention—including the aggregation operator, anchoring class, layer, attention head, attack step, and perturbation budget—on any plain ViT classifier with a single class token, in the timm or the Hugging Face layout. For the benchmark's six models, it places each measurement at its percentile among images of the same model, attack, operator, and outcome. Furthermore, we introduce an attention redistribution analysis based on three measures, each answering a different question: how far attention mass moved, whether the distribution changed at all, and whether attention spread or concentrated. As a proof of concept, we applied this analysis to 1,000 ImageNet images subjected to five attacks, using six ViT variants and two attention aggregation operators, one class-agnostic and one anchored on a class label. On the same images, the two operators disagree about whether a successful attack moves attention further than a failed one. How far attention moved therefore says something about an attack only when the operator and anchor that produced the measurement are reported.