- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how to leverage the activation magnitude distribution of intermediate representations in Vision Transformers to improve the cross-model transferability of adversarial attacks. The authors empirically observe that certain tokens exhibit unusually large feature norms, with their activations largely concentrated in a small subset of channels. Motivated by this observation, they propose MAAT to enhance adversarial transferability by calibrating highly activated channels, suppressing feature variations within highly activated channels and maximizing feature discrepancies across the remaining channels. Experiments show that integrating MAAT with existing transfer-based attacks can further improve their attack performance.
Critical strengths
- Different from prior studies on transferable adversarial attacks, this work explores how to exploit the concentration phenomenon of activation magnitudes in intermediate representations to improve attack transferability, and designs the attack objective from the perspective of the feature activation magnitude distribution.
- Superior attack performance and method compatibility. MAAT can be readily integrated with existing transfer attacks, including MI-FGSM, MTE, PNA, and PatchOut, consistently improving their average attack success rate across different target models and datasets.
Critical weaknesses
- Although the authors investigate the concentration of activations in intermediate representations of Vision Transformers, they do not sufficiently justify why this phenomenon warrants further investigation in the context of transferable adversarial attacks, nor do they clearly articulate how it relates to the limitations of existing attack methods.
- Separate ablations on suppressing variations in highly activated channels and maximizing discrepancies across the remaining channels are missing, making their individual contributions to attack performance unclear.
Questions for authors
- Could the authors further clarify why the concentration of activation magnitudes in intermediate representations deserves particular attention in the context of transferable adversarial attacks? Specifically, could exploiting this phenomenon help overcome certain limitations or technical challenges of existing methods, or provide new insights into improving adversarial transferability?
- Since the experiments only use DINOv2-L/14 and CLIP-L/14 as surrogate models, additional evaluations across different model scales (e.g. DINOv2-B/14, CLIP-B/14) would help verify the effectiveness and generality of MAAT.
- Existing work [1] shows that introducing register tokens mitigate the phenomenon of abnormally high-norm tokens in Vision Transformers. Could the authors evaluate MAAT on DINOv2 with registers, and compare the degree of activation concentration as well as attack performance before and after register incorporation, to examine whether MAAT remains effective when the feature structure of high-norm tokens changes?
[1] Darcet, Timothée, et al. "Vision transformers need registers." International conference on learning representations. Vol. 2024. 2024.