- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a method to improve adversarial transferability against Vision Transformers. It measures CLS–patch relationships using token representations and their variation across Transformer layers. The selected tokens receive gradient scaling at QKV and MLP outputs during adversarial example generation. Preliminary experiments on 100 ImageNet images show a modest improvement over MI-FGSM.
Critical strengths
- The paper addresses a relevant problem in transferable adversarial attacks.
- The proposed method explores CLS–patch relationships for selective gradient intervention.
- The preliminary evaluation includes ablation studies and uncertainty analysis.
Critical weaknesses
Incomplete evaluation of the proposed method. The paper describes an evaluation involving 1,000 ImageNet images and four surrogate models. However, the reported results cover only 100 images using ViT-B/16 as the surrogate. The results for the remaining surrogate models are unavailable. More importantly, the conclusion explicitly states that the experiments use an earlier nonstable-token selection variant rather than the complete method described in Section 3. Therefore, the reported results do not establish the effectiveness of the proposed method. This is a fundamental limitation, as the central technical contribution has not been fully evaluated.
Lack of competitive results against stronger baselines. The proposed method improves the average black-box attack success rate from 45.7% to 47.1% over MI-FGSM. However, Table 1 reports substantially higher success rates for existing methods, including TGR (70.3%), GNS (68.7%), and FPR (70.1%). These baseline results come from earlier experiments and are not directly comparable with the current pilot evaluation. Nevertheless, the available evidence does not demonstrate an advantage over stronger existing attacks. The practical significance of the proposed method therefore remains uncertain.
Insufficient validation of the token selection strategy. The proposed method selects tokens based on CLS–patch relationship strength and cross-layer variation. However, the paper does not establish why tokens with greater relationship variation should produce more transferable adversarial examples. Table 2 compares the earlier nonstable-token variant with random token selection, but does not isolate the contributions of relationship strength and cross-layer variation. The results therefore provide limited evidence that the specific selection criteria in Section 3 are responsible for improved transferability.
Limited and inconsistent transferability gains. The pilot experiment reports an overall improvement of 1.42 percentage points, with a 95% bootstrap confidence interval of [0.21, 2.70]. This provides some evidence of a positive gain on the evaluated images. However, the improvement on Transformer targets is only 1.00 percentage point, with a confidence interval of [-0.67, 2.70]. Several target models show little or no improvement. The observed benefits therefore appear modest and dependent on the target architecture. Their generality remains uncertain.
Incomplete implementation details. The method depends on the selected scoring layers and gradient-intervention layers, denoted by and . However, the experimental setup does not specify these layer sets. The token budget is also adjusted across stages for PiT-Ti without reporting the corresponding values. These configuration choices directly affect token selection and gradient propagation, making the method difficult to reproduce from the current description.
Questions for authors
- What experimental results establish the effectiveness of the complete method proposed in Section 3?
- How does the proposed method compare with TGR, GNS, and FPR under matched experimental settings?
- What evidence supports the use of both relationship strength and cross-layer variation for token selection?
- Is the advantage over random token selection statistically reliable across different target models?
- Which scoring and intervention layers are used for each surrogate architecture?