- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TIGA, a framework designed to improve targeted adversarial transferability against Multimodal Large Language Models (MLLMs) from both feature representation and optimization perspectives. Specifically, it introduces Global-Regional Collaborative Alignment to integrate global and regional visual features, and Neighborhood-Aware Gradient Fusion to incorporate the gradient information from a noise-perturbed neighbor. Experiments on 6 closed-source and 6 open-source MLLMs demonstrate improved attack success rates and semantic similarity compared with existing transfer attacks.
Critical strengths
- The paper investigates the challenging and practically relevant problem of black-box targeted transfer attacks against MLLMs, without requiring access to victim model parameters or gradients.
- The proposed framework achieves substantial improvements over existing baselines across a diverse set of closed-source and open-source MLLMs, particularly on commercial models.
Critical weaknesses
- Both proposed components primarily rely on straightforward feature augmentation and input-gradient augmentation rather than fundamentally new attack mechanisms. GRCA concatenates spatially pooled regional features with global representations, while NGF introduces a noise-perturbed input and combines the resulting gradients. These designs closely resemble established feature aggregation, input diversification, and gradient ensemble techniques in adversarial attacks [1,2].
- The paper argues that global feature alignment overlooks regional information relevant to language generation. However, the proposed GRCA simply applies fixed 2×2 average pooling and feature concatenation, without explicitly modeling regional semantic correspondence or interactions. The experiments mainly demonstrate performance improvements but provide limited evidence that these gains actually result from better alignment of region-level information used by victim MLLMs.
- The experimental evaluation is primarily limited to image captioning with a fixed prompt, restricting the scope of the findings. It remains unclear whether the improvements generalize to more challenging multimodal tasks, such as visual question answering and complex visual reasoning.
[1] Xie C, et al. Improving transferability of adversarial examples with input diversity. In CVPR 2019.
[2] Wang X, et al. Enhancing the transferability of adversarial attacks through variance tuning. In CVPR 2021.
Questions for authors
- What fundamentally distinguishes GRCA and NGF from existing feature aggregation, input diversification, and gradient ensemble techniques? Could the authors further clarify the methodological novelty of the proposed components?
- Could the authors provide more direct evidence that GRCA improves regional semantic alignment in victim MLLMs, beyond the performance gains? For example, would analyzing region-level feature similarities or comparing against alternative regional aggregation strategies help validate the proposed motivation?
- How well does TIGA generalize beyond image captioning? Could the authors provide additional evaluations on more challenging multimodal tasks, such as visual question answering or complex visual reasoning?
- The evaluated defenses are primarily conventional image preprocessing methods, which may not adequately reflect the robustness of modern MLLMs against adversarial attacks. Could the authors evaluate whether TIGA remains effective against more advanced adversarial defenses, such as DiffPure [1]?
[1] Nie W, et al. Diffusion models for adversarial purification. In ICML 2022.