- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DA-Attack, a targeted transfer attack that leverages a fixed ensemble of pretrained vision encoders and does not query the target MLLM during optimization. Its contrastive objective pulls the adversarial features toward those of the target image while pushing them away from those of the source image. RSE retains each original encoder and augments the ensemble with variants generated via random parameter pruning. ISNC assigns weights based on the cosine similarity between gradient directions; the normalized gradients are first aggregated within each model and then averaged across models, followed by a momentum update under an ℓ∞ constraint. The authors evaluate the method on twelve target MLLMs on NIPS 2017 and Flickr30K, using the semantic similarity of generated image captions as the metric, and report better targeted transfer performance than existing baselines.
Critical strengths
- Broad experimental coverage. The evaluation covers two datasets and twelve victim models, including both open-source and closed-source models. Component ablations and sensitivity analyses of the number of randomized variants and pruning probability provide evidence of the method’s effectiveness and transferability.
- The comparison of consensus strategies provides a useful empirical insight. Table 5 shows that consensus weighting among variants of the same surrogate performs well, whereas additionally applying consensus weighting across different surrogates reduces performance. This suggests that stabilizing update directions within a surrogate and preserving directional diversity across surrogates may serve distinct roles.
Critical weaknesses
- The methodological novelty is limited. RSE follows RaPA’s random parameter pruning strategy, while ISNC primarily introduces agreement-based gradient weighting over sampled variants. Previous work has already explored gradient conflict projection for vision-language model attacks, as in TGGR, and adaptive ensemble weighting with disparity-reduced filtering, as in AdaEA. Although the specific mechanisms differ, the paper lacks sufficient comparisons or deeper analysis to establish a substantive contribution beyond combining existing parameter randomization and gradient processing ideas.
- The source and computational cost of the improvements remain insufficiently established. Runtime, peak memory, and matched-compute comparisons are missing, making it difficult to assess the gains relative to the additional computational overhead. Furthermore, Table 5 does not adequately isolate the effects of computing losses independently for each surrogate, gradient normalization, and the proposed components, limiting the attribution of improvements.
Questions for authors
- Previous studies such as TGGR and AdaEA have explored gradient conflict resolution or adaptive aggregation to improve attack transferability. Could the authors provide a deeper analysis of why directional agreement among variants of the same surrogate improves transfer to unseen models? Is there evidence that higher within-surrogate gradient agreement corresponds to higher attack success on victim models?
- Were the baselines matched in total forward and backward evaluations? What are the per-image attack generation time and peak GPU memory usage? Which comparisons in Table 5 keep the loss formulation and gradient normalization unchanged?