- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes a universal multimodal attack that combines online vocabulary-constrained text optimization with image-perturbation learning conditioned on re-encoded edited captions. The framework also incorporates bidirectional competition and architecture-dependent fusion guidance. Experiments across six model configurations demonstrate strong white-box attack performance and varying degrees of transfer to other models and downstream tasks.
Critical strengths
The central idea is well motivated. The distinction between optimizing a continuous text representation and applying a discrete word replacement at inference is meaningful. Conditioning image updates on complete, re-encoded edited captions provides a sensible way to account for the replacement word’s contextual effects.
Critical weaknesses
-
Important comparisons are missing. Although the paper discusses DO-UAP and HRA, neither is evaluated experimentally. Both are closely related to the proposed approach. Downstream experiments compare only against C-PGC, despite ARG-Attack’s competitive retrieval performance, leaving the empirical advance incompletely established.
-
The full framework’s transfer advantage is also limited. Mean black-box I2T/T2I ASR is 40.00%/60.39%, compared with 40.55%/57.53% for ARG-Attack, so the average improvement is concentrated in T2I retrieval. In the ALBEF-source ablation, the full framework (A4) performs slightly below component D alone (A1): 50.58%/71.25% versus 50.95%/71.32%. Although bidirectional competition improves white-box performance, its additional complexity does not yield a consistent transfer benefit in the reported ablation.
Questions for authors
-
What are the values of all loss hyperparameters, how were they selected, and were they shared across source models?
-
How does the method compare with DO-UAP and HRA under matched attack constraints, checkpoints, and computational budgets? Can ARG-Attack be included in downstream comparisons?