- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
LEA combines LLM-guided semantic control with evolutionary search to generate targeted transferable adversarial images against vision-language models.
Critical strengths
This paper uses an LLM to adapt search regions, operators, and parameters based on optimization progress. Consistent improvements across three victim models.
Critical weaknesses
i) Limited scope. The experiments show that the attack performs well on VLP models but has limited effectiveness on LLaVA, with target ASR% 10.7 and Fool Rate 20.4%. The results corroborate my previous observation that attacks on VLP models differ substantially from those on multimodal large language models (MLLMs). Consequently, the work’s narrow focus on VLP models limits its relevance to future research, e.g., attacks on frontier MLLMs (GPT, Opus, etc.).
ii) Overly complex design without design intuition. The method combines an LLM with multiple evolutionary algorithms and search strategies, raising questions about its generalizability. With so many components and hyperparameters, it is unclear which design choices actually drive the performance gains and why.
Questions for authors
Please see the weaknesses above. I also wonder why Stable Diffusion is needed to generate a reference image when CLIP already aligns text and image representations? The rationale for this design, along with its connection to attack transferability, needs to be clarified.