- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates rain-based attacks targeting vision-language models. Specifically, it proposes a two-stage approach: optimizing global rain layer blending coefficients to reduce CLIP-style semantic margins, and using CMA-ES to optimize multiscale raindrops and spatial illumination parameters under perceptual and structural constraints. The evaluation covers zero-shot classification tasks based on multiple CLIP backbone networks, and the resulting images are transferred to image description and visual question-answering (VQA) models.
Critical strengths
- The paper introduces a structured and physically interpretable adversarial framework that jointly models multi-scale raindrops and illumination variations.
- Extensive experiments demonstrate strong attack effectiveness across multiple CLIP-based models and transferability to image captioning and VQA tasks.
Critical weaknesses
- The objective of this paper is not clear. While the paper claims to reveal the vulnerability of VLMs in rainy-day scenarios, its methodology is based on adversarial attacks using simulated raindrops rather than real-world rainy-day scenarios.
- What components does the proposed semantic decoupling mechanism actually decouple? The paper describes semantic decoupling as separating semantic manipulation from physical appearance synthesis. However, both stages modify image-space weather perturbations and optimize semantic similarity objectives.
Questions for authors
- Figure 2 does not clearly illustrate the optimization objectives and parameter interactions between the two stages, making the overall workflow less intuitive.
- Both arXiv:2009.09205 and arXiv:2302.14267 explore adversarial attacks that utilize simulated raindrop effects; the authors should discuss these studies.
- How to calculate the CMA-ES fitness values for candidates that did not make it into the Top-K subset.
- The number of model queries and computational costs of the two stages are not clearly compared with baseline methods.