- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes CFPatch, a VIS–IR adversarial patch combining shared curved-fractal geometry with spectrum-specific textures. It uses PSO with EOT to jointly attack zero-shot classification and evaluates the resulting adversarial samples on image captioning and VQA without further optimization.
Critical strengths
The joint success criterion requires both visible and infrared predictions to fail on the same sample, preventing strong results on one branch from masking failure on the other. Reusing the adversarial samples for captioning and VQA also provides useful evidence that the attack effects extend beyond classification.
Critical weaknesses
-
The main experiments use infrared images synthesized from visible images and general-purpose VLMs adapted through LoRA. Although the appendix includes real LLVIP data, the results do not establish effectiveness against dedicated infrared VLMs trained on real thermal imagery. This limits the evidence supporting the claimed contribution to VIS–IR VLM security.
-
The shape and texture design relies heavily on hand-crafted structures and numerous hyperparameters. The motivation for curved fractals and Fraser patterns is not sufficiently convincing, and their generality remains unclear. Other shape or color parameterizations could provide similar flexibility, but the paper does not establish why these particular choices are preferable.
Questions for authors
-
The main benchmark consists of COCO images and synthetic infrared counterparts. The real-data evaluation is limited to pedestrian classification on LLVIP, and the victim models are LoRA-adapted versions of general-purpose models. Dedicated infrared models such as IRGPT and Thermo-VL are not evaluated, leaving the applicability to real infrared perception unclear.
-
Thermal infrared intensity depends on temperature, emissivity, and surrounding radiation rather than visible color alone. CFPatch generates visible colors and infrared grayscale textures from a shared coordinate field, but does not establish a material-based relationship between these appearances. An insulating patch can produce a relatively uniform cool region, whereas the proposed grayscale texture contains spatial intensity variations whose physical realization is unexplained.
-
Table 1 reports substantially higher VIS-only ASR than TOUAP, but the comparison provides insufficient detail to explain this gap. TOUAP explicitly optimizes a grid of patch colors, offering considerable appearance flexibility. The limited information on baseline adaptation and optimization leaves me unconvinced that TOUAP was sufficiently optimized for this setting.
-
The comparison omits recent methods such as CDUPatch, despite citing it as related work.
-
The paper does not explain how the VQA questions are constructed or selected. Reusing the corresponding annotations does not specify the question set or selection procedure, reducing the reproducibility of the quantitative results.
-
Extending the attack to VLMs does not by itself establish substantial methodological novelty. The particular shape and color models still lack a convincing motivation. The term “cross-modal” is unclear here, since VIS and IR are both visual inputs, while VLMs combine vision and language.