- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes Thermal Plume Attack, a physics-inspired black-box attack against infrared vision-language models (IR-VLMs). The method simulates thermal plumes using a shared temperature field to generate coupled refractive and radiance perturbations, and employs CMA-ES to optimize five plume parameters without accessing model gradients. Experiments on FLIR and LLVIP demonstrate its effectiveness against LanguageBind Thermal, IRGPT, and several infrared-adapted CLIP models, revealing vulnerabilities of IR-VLMs to structured thermal disturbances.
Critical strengths
- This paper introduces a low-dimensional thermal plume attack framework that couples refraction and radiance through a shared temperature field, offering a physically motivated and potentially novel perturbation mechanism.
- The evaluation covers multiple infrared-specialized and infrared-adapted VLMs under direct black-box attacks, with a clear distinction between target-specific flips and open-set errors.
Critical weaknesses
- This paper lacks essential ablation studies on refraction-only, radiance-only, and uncoupled perturbations, making it difficult to isolate the contribution of the proposed shared temperature field.
- The plume renderer relies on simplified physical models and heuristic radiance approximations, without real-world validation using thermal sources and infrared cameras, leaving its physical realism and feasibility uncertain.
- The attack assumes access to ground-truth bounding boxes and continuous model scores, raising concerns about practical applicability and generalization.
Questions for authors
- Can the authors provide ablations isolating refraction, radiance, and their coupling, along with baseline comparisons under matched LPIPS and perturbation footprints?
- Can the authors validate the physical plausibility of the generated perturbations through real infrared imaging experiments or calibrated physical simulations?
- How effective is the proposed method when ground-truth bounding boxes or continuous model scores are unavailable?
3 comments
Thank you for your response. It addressed my questions, and I will maintain my score.
We are pleased to hear that our response has addressed your concerns. In the revised manuscript, we will incorporate the additional ablations, baseline comparisons, real infrared imaging results, and label-only evaluations, clarify the alternatives to ground-truth bounding boxes, and further refine the methodological descriptions, experimental details, and overall presentation.
Thank you for recognizing the thermal plume framework and for your suggestions on evidence and presentation. For presentation revisions, see our reply to Reviewer 4v4L (Readability comment).
A1
To isolate both components' roles, we added refraction and radiance ablations. All arms used 1,016 clean-correct LanguageBind Thermal crops (499 person-to-vehicle, 517 reverse), CMA-ES, ~1,000 candidates/crop and caps of LPIPS ≤0.1 and changed-pixel fraction ≤0.5. Refraction changes geometry; radiance approximates IR intensity.
Coupling improved ASR over each component in both seeds.
To compare mechanisms, we added separate baseline evaluations with common caps of LPIPS ≤0.1 and changed-pixel fraction ≤0.5. On the respective subsets, plume ASR was 41.7%; stripe noise and hot/cold blocks achieved 4.17% and 8.33%. Local-pixel and local smooth-random-field CMA-ES each achieved 54.17%. Mean LPIPS and mean changed-pixel fraction varied across methods.
Our plume parameters and IR image formation impose physically meaningful constraints. Real thermal plumes provide the physical basis, with disturbances not readily noticeable in real-world settings. Comparisons should consider ASR, physical plausibility and visibility.
A2
To evaluate physical plausibility, we used a mean temperature field following the Morton-Taylor-Turner (MTT) integral plume model of real plumes' mean buoyant rise and spreading, and added IR recordings from a stove combustion source. Visual comparison showed high similarity between real and simulated disturbances.
To further evaluate model effects, we added tests of all five models on the same recorded IR frames.
The HIKMICRO FALCON FH25 camera was ~1 m from the source between camera and person. All four CLIP encoders are IR-adapted.
All evaluated frames contained a person. Physical-scene ASR is the proportion classified as vehicle in person-versus-vehicle classification.
A3
The attack remains feasible. Without ground-truth boxes, automatic detection/segmentation can locate target regions; without continuous scores, a label-only variant can generate perturbations.
To test effectiveness without continuous scores, we added label-only tests on four IR-adapted CLIP models via local inference interfaces with original target crops, achieving 54.03%–74.32% ASR.
The six-parameter variant adds dimensionless radiance gain g to five plume parameters to control IR intensity-change amplitude. It uniformly samples all six, using only predicted labels as model feedback, and stops at the first candidate satisfying perturbation constraints and receiving the predetermined target label (at most 1,000 evaluations/sample).
Local label-only results
Both runs use the same n clean-correct samples per model. Values are two-run means; mean evaluations include all samples, counting failures as 1,000. Overall, our label-only variant remains effective.
Target regions and continuous scores can also be readily acquired. Without ground-truth boxes, YOLO, Grounding DINO or boxes enclosing segmentation masks can locate target regions for cropping and plume-parameter optimization. If the attacked model itself is locally deployable or returns scores, its class scores can guide CMA-ES. Score-interface examples are Hugging Face's
InferenceClient.zero_shot_image_classification(labels and scores) and CLIP-as-service'srank()(image-text matching scores). For a label-only attacked interface, use the variant above.