- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes JACK, a two-stage attack that exploits JPEG quantization to make images elicit benign behavior before compression and attacker-specified behavior afterward in vision-language and robotic models.
Critical strengths
A clever two-stage method that turns JPEG compression into an attack trigger. Broad experiments across vision-language tasks and robotic manipulation.
Critical weaknesses
- JPEG compression is not a convincing trigger: (i) unlike a pixel pattern that can be placed on a real road sign, it lacks a clear real-world counterpart explaining how an attacker would cause the victim to receive the compressed input. (ii) JPEG compression lacks trigger secrecy, since anyone can apply it without attacker-specific knowledge, which weakens the claimed practicality of the attack.
unintentional activation by benign users can expose a backdoor and introduces the False Trigger Rate to measure this issue.
- The attack assumes white-box access and a known JPEG quality factor, limiting its practicality, with robustness to different JPEG codecs, resizing, and other preprocessing operations untested.
Questions for authors
Please see the weaknesses above. My main concern is Weakness 1. If the authors provide a convincing justification addressing this concern, I would consider raising my score.
4 comments
Response to Reviewer Wv8t (Part 2/2)
W2: Robustness to different JPEG codecs and resizing.
Response: We have conducted additional experiments covering zero-shot transferability across JPEG quality-factor mismatches, chroma-subsampling mismatches, and image resizing (without re-optimization), alongside multi-quality joint optimization. Using ImageNet and LLaVA-v1.5-7B with 100 adversarial images per source quality factor (), we compare JACK against JPEG-aware PGD and Tan-Joint, reporting the mean SASR (%) across all five source qualities (and across tested offsets or resolutions where applicable):
| Transfer Setting | JACK | JPEG-aware PGD | Tan-Joint |
|---|---|---|---|
| Quality ±1 | 56.4 | 48.3 | 38.3 |
| Quality ±5 | 15.0 | 14.8 | 4.5 |
| JPEG 4:2:2 | 26.8 | 25.8 | 8.2 |
| JPEG 4:4:4 | 9.4 | 6.0 | 2.2 |
| Resize (384 / 448 / 512) | 24.7 | 21.5 | 15.9 |
(i) JPEG Quality-Factor Mismatch. At , JACK achieves 56.4% SASR, exceeding JPEG-aware PGD and Tan-Joint by 8.1 and 18.1 percentage points, respectively. At , JACK retains the highest mean SASR despite substantial degradation across all methods.
(ii) JPEG Configuration Mismatch. When images optimized under 4:2:0 subsampling are tested under 4:2:2 and 4:4:4, JACK consistently outperforms both baselines.
(iii) Resizing and Preprocessing Order. When JPEG compression precedes resizing from to 384, 448, and 512, JACK achieves 24.7% mean SASR, outperforming JPEG-aware PGD and Tan-Joint by 3.2 and 8.8 percentage points, respectively.
These results demonstrate partial cross-setting transferability but also reveal performance degradation under larger mismatches. This is expected because the original JACK is designed for a specified JPEG setting without explicitly optimizing cross-setting transferability.
(iv) Preliminary Exploration of Multi-Quality Joint Optimization. To investigate whether this limitation can be mitigated, we further explore extending JACK to jointly optimize a single perturbation for multiple JPEG qualities, using Q30 and Q50 as an illustrative setting.
Unlike Tan-Joint, which jointly optimizes the pre-JPEG benign loss and both post-JPEG targeted losses in one stage, JACK retains its two-stage framework. Stage I minimizes the mean JPEG-targeted loss across Q30 and Q50 to obtain a shared candidate , aiming to trigger the target response under both qualities. Stage II constructs the intersection of their quantization cells, . For feasible samples requiring refinement, JACK optimizes the pre-JPEG benign objective subject to the joint cell constraint and the perturbation budget using Dykstra-style alternating projection, aiming to preserve adversarial behavior under both qualities.
| Method | Optimized Qualities | Test Q30 | Test Q50 | Joint SASR (Q30 & Q50) |
|---|---|---|---|---|
| Tan-Joint (Joint) | Q30 + Q50 | 51.0% | 63.0% | 51.0% |
| JACK (Joint) | Q30 + Q50 | 87.0% | 89.0% | 87.0% |
JACK achieves 87.0% joint SASR, outperforming Tan-Joint by 36.0 percentage points. These preliminary results demonstrate the feasibility of extending JACK through multi-quality joint optimization and quantization-cell intersection, providing a promising direction for mitigating its dependence on a single JPEG quality setting.
These experiments and discussions are included in Appendix D of the revised manuscript.
Gentle Follow-Up on Our Rebuttal
Dear Reviewer Wv8t,
Thank you for your time and valuable feedback. As the discussion deadline approaches, we would greatly appreciate it if you could let us know whether our rebuttal has adequately addressed your concerns. If so, we would be grateful if you could reconsider your rating accordingly.
If any concerns remain, we would be happy to provide further clarification. Thank you again for your consideration.
Best regards,
The Authors
Looking Forward to Your Feedback
Dear Reviewer Wv8t,
With the discussion deadline approaching, please let us know if we have resolved all of your concerns or if you require any further clarification. We look forward to your feedback.
Best regards, The Authors
Response to Reviewer Wv8t (Part 1/2)
We thank the reviewer for the comments, and for recognizing the strengths of our work:
Please find the point-by-point response below.
Response: We thank the reviewer for this insightful comment. JACK is an inference-time, transformation-triggered adversarial attack against frozen VLMs, rather than a training-time model backdoor. Neither the victim model nor the JPEG codec is modified.
(i) Trigger Deployment. JPEG compression is a widely deployed operation, not an artificial trigger requiring attacker installation. JPEG is a widely used image compression standard [1], and image-sharing platforms routinely compress or re-encode uploaded images [2, 3]. Moreover, JPEG compression has been employed or evaluated as an adversarial defense in classical vision systems [4–6] and VLM security research [7, 8], while recent work highlights compressed-image understanding in deployed VLMs [9].
JACK exploits this existing processing step rather than introducing it. For example, an attacker may submit a crafted image to a service that automatically re-encodes it into JPEG before VLM inference. The attacker controls only the image, while the receiving pipeline performs compression. Related feasibility is supported by Duan et al. (AAAI 2024) [10] on JPEG-conditioned backdoors and Tan et al. (TMLR 2026) [11] on transformation-triggered adversarial examples. Unlike training-time backdoors, JACK requires no modification of the victim model.
(ii) Trigger Secrecy and Stealthiness. JACK does not rely on JPEG compression being secret; its effective trigger is the combination of an attacker-specific optimized image and JPEG processing. Compressing arbitrary benign images does not systematically induce attacker-specified behavior.
In the reviewer's road-sign analogy, a conventional adversarial patch may trigger malicious behavior immediately upon observation, whereas JACK additionally requires JPEG compression for activation. The crafted image elicits benign or task-consistent behavior before compression and attacker-specified behavior afterward. Thus, JPEG-conditioned activation provides behavioral stealthiness against pre-compression inspection, consistent with the motivation for conditionally activated physical attacks [12].
(iii) Unintended Activation (FTR). Unlike persistent model backdoors that may activate on arbitrary compressed inputs, JACK requires an individually optimized adversarial image. We assess unintended target-specific activation on unmodified inputs using clean-input SASR, which measures benign-to-target behavioral switching caused by JPEG compression alone. It is 0.0% on ImageNet (Table 2), 0.0% on MS-COCO VQA (Table 5 in the original submission; Table 11 in the revised manuscript), and 1.25%/1.22% on MM-SafetyBench (Table 3) for LLaVA/Qwen2.5-VL, respectively. These results indicate that unintended target-specific activation is rare on the evaluated clean inputs.
Overall, JACK reveals a distinct security risk: existing preprocessing or defensive operations can activate adversarial behavior concealed during direct, pre-compression inference.
[1] JPEG Committee, “100th JPEG Meeting Celebration Announcement,” JPEG.org, 2023.
[2] F. Bertini et al., “Are Social Networks Watermarking Us or Are We (Unawarely) Watermarking Ourself?,” Journal of Imaging, 2022.
[3] C. Pasquini et al., “Media Forensics on Social Media Platforms: A Survey,” EURASIP Journal on Information Security, 2021.
[4] C. Guo et al., “Countering Adversarial Images Using Input Transformations,” ICLR, 2018.
[5] N. Das et al., “SHIELD: Fast, Practical Defense and Vaccination for Deep Learning Using JPEG Compression,” KDD, 2018.
[6] Z. Liu et al., “Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples,” CVPR, 2019.
[7] L. Bailey et al., “Image Hijacks: Adversarial Images Can Control Generative Models at Runtime,” ICML, 2024.
[8] Y. Lan et al., “Phi: Preference Hijacking in Multi-modal Large Language Models at Inference Time,” EMNLP, 2025.
[9] Z. Zhang et al., “Benchmarking and Enhancing VLM for Compressed Image Understanding,” ICML, 2026.
[10] Q. Duan et al., “Conditional Backdoor Attack via JPEG Compression,” AAAI, 2024.
[11] Y. Tan et al., “Transform-Triggered Adversarial Examples,” TMLR, 2026.
[12] W. Zhu et al., “TPatch: A Triggered Physical Adversarial Patch,” USENIX Security, 2023.