- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies defenses against visual prompt injection in LVLMs. The authors introduce SafeRegionVPI for measuring benign evidence preservation and region-aware VPI localization. Based on an analysis of internal VPI signals, they further proposes VPIXDistill-X to locate and suppress injected visual text.
Critical strengths
-
The paper studies defenses against visual prompt injection while emphasizing the preservation of benign visual text and task utility. This is practical for real-world deployment.
-
SafeRegionVPI provides fine-grained annotations for both injected text and benign OCR regions.
-
The method is also effective against some jailbreak attacks.
Critical weaknesses
-
The paper provides no direct evidence that the model's self-reported bounding boxes reflect the evidence it actually used during inference.
-
S1's interpretation as instruction-to-region relevance is not justified. A high attention weight does not imply that the region causally affects the model’s prediction. Please add causal intervention experiments to support this interpretation
-
The need for distillation is not sufficiently motivated. For the model provider, the internal signals required by VPIXDistill-X are available, so the method could be implemented directly without distillation.
-
The web-agent experiments report AR and SR but do not include clean task-completion rates with and without the defense. While the results show that the defense reduces attack success, its impact on normal task performance remains unclear.
Questions for authors
-
Do the authors have evidence that the bounding boxes reported by the model in Protocol A correspond to the visual evidence it actually uses?
-
Which open-source teacher model was used to obtain the VPIXDistill-D detectors? Were the detection thresholds tuned separately for these closed-source models?
-
About VPIXDistill-D, I would like to see a baseline using the same classifier but trained only with the three-class cross-entropy loss.
-
See weaknesses.
7 comments
Author Response to Reviewer H6Vc (Part 1)
The paper provides no direct evidence that the model's self-reported bounding boxes reflect the evidence it actually used during inference.
Do the authors have evidence that the bounding boxes reported by the model in Protocol A correspond to the visual evidence it actually uses?
Yes. Masking tests show that the reported boxes hold the evidence the answer causally depends on, both without and under the defense. Following deletion-based checks of explanation faithfulness [1][2] and redaction checks of LLM self-explanations [3], we use clean TextVQA images from SafeRegionVPI. Each model answers under Protocol A and reports the boxes it used. We then cover selected boxes with gray rectangles, rerun the same prompt, and check which answer words remain. We track candidate words, i.e., content words of the answer that appear in the OCR text of a numbered box.
We mask k reported boxes or k unreported boxes matched in area, selected without looking at their text. In both conditions we track the candidate words found in the selected reported boxes. If the reports did not reflect the evidence the model uses, the two conditions would remove these words at similar rates. For the defended condition, we rerun Protocol A on each defended image, so the answer and the reports are those produced under the defense.
| Answer words retained after masking (%) | Qwen2.5-VL-7B | Qwen2.5-VL-7B + VPIXDistill-D | InternVL3-14B |
|---|---|---|---|
| Reported boxes masked | 27.7 | 30.7 | 33.1 |
| Matched unreported boxes masked | 82.0 | 81.5 | 85.3 |
| Paired difference (points) | 54.3 [48.8, 60.0] | 50.8 [43.7, 54.6] | 52.3 [46.6, 58.1] |
Masking the reported boxes removes most of the answer words they contain, while masking matched unreported boxes leaves these words in the answer. The gap exceeds 50 points on both models and stays close to its no-defense value under the defense (50.8 versus 54.3 points on Qwen2.5-VL-7B). The reported boxes also hold most of the answer's text-derived evidence. On average, 75.4% (Qwen2.5-VL-7B) and 83.7% (InternVL3-14B) of an image's candidate words occur in them. Omitted evidence does not bias BEUD. The reports cover most, but not all, of the benign text the model uses. To measure omissions, we also test every unreported box that contains a candidate word and record the share of candidate words that occur only in those boxes and disappear from the answer. On the same Qwen2.5-VL-7B images, this omission rate is 19.5% without the defense and 19.9% with it. Because BEUD compares each image with and without the defense, omissions affect both terms similarly and do not bias the change BEUD measures. Together, these tests support using BEUC to answer whether a defense preserves the benign evidence the model uses.
References
- [1] DeYoung, Jay, et al. ERASER: A benchmark to evaluate rationalized NLP models. Proceedings of the 58th annual meeting of the association for computational linguistics. 2020.
- [2] Petsiuk, Vitali, Abir Das, and Kate Saenko. Rise: randomized input sampling for explanation of black-box models (2018). arXiv preprint arXiv:1806.07421.
- [3] Madsen, Andreas, Sarath Chandar, and Siva Reddy. Are self-explanations from Large Language Models faithful?. Findings of the Association for Computational Linguistics: ACL 2024.
Author Response to Reviewer H6Vc (Part 2)
S1's interpretation as instruction-to-region relevance is not justified. A high attention weight does not imply that the region causally affects the model’s prediction. Please add causal intervention experiments to support this interpretation
We appreciate the reviewer's point and would like to clarify the role of S1. S1 ranks candidate regions in Stage 1, and our claim about it concerns ranking rather than causal influence. H1 (Appendix B.2) states that the injected region appears among high-S1 candidates. Appendix D.6 confirms this on all three benchmarks (Table 11), and Figure 7 shows that the S1 ranking reaches near-saturated injection recall.
Erasing the highest-S1 region changes the answer far more often than erasing a low-S1 region of matched area. To address the causal question directly, we ran a region-erasure experiment on 1,000 randomly sampled successful SafeRegionVPI attacks on LLaVA-1.5-7B. For each image, we held the question fixed and erased the text strokes in either the highest-S1 region or an area-matched text region drawn at random from outside the Stage 1 candidate set. We measured answer change relative to the answer before erasure.
| Text region erased | Answer changed (%) | Attack still succeeded (%) |
|---|---|---|
| Area-matched random region outside the Stage 1 candidate set | 4.5 | 98.2 |
| Highest-S1 region | 51.4 | 59.7 |
Erasing the highest-S1 region changed the answer in 51.4% of cases and disrupted 40.3% of attacks, compared with 4.5% and 1.8% for a low-S1 region. With 1,000 images, every rate in the table has a 95% binomial interval within ±3.1 points. Because the two regions are matched in area, the gap does not come from erasing more text, and the low-S1 condition shows that text removal itself rarely changes the answer. High-S1 regions therefore causally affect the prediction far more often than low-S1 regions. In the revision, we will describe S1 as an attention-based ranking signal and add the erasure results.
Author Response to Reviewer H6Vc (Summary)
We thank the reviewer for the insightful feedback and suggestions. We summarize our updates below and address each comment in detail in the following sections:
Validation of the self-reports: We have added deletion-based masking tests confirming that Protocol A boxes faithfully capture the causal evidence used by the model.
Role and causal impact of S1: We clarify S1's role as a Stage 1 ranking signal and have added region-erasure experiments showing that high-S1 regions causally affect predictions and attack success far more often than low-S1 regions.
Distillation motivation and trajectory latency: We clarify the black-box deployment and latency benefits of distillation and have added full-trajectory WebVoyager timing showing that VPIXDistill-D substantially reduces cumulative overhead compared with VPIXDistill-X.
Clean web-agent task completion: We have added an end-to-end evaluation on live WebVoyager tasks in Appendix D.12, confirming that VPIXDistill-D fully preserves the agent's clean task-completion rate.
Teacher setup and matched Label-only baseline: We clarify the teacher and threshold configuration for closed-source models and have added a matched Label-only baseline, showing that VPIXDistill-D achieves higher out-of-distribution recall across all settings.