acceptodds
Under review as a conference paper at ICLR 2027

VPIXDistill: Securing Visual Inputs against Prompt Injection via Explainability-Guided Distillation

Abstract

Large vision-language models (LVLMs) increasingly use visual text as an important source of task evidence across document, web, and physical-world applications. However, this strength also exposes LVLMs to text-form visual prompt injection (VPI), where malicious text in external visual content can bias the model toward an attacker-chosen answer or hijack it into following an attacker-specified instruction. A practical defense requires more than mitigating attacks. It must suppress malicious visual text while preserving the model's visual-text understanding and reasoning abilities. To make this requirement measurable, we introduce SafeRegionVPI, a bounding-box (bbox) level benchmark with explicit annotations for both injected text and benign text. SafeRegionVPI tests whether a defense distinguishes injected text from benign visual evidence while preserving visual-text understanding beyond a single task-level answer. We further propose VPIXDistill, an explainability-guided defense for visual prompt injection in LVLMs. VPIXDistill uses the LVLM's internal signals to distinguish injected visual text from benign visual evidence and distills them into a lightweight defense model for efficient inference time protection. We evaluate VPIXDistill on SafeRegionVPI, representative VPI benchmarks, and case studies on web agent hijacking and drone task manipulation. VPIXDistill reduces attack impact while preserving model utility with low inference overhead.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI security, visual language model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_H6VcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies defenses against visual prompt injection in LVLMs. The authors introduce SafeRegionVPI for measuring benign evidence preservation and region-aware VPI localization. Based on an analysis of internal VPI signals, they further proposes VPIXDistill-X to locate and suppress injected visual text.

Critical strengths
  1. The paper studies defenses against visual prompt injection while emphasizing the preservation of benign visual text and task utility. This is practical for real-world deployment.

  2. SafeRegionVPI provides fine-grained annotations for both injected text and benign OCR regions.

  3. The method is also effective against some jailbreak attacks.

Critical weaknesses
  1. The paper provides no direct evidence that the model's self-reported bounding boxes reflect the evidence it actually used during inference.

  2. S1's interpretation as instruction-to-region relevance is not justified. A high attention weight does not imply that the region causally affects the model’s prediction. Please add causal intervention experiments to support this interpretation

  3. The need for distillation is not sufficiently motivated. For the model provider, the internal signals required by VPIXDistill-X are available, so the method could be implemented directly without distillation.

  4. The web-agent experiments report AR and SR but do not include clean task-completion rates with and without the defense. While the results show that the defense reduces attack success, its impact on normal task performance remains unclear.

Questions for authors
  1. Do the authors have evidence that the bounding boxes reported by the model in Protocol A correspond to the visual evidence it actually uses?

  2. Which open-source teacher model was used to obtain the VPIXDistill-D detectors? Were the detection thresholds tuned separately for these closed-source models?

  3. About VPIXDistill-D, I would like to see a baseline using the same classifier but trained only with the three-class cross-entropy loss.

  4. See weaknesses.

7 comments
AuthorsOpenReview ↗

Author Response to Reviewer H6Vc (Summary)

We thank the reviewer for the insightful feedback and suggestions. We summarize our updates below and address each comment in detail in the following sections:

  1. Validation of the self-reports: We have added deletion-based masking tests confirming that Protocol A boxes faithfully capture the causal evidence used by the model.

  2. Role and causal impact of S1: We clarify S1's role as a Stage 1 ranking signal and have added region-erasure experiments showing that high-S1 regions causally affect predictions and attack success far more often than low-S1 regions.

  3. Distillation motivation and trajectory latency: We clarify the black-box deployment and latency benefits of distillation and have added full-trajectory WebVoyager timing showing that VPIXDistill-D substantially reduces cumulative overhead compared with VPIXDistill-X.

  4. Clean web-agent task completion: We have added an end-to-end evaluation on live WebVoyager tasks in Appendix D.12, confirming that VPIXDistill-D fully preserves the agent's clean task-completion rate.

  5. Teacher setup and matched Label-only baseline: We clarify the teacher and threshold configuration for closed-source models and have added a matched Label-only baseline, showing that VPIXDistill-D achieves higher out-of-distribution recall across all settings.

AuthorsOpenReview ↗

Author Response to Reviewer H6Vc (Part 1)

The paper provides no direct evidence that the model's self-reported bounding boxes reflect the evidence it actually used during inference.

Do the authors have evidence that the bounding boxes reported by the model in Protocol A correspond to the visual evidence it actually uses?

Yes. Masking tests show that the reported boxes hold the evidence the answer causally depends on, both without and under the defense. Following deletion-based checks of explanation faithfulness [1][2] and redaction checks of LLM self-explanations [3], we use clean TextVQA images from SafeRegionVPI. Each model answers under Protocol A and reports the boxes it used. We then cover selected boxes with gray rectangles, rerun the same prompt, and check which answer words remain. We track candidate words, i.e., content words of the answer that appear in the OCR text of a numbered box.

We mask k reported boxes or k unreported boxes matched in area, selected without looking at their text. In both conditions we track the candidate words found in the selected reported boxes. If the reports did not reflect the evidence the model uses, the two conditions would remove these words at similar rates. For the defended condition, we rerun Protocol A on each defended image, so the answer and the reports are those produced under the defense.

Answer words retained after masking (%)Qwen2.5-VL-7BQwen2.5-VL-7B + VPIXDistill-DInternVL3-14B
Reported boxes masked27.730.733.1
Matched unreported boxes masked82.081.585.3
Paired difference (points)54.3 [48.8, 60.0]50.8 [43.7, 54.6]52.3 [46.6, 58.1]

Masking the reported boxes removes most of the answer words they contain, while masking matched unreported boxes leaves these words in the answer. The gap exceeds 50 points on both models and stays close to its no-defense value under the defense (50.8 versus 54.3 points on Qwen2.5-VL-7B). The reported boxes also hold most of the answer's text-derived evidence. On average, 75.4% (Qwen2.5-VL-7B) and 83.7% (InternVL3-14B) of an image's candidate words occur in them. Omitted evidence does not bias BEUD. The reports cover most, but not all, of the benign text the model uses. To measure omissions, we also test every unreported box that contains a candidate word and record the share of candidate words that occur only in those boxes and disappear from the answer. On the same Qwen2.5-VL-7B images, this omission rate is 19.5% without the defense and 19.9% with it. Because BEUD compares each image with and without the defense, omissions affect both terms similarly and do not bias the change BEUD measures. Together, these tests support using BEUC to answer whether a defense preserves the benign evidence the model uses.

References

  • [1] DeYoung, Jay, et al. ERASER: A benchmark to evaluate rationalized NLP models. Proceedings of the 58th annual meeting of the association for computational linguistics. 2020.
  • [2] Petsiuk, Vitali, Abir Das, and Kate Saenko. Rise: randomized input sampling for explanation of black-box models (2018). arXiv preprint arXiv:1806.07421.
  • [3] Madsen, Andreas, Sarath Chandar, and Siva Reddy. Are self-explanations from Large Language Models faithful?. Findings of the Association for Computational Linguistics: ACL 2024.
AuthorsOpenReview ↗

Author Response to Reviewer H6Vc (Part 2)

S1's interpretation as instruction-to-region relevance is not justified. A high attention weight does not imply that the region causally affects the model’s prediction. Please add causal intervention experiments to support this interpretation

We appreciate the reviewer's point and would like to clarify the role of S1. S1 ranks candidate regions in Stage 1, and our claim about it concerns ranking rather than causal influence. H1 (Appendix B.2) states that the injected region appears among high-S1 candidates. Appendix D.6 confirms this on all three benchmarks (Table 11), and Figure 7 shows that the S1 ranking reaches near-saturated injection recall.

Erasing the highest-S1 region changes the answer far more often than erasing a low-S1 region of matched area. To address the causal question directly, we ran a region-erasure experiment on 1,000 randomly sampled successful SafeRegionVPI attacks on LLaVA-1.5-7B. For each image, we held the question fixed and erased the text strokes in either the highest-S1 region or an area-matched text region drawn at random from outside the Stage 1 candidate set. We measured answer change relative to the answer before erasure.

Text region erasedAnswer changed (%)Attack still succeeded (%)
Area-matched random region outside the Stage 1 candidate set4.598.2
Highest-S1 region51.459.7

Erasing the highest-S1 region changed the answer in 51.4% of cases and disrupted 40.3% of attacks, compared with 4.5% and 1.8% for a low-S1 region. With 1,000 images, every rate in the table has a 95% binomial interval within ±3.1 points. Because the two regions are matched in area, the gap does not come from erasing more text, and the low-S1 condition shows that text removal itself rarely changes the answer. High-S1 regions therefore causally affect the prediction far more often than low-S1 regions. In the revision, we will describe S1 as an attention-based ranking signal and add the erasure results.

Reviewer_nhpZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies text-form visual prompt injection while preserving legitimate visual-text evidence. It introduces SafeRegionVPI, with separate protocols for original-task accuracy, reported evidence usage, and injection localization. VPIXDistill-X combines four model-internal signals, while VPIXDistill-D learns a region detector from privileged teacher targets and region labels, then sanitizes images before a protected LVLM processes them. Evaluation covers typographic and goal-hijacking attacks, commercial models, adaptive attacks, and application case studies.

Critical strengths

1.Selective evidence preservation is made an explicit evaluation objective. Injection recall, benign-text false positives, and exact localization provide a more informative assessment than attacker-output suppression alone. 2.The deployment design is concrete and empirically supported. The distilled detector requires no protected-model internals at inference, and commercial-model experiments directly test this setting. Main results show both reduced attack success and improved user-task performance.

Critical weaknesses

1.The central evidence-preservation measure is an unvalidated self-report proxy. BEUC counts reported bbox IDs without checking transcription accuracy, answer grounding, or causal dependence. Format-validity filtering and defense-specific conditional populations further limit direct comparison. 2.The incremental value of privileged distillation is not isolated. The ablations lack an otherwise matched label-only region verifier. The complete signal combination is also not uniformly preferable: removing S2 improves recall, benign FPR, and exact localization on ARGUS banners in Table 13. Matched-FPR comparisons and uncertainty would strengthen the attribution claim.

Questions for authors

1.How does the method compare with a label-only verifier using matched features, proposals, data, training budget, and validation-based operating-point selection, particularly OOD at matched benign-text FPR? 2.Since injected text in SafeRegionVPI is additionally rendered while benign text primarily originates from the source images, could the authors evaluate hard negatives containing benign text matched in font, size, placement, and rendering procedure? This control would help determine whether the detector identifies malicious semantics or exploits visual artifacts associated with synthetic text insertion.

8 comments
AuthorsOpenReview ↗

Author Response to Reviewer nhpZ (Summary)

Summary of Our Responses

We thank the reviewer for the thoughtful feedback. We summarize the main findings below and address each comment in detail in the following sections.

  1. Validation of self-reports. Masking reported boxes reduced answer-word retention by over 50 percentage points relative to masking unreported boxes. Qwen’s omission rates were similar without and with the defense (19.5% and 19.9%).

  2. Matched baseline comparisons. Matching images separately for each baseline changed its estimates by at most 0.4 percentage points and preserved the direction of every Table 3 comparison with VPIXDistill-D.

  3. Label-only verifier. With Label-only matched to Full’s deployed benign-text FPR on each OOD set, Full achieved 4.3–9.2 percentage points higher injection recall across all four setups.

  4. Contribution of S2. At matched FPR, retaining S2 increased recall on SceneTAP-LingoQA and decreased it on ARGUS banners for both backbones. All four 95% paired bootstrap intervals for the recall differences excluded zero.

  5. Rendering-matched hard negatives. Benign replacements at the injection site were flagged in 32.2% of cases. Redrawing native words in place raised their flag rate from 6.4% to 14.1%. Actual VPI flag rates exceeded 92% in both controls.

AuthorsOpenReview ↗

Author Response to Reviewer nhpZ (Part 1)

The central evidence-preservation measure is an unvalidated self-report proxy. BEUC counts reported bbox IDs without checking transcription accuracy, answer grounding, or causal dependence.

We validated the self-reports with masking tests of causal dependence and answer grounding, both without and under the defense. Following deletion-based checks of explanation faithfulness [1][2] and redaction checks of LLM self-explanation [3], we use clean TextVQA images from SafeRegionVPI. Each model answers under Protocol A and reports the boxes it used. We then cover selected boxes with gray rectangles, rerun the same prompt, and check which answer words remain. We track candidate words, i.e., content words of the answer that appear in the OCR text of a numbered box.

Causal dependence and transcription. We mask k reported boxes or k unreported boxes matched in area, selected without looking at their text. In both conditions we track the words found in the selected reported boxes. An image qualified if it had at least one reported and one unreported box. For the defended condition, we rerun Protocol A on each defended image, so the answer and the reports are those produced under the defense.

Answer words retained after masking (%)Qwen2.5-VL-7BQwen2.5-VL-7B + VPIXDistill-DInternVL3-14B
Reported boxes masked27.730.733.1
Matched unreported boxes masked82.081.585.3
Paired difference (points)54.3 [48.8, 60.0]50.8 [43.7, 54.6]52.3 [46.6, 58.1]

Masking the reported boxes removes most of the answer words they contain, while masking matched unreported boxes leaves these words in the answer. The gap exceeds 50 points on both models and stays close to its no-defense value under the defense.

Answer grounding. On average, 75.4% (Qwen2.5-VL-7B) and 83.7% (InternVL3-14B) of an image's candidate words occur in the reported boxes. Together with the causal test, this shows that the answer is grounded in the boxes the model reports.

Omitted evidence. The reports cover most, but not all, of the benign text the model uses. To measure omissions, we also test every unreported box that contains a candidate word and record the share of candidate words that occur only in those boxes and disappear from the answer. On the same Qwen2.5-VL-7B images, this omission rate is 19.5% without the defense and 19.9% with it. Because BEUD compares each image with and without the defense, omissions affect both terms similarly and do not bias the change BEUD measures. Together, these tests support using BEUC/BEUD to answer whether a defense preserves the benign evidence the model uses.

References

  • [1] DeYoung, Jay, et al. ERASER: A benchmark to evaluate rationalized NLP models. Proceedings of the 58th annual meeting of the association for computational linguistics. 2020.
  • [2] Petsiuk, Vitali, Abir Das, and Kate Saenko. Rise: randomized input sampling for explanation of black-box models (2018). arXiv preprint arXiv:1806.07421.
  • [3] Madsen, Andreas, Sarath Chandar, and Siva Reddy. Are self-explanations from Large Language Models faithful?. Findings of the Association for Computational Linguistics: ACL 2024.
AuthorsOpenReview ↗

Author Response to Reviewer nhpZ (Part 2)

Format-validity filtering and defense-specific conditional populations further limit direct comparison.

Comparing each baseline with VPIXDistill-D on the same images preserved the main findings of Table 3. We repeated the Qwen2.5-VL-7B analysis using a separate matched set for each baseline. Sample sizes differ across rows because the baselines differ in format validity and task accuracy, so each comparison is within a row.

Reported benign-evidence coverage. For each baseline, we retained images with format-valid Protocol A responses under the baseline, VPIXDistill-D, and No Defense, and computed BEUC on this shared set. BEUD is the decrease in reported coverage relative to No Defense on the same images, so a positive value indicates a loss of coverage. As in Table 3, BEUD is computed from unrounded BEUC values.

Baseline paired with VPIXDistill-DPaired imagesBEUC: No DefenseBEUC: baselineBEUC: VPIXDistill-DBEUD: baselineBEUD: VPIXDistill-D
Prompt3,48759.451.961.07.5−1.7
OCR+Prompt3,48759.448.561.010.8−1.7
ARGUS2,78262.663.164.0−0.5−1.4
RIO-RT95954.727.256.227.5−1.6

Hidden evidence loss. Within each matched set above, we further retained only images for which both the baseline and VPIXDistill-D answered the original Protocol B question correctly.

Baseline paired with VPIXDistill-DJointly correct imagesBHEL_c: baselineBHEL_c: VPIXDistill-DBHEM_c: baselineBHEM_c: VPIXDistill-D
Prompt2,90641.832.414.39.3
OCR+Prompt2,81948.532.419.39.2
ARGUS2,29821.130.85.59.2
RIO-RT78174.633.732.98.9

Results. Matching changed the baseline estimates by at most 0.1 points for BEUC, 0.4 for BHEL_c, and 0.2 for BHEM_c relative to Table 3, and baseline BEUD was unchanged at the reported precision. For VPIXDistill-D, BEUD ranged from −1.7 to −1.4 across the four matched sets, compared with −1.7 in Table 3. BHEL_c ranged from 30.8 to 33.7 and BHEM_c from 8.9 to 9.3. ARGUS keeps lower BHEL_c and BHEM_c than VPIXDistill-D on its matched set, as in Table 3. Matching therefore does not change how any baseline compares with VPIXDistill-D.

Reviewer_s5Y3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript targets at text-based visual prompt injection (VPI) in large vision-language models: malicious text inside an image steers the model towards an attacker’s answer or instruction. A benchmark with bounding-box-level annotations of both injected and benign text is constructed. It evaluates whether a defence can separate the two while keeping the model’s visual-text understanding, rather than looking only at the final answer. In addition, it proposes VPIXDistill, an explainability-guided defence that uses the LVLM’s internal signals to identify injected text and distils this into a lightweight model for inference-time protection. The evaluation covers SafeRegionVPI, existing VPI benchmarks, and case studies on web-agent hijacking and aerial tracking prompt injection.

Critical strengths
  1. The method is interpretable and practical. Each flagged region comes with explicit risk signals. The distilled detector needs no access to the protected model’s internals, so it can be deployed in front of closed-source LVLMs.
  2. The evaluation is comprehensive. It covers open-source and closed-source LVLMs, both typographic and goal-hijacking attacks, adaptive attacks, and agentic and physical case studies. The results on goal hijacking are strong. I also appreciate that the appendix discusses failure cases openly, such as unstable localisation across frames in the web-agent study.
Critical weaknesses
  1. The detector may learn artefacts of how the injections were generated rather than what makes text malicious. I suggest evaluating harder cases.
  2. Both adaptive attacks are black-box query-based attacks with small budgets. Evaluating at least one white-box adaptive attack against the detector, and analysing these failure modes, would make the robustness claims much more convincing.
Questions for authors
  1. The distilled detector uses features from a frozen open-source LVLM. Have the authors tried a gradient-based attack that fools the target LVLM while lowering the detector’s score, with the backbone either known or approximated by a surrogate?
  2. I would liek to know where does detection fail? For the injections missed by VPIXDistill-D, what proportion are never flagged at Stage 1, and what proportion pass Stage 1 but are then rejected by the Stage 2 verifier?
4 comments
AuthorsOpenReview ↗

Author Response to Reviewer s5Y3 (Summary)

We thank the reviewer for the constructive feedback and insightful suggestions. We summarize our key updates below and address each comment in detail in the following sections:

  1. Rendering artifact ablation: We have added a re-rendering experiment in Appendix D.8. Re-rendering benign words yields a flag rate far below that of actual VPIs, confirming that rendering artifacts alone do not drive our detection performance.

  2. Gradient-based white-box adaptive attacks: We have updated §5.4 and Appendix D.10 with white-box adaptive attack evaluations targeting the fused score and individual signals. VPIXDistill-D substantially reduces targeted ASR across all settings and confirms the complementarity of our detection signals.

  3. Stage-wise failure mode analysis: We have added a breakdown of missed detections across Stage 1 and Stage 2 across multiple LVLMs and benchmarks. Overall missed detections are rare. Stage 1 passes nearly all injections to the verifier, with the few remaining corner-case misses occurring primarily at Stage 2.

AuthorsOpenReview ↗

Author Response to Reviewer s5Y3 (Part 1)

The detector may learn artefacts of how the injections were generated rather than what makes text malicious. I suggest evaluating harder cases.

We tested whether re-rendering benign text with the attack renderer would reproduce the high flag rate observed for actual VPIs. For each of TextVQA images from SafeRegionVPI, we randomly selected one eligible benign word. We compared the detector’s flag rate for the original word with its flag rate after erasing and re-rendering the same word in its original box using the attack renderer. We also measured the flag rate for actual VPI regions on the same set of images.

Text regionFlag rate
Original benign word6.4%
Same benign word re-rendered in its original box14.1%
Actual VPI region92.7%

Results. Re-rendering increased the benign flag rate by 7.7 percentage points. Even after re-rendering, the benign flag rate remained far below the 92.7% observed for actual VPIs. Artifacts introduced by the attack renderer alone therefore cannot explain the high VPI flag rate in this evaluation. Appendix D.8 provides the full setup and analysis.

AuthorsOpenReview ↗

Author Response to Reviewer s5Y3 (Part 2)

Both adaptive attacks are black-box query-based attacks with small budgets. Evaluating at least one white-box adaptive attack against the detector, and analysing these failure modes, would make the robustness claims much more convincing.

The distilled detector uses features from a frozen open-source LVLM. Have the authors tried a gradient-based attack that fools the target LVLM while lowering the detector’s score, with the backbone either known or approximated by a surrogate?

Yes. We added a gradient-based white-box attack with full access to both the victim and the detector, including the frozen backbone. When the attacker directly targeted the fused detector score while keeping the injected text fixed, VPIXDistill-D reduced cumulative targeted ASR from 38.2% to 23.7%.

We evaluated typographic injections from the SafeRegionVPI TextVQA test split using Qwen2.5-VL-7B and its distilled VPIXDistill-D detector. The attack jointly maximized the likelihood of the target answer and minimized the detector score of the injected region.

  • Rendering only. We kept the injected text fixed and optimized its position, scale, and color, using differentiable spatial transformations for position and scale [1]. Gradients were propagated through the rendered pixels to both the victim and the detector. Because CRAFT box extraction is non-differentiable, we recomputed the text proposals between optimization rounds.
  • Rendering + GCG suffix. To test whether adapting the text offers an additional gain, we also appended a GCG suffix under the same budget, following prior work on adversarial suffixes [2] and their use in text prompt injection [3]. Candidate suffixes were rendered into the image before scoring.

Results. Each entry reports cumulative targeted ASR (%) without / with defense.

Score targeted by the attackerRendering onlyRendering + GCG suffix
Fused score38.2 / 23.737.3 / 15.3
S2 (local conflict) only36.5 / 13.338.2 / 11.2
S3 (global inconsistency) only36.9 / 16.937.8 / 12.0
S4 (attention shift) only39.8 / 20.137.3 / 14.9

Compared with the black-box adaptive attack, the white-box attack reaches similar undefended ASR but evades the detector more effectively. Even so, VPIXDistill-D still reduces ASR from 38.2% to 23.7%. Targeting the fused score is the strongest strategy. Each single-signal objective yields lower defended ASR, consistent with S2, S3, and S4 capturing complementary aspects of VPI. Adding a GCG suffix did not increase defended ASR for any objective under the same budget, possibly because substitutions favored by text-token gradients have less effect when the model reads the suffix as rendered text.

We have added these results and the full attack protocol to section 5.4 and Appendix D.10 .

References

[1] Max Jaderberg et al. Spatial Transformer Networks. Advances in Neural Information Processing Systems.

[2] Andy Zou et al. (2023). Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043.

[3] Sizhe Chen et al. (2025). SecAlign: Defending Against Prompt Injection with Preference Optimization. Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security.

AuthorsOpenReview ↗

Author Response to Reviewer s5Y3 (Part 3)

I would liek to know where does detection fail? For the injections missed by VPIXDistill-D, what proportion are never flagged at Stage 1, and what proportion pass Stage 1 but are then rejected by the Stage 2 verifier?

We evaluated the Qwen2.5-VL-7B and LLaVA-1.5-7B versions of VPIXDistill-D on SafeRegionVPI across DocVQA, TextVQA, and DriveQA-V. For each model, we used the samples on which the original attack succeeded. We attribute a miss to Stage 1 if the injection region is not passed to the verifier, and to Stage 2 if the region reaches the verifier but is not flagged as an injection. The table reports detection and miss rates over all evaluated injections in each row.

DetectorAttackDetectedMissed at Stage 1Rejected by Stage 2
Qwen2.5-VL-7BTypographic87.7%4.2%8.1%
Qwen2.5-VL-7BGoal hijacking96.4%1.8%1.8%
LLaVA-1.5-7BTypographic85.0%2.4%12.6%
LLaVA-1.5-7BGoal hijacking98.2%0.0%1.8%

With a shared top-35% pass ratio, Stage 1 passed at least 95.8% of injections to the verifier in all four settings. Most missed typographic injections reached the verifier but were not flagged (66% of misses on Qwen2.5-VL-7B and 84% on LLaVA-1.5-7B). For goal hijacking, misses split equally between the two stages on Qwen2.5-VL-7B and occurred only at Stage 2 on LLaVA-1.5-7B. We analysis calibrating the pass ratio for each model further reduces Stage 1 misses (Figure 7). Most Stage 2 misses we inspected are short injections in text-dense images that match nearby text in content and typography, such as a date in a table or an item on a menu. Such corner cases are benign in isolation and become adversarial only for a specific question, so they are hard to distinguish from benign text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.