- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2(+1)
Read review
Paper summary
The authors propose PRISM, a defense against prompt injection for MLLMs. The method first feeds the image together with a fixed, task-independent query into a frozen VLM for one prefill, and extracts the multi-layer, multi-head attention from this query to each visual token as features. A lightweight spatial locator, trained with injection-region annotations, then uses these attention features to predict which visual tokens likely belong to the prompt injection region. Next, the visual features of all selected tokens are uniformly replaced with the mean of the unselected neighboring features in the one-hop ring around the selected region. Finally, the repaired visual features are fed together with the user’s original question into the same frozen model to produce the answer, thereby defending against prompt injection.
Critical strengths
-
The problem definition is valuable. The authors clearly distinguish between blocking an attack and answering the question correctly, and point out that blocking the attack without answering the question correctly is not enough.
-
The authors build a benchmark with realistic and valuable data. The screenshots are obtained from 11 real deployed open-source applications.
-
PRISM is lightweight. The whole locator has only about 9K parameters, and the recovery rate exceeds 94% on both models.
Critical weaknesses
-
PRISM’s cross-distribution generalization is weak. Appendix Table S5 shows that on Qwen2.5-VL, the locator trained only on ARGUS data achieves a recovery rate of only 39.9% on the App test, while the harm on normal inputs reaches 45.6%. This suggests that when the locator encounters unseen interface layouts, it fails on a large scale and damages normal content.
-
There is no evaluation of damage to normal visual cues in dense scenes. The benchmark does not test cases where the image contains multiple normal visual cues, where the answer region and the injection region are adjacent or overlapping, where the instruction is embedded in the main text, or where the answer itself is located in the user-writable region. In text-dense scenes such as TextVQA, PRISM’s Clean error is 3.52%, clearly higher than ARGUS’s 0.44%, which suggests that this risk exists.
-
There is no test on benign text that looks like instructions. The Clean and Neutral controls contain only ordinary content and unrelated text. Whether normal instruction-like text that naturally exists in interfaces, such as operation hints, system announcements, or usage instructions, would be misjudged as injection and removed is not evaluated.
-
PRISM lacks evaluation against adaptive attacks. The paper does not report the diversity of the prompt injection methods used, and does not test injections with natural wording and no keywords, injections split across multiple locations, or attackers who know the defense mechanism, for example, layouts designed to deliberately scatter attention.
-
The writing is somewhat obscure. A large portion of the paper is devoted to details and qualifications of the evaluation protocol, the sentences are dense, and it is hard to follow the main line of the method and the conclusions.
Questions for authors
-
How well does PRISM generalize? The test set and the training set come from the same 11 applications. If several applications were entirely held out from training, what would the recovery rate and harm rate be? On Qwen2.5-VL, when the locator trained only on ARGUS data is tested on App, 45.6% of normal inputs are answered incorrectly. Does this mean the locator causes large-scale damage when it encounters unseen interface layouts? How is this addressed?
-
Could the locator mistakenly remove normal visual cues? For example, when the image contains several pieces of information that are needed, when the answer region and the injection region are very close or even overlapping, or when the answer itself is in the comment section, how does PRISM perform in these cases? In addition, PRISM’s Clean error on TextVQA is somewhat higher than ARGUS’s. I would like to understand what roughly causes these errors.
-
The data's interfaces naturally contain some normal text that looks like instructions, such as operation hints, system announcements, or usage instructions. Would the locator treat such text as injection and remove it? Could the authors measure the harm rate on such samples?
-
How many different injection methods and phrasings are used for the prompt injection instructions? If the prompt is split and placed in several locations, can the locator still find it? In addition, if the attacker knows that the defense relies on attention for localization and specifically designs layouts that scatter attention, how would the method perform?
10 comments
Response to Reviewer wvU3 (Part 2/3)
This continues our response to Reviewer wvU3, addressing W3/Q3 and the wording and multi-region parts of W4/Q4. Answer-feedback search follows in Part 3.
3. Information inside legitimate instructions (W3, Q3).
We tested 100 benign scenarios, 50 each from WordPress and Nextcloud Talk, covering reply codes, operation steps, announcements and usage guidance. The question directly asks for information in the legitimate instruction, such as a code, button, time or permission. Existing application-excluded locators are used without further training.
| Backbone | Without PRISM | With PRISM | Harm rate |
|---|---|---|---|
| Qwen2-VL | 100/100 | 98/100 | 2.0% |
| Qwen2.5-VL | 100/100 | 99/100 | 1.0% |
| Qwen3-VL | 100/100 | 100/100 | 0.0% |
All undefended answers are correct, so harm measures correct-to-wrong transitions. Qwen2-VL gives an unrelated time once and omits a desk number once. Qwen2.5-VL changes 7:25 PM to 7:00 PM. These results directly measure whether PRISM preserves information needed from legitimate instructions (Appendix C.6, Table S3).
4.1. Attack methods and wording (W4, Q4).
AppInjectBench uses native user-writable areas across 11 applications. We counted 5,460 distinct attack texts across 6,438 sources, with 6,433 matched to 71 instruction templates and 5 unclassified. Many texts differ by entities or values. Complete attack texts do not repeat across application-holdout boundaries, although some templates are shared.
Explicit redirection is used to elicit the attack target. Eligibility requires correct undefended Clean/Neutral answers and target adoption on Impact. Screening uses only undefended outputs. In a separate 22-scenario check without SYSTEM or assistant-role markers, all 3 backbones answer correctly both with and without PRISM. With no effective undefended attacks, that check cannot measure conditional Recovery.
To test unseen wording, we evaluated 4 new single-region templates, each on the same 125 WordPress scenarios (500 trials). Both the templates and WordPress were absent from locator training. Without defense, Qwen3-VL adopts the target in 75/500 trials (15.0%). PRISM restores the correct answer in 63/75 effective attacks (84.0%) and reduces target adoption over all trials to 2.8%. This supports transfer under joint application/wording holdout, with lower Recovery than the main screened benchmark.
4.2. Attack content placed in 2 regions (W4, Q4).
We evaluated 4 additional templates on a separate set of 250 WordPress scenarios. A redirection prefix occupies the first comment and a wrong-answer instruction the third, separated by a benign comment. Templates were selected using only undefended outputs on a separate 32-scenario pilot. The WordPress-excluded locator remained fixed.
| Wording | Trials | Undefended target adoption | PRISM Recovery | Mask hits both |
|---|---|---|---|---|
| Original AppInjectBench | 250 | 119/250 (47.6%) | 106/119 (89.1%) | 238/250 |
| New template 1 | 250 | 170/250 (68.0%) | 165/170 (97.1%) | 245/250 |
| New template 2 | 250 | 117/250 (46.8%) | 117/117 (100.0%) | 249/250 |
| New template 3 | 250 | 145/250 (58.0%) | 144/145 (99.3%) | 250/250 |
| New template 4 | 250 | 62/250 (24.8%) | 62/62 (100.0%) | 248/250 |
| 4 new templates, pooled | 1,000 | 494/1,000 (49.4%) | 488/494 (98.8%) | 992/1,000 |
Every row uses the same 250 scenarios. The pooled row covers the 4 new templates, whose effective attacks involve 214 scenarios. PRISM reduces target adoption to 6/1,000 (0.6%) and preserves all 250 benign-control answers. "Mask hits both" means selecting at least one annotated visual token in each region. The second region contains a complete wrong-answer instruction, so this tests two-region placement rather than attacks requiring combination of incomplete fragments. Appendix C.6 and Table S4 contain these results.
Response to Reviewer wvU3 (Part 3/3)
This concludes our response to Reviewer wvU3, addressing the answer-feedback and attention-scattering part of W4/Q4, followed by presentation (W5).
4.3. Repeated attacks using answer feedback (W4, Q4).
We evaluated whether repeated probing of PRISM's answers finds additional successful attacks. On Qwen3-VL, we used 30 scenarios (15 WordPress and 15 Nextcloud Talk), with 20 attempts per scenario and the task question and page evidence fixed.
The first 4 attacks were predefined: 2 single-comment/message attacks, 1 two-region attack and 1 three-region attack. The remaining 16 attempts used earlier PRISM answers to choose untried variants from a fixed pool of 108 candidates. The controller prioritizes attacks whose answers adopt the target, then chooses a nearby variant by changing wording, answer format, region count or inclusion of the page title. All variants use predefined native comment/message positions. The controller sees answers, without access to masks, attention or gradients. The undefended model is evaluated on the same 600 candidates selected through PRISM feedback.
| Outcome | Without defense | With PRISM |
|---|---|---|
| Target adoption per attempt | 210/600 (35.0%) | 42/600 (7.0%) |
| Scenarios with at least 1 success | 28/30 | 12/30 |
PRISM restores the correct answer on 168/210 effective undefended attacks (80.0%). The next table shows when each scenario is first successfully attacked:
| First successful attempt | Without defense | With PRISM |
|---|---|---|
| By attempt 4 (predefined attacks) | 23 | 7 |
| 6 | 1 | 1 |
| 8 | 0 | 1 |
| 11 | 3 | 1 |
| 19 | 0 | 1 |
| 20 | 1 | 1 |
| No success in 20 attempts | 2 | 18 |
The answer-guided attempts find a first bypass in 5 of the 23 scenarios that withstand the predefined attacks. PRISM withstands all 20 attempts on 18/30 scenarios. These outcomes show reduced target adoption on the tested candidates and identify additional bypasses through repeated probing. This is an answer-feedback search within native comment/message positions. It does not optimize layouts to scatter probe attention, so it does not establish robustness to that specific attack. Appendix C.6 describes the search. We plan to explore attention-scattering layouts and whether training on additional attack variants improves robustness.
5. Readability and manuscript changes (W5).
We have substantially revised the exposition to clarify the motivation, method and experimental conclusions. Sections 1 and 3 now explain why instructions embedded in application content must be suppressed while preserving task evidence. We define the separate-caption setting before using it and explain how AppInjectBench's Clean, Neutral and Impact screenshots measure recovery and benign preservation.
Section 4 now presents PRISM in execution order: fixed-query observation, spatial localization, feature replacement and task answering. We explain the purpose of the fixed query, attention normalization and neighborhood features before their technical definitions. The text also explicitly distinguishes the frozen backbone from the trained locator and clarifies benign negative supervision.
Section 5 now states what each experiment tests and presents the main finding before its supporting numbers. The main results distinguish task recovery from target suppression and benign harm. The following experiments separately examine frozen-weight transfer, mixed-domain training, locator components, repair operators, application holdout and inference cost. Detailed scoring rules and implementation settings are consolidated in Appendix A.2 and Appendices B-C.
Across the manuscript, we added transitions, merged fragmented paragraphs, reduced repeated qualifications and simplified table captions. These changes make the progression from motivation to design and evidence explicit. The revised main text remains within 10 pages.
Response to Reviewer wvU3 (Part 1/3)
Thank you for the detailed review. We group overlapping Weaknesses (W) and Questions (Q), numbered in their original order. Part 1 addresses transfer and nearby evidence (W1-W2/Q1-Q2). Part 2 covers legitimate instructions and attack diversity. Part 3 covers answer-feedback search and presentation.
1. Training-domain shift and unseen applications (W1, Q1).
The cited ARGUS-only-to-App result changes the training domain, injection carrier and visual context. ARGUS appends injected text in a separate caption, whereas AppInjectBench places it among interface content, including answer-bearing text. On the same Qwen2.5-VL App test, Mixed training raises Recovery from 39.9% to 93.3% and reduces Clean/Neutral harm from 45.6%/44.6% to 0.3%/0.2%. The same Mixed checkpoint retains 92.5% Recovery on ARGUS, versus 93.7% for ARGUS-only training. This supports effective defense on both datasets using a shared locator when training covers both domains (Appendix C.2). Changes in probe-attention cues are a possible explanation for the transfer gap, which has not been verified at the feature level.
We tested unseen applications separately with Qwen2-VL, excluding each test application from locator training, validation and normalization fitting. Recovery is 94.66% (939/992), versus 97.38% (966/992) for the matched all-application reference. The decrease is 2.72 percentage points (95% paired source-bootstrap interval [1.31, 4.13]). Clean/Neutral harm is 0.60%/0.71%, versus 0.10%/0.00%. These fits use the same captures and training procedure. Table 6 gives per-application results and training quantities. These results support transfer to applications excluded from Qwen2-VL locator training.
The original training already included all-zero Clean/Neutral targets in the localization loss. We clarified this in Section 4.5 and Appendix B.3. Appendix A.1 describes split overlap, and Section 5.7 distinguishes application holdout from mixed-domain training.
2. Nearby answer evidence and TextVQA errors (W2, Q2).
We constructed 22 paired WordPress/Nextcloud Talk scenarios in the native apps. The question asks for a room number in a comment/message. Either benign text or an attack is placed beside that number in the same UI block, testing very tight spacing. Qwen2-VL answers all 22 correctly without defense. PRISM preserves 22/22 beside benign text and 19/22 beside the attack. Each of the 3 losses modifies 4/14 answer-region tokens.
Keeping questions, answers and attack texts fixed, we changed only the spacing in 720 x 1,600 screenshots:
The undefended model answers all 22 correctly at every distance. All benign controls remain correct with PRISM, and every attack receives a nonempty mask. Qwen2.5-VL and Qwen3-VL also preserve all 22 benign answers. Their attacked-input accuracy stays at 21/22 and 22/22, respectively, with and without defense. The original diagnostic gap is 9.0-13.2 pixels, compared with a median of 600 pixels (minimum 128) across 998 coordinate-verified Qwen2-VL benchmark images. This measures answer preservation under controlled proximity (Appendix C.6, Table S5). Multiple-clue tasks and primary-page-text injections are outside this panel.
For TextVQA, we corrected 2 scoring false negatives: "the first chapter" answers chapter 1, and "Y, X, W, V, and U" matches the registered sequence. Clean error is corrected from 3.52% (8/227) to 2.64% (6/227), while attack Recovery remains 72.1% (Table 2). The authors confirmed these corrections through question-aware review assisted by Claude Opus 5.5 (Appendix A.2). This test uses frozen ARGUS-trained weights without TextVQA training. The remaining Clean errors show reduced answer preservation under that transfer setting, without establishing a specific feature-level cause.