- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Pictionary, a prompt injection defense that renders untrusted documents and tool outputs as images. Trusted instructions remain in text. The method uses differences in instruction following across modalities to reduce attack success. The authors evaluate ten models on two benchmarks using static attacks, adaptive attacks, and human red teaming. They also examine task utility and the role of training.
Critical strengths
1.Important problem. Prompt injection through external documents and tool outputs is a practical concern for deployed language model agents.
2.Broad evaluation. The experiments cover adaptive attacks, human red teaming, benign task utility, and transcription accuracy.
Critical weaknesses
1.Limited novelty and mechanistic insight. Modality differences in instruction following and safety are widely studied. The paper does not clearly establish a substantive advance over this knowledge. Increased image-injection susceptibility after benign image-instruction fine-tuning is also an expected result.
2.Reduced utility on general tasks. Rendering external content as images adds recognition overhead and can impair tasks that require exact text processing. Table 4 reports a 9–10 percentage-point drop in SWE-bench success for two models.
Questions for authors
1.Contribution clarification: What new scientific conclusion goes beyond existing findings on multimodal jailbreaks and instruction following? Please distinguish a new application from a new mechanistic insight.
2.How can you ensure that rendering text as images does not degrade performance on benign tasks?
Thanks for taking the time to review our paper.
Could the reviewer point to the specific prior work and findings they have in mind? Our paper concerns prompt injection, not jailbreaking. The two are distinct problems, and findings on one cannot be assumed to hold for the other. Some existing studies of modality gap in jailbreaks (e.g., FigStep [5] ) treat it only as a vulnerability, and the gap they refer to is in safety training, not in instruction following. Balakrishnan et al. [6] report that typographic renderings is less effective than text, but only on two out of the four models tested, and while they name it "typographic prompt injection", the setup is a jailbreak evaluation.
Also, our defense stays effective under strong adaptive attacks, while many previous empirical defenses break down under adaptive attacks.
There have been some studies on modality differences, but they didn’t show that rendering the same text as an image can effectively defend against prompt injection. In particular, Kimura et al. [1] observed that textual prompt injections can be more effective than visual ones for goal hijacking, while Li et al. [2] and Gao et al. [3] investigated visual instruction following and cross-modal instruction conflicts. While these studies establish that input modality can influence prompt-injection behavior, they do not establish that converting untrusted textual content into images constitutes an effective prompt-injection defense.
Our main contribution is to systematically exploit this asymmetry as a practical, training-free defense. Rather than treating visual instructions as an attack surface, Pictionary renders untrusted textual documents and tool outputs as images while keeping trusted instructions textual. We demonstrate that this intervention substantially reduces attack success across ten models and two benchmarks, including under strong adaptive automated attacks and human red teaming. While prior defenses such as Spotlighting and StruQ also explore separating instructions from untrusted data, our approach leverages existing modality-dependent behavior without requiring model retraining or modifications to the underlying model.
Beyond the defense itself, we investigate the generality and origins of this behavior. We show that the effect extends to audio, where converting untrusted text into speech similarly reduces attack success rates, suggesting that the phenomenon is not specific to visual rendering or OCR. We further compare base and instruction-tuned models and show that benign image-instruction fine-tuning increases susceptibility to image-delivered prompt injections. While the latter result may be intuitive, it provides experimental evidence supporting our hypothesis that the modality gap is shaped by the distribution of instruction tuning. We do not claim to fully characterize the underlying mechanism, but rather provide evidence for a training-dependent explanation of the observed asymmetry.
We acknowledge that image rendering can degrade utility, particularly for tasks requiring exact text processing, and explicitly discuss this limitation in the paper (Section 6.3). As with other security defenses, there is no free lunch, and some security–utility tradeoff is unavoidable. However, our results show that this tradeoff is highly task-dependent: utility decreases by only 0.9–2.1 percentage points on τ²-Bench, while SWE-bench exhibits larger drops of 9–10 points for two models and no drop for the third. We further investigate these failures and identify transcription errors in exact-match code editing as an important contributing factor. Thus, while Pictionary may be less suitable for tasks requiring byte-exact processing, it provides substantial security benefits while largely preserving utility on tasks that primarily require semantic understanding.
[1] Kimura et al. Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection. NAACL 2024.
[2] Li et al. Text as Images: Can Multimodal Large Language Models Follow Printed Instructions in Pixels? NeurIPS 2024.
[3] Gao et al. Dissecting Dissonance: Benchmarking Large Multimodal Models Against Self-Contradictory Instructions. ECCV 2024.
[4] Pandey et al. Open Your Eyes: Benchmarking the Detection of Fabricated Realities and Weaponized Ethics in VLM Agents. ECCV 2026.
[5] Gong et al. FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts. AAAI 2025.
[6] Balakrishnan et al. Reading Between the Pixels: Linking Text-Image Embedding Alignment to Typographic Attack Success on Vision-Language Models. ICLR 2026 Workshop on Agents in the Wild.