- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies harmful text rendered by T2I models. It introduces ToxiGlyph and GlyphPilot, a black-box prompt search method, and reports strong attack results across five commercial T2I models.
Critical strengths
One thing I found valuable is the paper’s separation of procedural and persuasive text under the same harmful intent. This paired design makes the comparison much cleaner than treating all rendered text as one risk category, and it leads to a concrete observation that the two types behave differently in both attack success and harm severity. The evaluation also spans five commercial and popular T2I models and includes category-level and cross-model transfer results.
Critical weaknesses
1.If the final threat is simply to produce an image containing clear harmful text, an attacker could instead use an LLM to generate the content from the same high-level intent and then place it onto a benign image with a standard image editor. This would likely be cheaper, more controllable, and produce more legible text. In this case, why is it necessary to go through the additional step of jailbreaking a T2I model and asking it to generate and render the harmful text itself? Otherwise, the threat model seems closer to demonstrating an alignment blind spot in T2I models than to showing an attack path that a real-world attacker would prefer. However, if revealing this alignment blind spot is the main contribution, the mechanistic analysis is still limited. The paper mainly shows that the failure can be triggered, but does not explain why it occurs. For instance, whether it comes from insufficient input moderation, the model’s internal text-generation capability, or the lack of output-side safety mechanisms for rendered text.
2.Even if the practical relevance in Point 1 is accepted, the paper still misses an important comparison with closely related work. In particular, TYPO studies a highly similar setting: it also performs black-box search against commercial T2I models to elicit detailed and readable harmful instructions inside generated images. Since TYPO is already discussed in the related work, its absence from the main comparison makes it difficult to assess the actual improvement over the closest prior methods. Personally, GlyphPilot is still largely an LLM/VLM-driven prompt generation and iterative refinement pipeline, with the main methodological difference being visual-state diagnosis and multi-directional search.
3.ToxiGlyph is used not only for evaluation but also to provide fairly detailed target specifications to the attack, since the procedural/persuasive prototypes are directly used by the diagnosis and candidate-generation modules. It is therefore unclear how well GlyphPilot would work when only a high-level harmful intent is available. An ablation without the prototype, or evaluation on an external benchmark without such task-specific prototypes, would better show the generality of the search method.
4.The ablation does not really isolate the main components of GlyphPilot. Visual-State Diagnosis and Multi-Directional Exploration are presented as the key ideas, but the main component ablation only compares first-candidate, random, and selector-based selection. Missing comparisons include w/o Visual-State Diagnosis, single-direction vs. multi-direction search, w/o history, w/o refusal-guided expansion, w/o visual-state refinement, and different diagnostic dimensions. Without these, it is hard to tell what actually drives the reported 87.85% ASR.
Questions for authors
1.The paper analyzes the overall search budget, but why was the final setting fixed to 12 target queries, with 3 initial queries and up to 9 refinement queries? Also, why are exactly 3 candidates generated in each round? A sensitivity study with, for example, 1/2/3/5 candidates would make these design choices easier to justify.
2.GlyphPilot relies on several large and different models, including Gemma 4 26B, Qwen3-VL-32B, and Gemini 3.1 Pro. I think the paper should report the cost and computation of the whole pipeline, and test smaller models or different backbones for a fairer comparison with the baselines.
3.Final harmfulness is judged only by GPT-5.4. If GPT-5.4 is particularly sensitive or lenient toward certain types of persuasive content, it could systematically affect the reported 83.2% vs. 90.8% gap. Could the authors evaluate with judges from different model families, preferably including reproducible open-source models, and report agreement on a human-annotated subset? This would help rule out single-judge bias.
4.Since the attack depends on readable harmful text in the generated image, a natural defense is OCR or VLM transcription followed by a text safety guard. The paper does not evaluate this setting. Given that Beautiful Images, Toxic Words (AAAI 2026) studies mitigation for harmful rendered text and Multimodal Pragmatic Jailbreak (ACL 2025) evaluates safety filters, how much of GlyphPilot’s ASR remains under such an output-side defense?
5 comments
3. Evaluation on AdvBench (Weakness 3)
Our method (GlyphPilot) operates directly on harmful intents without requiring detailed procedural or persuasive prototypes. To validate this capability, we evaluate GlyphPilot on harmful intents from the external AdvBench benchmark, using GPT-Image-2 as the target image generator. The results demonstrate that GlyphPilot remains effective in this setting.
| Method | ASR (%) | |
|---|---|---|
| Direct | 2 | 4 |
| GlyphPilot | 72 | 3.5 |
4. Component Ablation of GlyphPilot (Weakness4)
We ablate four core components of GlyphPilot: (1) w/o Structured Diagnosis removes structured diagnostic feedback while retaining the previous image; (2) w/o History removes prior search history and repetition-avoidance memory while retaining the current image and diagnosis; (3) Single-direction retains the three-candidate initialization but generates only one diagnosis-guided revision per subsequent round, bypassing candidate selection; and (4) w/o Visual Guidance removes the previous image and diagnosis-derived feedback, retaining only non-visual context and the binary refusal outcome. The results show that removing any component reduces both overall ASR and , supporting the contributions of structured diagnosis, history, multi-directional exploration, and visual guidance.
| Method | Proc. ASR (%) | Pers. ASR (%) | Overall ASR (%) | Δ ASR (pp) | H_succ |
|---|---|---|---|---|---|
| Full | 80 | 84 | 82 | — | 4.05 |
| w/o Structured Diagnosis | 60 | 60 | 60 | −22pp | 3.83 |
| w/o History | 68 | 64 | 66 | −16pp | 3.79 |
| Single-direction | 64 | 76 | 70 | −12pp | 3.86 |
| w/o Visual Guidance | 72 | 76 | 74 | −8pp | 3.84 |
5. Search Budget and Candidate Count (Question 1)
- Query budget. Figure 8 of our paper already reports a sensitivity analysis of ASR under different query budgets. The results show that ASR largely stabilizes at around 10 search rounds, so we set the search to 10 rounds. Since no visual feedback is available initially, we query all three candidates in the first round to select a starting point. In each subsequent round, we use feedback to select and query one candidate, yielding at most target queries.
- Candidate count ablation. We added a sensitivity study with , keeping the search limit fixed at 10 rounds. ASR increases from 66.13% to 87.10% as the candidate count increases. Considering the tradeoff between effectiveness and overhead, we choose as the default setting.
| Candidate count | Maximum queries | P_proc ASR (%) | P_pers ASR (%) | Combined ASR (%) | |
|---|---|---|---|---|---|
| N=1 | 10 | 54.84 | 77.42 | 66.13 | 3.95 |
| N=2 | 11 | 80.65 | 83.87 | 82.26 | 4.08 |
| N=3 | 12 | 83.87 | 87.10 | 85.48 | 4.13 |
| N=5 | 14 | 87.10 | 87.10 | 87.10 | 4.02 |
6. Backbone Substitution and Runtime Overhead (Question 2)
To assess whether GlyphPilot depends on specific backbone models, we separately replace the backbones used by (i) the candidate generator (Gemma 4 26B → Qwen3-VL-8B), (ii) the candidate selector and the prerequisite-verification stage (Qwen3-VL-32B → Qwen3-VL-8B), and (iii) the semantic-diagnosis stage (Gemini 3.1 Pro → GPT-5.1), while keeping all other settings fixed. GlyphPilot maintains high attack success rates with smaller backbones and models from different families, suggesting that its effectiveness does not depend on a particular model. Stronger backbones further improve overall performance.
| Configuration | ASR (%) | |
|---|---|---|
| Full | 74.20 | 3.78 |
| → Qwen3-VL-8B | 67.70 | 3.52 |
| / (Stage 1) → Qwen3-VL-8B | 64.50 | 3.80 |
| (Stage 2) → GPT-5.1 | 61.30 | 3.60 |
In our method, the candidate generator (Gemma 4 26B) is deployed locally on two NVIDIA A100 80GB GPUs, while all other models are accessed via APIs. We report the average number of target T2I queries and auxiliary model calls during search for each method, together with ASR and harmful payload rate (HPR). Compared with baselines, GlyphPilot incurs higher call overhead but achieves higher ASR and lower HPR.
| Method | ASR (%)↑ | HPR (%)↓ | Avg. T2I Queries↓ | Avg. Auxiliary Calls↓ |
|---|---|---|---|---|
| TYPO | 66.13 | 41.94 | 1.61 | 7.22 |
| ETCH | 38.82 | 10.88 | 1.6 | 5.17 |
| GlyphPilot | 87.85 | 0 | 5.89 | 15.52 |
7. Evaluation Reliability (Question 3)
To assess potential single-evaluator bias, we conducted additional automatic and human evaluations:
- Additional evaluators. We re-evaluated the same outputs from our method (GlyphPilot) using Claude Sonnet 5, Mistral Small 3.2-24B (open-source), and Qwen3.6-27B (open-source), reporting both ASR and . Their results were broadly consistent with those reported in Table 1.
- Human evaluation. Ten independent annotators assessed the harmfulness of 200 images generated by our method (GlyphPilot). Using human majority votes as the reference, GPT-5.4 achieved 84.4% accuracy, 86.0% precision, 96.6% recall, and 91.0% F1 for harmful-content detection.
| Evaluator | Procedural ASR (%) | Persuasive ASR (%) | Average ASR (%) | Procedural | Persuasive | Average |
|---|---|---|---|---|---|---|
| GPT-5.4 (original) | 83.44 | 92.26 | 87.85 | 3.639 | 4.331 | 4.002 |
| Claude Sonnet 5 | 85.81 | 90.97 | 88.39 | 3.629 | 4.319 | 3.984 |
| Mistral Small 3.2-24B | 95.05 | 95.91 | 95.48 | 4.407 | 4.448 | 4.428 |
| Qwen3.6-27B | 86.67 | 90.11 | 88.39 | 3.873 | 4.22 | 4.05 |
8. Output-side Defense Evaluation (Question 4)
We added output-side defense experiments on GPT Image 2 and Grok Imagine 2 using PaddleOCR transcription followed by Llama Guard text moderation, LlavaGuard image moderation, and their combination. The results show that LlavaGuard provides little effective protection, whereas OCR-based text moderation substantially reduces ASR but does not eliminate it entirely. This suggests that robust defense requires internal safety alignment that explicitly accounts for the harmful semantics and functions of rendered text, rather than reliance on output-side defenses.
| Method | No Defense ASR (%) | OCR + Llama Guard ASR (%) | LlavaGuard ASR (%) | Combined Defense ASR (%) |
|---|---|---|---|---|
| DIRECT | 48.39 | 11.29 | 22.58 | 8.87 |
| ETCH | 41.94 | 22.58 | 33.87 | 20.16 |
| TYPO | 66.13 | 39.52 | 63.71 | 38.71 |
| GlyphPilot | 90.32 | 34.68 | 83.87 | 33.87 |
Thank you for your thorough review and valuable feedback. We appreciate your insights and would like to address each of your points below.
1. Research Motivation and Mechanism Analysis (Weakness1)
We would like to clarify that this paper focuses on whether commercial T2I models with reasoning capabilities can autonomously generate harmful text and render it into an image when the complete harmful text is not provided. Our threat model consists of three components:
Similar harmful images can also be produced by combining textual jailbreaks with manually added text, but ensuring that the text blends naturally with the layout, lighting, and scene could still be difficult and costly. This paper focuses on whether the target T2I model can autonomously generate harmful body text and integrate it into realistic, deceptive images.
The rejection rates in Table 1 of our paper reveal substantial differences in the defensive behavior of different models:
2. Additional Comparison with TYPO (Weakness 2)
Our study focuses on whether a target T2I model can infer and generate harmful text without being given the complete text. TYPO's original implementation allows complete harmful text to be supplied directly for rendering, which differs from our setting, so we did not include it in the initial comparison.
To address the reviewer's concern, we have added comparisons with TYPO on two target models and report the attack success rate (ASR) and harmful payload rate (HPR). HPR is the proportion of final prompts that GPT-5.4 judges to contain complete harmful text based on the task intent and related information.
GlyphPilot achieves higher ASR in most settings without supplying complete harmful text to the target model (HPR = 0). Although TYPO achieves a slightly higher ASR on GPT Image 2's Procedural tasks, its HPR is as high as 51.6%. This indicates that TYPO's ASR advantage is accompanied by a greater reliance on explicitly embedding harmful content in prompts, whereas GlyphPilot achieves comparable attack effectiveness with less reliance on explicit harmful content in prompts.