- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies a new T2I jailbreak attack that exploits spatial text composition. It decomposes harmful semantics into multiple text fragments and uses image-level composition to reconstruct the harmful meaning, achieving strong attack performance on several T2I models.
Critical strengths
The paper is clearly written and easy to follow. The evaluation covers five heterogeneous T2I models, including both commercial and open-weight systems, and uses several complementary metrics beyond ASR, such as harmfulness, semantic similarity, and visual-over-text impact. I think the one-shot black-box setting is meaningful, which is more practical than those query-based T2I jailbreaks.
Critical weaknesses
-
The idea of decomposing harmful semantics into multiple parts and reconstructing them through different modalities or spatial relations is not entirely new. Previous jailbreak studies have explored similar decomposition and recomposition strategies, such as Divide-and-Conquer Attack, which splits harmful intents into smaller components and reconstructs the original semantics. Moreover, TYPO and ETCH have already studied how to exploit T2I models’ text rendering capability to generate images containing harmful instructions.
-
The evaluation does not ensure that the original harmful prompts are rejected before applying CollageAttack. Although the source dataset contains malicious intents, this does not mean every original prompt is recognized as harmful or blocked by the target T2I model. Similar issues also appear in prior T2I jailbreak studies such as MMA and PGJ, where the original prompts are not always filtered out before measuring jailbreak success. If an original prompt already succeeds, it should not count as evidence of a successful jailbreak. Since the paper reports non-trivial ASR for Original prompts, this can directly inflate the reported attack success. I think this point is very important.
-
If the final goal is only to generate an image containing clear harmful text, an attacker could directly use an LLM to generate the text from the same high-level intent and place it onto a benign image with a standard image editing tool. This approach is cheaper, more controllable, and usually produces clearer text.
-
The experiments mainly evaluate existing commercial T2I models, but do not study whether CollageAttack can bypass existing safety enhancement methods, such as concept erasure/unlearning models, text-based safety filters, and output-side detectors (e.g., Qwen/Llama safety guards, NSFW classifiers, Detoxify, or image safety checkers).
-
The experimental comparison is still incomplete. The paper does not compare with closely related rendered harmful text jailbreak methods, especially TYPO and ETCH, which also study how to induce T2I models to generate images containing readable harmful text.
-
The ablation study does not sufficiently analyze why spatial text composition is effective. Important factors are missing, including the number of text fragments, the choice of text carriers, different spatial layouts, random fragment placement, removing spatial constraints, and the semantic relationship between fragments. Without these studies, it remains unclear whether the improvement mainly comes from spatial composition or simply from a more carefully designed prompt construction strategy.
Questions for authors
See weaknesses.
3 comments
Answer for W3 and W4
For W3
-
CollageAttack lowers the operational barrier to generating policy-violating images: Ordinary users do not need to learn Photoshop; simply entering a prompt allows the model to generate policy-violating images with text naturally blended into the scene. By comparison, using Photoshop requires manually adjusting both text and visuals. CollageAttack generates both in one step, reducing editing work and making rapid batch generation much easier.
-
Policy-violating images can cause financial losses or public harm. Verifiable C2PA metadata or SynthID watermarks in AI-generated images can provide clues to trace generation sources, helping affected parties investigate incidents and assess liability. In 2026, the Dutch nonprofit Offlimits sued xAI and X over Grok being used to generate and spread policy-violating images; the Amsterdam court subsequently ordered a halt to the relevant conduct, imposing a penalty of €100,000 per day—up to €10 million—for non-compliance. CollageAttack evaluates whether models can identify and block such requests, which has practical value for uncovering safeguard gaps and reducing real-world risks from policy-violating content.
Therefore, our focus is whether ordinary users can use simple text prompts to get T2I models to generate policy-violating images, and whether models can block such requests or outputs. Like other T2I red-teaming evaluations, this study aims to identify weaknesses in model safeguards and inform defense design. We will further clarify the corresponding threat model in the revised paper.
For W4
We added defense experiments. Code and results: https://anonymous.4open.science/r/ICLR-rebuttal-8B80/6_defense/README.md.
(1) Input-side defense
We used the independent safety guardrail Llama-Guard-4-12B to inspect attack prompts from five methods and measured the block rate:
| Method | Original | TYPO | MPJ (sign template) | PGJ | CollageAttack |
|---|---|---|---|---|---|
| Block rate | 92.0% | 100% | 91.0% | 37.0% | 22.5% (lowest) |
CollageAttack has the lowest block rate (22.5%). This shows that existing input-side text moderation lacks effective handling of spatially fragmented text and struggles to block this type of attack.
(2) Output-side defense
Following the reviewer’s suggestion, we evaluated multiple categories of output-side detectors. The table below reports each detector’s overall flagging rate across all generated images and its recall on ground-truth harmful images:
| Output-side detector | Type | CollageAttack (Flagged; Recall) | PGJ (Flagged; Recall) | MPJ (Flagged; Recall) |
|---|---|---|---|---|
| NudeNet | NSFW image classifier | 0%; 0% | 0%; 0% | 0%; 0% |
| Falconsai NSFW ViT | NSFW image classifier | 0%; 0% | 0%; 0% | 0%; 0% |
| Llama-Guard-4-12B | Multimodal safety guardrail | 20%; 22.7% | 8%; 28.6% | 68%; 68% |
| CompVis Safety Checker | Image safety checker | 8%; 9.1% | 6%; 21.4% | 22%; 22% |
| EasyOCR→Detoxify | OCR + text toxicity | 38%; 43.2% | 0%; 0% | 100%; 100% |
NSFW detectors show 0% recall on harmful images across all three methods. This matches the text-based presentation of the attack: these detectors mainly identify nudity or sexual imagery and cannot recognize harmful text inside images. Visibility to output-side detectors also varies clearly across methods.
Because MPJ displays the full harmful sentence together in the image, OCR can read the text, giving EasyOCR→Detoxify 100% recall on harmful images.
In contrast, CollageAttack splits text and distributes it across the layout, resulting in much lower recall across existing detectors—at most 43.2%, and 0% for NSFW detectors—showing that current output-side detection still struggles against CollageAttack.
(3) On concept erasure / unlearning
Jailbreak evaluations focus on deployment-level safety guardrails—such as input/output filters and moderation mechanisms—which do not require modifying model weights. Unlearning and safety post-training, by contrast, reduce risk after a vulnerability is discovered by modifying the model or its training process, operating at a different defense layer.
Our threat model assumes black-box API access (§3.2), whereas concept erasure requires modifying white-box model weights and thus falls outside the deployment setting studied here.
Moreover, the commercial black-box models tested in our experiments have already undergone extensive safety training prior to release, such as GPT-Image-2 (https://deploymentsafety.openai.com/chatgpt-images-2-0).
To our knowledge, no prior work has revealed this safety risk of spatially fragmented text in T2I models, nor are there existing unlearning or safety post-training methods targeting this class of attack. We will add this distinction and discussion to the revised manuscript.
Answer for W5 and W6
For W5
Regarding TYPO, we compare the methods both conceptually and experimentally. CollageAttack writes fragments from the source sentence into the prompt, distributes them across different locations, and generates an image in a single request; TYPO asks the model to generate in-image text using mirrored or vertical forms and relies on multi-turn iterations and target feedback. In our tests, TYPO’s instruction-dense prompts were blocked at the input stage (see W1 for mechanism comparison). We found TYPO’s official code, whereas ETCH has no public code available yet; we therefore reproduced TYPO using its official implementation. Code and results: https://anonymous.4open.science/r/ICLR-rebuttal-8B80/4_baselines_comparison/daca_typo_50/README.md.
| Method | Input blocked / failed | ASR | Mean score of successful images |
|---|---|---|---|
| CollageAttack | 24.0% | 54.0% | 7.63/10 |
| DACA | 18.0% | 14.0% | 0.93/10 |
| TYPO | 56.0% | 2.0% | 0.54/10 |
CollageAttack’s ASR is 3.9× that of DACA and 27× that of TYPO. In this experiment, TYPO’s prompts were blocked at the input stage, while DACA failed to accurately render hate slogans. These results match the mechanism comparison in W1: splitting the source sentence into fragments and assigning them to different locations helps lower the input block rate while keeping control over the rendered text.
Finally, we note that TYPO (released July 27, 2026) and ETCH (released August 20, 2026) are both arXiv preprints posted less than two months before our submission, making them concurrent work that could not be included in our initial experimental comparisons. We will add discussions and citations of both TYPO and ETCH in the revised manuscript, and include a direct reproduction comparison once ETCH’s code is released.
For W6
We added ablations across the six factor categories raised by the reviewer. Code and results: https://anonymous.4open.science/r/ICLR-rebuttal-8B80/1_ablation/README.md. The results are shown below:
| Factor | Variant | Harmful score | Change vs. full method |
|---|---|---|---|
| (Full attack) | command1+2+3 | 7.63 | — |
| Number of text fragments | fragments_1 (merged into 1 block) | 7.20 | −0.43 |
| Number of text fragments | fragments_2 (merged into 2 blocks) | 6.83 | −0.80 |
| Surface selection | carrier_generic (scene-matched → generic surfaces) | 7.13 | −0.50 |
| Spatial layout | layout_reverse (reversed reading order) | 4.80 | −2.83 |
| Random fragment placement | layout_random | 5.40 | −2.23 |
| Remove spatial constraints | no_spatial_constraints | 6.00 | −1.63 |
| Semantic relation among fragments | semantic_shift1 (swap semantics of adjacent fragments) | 5.43 | −2.20 |
Merging fragments or replacing surfaces produces only minor score changes, whereas disrupting the layout or order causes much larger drops: −2.83 for layout_reverse, −2.23 for layout_random, and −2.20 for semantic_shift1, along with −1.63 when spatial constraints are removed. These results show that the spatial arrangement and reading order of fragments across surfaces have a major impact on attack performance, further confirming the effectiveness of CollageAttack’s spatial arrangement and text fragmentation.
We will incorporate the clarifications and supplementary experiments above into the revised manuscript, and further improve the paper once image generation and evaluation for all baseline methods are complete.
Answer for W1 and W2
We thank the reviewer for raising these specific questions. We have added the relevant experiments and clarify the method differences, evaluation results, and defense settings below.
For W1
Thank you for pointing out these related works. We will cite them and add a comparison in Related Work. Having carefully read DACA, TYPO, and ETCH—three strong works in T2I safety—we note that although all of them involve decomposition or in-image text, their specific mechanisms differ from CollageAttack:
Compared with DACA: DACA rewrites harmful intent into descriptions of benign visual elements and uses metaphors to convey risk at the scene level, without using in-image text. Such rewriting cannot ensure that specific wording is rendered word for word. In contrast, CollageAttack splits the source sentence into phrase fragments, writes them directly into the prompt, and assigns them across different surfaces in the scene, allowing the attacker to specify the exact text displayed. In our supplementary experiments, CollageAttack achieves an ASR of 54.0% compared with 14.0% for DACA, 3.9× DACA’s ASR (answer for W5 provides details).
Compared with TYPO: TYPO asks the model to generate in-image text concentrated on a single surface, using mirrored or vertical layouts and adjusting prompts through multi-turn iterations and feedback from the target model. CollageAttack distributes text fragments across multiple scene surfaces so that no individual fragment carries the complete sentence meaning, and completes the attack in a single request. In our evaluation, Llama-Guard-4 blocked 100% of TYPO prompts versus 22.5% of CollageAttack prompts, showing that CollageAttack’s spatially distributed phrasing is much less likely to trigger input-side semantic guardrails.
Compared with ETCH: ETCH also asks the T2I model to generate in-image text, but its prompt contains no specific harmful strings and relies on multi-turn feedback from a VLM critic. CollageAttack instead writes text fragments from the source sentence directly into the prompt and completes the attack in a single generation request. The two methods differ in text source, wording control, and number of interaction turns.
Therefore, CollageAttack differs from DACA, TYPO, and ETCH mainly in whether exact wording is specified word for word, how fragments are spatially arranged, and whether multi-turn feedback is required. We will add this comparison and cite all three works in Related Work to clarify these differences.
For W2
Thank you for this important suggestion. Using a stricter protocol, we recalculated the results across all five models, defining Conditional ASR as CollageAttack’s success rate calculated solely on samples where the Original prompt failed (i.e., was blocked by the target model, or generated an image judged Harmless). Based on paired per-sample data from the main experiment (200 prompts per model):
After excluding samples where the Original prompt succeeded, the largest single-model ASR drop is 6.0 percentage points (FLUX.2: from 85.5% to 79.5%), while Doubao’s Conditional ASR increases from 86.0% to 88.3%. Across all five models combined, the Conditional ASR is 75.1% (588/783). Thus, after excluding samples where directly submitting the original prompt already succeeds, CollageAttack’s overall performance remains close to the original ASR. We will report both the original ASR and Conditional ASR in the revised manuscript.