- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses the problem of graphic design generation from multiple conditions, including a global prompt, a semantic layout (involving regional bounding boxes and prompts), textual contents, and a set of glyph and subject reference images.
The paper proposes a unified conditioning method that, instead of processing different conditions with distinct conditioning mechanisms as in prior work, represents all input conditions with spatially aligned two images: a semantic image containing a set of colored bounding boxes, where the global prompt, regional prompts and textual contents are rendered; an appearance image placing the glyph and subject references at their target locations.
Critical strengths
The quantitative results of the proposed method (as shown in Table 1) are promising.
Critical weaknesses
-
The paper’s motivation is questionable. This is no convincing discussion about fundamental shortcomings of the traditional conditioning method that processes different conditions through separate encoders. In other worlds, it is not clear why a unified conditioning method is needed. The proposed unified conditioning method is shown to outperform the existing methods (such as CreatiDesign), but the reasons for the performance improvement are not given. Some detailed discussions about the pitfalls of existing methods and the advantages of the proposed method should be added, e.g., in the introduction and experiment sections, to justify the necessity of the proposed unified conditioning method.
-
Technical novelty is limited. The main contribution of the paper is a simple visual representation of input conditions, whose technical novelty scale is not significant enough for an ICLR paper.
Questions for authors
-
What about rendering all input conditions into a single image (e.g., the top-left image in Figure 15), instead of the two images?
-
What are the ablation results on the three data sources used for training data construction (Section 3.4)?
3 comments
Response to Reviewer 1WPU Part 2
Q2: Ablation Results on the Three Training Data Sources
As detailed in Section 3.4 and Appendices D–G, replacing the base generator's native text encoder with a frozen visual reader requires first aligning the visual adapter on global captions (Table 5, Stages a–b), and then progressively integrating our three structured data sources: (1) CreatiDesign (394k) for multi-condition layout/subject grounding, (2) SAM 3 Structured Image-Caption Data (~1.14M, Appendix F) for open-domain regional semantic expansion, and (3) Online Synthetic Visual-Text Images (Appendix G) for exact character-level typography.
To track this cumulative data and resolution curriculum on a strictly matched evaluation set (since the CreatiDesign-only probe following caption alignment was trained at square resolution prior to multi-aspect training), we evaluate all three progressive checkpoints on the exact same 108 square benchmark designs (101 evaluated object boxes and 136 target text instances):
| Data-Source Configuration (All 108 Square Benchmark Designs) | CLIP-I ↑ | DINO-I ↑ | M-DINO ↑ | Layout Spatial ↑ | Color ↑ | Texture ↑ | Shape ↑ | Text Spatial ↑ | Sen. Acc ↑ | NED ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| 1. Base Caption Alignment + CreatiDesign Only (394k, ) | 85.93 | 80.09 | 54.46 | 75.25 | 53.47 | 55.45 | 55.45 | 50.46 | 6.94 | 72.11 |
| 2. + SAM 3 Structured Image-Caption Corpus (~1.14M, App. F, ) | 90.37 | 88.76 | 71.16 | 77.23 | 65.35 | 66.34 | 64.36 | 51.19 | 31.64 | 86.43 |
| 3. + Online Synthetic Visual-Text Stream (App. G, Full Mixture, , Ours) | 90.79 | 89.78 | 72.97 | 81.19 | 72.28 | 72.28 | 71.29 | 61.99 | 77.62 | 94.91 |
(Note: On the full 1,000-design multi-aspect benchmark at , adding the Online Synthetic Visual-Text Stream [Config. 2 → Config. 3] similarly improves performance across the board: DINO-I 88.40 → 89.81, M-DINO 69.66 → 72.76, Layout Spatial 76.09 → 80.57, Text Spatial 47.97 → 60.12, Sen. Acc 36.98% → 85.60%, and NED 86.73 → 97.15.)
This matched comparison clarifies the distinct role of each data source:
- 1. CreatiDesign (394k) Drives Core Layout & Subject Grounding: Even at resolution without SAM 3 regional image-caption data or synthetic text streams, training on CreatiDesign alone already achieves strong regional layout adherence (
75.25Layout Spatial,50.46Text Spatial) and subject fidelity (80.09DINO-I). (Furthermore, in our strictly matched 10k control on Qwen-Image-2.1 in Reviewer g2o9 Table R3—where the VLM is already aligned and no private corpus is needed—training on identical data improves Layout Spatial from80.47to91.05, confirming that spatial control stems from our dual-canvas representation.) - 2. SAM 3 Structured Image-Caption Corpus (~1.14M, Appendix F) Expands Regional Semantics & Subject Fidelity: Moving from to training with the regionally grounded image-caption corpus (Figure 3) and continued CreatiDesign fine-tuning boosts multi-subject fidelity by
+8.67 ppDINO-I (80.09 → 88.76) and+16.70 ppM-DINO (54.46 → 71.16), improves all four layout dimensions (+11.88 ppColor,+10.89 ppTexture), lifts Sen. Acc from6.94%to31.64%(86.43NED), and equips the visual reader with broad open-domain entity concepts that support zero-shot hand-drawn sketch control (Figure 5). (On our Qwen-Image-2.1 canvas variant, visual-canvas prompting similarly preserves general T2I quality at 82.15% on GenEval and 85.82% on DPG-Bench.) - 3. Online Synthetic Visual-Text Stream (Appendix G) Unlocks Exact Character-Level Typography: Because general image-caption pairs and commercial posters have limited string diversity, exact spelling remains moderate (
31.64%subset /36.98%full benchmark) prior to introducing synthetic text supervision. Adding the online synthetic visual-text stream (with character-level edit hard negatives in Table 6) drives a decisive leap in exact spelling—boosting Sen. Acc from31.64%to77.62%(+45.98 pp) on the 108 square designs and from36.98%to85.60%(+48.62 pp) on the full 1,000-design benchmark—while bringing Layout Spatial (81.19subset /80.57full) and Text Spatial (61.99subset /60.12full) to their peak.
Looking Forward to Your Feedback
Dear Reviewer 1WPU,
Thank you again for your time and constructive comments on our motivation, single- vs. dual-image design, and training data sources.
In our response (Parts 1–2), we clarified why semantic instruction reading and pixel appearance preservation require decoupled pathways and added a reference-routing diagnostic on the 152-design subset showing that routing subjects onto a single semantic image drops DINO-I from 89.90 to 42.76 (37.49 without VAE) while hurting Layout Spatial (Part 1), highlighted our matched cross-backbone controls isolating the visual interface's gains (+10.58 pp Layout Spatial on Qwen-Image-2.1 and +17.09 pp on Z-Image), and reported the progressive data-source comparison on the exact same 108 square designs demonstrating monotonic improvements across all 10 metrics (with synthetic visual-text boosting Sen. Acc by +45.98 pp, Part 2).
As the discussion period draws to a close, we would be truly grateful if you could let us know whether our response has addressed your questions. If anything remains unclear or unconvincing, we would be glad to provide further clarification. Thank you again for your time and help in improving our paper!
Best regards,
The Authors
Response to Reviewer 1WPU Part 1
We thank Reviewer 1WPU for the constructive comments on our motivation, single- vs. dual-image design, and data ablations.
W1, W2 & Q1: Motivation, Technical Novelty, and Single-Image vs. Dual-Image Conditioning
1. Why Two Images Instead of a Single Image? (Q1 & W1) At the user authoring level, designers work on a single unified canvas (Figure 1(a) and Appendix R, Figure 15), which our interface automatically splits into the Semantic Image and Appearance Image. At the model encoding level, however, feeding everything into a single image fails because reading semantic instructions and preserving instance appearance require conflicting feature invariance:
A new reference-routing diagnostic on the 152-design ablation subset (holding the checkpoint, prompts, glyphs, and seed 42 fixed, varying only subject placement) confirms this quantitatively alongside Table 3:
Using a single semantic encoder without the VAE pathway or routing subject references onto the semantic image causes DINO-I to plummet from 89.90 to 37.49 / 42.76 and M-DINO from 73.55 to 13.90 / 16.46, while also degrading Layout Spatial from 82.19 to 71.92 / 80.14. Conversely, duplicating subjects across both images yields no subject gain while hurting Layout Spatial (79.45 vs. 82.19). Decoupled dual-image conditioning is therefore essential to achieve both high layout adherence and exact identity retention.
2. Technical Novelty and Advantage Over Prior Methods (W1 & W2) Conventional design pipelines rely on separate modality encoders and handcrafted routing rules (e.g., serialized coordinates or regional attention masks), while prior visual-native methods (Appendix A, Table 4) either omit text prompts (Prompt-Free Diffusion), handle only global prompts (UniModel), or lack full-page multi-reference design support (UniVL). Our novelty lies in: