acceptodds
Under review as a conference paper at ICLR 2027

All-in-Image: An Image-Native Generator for Graphic Design

Abstract

Graphic design generation combines requirements for content, layout, and typography with visual references whose appearance should be retained. Existing systems commonly encode these conditions through separate language, layout, and reference interfaces. Our observation is that different control objectives can share a unified semantic representation. Various requirements can all be expressed on a single spatial canvas, which describes the desired design without depicting its final appearance. Based on this observation, we introduce All-in-Image, a unified visual conditioning method that represents design conditions with two aligned images. On the semantic image, a global prompt, regional prompts, and text-rendering instructions are drawn together with their target regions, allowing semantic content and spatial extent to be represented in the same image. On the appearance image, subject and glyph references are placed at their target locations to provide appearance and glyph structure. A semantic encoder reads the written instructions and their spatial context, while a pixel encoder extracts features that preserve reference appearance. Their tokens jointly condition a single-stream diffusion transformer, allowing language, spatial regions, and appearance references to enter the generator through image space without a conventional text encoder or numerical bounding-box inputs. Experiments on the CreatiDesign benchmark show this formulation supports spatial control, text rendering, and subject fidelity jointly, ranking first on nine of ten metrics.

Primary area:
generative models
Keywords:
Diffusion models, Controllable image generation, Visual conditioning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_1WPUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the problem of graphic design generation from multiple conditions, including a global prompt, a semantic layout (involving regional bounding boxes and prompts), textual contents, and a set of glyph and subject reference images.

The paper proposes a unified conditioning method that, instead of processing different conditions with distinct conditioning mechanisms as in prior work, represents all input conditions with spatially aligned two images: a semantic image containing a set of colored bounding boxes, where the global prompt, regional prompts and textual contents are rendered; an appearance image placing the glyph and subject references at their target locations.

Critical strengths

The quantitative results of the proposed method (as shown in Table 1) are promising.

Critical weaknesses
  1. The paper’s motivation is questionable. This is no convincing discussion about fundamental shortcomings of the traditional conditioning method that processes different conditions through separate encoders. In other worlds, it is not clear why a unified conditioning method is needed. The proposed unified conditioning method is shown to outperform the existing methods (such as CreatiDesign), but the reasons for the performance improvement are not given. Some detailed discussions about the pitfalls of existing methods and the advantages of the proposed method should be added, e.g., in the introduction and experiment sections, to justify the necessity of the proposed unified conditioning method.

  2. Technical novelty is limited. The main contribution of the paper is a simple visual representation of input conditions, whose technical novelty scale is not significant enough for an ICLR paper.

Questions for authors
  1. What about rendering all input conditions into a single image (e.g., the top-left image in Figure 15), instead of the two images?

  2. What are the ablation results on the three data sources used for training data construction (Section 3.4)?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 1WPU Part 1

We thank Reviewer 1WPU for the constructive comments on our motivation, single- vs. dual-image design, and data ablations.

W1, W2 & Q1: Motivation, Technical Novelty, and Single-Image vs. Dual-Image Conditioning

1. Why Two Images Instead of a Single Image? (Q1 & W1) At the user authoring level, designers work on a single unified canvas (Figure 1(a) and Appendix R, Figure 15), which our interface automatically splits into the Semantic Image and Appearance Image. At the model encoding level, however, feeding everything into a single image fails because reading semantic instructions and preserving instance appearance require conflicting feature invariance:

  • Semantic encoders (DeepEncoder V2 / CLIP) abstract high-level concepts while discarding pixel-level geometry (e.g., placing a rectangular dining-table photo into the semantic image turns it into a round table in Appendix Q, Figure 14).
  • Pixel encoders (Spatial VAE) preserve exact local structure and texture on the target latent grid.

A new reference-routing diagnostic on the 152-design ablation subset (holding the checkpoint, prompts, glyphs, and seed 42 fixed, varying only subject placement) confirms this quantitatively alongside Table 3:

Conditioning / Routing Configuration (152-Design Ablation Subset)CLIP-I ↑\uparrowDINO-I ↑\uparrowM-DINO ↑\uparrowLayout Spatial ↑\uparrow
Single Encoder / No-VAE Pathway (DeepEncoder V2 only, Table 3)75.3037.4913.9071.92
Single-Image Subject Routing (Subjects placed on Semantic Image)76.6842.7616.4680.14
Duplicated Routing (Subjects placed on Both Images)90.9990.0873.8579.45
Decoupled Dual-Image (Subjects on Appearance Image via VAE, Ours)91.0289.9073.5582.19

Using a single semantic encoder without the VAE pathway or routing subject references onto the semantic image causes DINO-I to plummet from 89.90 to 37.49 / 42.76 and M-DINO from 73.55 to 13.90 / 16.46, while also degrading Layout Spatial from 82.19 to 71.92 / 80.14. Conversely, duplicating subjects across both images yields no subject gain while hurting Layout Spatial (79.45 vs. 82.19). Decoupled dual-image conditioning is therefore essential to achieve both high layout adherence and exact identity retention.

2. Technical Novelty and Advantage Over Prior Methods (W1 & W2) Conventional design pipelines rely on separate modality encoders and handcrafted routing rules (e.g., serialized coordinates or regional attention masks), while prior visual-native methods (Appendix A, Table 4) either omit text prompts (Prompt-Free Diffusion), handle only global prompts (UniModel), or lack full-page multi-reference design support (UniVL). Our novelty lies in:

  1. Formulating a dual-canvas image-native interface that unifies global prompts, dense overlapping regional prompts, exact typography, and multi-subject references in a shared coordinate frame without numerical box inputs or regional attention masks.
  2. Proving through controlled experiments (Reviewer g2o9, Tables R1–R3) that our visual interface outperforms Text + Coordinate conditioning by +17.09 pp Layout Spatial and +23.86 pp Text Spatial on Z-Image, generalizes to Qwen-Image-2.1 (91.05 Layout Spatial, +10.58 pp over matched Text + Coordinates), and enables zero-shot sketch control (Figure 5).
AuthorsOpenReview ↗

Response to Reviewer 1WPU Part 2

Q2: Ablation Results on the Three Training Data Sources

As detailed in Section 3.4 and Appendices D–G, replacing the base generator's native text encoder with a frozen visual reader requires first aligning the visual adapter on global captions (Table 5, Stages a–b), and then progressively integrating our three structured data sources: (1) CreatiDesign (394k) for multi-condition layout/subject grounding, (2) SAM 3 Structured Image-Caption Data (~1.14M, Appendix F) for open-domain regional semantic expansion, and (3) Online Synthetic Visual-Text Images (Appendix G) for exact character-level typography.

To track this cumulative data and resolution curriculum on a strictly matched evaluation set (since the CreatiDesign-only probe following caption alignment was trained at 5122512^2 square resolution prior to 102421024^2 multi-aspect training), we evaluate all three progressive checkpoints on the exact same 108 square benchmark designs (101 evaluated object boxes and 136 target text instances):

Data-Source Configuration (All 108 Square Benchmark Designs)CLIP-I ↑DINO-I ↑M-DINO ↑Layout Spatial ↑Color ↑Texture ↑Shape ↑Text Spatial ↑Sen. Acc ↑NED ↑
1. Base Caption Alignment + CreatiDesign Only (394k, 5122512^2)85.9380.0954.4675.2553.4755.4555.4550.466.9472.11
2. + SAM 3 Structured Image-Caption Corpus (~1.14M, App. F, 102421024^2)90.3788.7671.1677.2365.3566.3464.3651.1931.6486.43
3. + Online Synthetic Visual-Text Stream (App. G, Full Mixture, 102421024^2, Ours)90.7989.7872.9781.1972.2872.2871.2961.9977.6294.91

(Note: On the full 1,000-design multi-aspect benchmark at 102421024^2, adding the Online Synthetic Visual-Text Stream [Config. 2 → Config. 3] similarly improves performance across the board: DINO-I 88.40 → 89.81, M-DINO 69.66 → 72.76, Layout Spatial 76.09 → 80.57, Text Spatial 47.97 → 60.12, Sen. Acc 36.98% → 85.60%, and NED 86.73 → 97.15.)

This matched comparison clarifies the distinct role of each data source:

  • 1. CreatiDesign (394k) Drives Core Layout & Subject Grounding: Even at 5122512^2 resolution without SAM 3 regional image-caption data or synthetic text streams, training on CreatiDesign alone already achieves strong regional layout adherence (75.25 Layout Spatial, 50.46 Text Spatial) and subject fidelity (80.09 DINO-I). (Furthermore, in our strictly matched 10k control on Qwen-Image-2.1 in Reviewer g2o9 Table R3—where the VLM is already aligned and no private corpus is needed—training on identical data improves Layout Spatial from 80.47 to 91.05, confirming that spatial control stems from our dual-canvas representation.)
  • 2. SAM 3 Structured Image-Caption Corpus (~1.14M, Appendix F) Expands Regional Semantics & Subject Fidelity: Moving from 5122512^2 to 102421024^2 training with the regionally grounded image-caption corpus (Figure 3) and continued CreatiDesign fine-tuning boosts multi-subject fidelity by +8.67 pp DINO-I (80.09 → 88.76) and +16.70 pp M-DINO (54.46 → 71.16), improves all four layout dimensions (+11.88 pp Color, +10.89 pp Texture), lifts Sen. Acc from 6.94% to 31.64% (86.43 NED), and equips the visual reader with broad open-domain entity concepts that support zero-shot hand-drawn sketch control (Figure 5). (On our Qwen-Image-2.1 canvas variant, visual-canvas prompting similarly preserves general T2I quality at 82.15% on GenEval and 85.82% on DPG-Bench.)
  • 3. Online Synthetic Visual-Text Stream (Appendix G) Unlocks Exact Character-Level Typography: Because general image-caption pairs and commercial posters have limited string diversity, exact spelling remains moderate (31.64% subset / 36.98% full benchmark) prior to introducing synthetic text supervision. Adding the online synthetic visual-text stream (with character-level edit hard negatives in Table 6) drives a decisive leap in exact spelling—boosting Sen. Acc from 31.64% to 77.62% (+45.98 pp) on the 108 square designs and from 36.98% to 85.60% (+48.62 pp) on the full 1,000-design benchmark—while bringing Layout Spatial (81.19 subset / 80.57 full) and Text Spatial (61.99 subset / 60.12 full) to their peak.
AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer 1WPU,

Thank you again for your time and constructive comments on our motivation, single- vs. dual-image design, and training data sources.

In our response (Parts 1–2), we clarified why semantic instruction reading and pixel appearance preservation require decoupled pathways and added a reference-routing diagnostic on the 152-design subset showing that routing subjects onto a single semantic image drops DINO-I from 89.90 to 42.76 (37.49 without VAE) while hurting Layout Spatial (Part 1), highlighted our matched cross-backbone controls isolating the visual interface's gains (+10.58 pp Layout Spatial on Qwen-Image-2.1 and +17.09 pp on Z-Image), and reported the progressive data-source comparison on the exact same 108 square designs demonstrating monotonic improvements across all 10 metrics (with synthetic visual-text boosting Sen. Acc by +45.98 pp, Part 2).

As the discussion period draws to a close, we would be truly grateful if you could let us know whether our response has addressed your questions. If anything remains unclear or unconvincing, we would be glad to provide further clarification. Thank you again for your time and help in improving our paper!

Best regards,

The Authors

Reviewer_g2o9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes All-in-Image, which represents all conditions for graphic design generation as two spatially aligned images: (1) a semantic image, on which the global prompt, regional prompts, and text-rendering instructions are typeset together with same-colored bounding boxes; and (2) an appearance image, on which subject references (cutouts or crops) and glyph references are placed at their target locations. The semantic image is read by a frozen OCR-pretrained encoder (DeepEncoder V2) with a trainable linear adapter, and the appearance image by the frozen VAE. Both token streams are concatenated with the noisy target tokens and processed by a single-stream DiT initialized from Z-Image with its text-encoder path removed. Training proceeds in four stages: adapter alignment, global-prompt-only generation, regional/text conditioning, and joint semantic–appearance conditioning, using CreatiDesign, ~1.14M internal image–caption pairs structured with a VLM and SAM 3, and online synthetic text images. On the CreatiDesign benchmark, the method ranks first on nine of ten metrics and top-two on all, with fewer parameters (7.05B), lower latency, and lower memory than CreatiDesign. The paper also shows zero-shot hand-drawn canvases, differential-attention visualizations, encoder ablations, and a user study.

Critical strengths

1.Strong main results. The method outperforms CreatiDesign on all ten metrics. Relative to the zero-shot Z-Image starting point, Layout Spatial rises from 59.00 to 80.57 and Text Spatial from 18.15 to 60.12, showing that the full pipeline (representation + task training + data) is effective. The joint pass-rate analysis (App. K) is well done: both methods are evaluated locally on the same 867 designs, with paired bootstrap CIs (+13.26 pp, 95% CI [9.34, 17.07]) and robustness across thresholds 0.70–0.90. (The comparison with CreatiDesign, however, also differs in backbone and data; see W1.) 2.Lighter deployment. 7.05B vs. 17.36B total parameters, 41.7 s vs. 53.6 s per image, and 15.6 vs. 36.7 GiB peak memory, with a carefully specified measurement protocol (App. M). This advantage, however, comes mainly from a smaller backbone and a faster implementation; see W1.

Critical weaknesses

W1. No controlled comparison separates the representation from the backbone, data, and implementation, for either the main results or the efficiency gains. The central claim is that the image-space representation drives the gains, but the comparison with CreatiDesign conflates several factors: (a) a different backbone (Z-Image vs. FLUX), and the paper neither explains the choice of Z-Image nor reports results on FLUX; (b) different training data (an additional 1.14M internal image–caption pairs and synthetic text data targeted at text rendering, including strings the model previously rendered incorrectly); and (c) different inference-time inputs (see W2). All ablations (Tables 2–3, Fig. 6) swap components within Z-Image; none separates backbone effects from method effects. The Z-Image baselines in App. J are zero-shot prompt serializations and are not a fair control.

The backbone matters especially for text. In Table 1, the zero-shot Z-Image base already beats FLUX.1-dev by ~22 points in Sen. Acc (79.84 vs. 57.95) and in NED (92.58 vs. 81.52), while CreatiDesign is built on FLUX. A substantial part of the text-metric lead over CreatiDesign (85.60 vs. 78.30) may therefore come from the backbone. On layout, the two zero-shot starting points are similar (59.00 vs. 60.02), so those gains are more attributable to the method. The efficiency gains are largely attributable to the backbone and implementation. The parameter gap mainly reflects DiT size (~12B for FLUX vs. ~6B for Z-Image); the method's own contribution is replacing the ~4B Qwen3 text encoder with a smaller DeepEncoder, which is a real but small part of the savings. The latency comparison also mixes implementation differences: All-in-Image uses variable-length FlashAttention and batches both CFG branches, whereas CreatiDesign uses SDPA with a structured attention mask and runs the branches sequentially, even though All-in-Image processes more condition tokens (4,096 reference + up to ~1,120 semantic, vs. 1,024 reference for CreatiDesign). For memory, CreatiDesign's text encoder retains FP32 components. The paper also does not report the latency and memory of the original Z-Image with its Qwen3 text encoder.

Missing experiments: (i) a baseline with the same Z-Image backbone, the same data, and the same training budget that uses a conventional text encoder plus region conditioning (e.g., fine-tuning on serialized structured prompts, or CreatiDesign-style structured attention/region masks), or conversely All-in-Image on a FLUX backbone; and (ii) latency and memory of the original Z-Image under the same setup, to quantify the savings from removing the text encoder alone. Without these, it is hard to tell how much of the improvement comes from the All-in-Image representation itself, and the efficiency gains should not be presented as an advantage of the method.

W2. Glyph references at evaluation provide extra information. Per App. E/H, at evaluation the appearance image contains glyph references rendered from the target strings inside their boxes. This gives the model the target text in pixel form, which directly helps Sen. Acc, NED, and Text Spatial, while baselines such as CreatiDesign and BizGen do not receive such input. The text-metric gains (e.g., Text Spatial 60.12 vs. 56.90) therefore cannot be fully attributed to the semantic-image representation. Please report results without glyph references at evaluation (training already includes a 10% no-text-reference mode, so this should be straightforward).

W3. The benefits of replacing the text encoder with an OCR encoder are not demonstrated, and the potential costs are not analyzed. The core design choice is to typeset all prompts onto the semantic image and read them with an OCR-pretrained encoder (DeepEncoder V2), replacing Z-Image’s original Qwen3 text encoder. However, the paper does not directly address the key question: what does this gain over a text encoder, and what does it lose?

Questions for authors

1.Can you provide a same-backbone, same-data baseline with a conventional text encoder plus region conditioning (W1), even at a smaller scale? 2.Why was Z-Image chosen over FLUX? Can you report All-in-Image on a FLUX backbone, or CreatiDesign-style conditioning on Z-Image, to separate backbone effects from representation effects? 3.What are the text metrics when glyph references are removed at evaluation (W2)?

11 comments
AuthorsOpenReview ↗

Response to Reviewer g2o9 Part 1

We thank Reviewer g2o9 for the rigorous feedback. Below, we provide direct same-backbone controlled comparisons, clarify why Z-Image was chosen, quantify the exact efficiency savings of replacing the text encoder alone, and report full-benchmark text evaluation without glyph references.

W1, Q1 & Q2: Controlled Same-Backbone Comparisons, Backbone Choice, and Efficiency Breakdown

1. Why We Chose Z-Image (Q2) At the time of submission, Z-Image was a newly released, compact (~6B DiT parameters vs. ~12B for FLUX) single-stream diffusion transformer with strong baseline generation and text capabilities. Its single-stream architecture allows target latent tokens, semantic tokens, and reference pixel tokens to be concatenated and jointly processed via full self-attention without auxiliary cross-attention branches, while its compact footprint enabled tractable progressive training and ablations.

2. Controlled Comparisons: Visual Canvas vs. Conventional Text + Coordinates (W1(i), Q1, Q2) We provide two complementary comparisons: (i) a strictly matched single-variable control on Qwen-Image-2.1 (Table R3 in Part 2 below) that holds the pretrained checkpoint, frozen VLM, native VAE appearance pathway/inputs, training data, and 10k optimization budget 100% identical, and (ii) a same-backbone fine-tuned baseline on native Z-Image (Tables R1–R2 below):

  • On Z-Image (Tables R1–R2): Because native Z-Image retains its pretrained Qwen3-4B text encoder (which is already aligned with the DiT, unlike our newly attached frozen OCR reader that required progressive alignment stages), we directly fine-tuned native Z-Image (Qwen3 text encoder + serialized layout coordinates) on the target CreatiDesign training set (10k steps) and compared it against All-in-Image on both the full 1,000-design benchmark (Table R1) and the 152-design subset under an identical deterministic evaluator (Table R2). While native Z-Image lacks a VAE reference pathway and uses single-stage target-domain fine-tuning (whereas Table R3 in Part 2 isolates the representation under identical training data and appearance inputs), both models in Tables R1–R2 receive the exact same target-domain layout and text specifications on the native Z-Image backbone.

Table R1: Native Z-Image Fine-Tuned vs. All-in-Image (Full 1,000-Design Benchmark, CFG 4 / 50 Steps / Seed 42)

MethodCLIP-I ↑\uparrowDINO-I ↑\uparrowM-DINO ↑\uparrowLayout Spatial ↑\uparrowColor ↑\uparrowTexture ↑\uparrowShape ↑\uparrowText Spatial ↑\uparrowSen. Acc ↑\uparrowNED ↑\uparrow
Z-Image (Zero-Shot Caption-Only)75.6844.3517.6459.0049.1449.9547.1018.1579.8492.58
Z-Image (Zero-Shot + Serialized Boxes, App. J Table 7)75.6846.7319.3857.0747.0047.0043.8515.7076.4292.00
Z-Image (Fine-Tuned Text + Coordinates on CreatiDesign)81.5961.9030.6163.4852.2951.5849.7536.2687.7397.30
All-in-Image (Z-Image, Ours)90.7489.8172.7680.5773.0471.8270.8060.1285.6097.15

Table R2: Controlled Layout Comparison on the 152-Design Ablation Subset (Identical Deterministic Evaluator)

Method (Z-Image Backbone)Layout Spatial ↑\uparrowLayout Color ↑\uparrowLayout Texture ↑\uparrowLayout Shape ↑\uparrow
Z-Image (Fine-Tuned Text + Coordinates)71.9260.2759.5956.85
All-in-Image (Z-Image, Ours)82.1976.0373.9772.60
  • Substantial Gains on Matched Spatial & Text Inputs: While higher subject fidelity (89.81 vs. 61.90 DINO-I) reflects the addition of our spatial VAE reference pathway, both models receive the exact same layout and text specifications. On these shared spatial tasks, All-in-Image outperforms fine-tuned Text + Coordinates by +17.09 pp on Layout Spatial (80.57 vs. 63.48) and +23.86 pp on Text Spatial (60.12 vs. 36.26) on the full benchmark (Table R1), and leads across every layout dimension on the 152-design subset (Table R2: +10.27 pp Spatial, +15.76 pp Color, +14.38 pp Texture, +15.75 pp Shape). Conversely, native Qwen3 retains a slight edge in raw character spelling (87.73 vs. 85.60 Sen. Acc), illustrating a natural trade-off where visual-spatial conditioning substantially strengthens geometric grounding while maintaining comparable spelling accuracy (97.15 vs. 97.30 NED).
AuthorsOpenReview ↗

Response to Reviewer g2o9 Part 2

Table R3: Matched 10k vs. 10k Control & Cross-Backbone Generalization on Qwen-Image-2.1 (Full 1,000-Design Benchmark, CFG 7 / 50 Steps / Seed 42)

To further isolate our visual conditioning representation with zero architectural changes, we also evaluated a controlled 10k vs. 10k comparison on Qwen-Image-2.1. Because Qwen-Image-2.1 natively uses a frozen Vision-Language Model (Qwen3-VL) as its conditioner, both models start from the exact same pretrained checkpoint, identical transformer architecture, frozen Qwen3-VL encoder, native VAE appearance pathway, training data, and 10k optimization budget—varying only whether global/regional prompts, Writes strings, and normalized xyxy coordinates are fed as serialized text or rendered onto our aligned visual canvases:

Method (Full 1,000-Design Benchmark)CLIP-I ↑\uparrowDINO-I ↑\uparrowM-DINO ↑\uparrowLayout Spatial ↑\uparrowColor ↑\uparrowTexture ↑\uparrowShape ↑\uparrowText Spatial ↑\uparrowSen. Acc ↑\uparrowNED ↑\uparrow
CreatiDesign (FLUX-based Reference, Table 1)89.3986.4865.7578.9466.0266.9465.8256.9078.3094.68
All-in-Image (Z-Image, Ours, Table 1)90.7489.8172.7680.5773.0471.8270.8060.1285.6097.15
Qwen-Image-2.1 (Text + Coordinates, 10k)91.9290.8975.3480.4777.0175.8974.9756.0163.8093.11
Qwen-Image-2.1 (All-in-Image Visual Conditioning, 10k, Ours)91.9191.0075.5791.0587.4986.2786.9859.9574.1895.36
Δ\Delta (Qwen-Image-2.1: All-in-Image vs. Text + Coordinates)-0.01+0.11+0.23+10.58+10.48+10.38+12.01+3.94+10.38+2.25
  • Identical Appearance Inputs & Pathway Confirm Strictly Matched Control: Both Qwen-Image-2.1 variants receive the exact same appearance canvas inputs (including identical subject and glyph reference pixels) through the identical native VAE appearance pathway, achieving virtually identical subject fidelity (91.91 vs. 91.92 CLIP-I; 91.00 vs. 90.89 DINO-I; 75.57 vs. 75.34 M-DINO).
  • Large Gains Across Every Layout and Text Metric: With the generator, frozen VLM conditioner, appearance inputs/pathway, training data, and 10k budget held strictly identical, All-in-Image outperforms Text + Coordinate conditioning on 9 of the 10 metrics—gaining +10.38 to +12.01 pp across all four layout metrics (91.05 vs. 80.47 Layout Spatial) and +10.38 pp in sentence accuracy (74.18 vs. 63.80 Sen. Acc; +3.94 pp Text Spatial), while our Z-Image and Qwen-Image-2.1 All-in-Image models together rank first on 9 of 10 metrics across the full table. This provides direct single-variable evidence for the representational advantage of our visual-spatial interface.

3. Controlled Efficiency Comparison: Replacing the Text Encoder Alone (W1(ii)) We agree with the reviewer that the cross-backbone runtime and memory gap against CreatiDesign (Appendix M, Table 11) reflects system-level differences (6B vs. 12B DiT and BF16 vs. FP32 text encoder components), and we will revise the text to frame Table 11 strictly as a system-level comparison. To isolate the exact savings of replacing the Qwen3-4B text encoder with DeepEncoder V2 under matched non-reference inputs, we benchmarked Native Z-Image (processing full global + regional + xyxy text prompts) against All-in-Image with its reference branch disabled (0 reference tokens) on the exact same physical NVIDIA H800 GPU (102421024^2, 50 steps, BF16, 8 conditions ×\times 2 seeds):

AuthorsOpenReview ↗

Response to Reviewer g2o9 Part 3

Method (Same NVIDIA H800 GPU, 102421024^2, 50 steps)Conditioning InputsResident Params (B) ↓\downarrowStep Latency (ms) ↓\downarrowImage Latency (s) ↓\downarrowPeak Allocated (GiB) ↓\downarrowPeak Reserved (GiB) ↓\downarrow
Native Z-Image (w/ Qwen3-4B Text Encoder)Full Global + Regional + xyxy Text10.260469.65 ±\pm 12.8823.7621.7025.51
All-in-Image (Z-Image + DeepEncoder V2, Ours)Semantic Canvas (0 ref tokens)7.051403.92 ±\pm 0.8620.6015.7118.28

Under matched non-reference inputs, replacing the native Qwen3-4B text encoder with DeepEncoder V2 improves every single efficiency metric: it reduces resident parameters by 3.21B (-31.3%), lowers peak allocated GPU memory by 5.99 GiB (-27.6%), speeds up end-to-end generation by 13.3% (20.60 s vs. 23.76 s), and cuts step-latency standard deviation by 15× (±0.86 ms vs. ±12.88 ms, as visual token count is bounded whereas serialized coordinate text expands dynamically). (Note: In this same-machine rerun, enabling the optional 4,096-token spatial appearance branch brings total latency to 41.55 s with 15.71 GiB peak allocated memory—closely matching Table 11's 41.71 s / 15.56 GiB and still substantially below CreatiDesign's 53.56 s / 36.71 GiB.)

Comments on the paper

6 comments
AuthorsOpenReview ↗

Response to Reviewer 6Kgh Part 1

We thank Reviewer 6Kgh for the encouraging review and constructive suggestions on isolating interface benefits and characterizing operating boundaries.

W1 & Q1: Matched Training Comparison (Semantic Canvas vs. Text-Plus-Box Conditioning)

To directly answer your question on isolating the semantic canvas while retaining the exact same generator, appearance pathway, training data, and 10k optimization budget, we conducted a strictly matched control on Qwen-Image-2.1 (where both variants share the identical frozen Qwen3-VL conditioner and native VAE appearance pathway), alongside a same-backbone layout/text comparison against fine-tuned native Z-Image (detailed in our response to Reviewer g2o9, Tables R1–R3):

Conditioning Interface (Full 1,000 Designs)CLIP-I ↑\uparrowDINO-I ↑\uparrowM-DINO ↑\uparrowLayout Spatial ↑\uparrowColor ↑\uparrowTexture ↑\uparrowShape ↑\uparrowText Spatial ↑\uparrowSen. Acc ↑\uparrowNED ↑\uparrow
Qwen-Image-2.1: Text + Coordinates (10k)91.9290.8975.3480.4777.0175.8974.9756.0163.8093.11
Qwen-Image-2.1: Semantic Canvas (10k, Ours)91.9191.0075.5791.0587.4986.2786.9859.9574.1895.36
Gain (Δ\Delta on Qwen-Image-2.1)-0.01+0.11+0.23+10.58+10.48+10.38+12.01+3.94+10.38+2.25
Z-Image: Fine-Tuned Text + Coordinates*81.5961.9030.6163.4852.2951.5849.7536.2687.7397.30
Z-Image: Semantic Canvas (All-in-Image, Ours)90.7489.8172.7680.5773.0471.8270.8060.1285.6097.15

*Native Z-Image fine-tunes its native Qwen3 text encoder on CreatiDesign without a VAE reference pathway, isolating shared layout and text spatial tasks.

Key Takeaway (Matched 10k Control on Qwen-Image-2.1):

  • Identical Appearance Fidelity: Because both variants share the exact same native VAE appearance pathway and inputs, subject similarity is virtually identical (~91.0 DINO-I), confirming a strictly controlled comparison.
  • Substantial Spatial & Text Gains: Varying only the conditioning interface yields +10.38 to +12.01 pp improvements across all layout metrics (reaching 91.05 Layout Spatial) and +10.38 pp in sentence accuracy (74.18 vs. 63.80), directly isolating the representational advantage of spatially aligned visual conditioning.
AuthorsOpenReview ↗

Response to Reviewer 6Kgh Part 2

W2 & Q2: Canvas Operating Limits, Density Stability, and Controlled Perturbations

Following your suggestions, we conducted four targeted studies to establish the operating boundaries of our semantic canvas interface and calibrate our joint metric against human judgment:

1. Systematic Readability Sweep Across Canvas Density and Font Size Across 27 configurations of canvas resolution (10242–204821024^2\text{--}2048^2) and font size (8px–60px) under a fixed 832-token budget, character readback accuracy remains >96% up to ~3,300 rendered text tokens (down to 8px at 102421024^2 and 11px at 153621536^2) and only degrades once aggressive downsampling shrinks glyph height below ~6–7 pixels (e.g., 9px at 153621536^2 drops to 56.0%; 11px at 204822048^2 drops to 11.9%). Our standard rendering (1536px long side, 24px font) operates safely in the >99.6% accuracy regime.

2. Instruction Density Stratification Across All 867 Multi-Condition Designs We stratified all 867 multi-condition benchmark designs (Appendix K) by total region count (annotated object boxes + target text instances), reporting the Location-Aware Joint Pass rate (Equation 14) at both the default threshold τ=0.80\tau = 0.80 (Table 8, 23.30% [202/867]) and τ=0.70\tau = 0.70 (Table 9, 25.72% [223/867]):

Density Stratum (All 867 Multi-Condition Designs)nnDINO-I ↑Layout Spatial ↑Text Sen. Acc ↑Location-Aware Joint Pass (τ=0.80\tau = 0.80, Table 8) ↑Location-Aware Joint Pass (τ=0.70\tau = 0.70, Table 9) ↑
Low Density (1–2 regions)46390.1681.6484.6726.13% (121/463)28.94% (134/463)
Medium Density (3–4 regions)39289.8778.8585.2520.15% (79/392)22.19% (87/392)
High Density (≥5\ge 5 regions)1287.8490.9183.3316.67% (2/12)16.67% (2/12)
Total (All Multi-Condition Designs)86790.0080.5784.9123.30% (202/867)25.72% (223/867)

Individual constraint adherence remains highly stable as region density grows—DINO-I (90.16 → 87.84) and Text Sen. Acc (84.67 → 83.33) vary by less than 2.4 pp across strata (note n=12n=12 in the ≥5\ge 5 stratum). Because Equation 14 requires simultaneous satisfaction of image-level mean subject similarity (mean DINO-I≥τ\text{mean DINO-I} \ge \tau), all per-object layout checks, all target-string content checks, and all per-box text IoU checks (IoU≥0.5\text{IoU} \ge 0.5), the conjunctive pass rate naturally decreases as the number of evaluated object and text regions per design grows (26.13% → 20.15% → 16.67% at τ=0.80\tau = 0.80) even when per-region accuracy remains stable.

3. Controlled Canvas Perturbation Study (152-Design Ablation Subset, Seed 42) Holding all semantic prompts, bounding boxes, appearance references, and seed 42 fixed, we varied only the semantic canvas rendering via (1) Color-Assignment Permutation (shuffling the color palette assigned to prompt-region pairs while preserving 1-to-1 box-text color binding) and (2) Font-Size Scaling (24px → 18px):

Semantic Canvas Variant (152-Design Ablation Subset, Seed 42)DINO-I ↑Layout Spatial ↑Text Sen. Acc ↑Location-Aware Joint Pass* ↑
Default Canvas (24px, Standard Color Palette)89.8682.1980.4832.56% (42/129)
Color-Assignment Permutation (Shuffled Box-Text Colors)89.84 (-0.02)83.56 (+1.37)79.93 (-0.55)31.01% (-1.55 pp)
Font-Size Scaling (18px vs. 24px)89.84 (-0.02)83.56 (+1.37)81.47 (+0.99)31.78% (-0.78 pp)

*Location-aware joint pass is evaluated on the 129 designs in the 152-design subset that contain all three annotated condition types (subject, object layout, and text).

Key Takeaway (Perturbation Invariance): Across both perturbations, all four metrics remain virtually invariant—DINO-I shifts by only -0.02 pp, Text Sen. Acc stays within ±1.0 pp, Layout Spatial remains consistently strong (83.56 vs. 82.19), and Location-Aware Joint Pass varies by at most -1.55 pp (1–2 designs out of 129). This shows that the generator effectively uses color as a relational binding cue rather than memorizing fixed color indices or font scales.

AuthorsOpenReview ↗

Response to Reviewer 6Kgh Part 3

4. Blind Human Calibration of Location-Aware Joint Pass Decisions To calibrate our automatic location-aware joint metric (τ=0.80\tau = 0.80, Appendix K) against human judgment, we conducted a blind multi-rater study on 20 stratified benchmark outputs from our main model (10 auto-pass and 10 auto-fail designs; 30 questionnaires yielding 150 complete image-level evaluations, with 7–8 independent raters per image). Raters evaluated four explicit criteria per image, and an evaluation counts as a strict joint pass only if a rater answers Yes to all four criteria simultaneously.

Table 4a: Image-Level Majority-Vote Human–Automatic Calibration (20 Stratified Designs)

Resolved AgreementCohen's κTrue Pos. (TP)False Pos. (FP)True Neg. (TN)False Neg. (FN)Unresolved Ties
83.33% (15/18)0.66790632 (4/8 votes each)

Table 4b: Per-Criterion Human Audit on Auto-Pass Designs & Inter-Rater Agreement

Human Evaluation CriterionTargeted Metric Check (Addressing W2)Human Endorsement on Auto-Pass (n=74n=74 evals) ↑Inter-Rater Pairwise Agreement (490 pairs) ↑
1. Subject IdentityTests if mean DINO-I hides individual subject failures100.0% (74/74)98.57% (483/490)
2. Text Content & PlacementExact string spelling and target box localization93.24% (69/74)86.33% (423/490)
3. No Extra / Garbled TextTests if substring OCR hides unrequested extra text89.19% (66/74)78.37% (384/490)
4. Regional Object LayoutPresence and placement of all prompted objects85.14% (63/74)66.94% (328/490)
  • Substantial Majority-Vote Agreement with Zero Observed False Positives (Table 4a): Across the 18 resolved designs in this calibration sample, majority-vote human judgments agree with the automatic metric on 83.33% (15/18) of designs (κ = 0.667). Notably, all 3 disagreements are false negatives (TP = 9, FP = 0, TN = 6, FN = 3)—cases where the automatic metric penalized unmerged multiline OCR boxes or strict VLM region checks, but a majority of human raters (5/7 to 7/8) judged all four criteria as satisfied.
  • Criterion-Level Audit on Auto-Pass Designs (Table 4b): Among the evaluated auto-pass designs, human raters endorsed subject fidelity in 100.0% (74/74) of evaluations, text accuracy/placement in 93.24% (69/74), and the absence of extra/garbled text in 89.19% (66/74)—with 79.73% (59/74) of individual evaluations simultaneously passing the three core checks (subject, object, text) and 68.92% (51/74) passing all four checks including no extra text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.