acceptodds
Withdrawn from ICLR 2027

DesignX: Bridging Coding and Generation for Recommendation-based Visual Design

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Despite recent advances in AI visual generation, professional design still faces two challenges: enabling non-experts to create professional-quality designs and converting raster images into unified, editable representations of raster graphics, vector graphics, and text. To address these challenges, we present DesignX, a recommendation-based visual design agent that enables both professional design creation and editable visual reconstruction. For the design agent, we construct a large-scale template library covering diverse visual design categories and develop three specialized skills: a cataloging skill for organizing and continuously expanding the template library, a coarse-to-fine retrieval skill for efficiently identifying relevant templates, and a generation skill that adapts templates according to user requirements. Beyond raster-only generation, we introduce an editable visual decomposition framework consisting of a vision-language model (VLM), an RGBA-based diffusion transformer (DiT), and a text-attribute prediction network. Specifically, we design a structured decomposition protocol that enables the VLM to parse any design into seven editable elements with spatial relationships and semantic descriptions while directly generating code for simple vector primitives. The RGBA-based DiT extracts individual layers from VLM descriptions and completes occluded regions, while the text-attribute prediction network jointly predicts typography attributes through a multi-head Transformer architecture. Furthermore, we introduce synthetic collage data augmentation to improve model robustness. Finally, we establish a comprehensive visual design benchmark spanning diverse categories to facilitate future research. Extensive experiments demonstrate that DesignX significantly improves visual design quality, and our decomposition framework surpasses existing SOTA commercial solutions. We have released the code and template library to promote research in this direction.

Primary area:
generative models
Keywords:
Visual Generation, Design Agent

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Vz3rOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents DesignX, which has two parts.

(i) A template-based design agent. The agent tags an internal template library along seven facets: Type, Scenario, Purpose, Domain, Subject, Visual Style and Color Palette. It retrieves five candidates coarse-to-fine and adapts the selected template to the user’s request, producing raster designs.

(ii) A framework that converts raster designs into editable elements. A VLM (Qwen3.6-35B-A3B) parses a design into typed elements. Training uses 2.2M real layered designs and 2.1M synthetic collages.

Evaluation. The agent is evaluated by human preference against GPT-Image-2 (82.96% preferred). The decomposition is evaluated on a new 150-image benchmark against Qwen-Image-Layer, MagicLayer and GPT6-Astra Ultra. The metrics are LPIPS and valid-layer counts judged by GPT-6 Astra.

Critical strengths
  1. The problem matters in practice. The method recovers editable text and simple shapes, not only image layers.
  2. The font prediction is the strongest part. A dedicated classifier picks among 6,279 font families instead of asking the VLM. A normalization (Eq. 1–2) keeps size and spacing predictions valid when crops are resized.
  3. The training data is large. The ablation suggests that real and synthetic data complement each other (Tab. 2), and the comparisons include commercial systems.
Critical weaknesses

1. The core design choices are not justified, and the agent does not serve its stated goal.

  • The paper generates designs as images and then converts them into editable code. It never explains why this beats producing editable designs directly, which is an established alternative in research (e.g., COLE, ART, StarVector, OmniSVG) and in commercial tools (e.g., Canva's design model, Recraft V4, Claude Design). Even the image-first workflow already exists (Accordion, ICCV 2025).
  • The agent, framed as "skills", is retrieval-augmented generation over a fixed internal library. Its stated goal is to help users who "may struggle to articulate their preferred visual style" (l. 78–79). Yet it narrows the library by keyword-matching the user's request against seven broad tags (l. 229–238). Those tags leave out layout, density, text capacity, typography and imagery.
  • Beyond picking one of five templates, nothing captures or learns the user's preferences. Work that targets this problem directly is not discussed (e.g., Creo, DesignScape, Sequential Gallery, AesthetiQ, DesignPref).

the pipeline is the paper's main contribution. It is not justified against direct editable generation, and it has no mechanism that models what users want, so it is unclear what the system does better than existing approaches.

2. The evaluation does not support the claims.

  • The agent is evaluated with a single A/B test. It is compared only against plain GPT-Image-2 (Tab. 1).
    • The test reports no criteria, rater details, agreement or confidence intervals.
    • No baseline isolates the effect of retrieval, such as a random template, embedding-based retrieval or filling the template directly.
    • There are no side-by-side outputs from the compared methods for the same prompt. There are also no template/output pairs that would show whether a template is adapted or simply copied, and the paper has no failure cases or appendix..
  • The full system is never tested as a whole. The preference study judges images only, and the decomposition is tested on a separate set of 150 images. Nothing evaluates the final editable file.
  • The decomposition numbers are weak. They are counts of "valid" layers from an LLM judge that is not checked against humans, and several reported numbers contradict the text.

the main claims rest entirely on this evidence: the 82.96% preference, outperforming commercial systems, and bridging generation and editing. As presented, the evidence cannot separate the method from simpler alternatives, and readers cannot inspect the outputs.

Questions for authors
  1. Why generate raster designs and then decompose them, instead of generating editable designs directly? How does DesignX differ from Accordion (ICCV 2025), which also converts AI-generated designs into editable layers?
  2. How many templates does the library contain, and how were they selected? Beyond the tags, what does the “careful inspection” step (l. 237–238) consider, and does the system use the user’s choices or feedback in any way?
  3. How many raters and judgments were collected for Tab. 1, with what expertise and agreement, and how does the agent compare with a random template of the same type? Could you show side-by-side outputs (prompt, retrieved template, DesignX, GPT-Image-2) for randomly sampled requests, including failure cases?
  4. Could you define in the paper the model variants (DesignX V1–V3, Text Predictor V1–V8) and reconcile the numbers that contradict the text? These include the vector-layer claim at l. 376–377 (GPT6-Astra Ultra has 2,345 vs 2,233), the V1 total in Tab. 2 (4,294 vs 4,744), the LPIPS of the same model in Tabs. 2 and 3 (0.1400 vs 0.1382), and the claim at l. 454–455, which Tab. 4 contradicts.
Reviewer_ss1xOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DesignX combines template-guided design generation with decomposition into raster layers, editable text, and simple vector graphics. The decomposition pipeline uses a VLM, an RGBA diffusion model, and a text-attribute predictor, trained with real designs and synthetic collages.

Critical strengths
  1. Joint recovery of raster assets, editable text, and vector primitives addresses a practical gap between image generation and design editing.

  2. The data and typography ablations provide useful evidence for the decomposition pipeline, with encouraging reconstruction and preference results.

Critical weaknesses
  1. Evaluation needs stronger validation. GPT-based layer judgments lack reported agreement with human annotations, and layer counts may depend on decomposition granularity. The human evaluation protocol and train–test separation also need clarification.

  2. Editability and positioning need clarification. Reconstruction quality alone does not establish practical editing benefits. Related work should cover image-to-SVG methods such as StarVector, OmniSVG, and InternSVG, clarifying how joint raster/text/vector recovery differs from SVG generation.

Questions for authors
  1. Could you clarify the benchmark separation and human preference protocol, and validate the automatic layer judgments against blinded human annotations on a representative subset?

  2. Could you report success rates and manual correction effort for a small set of practical edits, such as text replacement, shape recoloring, and object relocation?

Reviewer_4J3GOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the problem of layered graphic design generation from input user requirements.

The main contribution is a template-based generation method, which first generates raster design images by retrieving and adapting existing designs from a large-scale template library, and then decomposing the raster images into editable layers with raster images, editable text and simple vector shapes.

Critical strengths
  1. Graphic design generation is an important research problem to study.

  2. The idea of using templates for graphic design generation is novel, which is in contrast to previous methods that largely rely on generative models.

Critical weaknesses
  1. The novelty of the paper is limited. First, the workflow of first generating raster design images and then decomposing them into editable layers has already been explored in prior work (Suzuki et al., 2025; Chen et al., 2025a). Second, both raster images and editable text have already been considered in prior work on graphic design image decomposition (Chen et al., 2025a), and thus the novelty involved in the proposed decomposition method diminishes since it only additionally considers simple vector shapes in decomposition results. Third, while the idea of template-based generation is novel, its implementation lacks innovative technical methods, although it does involve some engineering effort for designing the skills.

  2. The design choice of the RGBA-based DiT in the proposed decomposition method is questionable. Since the RGBA-based DiT is conditioned on a local cropped image region without observing the full design image, it raises a concern about layer cohesion since individually extracted layers may conflict with each other. To address this concern, a variant of the RGBA-based DiT should be tested in the experiments, which includes the full design image as input.

  3. The diversity of the visual design agent is not evaluated. The visual design agent employs a library of templates, which can improve sample fidelity but inevitably restricts sample diversity. Hence, it is important to specially evaluate the diversity of generated outputs, which is missing in the current experiments.

  4. Some important implementation details are missing. The details of the proposed three skills (cataloging, retrieval, generation) are not provided in the paper.

Questions for authors

Which VLM model is used for the visual design agent?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.