acceptodds
Under review as a conference paper at ICLR 2027

PLAYGROUND: Progressive Layout Generation with Render-Grounded Planning

Abstract

Generating editable graphic designs requires arranging visual and textual elements into a coherent composition while preserving their content and editability. Although recent MLLM-based approaches employ iterative generation and visual refinement to reflect the real-world design workflows, their high-level plans are predetermined prior to execution, limiting their adaptability as the design evolves. To address this limitation, we propose PlayGround, a progressive layout generation framework that dynamically adapts to the current rendered canvas. PlayGround establishes a progressive design loop comprising a Progressive Planner, a Layout Generator, and a Visual Verifier, guided by a Structure-Aware Retriever. Given an initial set of unordered elements, the Structure-Aware Retriever initially infers a coarse layout structure to retrieve structurally similar reference designs. Subsequently, at each turn, the Progressive Planner examines the current rendered canvas alongside the remaining elements, forms the next functional group, and plans its composition. The Layout Generator then predicts editable layout attributes, while the Visual Verifier evaluates rendered candidates and provides feedback when refinement is necessary. To train the Progressive Planner with step-wise supervision, we construct a two-stage data generation pipeline that extracts functional groups and turn-level planning trajectories from completed designs. Experiments on Crello and LICA demonstrate the effectiveness of PlayGround over existing methods. Our code and weights will be publicly released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal large language models, layout generation, graphic design

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_bGdJOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PlayGround, a workflow for arranging multimodal elements, such as text and images, in graphic design. Unlike prior approaches that directly predict the attributes of all elements in a single pass, PlayGround decomposes the design process into multiple steps and learns to place a subset of elements at each step. The Structure-Aware Retriever predicts a coarse layout structure, which is used to retrieve the Top-5 reference designs from a constructed reference pool. These references are subsequently provided as contextual input to the Progressive Planner, which predicts a group-level plan and selects the group of elements to be rendered at the current step. The Layout Generator then predicts the layout and text attributes of the elements in that group, conditioned on the Planner’s output and the retrieved references. In addition, the Visual Verifier evaluates the current predictions and, if they are rejected, provides feedback for revision and regeneration. The evaluations on Crello and LICA show that PlayGround outperforms the baselines on multiple VLM-based metrics.

Critical strengths
  1. Comprehensive coverage of related work and competitive empirical comparisons. The paper provides a comprehensive discussion of related work and compares PlayGround against several competitive open-source methods.
  2. Extensive evaluation of the proposed components. The paper provides a relatively comprehensive analysis of the effects of its individual modules, demonstrating how each component contributes to the final performance.
Critical weaknesses
  1. Limited methodological novelty. My primary concern is the novelty of the proposed method. PlayGround remains a workflow that combines multiple domain-specialized VLM modules, and its overall architecture does not appear fundamentally different from prior agentic systems. More specifically, it follows a familiar Planner–Predictor–Verifier structure. Moreover, the main advantage claimed by the paper—iterative design generation—have already been explored in PSDesigner. Therefore, it is unclear whether the paper provides a sufficiently substantial conceptual or algorithmic advance beyond existing systems.
  2. Evaluation is limited to relatively simple design tasks. The Crello dataset is limited in terms of element and attribute types, layout diversity, and the number of layers. This raises concerns about whether a system specifically designed and evaluated in this setting can generalize to more complex design tasks. For formats such as HTML, PowerPoint, and PSD, the substantially larger number of layers and greater diversity of element types introduce much greater complexity. In my opinion, PlayGround may be more suitable for simple layout tasks, which limits its applicability to realistic design scenarios.
  3. The system may not scale to more complex inputs. Several components, including the Structure-Aware Retriever and Progressive Planner, need to process the complete element inventory. This can substantially increase memory consumption and computational cost and may eventually exceed the effective context or processing capacity of the underlying pretrained model. Although the relative simplicity of Crello may mitigate this issue, scalability could become a serious limitation as the number of input assets increases.
  4. Pure SFT training may lead to a substantial training–inference gap. During inference, prediction errors from earlier steps will inevitably affect subsequent design decisions because the proposed components are trained only on ground-truth trajectories. Although the method introduces a Visual Verifier implemented with GPT-5.6 Luna to mitigate this issue, the effectiveness of this verifier in the graphic-design domain has not been sufficiently validated. More importantly, it does not fundamentally address the underlying training–inference discrepancy.
Questions for authors
  1. Generalization to more complex designs. Has the method been evaluated on more challenging design benchmarks, particularly those containing substantially more layers than are commonly observed in the training set? If the method performs well only on relatively simple evaluation sets such as Crello, its practical value in real-world design applications would remain unclear. Evidence on such settings could materially improve my assessment.
  2. Construction and role of the reference pool. How is the reference pool constructed? Why is the coarse layout predicted by the Structure-Aware Retriever used only for retrieval rather than directly guiding the downstream modules? What are the respective limitations of directly conditioning on the predicted coarse layout and conditioning on retrieved reference designs? In addition, providing five reference designs may consume a considerable number of visual tokens, resulting in substantial memory and computational overhead. Please provide an analysis of this trade-off.
  3. Revision of previously placed elements. How does PlayGround handle cases in which a newly inserted group makes the layout of previously placed elements suboptimal? The current design appears to revise only the group being processed in the current iteration, while elements accepted in previous iterations remain fixed and cannot be updated.
  4. Training–inference gap. How does PlayGround address the training–inference gap? Without self-forcing, the method may be sensitive to inference-time prediction noise, such that an incorrect element placement at one step affects all subsequent predictions. Please provide experiments or analyses that quantify this issue and demonstrate whether the Visual Verifier is sufficient to mitigate error accumulation.
6 comments
AuthorsOpenReview ↗

Author Rebuttal(1/2)

We thank the reviewer for recognizing the breadth of our baseline comparisons and ablations. Your comments help us clarify the methodological contribution and the limits of the current system.

[W1] Methodological contribution

We appreciate the reviewer's comparison to existing Planner–Generator–Verifier frameworks. However, our key contribution goes beyond the generic architecture or iterative refinement. Specifically, we introduce a render-grounded progressive policy that jointly determines which remaining elements to group, when to place them, and how to arrange them based on the evolving canvas. To enable this, we introduce functional groups as adaptive decision units and, as also recognized by Reviewer LEAE, an automatic annotation pipeline that constructs the required step-wise supervision.

Furthermore, as discussed in Appendix A, prior retrieval-based layout methods primarily predict element bounding boxes (e.g., text, logos, and underlays) on a given background, without requiring the actual element content. They can therefore retrieve similar layouts using spatial cues from the background, such as the main subject's position. In contrast, our setting provides only a blank canvas and predefined elements with unknown positions and scales, making such retrieval strategies inapplicable. To address this, we predict coarse functional group–region pairs from the supplied elements to retrieve structurally similar designs. To the best of our knowledge, we are the first to introduce retrieval-based guidance for asset-given element-to-design composition.

[W2–W3, Q1] Complexity and generalization

Table B1. Overall VLM Scores Across Element Counts

\hline \text{Element Count} & \text{Samples} & \text{PLAYGROUND} & \text{FlexDM} & \text{LaDeCo} & \text{PosterCopilot} & \text{VFLM} \\\\ \hline 1\text{--}10 & 1,312 & \mathbf{4.063} & - & - & - & - \\\\ 11\text{--}20 & 956 & \mathbf{3.940} & - & - & - & - \\\\ 21\text{--}30 & 85 & \mathbf{3.771} & - & - & - & - \\\\ 31\text{--}50 & 22 & \mathbf{3.962} & - & - & - & - \\\\ \hline \text{Overall} & 2,375 & \mathbf{4.001} & 2.970 & 3.378 & 3.263 & 3.531 \\\\ \hline \end{array}$$ We understand the reviewer's concern that Crello contains relatively simple designs. To examine whether our performance degrades as layout complexity increases, we analyze the VLM scores in Table 1 by the number of input elements. As shown in Table B1, **PLAYGROUND consistently achieves scores above 3.771 across all element-count ranges**, exceeding the overall scores of the baselines: FlexDM (2.970), LaDeCo (3.378), PosterCopilot (3.263), and VFLM (3.531). This indicates that our method maintains strong performance even on more element-dense designs within Crello. **To further evaluate generalization beyond the complexity observed during training**, we consider LICA, where 16.8% of test designs contain more than 25 elements—the maximum in our Crello training set—including designs with over 50 elements. Despite these more challenging inputs, PLAYGROUND achieves the highest overall VLM score among the compared methods **without training on LICA**. These results provide encouraging evidence of generalization to more complex designs and suggest that further training on such datasets, enabled by our automatic annotation pipeline, could yield additional improvements. >**[W3, Q2] Reference Pool, Conditioning, and Computational Trade-off** **Table B2. Direct coarse-layout conditioning versus retrieved references.** $$\begin{array}{lc} \hline \text{Conditioning} & \text{Overall VLM} \\\\ \hline \text{Top-5 references} & \mathbf{3.958} \\\\ \text{No references} & 3.901 \\\\ \text{Lowest-5 references} & 3.867 \\\\ \text{Direct coarse layout} & 3.908 \\\\ \hline \end{array}$$ **[Reference Pool Construction and Direct Coarse Layout Conditioning.]** Our reference pool consists exclusively of the **Crello training set** across all experiments. We use predicted coarse group–region pairs for retrieval rather than directly conditioning downstream modules, as uncertain initial spatial predictions may constrain subsequent layout decisions. Instead, retrieved designs provide richer visual guidance while allowing flexible, canvas-conditioned planning. To validate this design choice, we conducted an additional ablation that directly conditions the planner and predictor on the predicted coarse layout without retrieving reference designs. **Direct conditioning (3.908) outperforms conditioning on the lowest-ranked references (3.867), but performs comparably to using no references (3.901)**, while our retrieval-based approach achieves **3.958**. These results support our design choice of using coarse layout predictions to retrieve informative reference designs rather than directly constraining downstream generation. We have added these analyses in Sec. 5.3 (Table 2)
AuthorsOpenReview ↗

Author Rebuttal(2/2)

Table B3. Quality and resource trade-off of reference retrieval.

\hline \text{Metric} & \text{Top-5 references} & \text{No references} \\\\ \hline \text{Overall VLM} & \mathbf{3.958} & 3.901 \\\\ \text{Compute (TFLOPs/design)} & 425 & 346 \\\\ \text{Peak GPU memory (GiB)} & 24.0 & 22.4 \\\\ \text{Verifier retries / design} & 3.05 & 3.57 \\\\ \text{Latency (s/design)} & 176 & 193 \\\\ \text{GPT API cost (USD/design)} & 0.0129 & 0.0138 \\\\ \hline \end{array}$$ **[Computational Overhead.]** Using five references increases computation by 23% (346 → 425 TFLOPs per design) and peak GPU memory by 7% (22.4 → 24.0 GiB). However, reference guidance reduces Visual Verifier retries by 15% (3.57 → 3.05), offsetting the additional computational overhead. Consequently, end-to-end latency decreases by 9% (193 → 176 s/design), while GPT API cost also decreases by 7% (\$0.0138 → \$0.0129 per design). Meanwhile, retrieval yields a statistically significant quality improvement over the no-reference setting (Δ = +0.058, 95% CI: [0.024, 0.093], p = 0.0012). These results demonstrate that reference retrieval improves design quality while reducing end-to-end latency and API cost, despite the additional GPU computation and memory requirements. We have added these analyses in Appendix D.4 (Table 14). >**[Q3–Q4] Previously accepted groups and the training–inference gap** **Table B4. Stage-wise performance after an injected adverse intermediate state.** $$\begin{array}{lccccc} \hline \text{Stage} & \text{PLAYGROUND} & \text{A (Error + Verifier)} & \text{B (Error + Random)} & \Delta_A & \Delta_B \\\\ \hline \text{Intervention } & 3.261 & 3.175 & 3.175 & 0.086 & 0.086 \\\\ \text{Subsequent (avg.)} & 3.778 & 3.701 & 3.613 & 0.077 & 0.165 \\\\ \text{Final} & \mathbf{3.958} & 3.885 & 3.712 & \mathbf{0.073} & \mathbf{0.246} \\\\ \hline \end{array}$$ ****[Scope of Revision.]**** PLAYGROUND adopts a **closed-loop, group-wise generation process**, where revisions are restricted to the current group's elements. To ensure reliable placement within this constraint, our Progressive Planner adaptively generates layout plans based on the evolving canvas, while the Visual Verifier selects among five candidates and provides render-grounded feedback to refine the current layout instruction. We acknowledge that revising previously placed elements could further improve design quality and consider this a promising future direction. ****[Training-Inference Gap: Error Injection.]**** To assess whether PLAYGROUND can **adapt to imperfect intermediate states without revising previously placed elements**, we conducted an error-injection experiment using PLAYGROUND's generated trajectories from the Crello 600-design ablation subset. **For each design, we selected an intervention turn at the midpoint of its generation trajectory**, where we used GPT-6 Luna to identify the **most erroneous candidate among the four non-selected layouts** and forcibly committed it before continuing generation. Since PLAYGROUND ordinarily selects a verifier-approved candidate rather than the worst alternative, this intervention creates a more challenging intermediate state than typically encountered during generation. Despite this challenging setting, **PLAYGROUND maintains strong performance (A: 3.885 vs. Ours: 3.958)**, with only a 0.073-point decrease. To examine the contribution of the Visual Verifier, we introduce **variant B, which removes the Visual Verifier** and randomly selects among five candidates in subsequent turns, starting from the **same erroneous canvas as A**. The substantial performance degradation in B (3.712 vs. 3.885 in A, Δ = 0.173) indicates that **the Visual Verifier is critical to maintaining design quality even when generation starts from an erroneous intermediate state**. ****[Error Accumulation Analysis.]**** To further analyze error propagation, we evaluate layouts at different generation stages, using **the overall VLM score gap relative to unperturbed PLAYGROUND as a proxy for error impact**. With the Visual Verifier (A), the gap remains stable and slightly decreases (0.086 → 0.077 → 0.073). In contrast, without the Visual Verifier (B), the gap progressively increases (0.086 → 0.165 → 0.246). **While absolute VLM scores naturally increase as more elements are placed on the canvas, these relative gaps reveal how the impact of injected errors evolves.** These results demonstrate that **the Visual Verifier effectively handles error accumulation by mitigating the propagation and amplification of injected errors throughout progressive generation**. We have added this experiment in Appendix D.4 (Table 15).

Dear Reviewer bGdJ, we sincerely appreciate your thoughtful feedback and the opportunity to further clarify our work. In response to your suggestions, we have clarified the novelty of our approach and conducted additional experiments to examine its effectiveness, computational cost, and robustness. We hope these clarifications and new results help address your questions. As the discussion period draws to a close, we would be grateful to hear your assessment of our updated response and would be happy to address any remaining questions.

Reviewer_LEAEOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes PlayGround, a framework for composing editable graphic designs from a given set of image and text elements. The main idea is to move planning inside the generation loop: instead of fixing a global plan upfront, a Progressive Planner looks at the current rendered canvas and the remaining elements at each turn, picks the next "functional group" (one of six roles such as background, visual/hero, title/headline), and writes a natural-language layout instruction for it. A Layout Generator then predicts K=5 candidate layouts for that group, and a Visual Verifier (GPT-5.6 Luna) picks the best rendered candidate and either accepts it or rewrites the instruction for up to three rounds. Before generation, a Structure-Aware Retriever predicts coarse role–region pairs and retrieves five reference designs.

Since existing datasets have no step-wise supervision, the authors use GPT-5.6 Luna to annotate Crello designs with functional groups, execution order, and turn-level rationales and instructions. The Retriever, Planner, and Generator are all LoRA-tuned Qwen3-VL-8B models.

On Crello and zero-shot LICA, PlayGround outperforms FlexDM, LaDeCo, PosterCopilot, and VFLM on most of six GPT-5-judged criteria. It is also preferred in a user study with 32 participants on 25 samples. Ablations cover retrieval, number of candidates, the verifier, a static planner, and grouping granularity.

Critical strengths
  1. The data construction pipeline is a useful contribution in its own right. Turning finished Crello designs into functional groups, execution orders, cumulative canvas states, and turn-level instructions gives step-wise supervision that existing layout datasets lack. The role definitions in Appendix B are careful, and the statistics (Figs. 7–9) suggest the decomposition is reasonable rather than degenerate. I expect others working on sequential or agentic design generation could reuse this.

  2. The empirical results are fairly convincing at the system level. PlayGround leads on most VLM metrics on both datasets, including zero-shot on LICA, and the user study points the same way (ranked first in roughly 80% of responses). The qualitative results show fewer of the typical failure modes, such as text placed over salient visuals or fragmented compositions. Overall, the paper is clearly written and easy to follow.

Critical weaknesses
  1. It is not clear that the gains come from progressive planning, which is the paper's main claim. In Table 2, replacing the progressive planner with a static one changes the mean VLM score only from 3.958 to 3.894, and removing retrieval gives 3.901. Removing the verifier, however, drops it to 3.698. So most of the improvement seems to come from best-of-K sampling plus GPT-5.6 Luna selection and rewriting. With per-sample std around 0.6, a single run, and 600 samples, I am not sure the 0.06 gap for the planner is meaningful without a significance test or multiple seeds.

    This connects to a fairness issue in Table 1. PlayGround can generate up to 15 candidates per group and uses a strong proprietary model to pick among them, while all baselines produce a single output. The verifier also selects using criteria (readability, avoiding harmful overlap, composition) that closely match what the GPT-5 judge scores. In addition, the judge, verifier, and data annotator all come from the same model family, so some self-preference bias cannot be ruled out. I would want to see a compute-matched comparison, for example a baseline such as LaDeCo with the same number of samples and the same verifier, and some report of inference cost and latency, which is currently missing.

  2. Several components and claims are weaker than presented. In the retrieval score, the role-IoU term (the only one that uses the predicted structure) has weight 0.10, while aspect ratio and word/image counts get 0.45 each. The "structure-aware" part therefore contributes little, which is consistent with the small retrieval ablation. The rule-based metrics also do not fully back up the claims: on LICA, PlayGround has the lowest validity (0.8036), higher overlap than LaDeCo, and text occlusion about 12x that of GT. Finally, the paper motivates itself against methods with fixed plans (DesignAsCode, PSDesigner) but does not compare with them. The only evidence against fixed plans is the internal static-planner ablation discussed above.

Questions for authors
  1. Could you report a compute-matched comparison, e.g., a baseline (LaDeCo or the static-planner variant) given the same number of candidates and the same GPT-5.6 Luna verifier? How many model calls and how much time/API cost does PlayGround need per design compared with the baselines?
  2. Is the gap between the progressive and static planner (3.958 vs. 3.894) statistically significant? Results over multiple seeds or a paired test would help.
  3. Have you checked for self-preference bias, for example by evaluating with a non-OpenAI judge?
  4. Can we put the DesignAsCode and PSDesigner results as baselines, given that they are the main motivation for the paper?

I am willing to raise my score if all the questions are addressed adequately.

4 comments
AuthorsOpenReview ↗

Author Rebuttal(1/2)

We thank the reviewer for recognizing the value of our data construction pipeline and empirical results. We particularly appreciate the suggestions to strengthen the baselines and examine evaluation reliability.

[W1, Q2] Statistical support for planning and retrieval

[Progressive planning.] The three-run comparison with the Static Planner is reported in Table J1 of the general response. The reference-retrieval ablation is reported below.

Table L1. PLAYGROUND versus the no-reference variant across generation runs.

\hline \text{Generation} & \text{Ours} & \text{No ref.} & \Delta & \text{95\\% CI} & p \\\\ \hline \text{Run 1} & 3.958 & 3.901 & +0.058 & [0.024,0.093] & 0.0012 \\\\ \text{Run 2} & 3.950 & 3.903 & +0.047 & [0.012,0.081] & 0.0086 \\\\ \text{Run 3} & 4.022 & 3.903 & +0.118 & [0.084,0.152] & < 10^{-5} \\\\ \text{3-run mean} & \mathbf{3.977} & 3.902 & \mathbf{+0.074} & [0.051,0.096] & < 10^{-5} \\\\ \hline \end{array}$$ **[Contribution of reference retrieval.]** We conducted three independent generation runs with and without reference retrieval. PLAYGROUND consistently and significantly outperforms the no-retrieval variant across all three runs. The three-run average shows a significant improvement (Δ = +0.074, 95% CI: [0.051, 0.096], p < 0.00001). These results provide strong evidence that structure-aware retrieval consistently contributes to downstream design quality. We have added this statistical analysis in Appendix D.4 (Table 12). >**[W1, Q1] Verifier-augmented LaDeCo & Inference Cost** **Table L2. Verifier-augmented LaDeCo on the full Crello test set.** $$\begin{array}{lccccccc} \hline \text{Method} & \text{Layout} & \text{Typo.} & \text{Elem.} & \text{Color} & \text{Style} & \text{Intent} & \text{Overall} \\\\ \hline \text{LaDeCo} & 3.103 & 2.615 & 3.451 & 3.979 & 3.592 & 3.528 & 3.378 \\\\ \text{LaDeCo + verifier} & 3.449 & 3.098 & 3.710 & 4.167 & 3.863 & 3.974 & 3.710 \\\\ \text{PLAYGROUND} & 3.794 & 3.503 & 3.899 & 4.381 & 4.183 & 4.248 & \mathbf{4.001} \\\\ \hline \end{array}$$ **Table L3. Verifier-augmented LaDeCo and the no-verifier ablation.** $$\begin{array}{lccc} \hline \text{Method} & K & \text{Verifier} & \text{Overall VLM} \\\\ \hline \text{LaDeCo + verifier} & 5 & \checkmark & 3.632 \\\\ \text{PLAYGROUND, no verifier} & 5 & \times & \mathbf{3.698} \\\\ \text{PLAYGROUND} & 5 & \checkmark & \mathbf{3.958} \\\\ \hline \end{array}$$ **[Verifier-augmented LaDeCo.]** Following your suggestion, we strengthened LaDeCo with best-of-five generation and a Visual Verifier. As shown in Table L2, although this improves LaDeCo's performance, PLAYGROUND consistently achieves higher scores across all six criteria on the full Crello test set (4.001 vs. 3.710). Notably, as shown in Table L3, PLAYGROUND without the Visual Verifier still outperforms verifier-augmented LaDeCo (3.698 vs. 3.632) on the 600-design Crello ablation subset. These results demonstrate that PLAYGROUND's improvements cannot be attributed solely to best-of-K sampling and verification, further highlighting the contributions of our progressive planning and structure-aware retrieval. We have added these results in Tables 1 and Sec.5.3. **Table L4. Model calls, latency, and API charges per design.** $$\begin{array}{lccc} \hline \text{Method} & \text{Calls/design} & \text{Latency (s)} & \text{API (USD)} \\\\ \hline \text{FlexDM} & 1\text{ forward} & 0.039 & 0 \\\\ \text{LaDeCo} & 6.88\text{ GPT-4o}+5\text{ gen.} & 21.4+\text{planning} & 0.0092 \\\\ \text{PosterCopilot} & 1\text{ generation} & 18.3 & 0 \\\\ \text{VFLM} & 3.97\text{ completions} & 93.6 & 0 \\\\ \text{DesignAsCode} & 1P+3\text{ GPT}+1.44\text{ edits} & \sim179 & 0.1030 \\\\ \text{PLAYGROUND} & 4.71P+(6.9\text{--}7.3)G+7.39V & 174.9 & \mathbf{0.0122} \\\\ \hline \end{array}$$ <P: planner; G: predictor; V: visual verifier. Latency excludes rendering> ****[Inference cost comparison.]**** We thank the reviewer for raising this important question. We provide a detailed comparison of model calls, inference latency, and API costs per design in the Table L4. While PLAYGROUND requires more inference steps and computational resources than non-iterative baselines(FlexDM, PosterCopilot), this additional cost is reasonable by its substantially stronger design quality, as demonstrated in Table 1. Importantly, compared with DesignAsCode, a closely related iterative agentic framework, **PLAYGROUND achieves comparable inference latency (174.9 vs. ~179 s/design) while reducing API costs by 8.4× (\$0.0122 vs. \$0.1030 per design)**. These results demonstrate that PLAYGROUND achieves strong design quality with a reasonable inference-time overhead and competitive API efficiency. We have added these results in Appendix D.4 (Table 16).
AuthorsOpenReview ↗

Author Rebuttal(2/2)

[W1, Q3] Evaluation with a non-OpenAI judge

Table L5. Evaluation with the non-OpenAI judge.

\hline \text{Method} & \text{Layout} & \text{Typo.} & \text{Elem.} & \text{Color} & \text{Style} & \text{Intent} & \text{Overall} \\\\ \hline \text{FlexDM} & 2.21 & 1.80 & 2.53 & 2.92 & 2.54 & 2.67 & 2.443 \\\\ \text{PosterCopilot} & 2.57 & 2.40 & 2.64 & 3.28 & 2.80 & 2.87 & 2.762 \\\\ \text{LaDeCo} & 2.68 & 2.42 & 2.86 & 3.36 & 3.03 & 3.07 & 2.903 \\\\ \text{VFLM} & 3.05 & 2.71 & 3.25 & 3.55 & 3.25 & 3.17 & 3.164 \\\\ \text{PLAYGROUND} & 3.37 & 3.43 & 3.48 & 3.89 & 3.62 & 3.59 & \mathbf{3.562} \\\\ \text{GT (revised)} & 3.81 & 3.93 & 3.69 & 4.08 & 3.95 & 3.81 & 3.880 \\\\ \hline \end{array}$$ **[Non-OpenAI judge evaluation.]** We additionally evaluated the methods using Claude Haiku 5.5 on the 600-design Crello test subset. PLAYGROUND remains the highest-scoring generated method across all six criteria. These results demonstrate that our performance gains generalize to a non-OpenAI judge, suggesting that the improvements are not merely attributable to self-preference bias in OpenAI-based evaluation. We have added this evaluation in Appendix D.2.2 (Table 9). >**[W2] Interpretation of rule-based metrics** ****[Metric limitations.]**** As discussed in LaDeCo[1] and LICA[2], rule-based metrics serve as useful sanity checks but cannot fully capture design quality. For example, **Validity** may penalize intentionally small elements, while **Overlap** may penalize intentional layering, such as underlays used to emphasize text. Therefore, these metrics are not necessarily monotonic measures of quality, and proximity to GT can sometimes be a more meaningful reference. We apologize for the potentially misleading direction arrows and will revise them accordingly. [1] Lin, Jiawei, et al. "From elements to design: A layered approach for automatic graphic design composition." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025. [2] Hirsch, Elad, et al. "Lica: Layered image composition annotations for graphic design research." arXiv preprint arXiv:2603.16098 (2026). ****[Complementary evaluation.]**** To complement these metrics, we additionally report more interpretable measures, including **text occlusion (T. Occ.)** and **readability (Rea.)**, alongside VLM-based evaluations and a human user study. We have added this discussion in Appendix D.2.3 (Table 10). >**[W2] Contribution of the structural retrieval term** **Table L6. Change in Top-5 retrieval after removing role-IoU.** $$\begin{array}{lc} \hline \text{Top-5 comparison} & \text{Queries (\\%)} \\\\ \hline \text{Identical members and order} & 2.2 \\\\ \text{Same members, different order} & 5.9 \\\\ \text{Different members} & \mathbf{91.9} \\\\ \hline \end{array}$$ **Table L7. Downstream role-IoU ablation: the reported paper-run comparison.** $$\begin{array}{lccc} \hline \text{Setting / contrast} & \text{Score / difference} & \text{95\\% CI} & p \\\\ \hline \text{Full score} & 3.958 & \text{--} & \text{--} \\\\ \text{Without role-IoU} & 3.917 & \text{--} & \text{--} \\\\ \Delta\\ (\text{Full}-\text{No role-IoU}) & +0.042 & [0.008,0.075] & 0.016 \\\\ \hline \end{array}$$ ****[Influence on reference selection.]**** Although the role-IoU term has a relatively small weight (0.1), removing it and renormalizing the remaining weights changes the **Top-5 reference membership for 91.9% of 2,375 queries**, indicating its substantial influence on reference selection. ****[Downstream comparison.]**** On the 600 Crello subset, removing this term leads to a **statistically significant performance drop** (3.958 vs. 3.917, Δ = +0.042, 95% CI: [0.008, 0.075], p = 0.016). These results demonstrate that the structural term not only influences reference selection but also contributes to downstream design quality. We have added this analysis in Appendix D.4 (Table 13). >**[W2, Q4] PSDesigner & DesignAsCode** Please see the general response under **[Comparison with PSDesigner and DesignAsCode]**, including Table J6, for the shared comparison and experimental setup.
Reviewer_LEAEOpenReview ↗

Response the authors

Thanks the authors for the very detailed experimental results. I appreciate the efforts of the authors. I feel like my concerns for this paper are partly addressed. So I will raise my score to 3 (Weak Acceptance), I think this paper is above the boarderline.

I still think much of the improvement comes from the Visual Verifier rather than from progressive planning, and I encourage the authors to state this clearly and to discuss inference latency as a limitation in the final version.

We sincerely thank the reviewer for their thoughtful follow-up, recognition of our additional experimental efforts, and willingness to raise the score. In the final version, we will explicitly discuss inference latency as a limitation of our approach.

Reviewer_JDEMOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PlayGround addresses element-to-design composition: given canvas metadata and unordered editable image/text elements, it predicts bounding boxes, text attributes, and stacking order. A Structure-Aware Retriever first predicts coarse role-region groups and retrieves five references using role-aware IoU, aspect-ratio, and word/image-count similarity. A Qwen3-VL-8B-based Progressive Planner then repeatedly selects a functional group and writes a canvas-conditioned plan; a separately fine-tuned Layout Generator predicts five candidates, and an untrained GPT-5.6 Luna Visual Verifier selects, accepts, or requests feedback for up to three rounds.

Critical strengths
  1. A well-motivated mechanism for reducing error propagation while preserving editability.
  2. Complementary evidence supports the overall system, especially on Crello.
Critical weaknesses
  • The evidence for the central progressive-planning and grouping contribution is modest and not fully isolated. In Table 2, replacing the Progressive Planner with a Static Planner changes mean VLM score only from 3.958 to 3.894, whereas removing the verifier causes a much larger drop to 3.698. In Table 3, functional grouping scores 3.578 versus 3.532 for LaDeCo-style grouping. No run-level variance or significance testing is reported for these differences on the 600-sample subset. Moreover, the static-planner baseline changes the entire planning schedule; without a variant that fixes groups/order initially but still replans each group's placement from the current canvas, the experiment cannot separate the benefit of dynamic regrouping from that of canvas-conditioned layout replanning. This weakens the paper's main causal attribution, rather than the evidence for the pipeline as a whole.
  • The LICA evaluation uses target-derived metadata and does not provide a clean equal-input zero-shot comparison. Appendix D.1 states that PlayGround's required LICA document titles are generated using the ground-truth rendered image and then used during inference. Even a short title is therefore derived from the target design, and the paper does not establish that equivalent information is supplied to every baseline. In addition, VFLM receives a ground-truth non-text background and PosterCopilot receives text rendered with ground-truth attributes. These native-input adaptations are disclosed, but together they limit what Table 1 can establish about comparative performance and zero-shot generalization. A LICA control using no title or titles derived only from inference-available inputs is needed to rule out an oracle-metadata effect.
Questions for authors
  1. Can you add an ablation that fixes functional groups and their order at initialization but retains canvas-conditioned, per-turn layout planning (and, if feasible, the converse)? Which factor explains the Table 2 difference between 3.958 and 3.894, and is that difference significant across generation and judge seeds?
  2. Why is the ground-truth LICA render used to generate an inference-time title? What are the LICA results when titles are removed or generated solely from the supplied elements and other inference-available metadata, with identical metadata provided to all compared methods?
  3. Why are DesignAsCode and PSDesigner—the closest plan/refine systems discussed in Secs. 1–2—not included in the quantitative comparison? If their implementations are unavailable, can the authors otherwise substantiate the relative advantage through a controlled reproduction or a stronger static-planning baseline?
  4. For Fig. 14, how were participants recruited and instructed, how were method order and sample order randomized, and how many samples did each participant evaluate? Can you report participant- and sample-aware confidence intervals or significance tests for the reported 81–84% rank-1 rates?
4 comments
AuthorsOpenReview ↗

Author Rebuttal(1/2)

We thank the reviewer for recognizing the motivation and overall effectiveness of PLAYGROUND. Your questions have helped us strengthen the statistical evaluation and clarify the scope of our claims.

[W1, Q1] Progressive planning and evaluation variability

[Generation variability.] Please see Table J1 and the shared progressive-planning response in the general response. We address judging variability and the separation of grouping from layout planning below.

Table J2. Mean of three independent judge evaluations of fixed renders.

\hline \text{Method / contrast} & \text{Score} & \text{95\\% CI} & p \\\\ \hline \text{Static Planner} & 3.905 & [3.869,3.941] & \text{--} \\\\ \text{PLAYGROUND} & 3.949 & [3.911,3.988] & \text{--} \\\\ \Delta\\ (\text{Ours}-\text{Static}) & \mathbf{+0.044} & [0.009,0.079] & 0.016 \\\\ \hline \end{array}$$ **[Judging variability.]** We repeated the VLM evaluation three times on the same generated designs. PLAYGROUND consistently outperforms the Static Planner across all three judge runs (3.958 vs. 3.894, 3.944 vs. 3.910, and 3.945 vs. 3.912). The three-run average shows a statistically significant improvement (Δ = +0.044, 95% CI: [0.009, 0.079], p = 0.016), demonstrating that our performance gains remain robust across repeated VLM evaluations. We have added these analyses in Appendix D.4 (Table 12). **Table J3. Separating adaptive grouping from canvas-conditioned planning.** $$\begin{array}{lcc} \hline \text{Grouping / order} & \text{Static plan} & \text{Canvas-conditioned, per-turn plan} \\\\ \hline \text{Fixed} & 3.894 & 3.921 \\\\ \text{Dynamic} & \text{N/A} & \mathbf{3.958} \\\\ \hline \end{array}$$ ****[Separating grouping from layout planning.]**** Following the reviewer's suggestion, we introduce an additional variant that fixes functional group membership and execution order at initialization while retaining canvas-conditioned, per-turn layout planning. As shown in Table J3, **canvas-conditioned planning improves performance over the Static Planner** (3.921 vs. 3.894, +0.027), while **dynamically adapting group membership and order further improves performance** (3.958 vs. 3.921, +0.037). These results suggest that both components contribute to the improvement, with their joint use achieving the best performance. Regarding the converse setting, layout instructions are generated specifically for each group's constituent elements. Therefore, dynamically changing group membership while keeping the corresponding layout instructions fixed is not directly compatible with our formulation. We have added this experiment in Sec. 5.3(Table 3). >**[W1] Functional grouping** **Table J4. Functional grouping versus LaDeCo-style grouping on complex LICA designs.** $$\begin{array}{lccccccc} \hline \text{Grouping} & \text{Layout} & \text{Typo.} & \text{Elem.} & \text{Color} & \text{Style} & \text{Intent} & \text{Overall} \\\\ \hline \text{Functional (Ours)} & 3.602 & 3.389 & 3.695 & 4.328 & 4.086 & 3.906 & \mathbf{3.834} \\\\ \text{LaDeCo-style} & 3.445 & 3.222 & 3.438 & 4.258 & 3.977 & 3.734 & 3.679 \\\\ \hline \end{array}$$ **[Evaluation beyond Crello.]** We acknowledge that the improvement from functional grouping over LaDeCo-style grouping in Table 3 is modest. This may be because LaDeCo-style grouping is specifically designed around Crello's semantic layer structure, making it well-suited to the Crello dataset and potentially narrowing the performance gap in this setting. To further validate our grouping strategy, we conducted additional zero-shot experiments using GPT-5.6 Luna on 128 challenging LICA designs with more than 30 elements. As shown in Table J4, functional grouping achieves a statistically significant improvement over LaDeCo-style grouping (3.834 vs. 3.679, Δ = +0.155, 95% CI: [0.040, 0.265], p = 0.0070). Unlike LaDeCo-style grouping, which separates elements into predefined semantic layers, our approach organizes elements by functional roles and allows multiple groups per role. These results support the effectiveness and flexibility of our grouping strategy on more complex designs beyond Crello. We have added these experiments in Sec. 5.3 (Tables 5).
AuthorsOpenReview ↗

Author Rebuttal(2/2)

[W2, Q2] Target-derived LICA titles & input settings

Table J5. Title ablation on the full LICA evaluation.

\hline \text{Metric} & \text{With Title} & \text{No Title} & \Delta\\ [95\\%\\ \text{CI}] & p \\\\ \hline \text{Layout} & 3.698 & 3.692 & -0.006\\ [-0.045,+0.033] & 0.81 \\\\ \text{Typography} & 3.480 & 3.467 & -0.014\\ [-0.060,+0.032] & 0.60 \\\\ \text{Element} & 3.970 & 3.955 & -0.015\\ [-0.055,+0.024] & 0.50 \\\\ \text{Color} & 4.378 & 4.433 & +0.055\\ [+0.020,+0.091] & 0.003 \\\\ \text{Style} & 4.217 & 4.221 & +0.004\\ [-0.035,+0.043] & 0.88 \\\\ \text{Intention} & 4.224 & 4.225 & +0.001\\ [-0.046,+0.048] & 1.00 \\\\ \hline \text{Overall} & \mathbf{3.994} & \mathbf{3.999} & \mathbf{+0.004}\\ [-0.026,+0.034] & 0.78 \\\\ \hline \end{array}$$ ****[Removing target-derived titles.]**** We thank the reviewer for pointing out this concern. Following the suggestion, we re-evaluated PLAYGROUND **without document titles** on full LICA designs. As shown in Table J5, removing titles yields comparable performance (3.994 vs. 3.999), suggesting that our results are not driven by target-derived metadata. ****[Native input settings.]**** Regarding input fairness, existing methods differ in their required inputs, element representations, and prediction targets, making a strictly identical input setting difficult to establish without modifying their original task formulations. Importantly, some baselines also receive privileged information: **PosterCopilot uses text rendered with GT attributes, while VFLM receives a GT non-text background.** Therefore, PLAYGROUND is not uniformly advantaged by its input setting. We clarify that Table 1 compares methods under their **respective native inference settings**, rather than claiming a strictly equal-input comparison. We have added these clarifications and results in Appendix D.1 (Table 8). >**[Q4] User-study protocol and uncertainty** **Table J7. Participant- and design-aware user-study confidence intervals.** $$\begin{array}{lcc} \hline \text{Criterion} & \text{Rank-1 (\\%)} & \text{95\\% CI (\\%)} \\\\ \hline \text{Layout composition} & 82.1 & [74.7,89.1] \\\\ \text{Typography} & 84.4 & [77.5,90.5] \\\\ \text{Visible elements} & 81.5 & [73.5,88.7] \\\\ \text{Color harmony} & 81.4 & [73.4,88.4] \\\\ \text{Style coherence} & 81.5 & [73.2,89.0] \\\\ \text{Intention alignment} & 81.7 & [73.6,88.7] \\\\ \text{All six pooled} & \mathbf{82.1} & [74.7,88.8] \\\\ \hline \end{array}$$ **[Evaluation protocol.]** Participants were randomly recruited and instructed to rank the five methods using the same six criteria as our VLM evaluation. The study included 32 participants and 25 design sets, yielding 661 participant–design evaluations; 24 participants completed all 25 sets. We computed participant- and design-aware 95% CIs using 10,000 two-way cluster bootstrap resamples, independently resampling participants and designs. The results support a strong preference for PLAYGROUND after accounting for both sources of variability. We have added these details and results in Appendix D.2.4 (Table 11).

Dear Reviewer JDEM, thank you again for your constructive review and valuable suggestions. As the discussion deadline approaches, we wanted to follow up on our response, which includes additional statistical analyses, ablations separating dynamic grouping from canvas-conditioned planning, and comparisons with DesignAsCode. If you have had a chance to review these results, we would appreciate your thoughts on whether they address your questions. We would be glad to clarify any remaining points before the deadline.

Dear Reviewer,

As the discussion period draws to a close, we kindly ask you to review our rebuttal and additional experimental results. We sincerely hope that our clarifications help address your concerns and highlight the significance of our progressive, reference-guided framework for editable graphic design. We would greatly appreciate your reconsideration of the overall score.

If you have any remaining concerns, we would be happy to discuss them further and provide additional clarification.

Thank you again for your time and thoughtful feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their constructive feedback and for recognizing the motivation of our approach (JDEM), its empirical effectiveness (JDEM, LEAE), the value of our automatic data construction pipeline (LEAE), and the breadth of our comparisons and ablations (bGdJ). We address the shared concerns of JDEM and LEAE below, followed by reviewer-specific responses.

[Statistical Protocol.] Throughout the rebuttal, we compute 95% CIs using 10,000 bootstrap resamples and two-sided p-values using 100,000 paired permutations. Comparisons are paired by design, with repeated-run scores averaged for each design before testing. User-study CIs use a two-way cluster bootstrap over participants and designs.

[Progressive versus Static Planning: JDEM W1/Q1; LEAE W1/Q2.]

Table J1. Progressive versus Static Planner across generation runs.

\hline \text{Generation} & \text{Ours} & \text{Static} & \Delta & \text{95\\% CI} & p \\\\ \hline \text{Run 1} & 3.958 & 3.894 & +0.064 & [0.028,0.100] & 0.0006 \\\\ \text{Run 2} & 3.950 & 3.903 & +0.047 & [0.013,0.082] & 0.0096 \\\\ \text{Run 3} & 4.022 & 3.902 & +0.119 & [0.083,0.157] & < 10^{-5} \\\\ \text{3-run mean} & \mathbf{3.977} & 3.900 & \mathbf{+0.076} & [0.051,0.102] & < 10^{-5} \\\\ \hline \end{array}$$ **[Generation variability.]** We thank the reviewer for suggesting a more rigorous evaluation. We conducted three independent generation runs for both PLAYGROUND and the Static Planner. As shown in Table J1, **PLAYGROUND consistently achieves statistically significant improvements across all three runs**. Averaging the three runs yields a significant improvement (Δ = +0.076, 95% CI: [0.051, 0.102], p < 0.00001). These results demonstrate that the benefits of progressive planning are consistent across independent generations, rather than arising from a single favorable run. The performance gap between the Progressive Planner and the Static Planner may appear modest compared with **PLAYGROUND's larger improvements over other baselines in Table 1**. We attribute this relatively small gap to their shared supervision. Specifically, although the two planners use different training formats, both are trained on the same high-quality planning trajectories constructed by our automatic annotation pipeline. The Static Planner therefore already benefits from strong step-wise planning supervision, making it a strong, controlled baseline. We have added these analyses in Appendix D.4 (Table 12). >**[Comparison with PSDesigner and DesignAsCode: JDEM Q3; LEAE W2/Q4.]** **Table J6. Comparison with DesignAsCode under shared generated assets.** $$\begin{array}{lcc} \hline \text{Metric} & \text{DesignAsCode} & \text{PLAYGROUND} \\\\ \hline \text{Layout} & 3.690 & \mathbf{3.753} \\\\ \text{Typography} & 3.633 & \mathbf{3.693} \\\\ \text{Visible elements} & 3.853 & \mathbf{3.865} \\\\ \text{Color harmony} & 4.282 & \mathbf{4.365} \\\\ \text{Style coherence} & 3.937 & \mathbf{4.005} \\\\ \text{Intention alignment} & 3.973 & \mathbf{4.057} \\\\ \text{Overall VLM} & 3.895 & \mathbf{3.956} \\\\ \text{API cost (USD/design)} & 0.1030 & \mathbf{0.0122} \\\\ \hline \end{array}$$ ****[Challenges in Direct Comparison.]**** We appreciate the reviewer's suggestion to compare with these closely related methods. A direct comparison with PSDesigner is currently difficult due to the unavailability of its implementation. Meanwhile, DesignAsCode targets **intention-to-design generation**, where a trained planner specifies the required assets and layout, followed by image synthesis (GPT-Image-1) and GPT-based refinement (GPT-5 and GPT-4o). This differs from our primary **asset-given layout composition** task. ****[Comparison with DesignAsCode.]**** To enable a more comparable evaluation, we use PLAYGROUND's intention-to-design extension described in Appendix F. Specifically, we generate assets from DesignAsCode's planner-generated descriptions using Qwen-Image-2.1 instead of GPT Image to reduce image-generation costs, and provide **the same generated assets to both pipelines**. On the **Crello 600-design ablation subset**, PLAYGROUND achieves a higher overall VLM score (3.956 vs. 3.895) while requiring **8.4× lower API cost**. Notably, PLAYGROUND achieves superior design quality despite DesignAsCode's reliance on powerful, higher cost closed source models (GPT-5 and GPT-4o), highlighting the effectiveness and cost efficiency of our composition pipeline. We have added the comparison in Sec. 5.3 (Table 6) and Appendix F (Fig. 19).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.