- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors propose PlanYourPixel (PYP), a training-free pipeline that uses sequential LLM agents to decompose a prompt into a structured scene representation S(T) = {N,NP,R}. They first use a prompt parse that extracts object nodes and enrich them with LLM's prior knowledge, then PYP considers the relation of the nodes, it converts each relation into a directed edge with explicit geometric constraints like dominant axis, ordering , size etc and finally it uses its layout generator to generate then check the bounding boxes on a 512 canvas grid to place the objects.
PYP proposes two applications:1. Layout-guided generation, and 2. failure localization. The layout guided generation just applied to the generated layout using PYP's scene representation to the layout2image models like GLIGEN, MIGC etc. The failure localization uses the scene representation to generate atomic visual questions and use VLM judge to answer to identify the fine-grained errors in the results.
Based on the scene representation and the failure localization application, the authors propose PYP-Bench to perform systematic evaluation for image generation with fine-grained failure localization which are ignored by the current evaluation benchmarks.
Quantitatively, PYP can generate more accurate layouts and boost the performance of the Layout2image generators.
Critical strengths
-
The authors motivation is well defined and useful, that the current evaluation benchmarks are not aware of the fine-grained failures such as inter-object geometry (contact, relative size).
-
The design of the scene representation is universal, it fits in current generation pipelines as well as the evaluation pipeline.
-
Improvement in downstream generation on Layout2Image models is significant.
Critical weaknesses
-
I believe the authors should also include the layout generation pipeline in previous agentic image generation like GenArtist, CompAgent, LayerCraft or even the CoT reasoning pipeline from close sourced models like GPT image 2 and Nanobanana.
-
I think the overall framework is still only using the LLM's text reasoning ability, there must be a gap between the real image level reasoning, defining a more structured representation is still somehow incremental.
Questions for authors
I believe the comparison between PYP layout and LaySyn layout for downstream task is not fair. in the Figure 4. PYP Layout has label "a red cow", but the others is just "a cow" without any other descriptions. I want to know how this differences changes the downstream performance. Could the authors elaborate more on this?
What if the LLM itself is not able to generate correct relations or it cannot refine the output from its own hallucinations? I hope authors can provide some LLM failure cases in the revised version.
I will raise the score if the authors solve my questions.
Response to Reviewer Concerns and Weaknesses
[W1]
See SKE-Layout discussion in Q1. For layout generation, we have expanded our comparisons to include GenArtist and LayerCraft. We also investigated CompAgent, but its official GitHub repository contains only a placeholder README without publicly available implementation code. See updated results in the revised paper Table 2.
For evaluation methods, our PYP is different in the following aspects:
Fine-grained failure modes. PYP-VQA reports more precise failure modes (e.g., object presence, attribute swap) and more fine-grained categories related to attributes compared to existing work such as TIFA, DSG, L-VQAScore, and FineGRAIN. For example, in Figure 5, PYP can report three different types of attribute errors, while other tools just say “attribute failures”. As a result, model designers can understand detailed model weaknesses.
Shared scene representation. PYP-VQA uses the same scene representation for planning and evaluation stages. We therefore can locate precise failures during planning and/or in existing layouts, which is not done in prior work like TIFA, DSG, L-VQAScore, and FineGRAIN.
[W2]
Part 1: Yes, good point. When we calculate the overall score to judge whether a generation fails, we focus on relational constraints directly from the prompt. We limit the influence of the relative size (RS) and physical contact (CC) which sometimes can come from enriched contents and sometimes directly from prompt. As shown in Table 4, our PYP outperformed the baselines regarding evaluation.
Part 2: In addition to overall human ratings provided by the ground truth given in Table 4, we have done an additional manual evaluation on 300 images from six T2I-CompBench categories, covering 2,197 questions across seven diagnostic types. Two independent reviewers have done the annotations and then a third reviewer evaluated cases where there is a disagreement. The final manual label goes with the majority voting among the three reviewers.
[W3]
We added experiments on the same GPT-4o-mini backbone, including two new baselines GenArtist (NeurIPS 2024) and LayerCraft (NeurIPS 2025). We also updated our results in the revised paper Table 2.
Downstream baseline selection. We selected LaYouSyn because its layout accuracy is closest to PYP among the compared baselines (Table 2). We tested both methods across four image-generation backbones to examine whether similar layout accuracy leads to similar prompt adherence. PYP generally performs better, particularly on spatial relationships (Table 3).
We address the planning-component ablation in our response to Q3 (Appendix A1.1).