- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper argues that text-to-image models fail on prompts that require computation because the answer never reaches the renderer in a usable form. On SynBool, a Boolean program in the prompt selects a shape’s colour. There, FLUX.2-Klein drops from 60.8% at 3 operations to chance (about 25%) at 7 or more, but reaches 80.7% when the answer is given. More denoising steps do not help, and Qwen-Image behaves the same way. They formalize this via three theorems in a toy flow-matching model. The paper then proposes Plan+Paint, where an LLM resolves the instruction first and the renderer adds objects one edit at a time. They report gains on synthetic grids, GenEval, and T2I-ReasonBench.
Critical strengths
S1. A clean diagnosis of instruction following versus drawing. SynBool keeps the visual target fixed and changes only how the answer is given. This separates following the instruction from drawing the image. The controls are convincing. The model draws an injected wrong colour 86.8% of the time, and a shuffled reasoning trace works as well as an ordered one (78.7% vs. 78.1%). So the renderer follows the final decision, not the reasoning. Step sweeps and a second model rule out sampler- or model-specific effects. SynBool could be a useful diagnostic tool for others.
S2. A simple error decomposition. End-to-end accuracy is roughly planner accuracy times rendering accuracy on correct plans, and the renderer almost never fixes a wrong plan (). This gives a clear rule for where to spend test-time compute in such pipelines.
Critical weaknesses
W1. The main remedy is already standard, and the theory makes no testable prediction. Using a language model to resolve or rewrite a prompt before rendering is already common. DALL-E 3 rewrites prompts with an LLM (Betker et al., 2023), and the paper itself cites planner-based and reasoning-first systems such as RPG, LMD, T2I-R1 and GoT. The contribution therefore rests on whether the theory explains something new. However, each theorem is used to explain results after they are observed rather than to predict them. The theory does not explain why accuracy collapses between 3 and 7 operations. The Fourier analysis in Appendix B.1 could predict which SynBool programs a simple readout can solve, but the paper never compares such predictions with per-task accuracy. Without a prediction that the data could have contradicted, the theory restates the empirical finding rather than explaining it.
W2. Theorem 1 is not connected to a real model. Theorem 1 says a model that cannot represent the answer cannot use it, which is nearly true by construction. Its relevance therefore depends on knowing what the model can represent, but the paper never identifies or measures this for FLUX or Qwen-Image. The 60.8% accuracy at 3 operations also shows that the model does compute part of the answer.
W3. The assumptions of Theorem 2 do not hold for FLUX. Theorem 2 assumes the model has fewer sequential steps than the program depth. FLUX runs dozens of layers 28 times, which is far more than the depth of these programs. The bound also assumes each step is a hidden lookup table, whereas SynBool writes every step in the prompt.
W4. The “16× more energy” result follows directly from the guidance gain. Energy grows with the square of the guidance gain. Raising the gain from 1 to 4 therefore gives times more energy at any disclosure time. This number says nothing about timing and should not be presented as support for Theorem 3.
W5. The timing experiments do not test Theorem 3. Theorem 3 concerns an optimal control and implies that late correction remains possible at finite cost. The experiments use standard guidance instead. This guidance fails at even with 4.65× the energy of early guidance. That points to the model committing to colour and layout early, which the theory does not model.
W6. A simpler explanation is not ruled out. The same Qwen3 model stays at chance when it answers without chain-of-thought, and the text-encoder probes collapse with depth. A single LLM forward pass failing on deep Boolean programs is a known result (Feng et al., 2023a; Li et al., 2024). This explains the main finding without any diffusion-specific theory.
W7. The evidence for an extra gap inside the renderer is weak. The only evidence that the renderer adds its own gap is at shallow depth, where the probe reaches 0.927 but rendering reaches 0.646. This probe is a four-layer Transformer that can compute the last gates itself. The linear and MLP probes described in Appendix C.4 would show what is directly readable, but their results are not reported.
W8. No experiment separates “cannot” from “was never trained to”. The paper claims the failure is structural, but current models were never trained on such prompts. Fine-tuning on unresolved SynBool prompts would test this. If shallow programs improve while deep ones stay at chance, the claim would be supported. Without this experiment, both readings fit the data equally well. Running this experiment, and the reframing it may require, goes beyond what a rebuttal can fix.
W9. The compute budgets of the compared methods are not matched. Plan+Paint renders up to five times per image while Plan renders once, and no compute cost is reported. Part of the gain may come from the extra render passes. Comparisons at equal budget are needed, for example Plan with best-of-N sampling and a verifier.
W10. Important baselines are missing. A layout-grounded renderer (LMD, GLIGEN) might solve the crowding problem in a single pass, which would directly test the paper’s explanation for why Plan fails. Existing planner and editing systems (LMD, RPG, SLD, GenArtist) are also not compared, although Plan+Paint is close to GenArtist and SLD.
W11. The T2I-ReasonBench results contradict the paper’s own explanation. These prompts contain a single entity and no crowding, so incremental painting should add little. Yet Plan+Paint beats Plan by 22 points, and Plan is 23 points below the oracle. This suggests the gain comes from differences in prompts or render passes rather than from the span/work trade-off.
W12. The natural-prompt benchmarks do not test serial reasoning. GenEval position prompts contain a single spatial relation. The T2I-ReasonBench entity prompts mostly require recognising one entity from its description. Neither involves the serial dependencies the paper is built around. The gains of Plan on these benchmarks likely come from clearer, more explicit prompts rather than from resolving serial depth, so they do not support the theory.
Minor issues: The step sweep is reported as 24.8–27.1% in Section 3.3, but 24.3–27.1% in Appendix A.3. Figure 5c shows 28.5% while the text says 28.7%. Several items are missing or undefined, including the 16-cell table behind “15 of 16 cells”, the per-cell SynBool results, the “early7” condition, and the flow-control task setup. The symbol means both noise level and σ-algebra, and is never defined. The title says “Plan Before You Denoise” while the method is called “Plan Before You Draw”. The planner’s token budget is not given, so truncation could explain the 31.2% planner accuracy at 31 operations. Please refine the writing. It was very hard to understand and appreciate your contributions due to the poor presentation.
Questions for authors
Q1. What does the probe predict, and what is its chance level? Figure 2 marks 0.5, but rendering is four-way with chance 0.25. If the probe is four-way, 0.481 at deep strata is well above chance, which changes Finding 4. Please also report the linear and MLP probes, the layer used, and the number of tasks per depth level.
Q2. Why does Plan+Paint beat Plan by 22 points on single-entity prompts, and what do its edit steps do there? Please report planner accuracy, the compiled prompts, the planner and editor models for each benchmark, and how FLUX.2-Klein performs the edits.
Q3. What is the total cost of each method (render calls, tokens, time)? How does Plan with best-of-N and a verifier compare at equal cost?
Q4. Which readout class describes FLUX, and can it be measured? How does Theorem 2’s assumption of fewer steps than program depth hold for FLUX?
Q5. Unresolved SynGrid-2×2 accuracy rises from 0.081 to 0.360 between 3 and 4 objects. Is this a scoring or label-balance artefact? Please also add per-object planner and oracle-steps accuracy to Table 1.
2 comments
Thank you for running all the experiments I suggested and for the candid response. The regional prompting baseline and the fine-tuning details are useful additions, and the paper is clearer than the original submission.
However, the new results confirm my central concerns rather than resolve them. The response itself now steps back from the paper's main claims.
- The failure is in the text encoder, not in denoising. The response agrees that the bottleneck is text conditioning, and the joint linear probe (0.622) nearly matches rendering (0.646). This is the known limit of a single Transformer pass, so the central finding is not new.
- The theory no longer explains the results. The response now presents the theorems as intuition. Fourier weight explains most of the span effect (the L=1 vs. L=3 gap drops from 21.8 to 8.2 points). Yet the theory's own quantity improves out-of-distribution prediction by only 0.019 nats, against 0.094 for the empirical probe.
- The method gains are small once baselines are fair. On T2I-ReasonBench, a short-caption Plan already reaches 0.745, so the second pass adds only 2.5 points, and a matched oracle reaches 0.880 to 0.900. On SynGrid, PLAN+PAINT beats regional prompting by about 7 points at matched latency, and only on dense synthetic grids. Planning before rendering is already standard, as the response agrees.
- The "cannot" claim is not supported. Joint fine-tuning fits 82.6% of 7-operation training circuits but only 27.7% of test circuits. This shows poor generalization, not a capacity limit.
The current manuscript still makes the claims the response concedes, namely that the theorems explain the gap, that PLAN+PAINT matches the explicit-entity oracle, and that a single-pass capacity limit exists. Aligning the paper with these findings would require a substantial reframing.
What remains is a useful synthetic diagnostic and an incremental planning-plus-editing pipeline, which I do not think meets the high bar of ICLR. I therefore keep my score. Thank you again for the constructive discussion.
Thank you for the detailed response and the revised manuscript. Several of my concerns are now resolved. The probe clarifications answer Q1, and the revised timing section, together with the new patching test, addresses W4 and W5. The fine-tuning study, the cost table with 6-step PLAN+PAINT, the per-task Fourier analysis and the explanation of the 2×2 jump (Q5) also make the paper stronger, and the writing is much clearer.
My main concerns, however, are about the core claims, and the new results make them sharper rather than weaker.
The first is where the failure actually happens. The response agrees that deep rules fail because of the known limit of a single Transformer pass (response to W6). The new linear probe points the same way. It reaches 0.784 per bit at shallow depth, which is roughly 0.61 for both bits and close to the 0.646 rendering accuracy. In other words, the renderer seems to read about as much as a linear probe can read from the text encoder. The diffusion-side effects listed in the response (prompt interference, crowding, late disclosure) are real, but they concern drawing the answer, not working it out. The execution gap therefore appears to come from the text encoder rather than from denoising.
The second concern is about the theory. The new evidence does not support the span-versus-work view behind Theorem 2. In Study 2 (Table 5), span is fixed at 2, yet rendering already falls to 29.4% at 7 operations, below the matched chain (39.6%), while the probe stays at 0.92. Rendering failure thus tracks the number of XOR operations, not serial span. Section 3.3 also leaves this result out and reports only the probe. Since the response now explains the timing effect through early commitment rather than the Gramian bound, the theory adds little beyond the observation that the DiT reads text features linearly.
The third concern is the T2I-ReasonBench comparison. The revised Section 5.2 explains the 22-point gain (W11), but the explanation suggests the comparison is not fair. Plan puts the entity, clues and rubric details into one long caption, while PLAN+PAINT draws a short subject caption and then refines it in a second pass. The gain may therefore come from the shorter caption and the extra pass rather than from planning.
The numbers point the same way. The planner is correct on 83.5% of prompts, and wrong plans are rarely rendered correctly (Section 3.5). To reach 0.770, PLAN+PAINT would need to render about 92% of correct plans correctly. That is well above the single-pass oracle that names the entity directly (0.775). W12 also remains open, since neither natural benchmark requires serial reasoning.
Finally, the response agrees that prompt rewriting is standard and that LMD, GLIGEN and RPG share the same division of work as PLAN+PAINT. What remains new is placing objects across several passes instead of within one. This choice is never compared with a single-pass layout-grounded method. Training-free regional prompting already runs on FLUX-style DiTs, so a fair comparison on the same backbone is feasible.
The following experiments would help clarify the contribution:
I appreciate the effort that went into this revision. Since my remaining concerns are about the core claims rather than details, I keep my score for now, but I am happy to continue the discussion. I am inclined to increase my score if these very central issues can be addressed properly. Thanks again.