- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies text conditioning for text-to-image diffusion and argues that caption information, rather than caption length, is the relevant axis for scaling. It introduces structured prompts (SP), a typed JSON caption interface, and two measures of caption informativeness: Grounded Perplexity Gain (GPG) and Effective Detailness (ED). In a 15-configuration BAGEL sweep, the authors calibrate converged diffusion MSE against GPG and ED to characterize text-conditioning scaling within that setup. They frame generation quality as Diffusability x Promptability: SP annotations improve diffusability, while an LLM prompter trained to map user requests to SPs improves promptability. The evaluation includes controlled caption sweeps, held-out spatial/field variants, matched natural-language retraining on Qwen-Image, prompter-training ablations, and an inference-time refine-render-judge loop. The paper reports results on GenEval, GenEval2, DPG-Bench, TIIF, WISE, and T2I-CoReBench. The evaluation spans several settings, but the scaling claim remains specific to the tested recipe, and proprietary components limit reproducibility.
Critical strengths
-
The controlled design tests whether caption organization matters independently of caption length. It fixes the backbone and training budget, evaluates 15 caption configurations using two complementary metrics, tests held-out variant prediction, and includes matched NL retraining. Together, these results support the claim that caption organization contributes beyond caption length.
-
The end-to-end evaluation spans GenEval, GenEval2, DPG-Bench, TIIF, WISE, and T2I-CoReBench, and includes matched controls. The SP system outperforms an NL interface trained with the same Qwen-Image architecture, data, stages, and budget. This comparison makes additional training alone a less plausible explanation for the gains.
Critical weaknesses
-
The scaling claim extends beyond the evidence. The reported properties are fitted to a single BAGEL sweep of 15 configurations, with each configuration trained once, and are explicitly dependent on the judge, extractor, and training recipe. Qwen-Image is not re-fitted, and the held-out variants come from the same annotation suite. These results do not yet justify treating the fitted properties as general.
-
The study provides limited evidence on reproducibility and confound control. The pipeline relies on proprietary VLMs and APIs, a Qwen3.5-397B prompter, and large annotation infrastructure. Corpus size, annotation quality, judge variance, and seed-level training variance are not reported in sufficient detail. GPG and ED also depend on frozen judges and extractors, leaving the central calibration without independent validation.
Questions for authors
-
Can the authors evaluate whether the GPG/ED calibration transfers across backbones, for example, Qwen-Image or another DiT family? If the coefficients do not transfer, how should the claim about scaling properties be scoped?
-
What are the exact training corpus sizes, the number of L10 SPs, and the annotation quality statistics?
-
For the key BAGEL sweep points, can the authors report run-to-run or seed-level variance and compare it with the reported held-out MAEs of 5.0e-4 and 8.1e-4?
-
Are GPG/ED evaluated using judges and extractors held out from the calibration fit? Can the authors report results on an independent validation set, or at least assess judge robustness for the end-to-end benchmark gains?
-
Can the authors add human evaluation or a stronger judge-robustness analysis for the main benchmark comparisons, given that most gains are measured by automated judges?