- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work tackles the labor-intensive nature of existing text-to-image post-training and the poor cross-model generalization of handcrafted post-training datasets. They present a fully automated, model-, benchmark-, and architecture-agnostic RSI framework built upon three collaborative agents. Their framework achieves higher average performance than the manually curated RL dataset baseline on all tested diffusion models.
Critical strengths
(1) Propose a model-, benchmark-, and architecture-agnostic RSI framework for automated text-to-image post-training with recursive self-improvement. (2) Design an iterative loop with three collaborative agents. Then create a diagnostic evaluation dataset along with a hierarchical scene taxonomy, and generate diverse prompts across multiple descriptive dimensions. The pipeline automatically yields prompt-rubric pairs for diagnosis, independent of public benchmarks.
Critical weaknesses
(1) While empirical results show promising performance, the work neglects inherent errors from the agent-based evaluator. (2) It remains unvalidated whether the proposed hierarchical taxonomy comprehensively spans all critical evaluation dimensions for text-to-image generation. (3) The alignment between judgments from the diagnosis agent and human preferences is not verified.
Questions for authors
(1) Is the superiority of the automated results over the manually curated training set confirmed via human double-blind evaluation? (2) What is the construction process of the manually curated RL training baseline?
2 comments
Q1: Is the superiority over the manually curated training set confirmed via human double-blind evaluation?
We conducted a human double-blind evaluation (Table C1). We sampled 1,000 prompts from the five evaluation benchmarks (GenEval, GenEval2, OneIG-EN, DPG, and CVTG-2K) and generated images with the Base model, the Handcrafted RL model, and APT using the same seed and inference settings. Six PhD students from different research fields served as annotators. For each prompt, APT was compared with each baseline, giving 1,000 pairs per comparison, and each pair was rated by one annotator. The two images were shown side by side in random order without method labels, and annotators chose win, tie, or loss on prompt alignment, visual quality, and overall preference. APT is preferred over the handcrafted baseline in 64.6% of comparisons on prompt alignment and 63.1% on overall preference.
Table C1. Human double-blind evaluation (Lens; 1,000 pairs per comparison, 6 annotators). Win / Tie / Loss (%) of APT.
| Comparison | Prompt alignment | Visual quality | Overall preference |
|---|---|---|---|
| APT vs. Base | 81.1/9.8/9.1 | 71.3/8.8/19.9 | 80.5/8.5/11.0 |
| APT vs. Handcrafted RL | 64.6/12.2/23.2 | 60.2/14.5/25.3 | 63.1/12.5/24.4 |
Q2: Construction process of the manually curated RL training baseline
The handcrafted baseline is the RL training set released with Lens[2], which was curated by human experts to cover diverse generation scenarios. As described in the Lens[2] paper, it is a static RL prompt set built with a taxonomy-driven pipeline:
- Human experts defined a taxonomy of generation-scenario categories (e.g., Human, Object, Text, UI & Graphic Design). Each category has fine-grained sub-categories and concrete items.
- They defined description dimensions for enriching prompts: Attribute, Spatial Relationship, Count, Interaction, and Color.
- For each item, a few dimensions were sampled at random, and GPT-4.1 turned them into an image-generation prompt.
- They manually reviewed the prompts and added prompts from categories where the base model performs poorly.
In the Lens RL dataset, human experts fix the training distribution in advance, and it stays the same across models and throughout training. APT covers the same scenarios but adapts its distribution at every round to the current model's diagnosed weaknesses. We will add these details to the appendix.
We thank the reviewer again. All new results and clarifications will be included in the revised manuscript.
[1] RubricRL: Simple Generalizable Rewards for Text-to-Image Generation [2] Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models
We thank the reviewer for the helpful comments on evaluator reliability and human alignment. We respond to each point below.
W1: Inherent errors of the agent-based evaluator
We agree that the evaluator is imperfect. Several properties of APT make it robust to evaluator noise, and we now quantify that noise.
(i) Decomposed binary judgments. Instead of asking for a holistic score, the evaluator answers narrow, verifiable yes/no questions such as "exactly two puppies are present." Such questions are less ambiguous than holistic ratings.
(ii) Aggregation. The training reward averages up to 10 rubric judgments. The Data Generation Agent does not act on individual judgments; it identifies recurring failure patterns across about 120 prompts and up to 1,200 rubric judgments per round. Isolated errors are therefore smoothed out.
(iii) Independent final evaluation. None of the reported results are computed with our GPT-4.1-mini evaluator. All numbers come from the official protocols of five public benchmarks: detector- and classifier-based checks in GenEval, Soft-TIFA in GenEval2, MLLM-based VQA in DPG, OCR word accuracy in CVTG-2K, and dimension-specific scorers in OneIG-EN. If evaluator errors were driving the training, the gains would be unlikely to transfer to these independent metrics.
(iv) Measured error rate. Following RubricRL[1], we conduct a human evaluation to validate the quality of generated rubrics and the reliability of our VLM grader (GPT-4.1-mini). We sample 100 prompts and generate 10 rubrics per prompt, yielding 1,000 prompt–rubric pairs. Human annotators assess prompt–rubric consistency, rubric hallucination, and grader correctness. Three annotators independently labeled all 1,000 prompt–rubric pairs, and we report the average over annotators. On average, 97.2% of the generated rubrics are consistent with their prompts, only 0.83% contain hallucinated constraints, and the VLM grader agrees with human judgments on 95.7% of pairs, demonstrating the reliability of our rubric-based evaluation.
W2: Whether the hierarchical taxonomy spans all critical evaluation dimensions
No finite taxonomy can be proven complete, and we will soften the wording accordingly. Our goal is broad and extensible coverage rather than exhaustive coverage, and we provide the following evidence for it.
(i) Scale and structure. The evaluation set spans 10 categories, 219 sub-categories, and 2,609 prompt–rubric pairs. The rubrics cover OCR, counting, attributes, actions, and spatial relations (Figure 2, Appendix E.2).
(ii) Generalization to unseen benchmarks. None of the five benchmarks is used to build the evaluation set, yet a single APT run improves the base model on almost all of them across four models.
(iii) Comparison with benchmark evaluation sets. Replacing our evaluation set with the OneIG-EN or GenEval sets lowers the average and produces biased improvements (Table 3). This indicates that our set is broader than either single benchmark.
(iv) Extensibility. Users can specify additional scopes, and the Evaluation Set Generation Agent will extend the taxonomy.
W3: Alignment between the diagnosis judgments and human preferences
We add the human agreement study described in W1(iv): 97.2% of the generated rubrics are consistent with their corresponding prompts, while only 0.83% contain hallucinated constraints. Moreover, our VLM grader achieves 95.7% agreement with human judgments, demonstrating the reliability of our rubric-based evaluation. The benchmark metrics we report are also known to be human-aligned. For example, GenEval2 reports that Soft-TIFA_GM, the metric we use, achieves higher AUROC against prompt-level human judgments than the arithmetic-mean variant.