- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Fine-T2I is a large-scale dataset for text-to-image fine-tuning, combining 6,145,693 synthetic samples with 168,424 curated real images. Its construction pipeline integrates controlled prompt generation, semantic deduplication, prompt enhancement, multiple aspect ratios and resolutions, aesthetic filtering, and reasoning-VLM-based verification. The authors evaluate the dataset by fine-tuning LlamaGen, SD-XL, and DeCo, reporting improvements in human preference, particularly in visual quality, alongside mixed results on GenEval. The main contribution is a substantial data resource and an integrated curation pipeline intended to support open text-to-image research.
Critical strengths
-
A substantial data contribution addressing a practical research need. Fine-T2I brings together millions of synthetic samples and curated real photographs, with coverage across content categories, visual styles, prompt granularities, and aspect ratios. This combination of scale and structured coverage makes it a potentially useful resource for studying text-to-image fine-tuning. Its release could reduce the substantial cost of assembling comparable training data and enable further research on data selection and curation.
-
Encouraging results across different model families. Experiments on LlamaGen, SD-XL, and DeCo demonstrate the dataset’s applicability to different generation frameworks. Human evaluation indicates improved visual quality after fine-tuning, particularly for LlamaGen and SD-XL, supporting the dataset’s practical utility.
Critical weaknesses
-
Insufficient validation of the filtering strategy. The pipeline combines Aesthetic Predictor V2.5 with reasoning-VLM verification, but provides no controlled comparison with alternatives such as HPSv3 or HPSv3++. The limitations of scalar reward models do not, by themselves, establish the effectiveness of the proposed approach. Given the reported preference-prediction and downstream optimization results of these alternatives, a comparison on the same candidate pool, with matched retained-data sizes and independent quality assessment, would better justify the filtering choices.
-
Limited evaluation coverage. The evaluation relies primarily on human preferences over 500 prompts, while GenEval is the only automatic benchmark reported. Human evaluation is useful for assessing visual quality, but the lack of complementary benchmarks leaves complex instruction following, text rendering, and other claimed capabilities insufficiently evaluated. Results on a representative selection of DPG-Bench, TIIF-Bench testmini, OneIG-EN, or Qwen-Image-Bench would strengthen the evidence. Confidence intervals and inter-annotator agreement would also help assess the reliability of the human evaluation.
-
Declines in basic instruction-following capabilities. GenEval evaluates basic capabilities such as counting, spatial relations, and attribute binding, which remain essential to text-to-image generation regardless of aesthetic quality. DeCo’s overall score decreases from 0.82 to 0.77, with declines in counting (0.66 to 0.60), position (0.72 to 0.54), and attribute binding (0.77 to 0.68). Correctly satisfying these constraints is compatible with producing aesthetically appealing images. I therefore do not consider improvements in visual preference a sufficient justification for these declines, nor should they be dismissed as merely a benchmark-specific effect.
-
Confounded and incomplete dataset comparisons. Fine-T2I, T2I-2M, and BLIP3o-60k are each used for one epoch despite substantial differences in dataset size, making it difficult to separate data quality from training scale. BLIP3o-60k’s lower training loss does not resolve this issue, since losses on different data distributions are not directly comparable measures of downstream utility. Comparisons with matched training budgets and matched sample counts are needed. Including representative text-to-image subsets from OpenGPT-4o-Image, ShareGPT-4o-Image, or Echo-4o-Image would also provide a more complete assessment against relevant synthetic-data resources.
-
Limited validation on contemporary latent-space flow-matching models. Although DeCo incorporates a DiT and a flow-matching objective, the study lacks fine-tuning experiments on representative latent-space flow-based transformers, such as FLUX.1-dev from the FLUX family. Evaluating such a backbone would help establish whether the dataset’s benefits extend to widely used models with stronger initial generation capabilities, strengthening the evidence for its generalizability.
Questions for authors
-
What controlled experiments show that the proposed filtering pipeline produces better training data than HPSv3 or HPSv3++?
-
Do the reported gains hold on complementary benchmarks such as DPG-Bench, TIIF-Bench testmini, OneIG-EN, or Qwen-Image-Bench?
-
Does Fine-T2I retain its advantage under matched training budgets and data sizes, including against recent datasets such as OpenGPT-4o-Image, ShareGPT-4o-Image, or Echo-4o-Image?
-
Have the authors tested whether the gains extend to latent-space flow-based models such as FLUX.1-dev?
6 comments
Rebuttal (part 2 / 3)
W3: Declines in basic instruction following. We agree these obersvations should not be ignored, and we have identified the reason behind.
BLIP3o-60k includes geneval_train set (7.1k samples, 11.8%) from the GenEval prompt, and DeCo's checkpoint was finetuned on BLIP3o-60k. We added this set in and out at a matched budget of 28k samples, including continued fine-tuning of DeCo itself:
| Backbone | Fine-tuning data (28k) | GenEval | Counting | Position | GenEval (paraphrased) | DPG-Bench |
|---|---|---|---|---|---|---|
| DeCo | none (public checkpoint) | 79.1 | 61 | 70 | 75.5 | 81.7 |
| DeCo | Fine-T2I | 73.5 | 52 | 55 | 73.0 | 82.5 |
| DeCo | Fine-T2I + 11.8% compositional | 80.4 | 64 | 72 | 75.9 | 82.9 |
| LlamaGen-XL | Fine-T2I | 40.7 | 30 | 8 | 38.2 | 67.6 |
| LlamaGen-XL | Fine-T2I + 11.8% compositional | 49.4 | 40 | 15 | 46.6 | 67.7 |
| LlamaGen-XL | BLIP3o-60k | 48.1 | 39 | 12 | 46.3 | 65.8 |
| LlamaGen-XL | BLIP3o-60k w/o geneval_train | 39.8 | 28 | 7 | 39.5 | – |
- Cause. Without its
geneval_trainpart, BLIP3o-60k falls below Fine-T2I on GenEval (39.8 vs. 40.7). - Fix. Mixing these samples into Fine-T2I removes the decline on DeCo (80.4 vs. 79.1, position 72 vs. 70). We employed GPT-4.1 and GPT-5.5 to further evaluate, and we see the preference gains are real. GPT-4.1 prefers our counterpart over the public DeCo in visual quality (+17.2, p < 0.001, confirmed by GPT-5.5: +15.2) and in alignment (+8.8, p = 0.04). We also improve HPSv3, ImageReward, PickScore, Aesthetic and CLIPScore metrics.
- Not a general loss. Even without mixing
geneval_train, Fine-T2I improves DPG-Bench from 81.7 to 82.5, and the gains persist on our paraphrased GenEval.
We will remove the "benchmark-specific" word and add the new mixing experiments.
W4 & Q3: Matched comparisons, including GPT-4o-distilled datasets. Thanks for pointing out this and we will remove the presentation of training-loss. Here, we fine-tuned LlamaGen-XL on exactly 28k samples from each of seven datasets, with identical settings (580 steps, batch 192, lr 3e-5, 512², ≤120-token captions). We report Fine-T2I win rate (win-loss) evaluated by GPT-4.1 comapred with other datasets in the first two columns:
| Data (28k each) | We Win Alignment | We Win Visual quality | ImageReward | PickScore | Aesthetic |
|---|---|---|---|---|---|
| none (base) | +33.6** | +40.0** | 0.102 | 0.788 | 5.24 |
| BLIP3o-60k | −0.6 | +15.0** | 0.531 | 0.809 | 5.38 |
| T2I-2M | +17.0** | +7.8 | 0.464 | 0.808 | 5.40 |
| ShareGPT-4o-Image | +3.4 | +14.8** | 0.526 | 0.806 | 5.32 |
| OpenGPT-4o-Image | −2.2 | +5.4 | 0.572 | 0.809 | 5.28 |
| Echo-4o-Image | +25.8** | +16.4** | 0.456 | 0.801 | 5.33 |
| Fine-T2I | — | — | 0.612 | 0.812 | 5.44 |
Answer to Q3. According to above table, our Fine-T2I significantly better than all other datasets across different evaluations and benchmarks.
- It is significantly better in visual quality over BLIP3o-60k, ShareGPT-4o-Image and Echo-4o-Image, and in alignment over T2I-2M and Echo-4o-Image.
- It is compared with OpenGPT-4o-Image in pairwise judgment, while higher on DPG-Bench (67.6 vs. 67.4) and on all three reward models.
- It obtains the highest ImageReward, PickScore and Aesthetic scores of all seven datasets.
Rebuttal (part 3 / 3)
W5 & Q4: Latent flow-matching models. We also finetuned FLUX.1-dev with LoRA (rank 32, 750 steps, batch 16) on 22k Fine-T2I images at native aspect ratios. Both models are sampled with identical settings (1024², 28 steps, guidance 3.5, same seeds). The last two columns compare the fine-tuned model with the base model in alignment (win − loss):
| Model | GenEval | Counting | Position | DPG-Bench | TIIF (short / long) | LongText | ImageReward | GPT-4.1 | GPT-5.5 |
|---|---|---|---|---|---|---|---|---|---|
| FLUX.1-dev | 63.1 | 64 | 20 | 83.3 | 73.2 / 75.3 | 0.566 | 1.031 | — | — |
| + Fine-T2I | 67.9 | 74 | 27 | 84.5 | 75.7 / 79.3 | 0.653 | 1.107 | +19.8 | +24.8 |
Both judge results have p < 0.001.
Even on the strong model FLUX.1-dev, we can see Fine-T2I improves instruction following and text rendering abilities significantly.
- Non-square generation. A LoRA trained on square and random-aspect-ratio Fine-T2I images is preferred over the base model in alignment at 16:9 (+20.5, p = 0.003) and at 9:16 (+13.0).
- Half-strength LoRA. At scale 0.5, the LoRA keeps the base model's aesthetic score (6.15 vs. 6.12) while retaining most of the gains (GenEval 67.5, DPG-Bench 84.1).
- Not self-distillation. FLUX.1-dev is not one of the generators used to build Fine-T2I.
We will add all of these results to the revision. Thank you again for the insightful review and questions, which makes our work stronger. We hope our responses address your concerns and would be grateful if you would consider raising your score. We are happy to discuss any further questions.
Best,
The Authors
Rebuttal (part 1 / 3)
Dear Reviewer aWUf,
Thank you for your valuable questions and suggestions. Here is our rebuttal. To address your concerns and questions, we ran 15 new fine-tuning experiments on three different backbones (LlamaGen-XL, DeCo, FLUX.1-dev), generated and evaluated ~100k new images, added four benchmarks and five reward models, and directly validated our filtering strategy.
W1 & Q1: Validation of the filtering strategy. We will revise the reward-hacking paragraph accordingly. To validate the filtering strategy, we have compared filters on 5,530 images generated using the GenEval prompts by three models (LlamaGen-XL, DeCo, FLUX.1-dev). We labelled correct or incorrect by GenEval's official detector. Our verifier (Qwen3-VL-8B-Thinking with exactly the Fig. 11 system prompt) is compared with top-k selection by five scalar rewards, all keeping the same 59.0% of the images:
Clearly, our method with our system prompt significantly outperforms the strongest method HPSv3 by 5.0 points. Besides, these only care about alignment, but our verifier also rejects artifacts that they don't care.
W2 & Q2: Evaluation coverage, CIs and agreement. We added additional benchmarks, including DPG-Bench, TIIF-Bench testmini, LongText-Bench and paraphrased GenEval (the latter is reported under W3):
On TIIF-Bench, the biggest improvements are from long prompts and on the text dimension (+15.8 / +14.0 for short / long prompts). Fine-T2I improves LongText-Bench in all eight text categories.
CIs for the original human study (win − loss with 95% CI, visual quality / alignment): LlamaGen +73.5 [68.4, 78.6] / +52.9 [46.7, 59.1]; SD-XL +48.9 [42.4, 55.4] / +33.1 [26.3, 39.9]; DeCo +42.3 [36.1, 48.5] / +6.8 [0.5, 13.1]. All of them are significant.
Agreement for the new results: we additionally evaluated 4,500 pairs with an independent SOTA model GPT-5.5. For all comparisons, they agree on 74.1% for alignment and 68.7% for quality, showing great agreement.