acceptodds
Withdrawn from ICLR 2027

Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Few-step distillation has emerged as a critical component in the development of advanced visual generative foundation models, substantially reducing inference overhead while enabling real-time generation and cost-efficient deployment across a broad range of practical scenarios. However, prior work has predominantly focused on advancing training objectives, while comparatively overlooking the training recipe, which has become increasingly critical in the era of large-scale foundation models. In this work, we systematically revisit the training recipe under the well-established distribution matching distillation (DMD) framework for both text-to-image generation and image editing, focusing on three key dimensions: training data composition, teacher guidance within DMD, and task mixture. Our empirical analysis reveals several non-obvious and counterintuitive phenomena, ultimately motivating the development of Qwen-Image-Flash. These findings highlight that effective few-step distillation depends not only on carefully designed objectives, but also on a principled training recipe.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Few-Step Distillation, training recipe, Qwen-Image-Flash

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8BzvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies data composition, teacher guidance, and task mixture for few-step DMD using Qwen-Image-2.0. The resulting Qwen-Image-Flash supports both image generation and editing.

Critical strengths
  1. It addresses the practical challenge of preserving generation and editing capabilities under few-step distillation.
  2. It reports an interesting finding that text-centric training data can degrade even text-centric performance.
  3. It demonstrates competitive four-NFE performance on public generation and editing benchmarks.
Critical weaknesses
  1. This paper provides no theoretical analysis and evaluates only one model family at a fixed four-step budget. If there is no theoretical guarantee, I think the authors should include more base models and distillation settings to support broader claims about the training recipe.
  2. Landscape + Text-centric and Portrait + Text-centric are missing, making it difficult to distinguish text-centric data effects from category-mixing effects. Differences in prompt difficulty, dataset size, and repetition rates further limit this interpretation.
Questions for authors
  1. In sec 4.3, how do you choose multiple teachers? Does the proposed schedule outperform a fixed base/RL mixture under matched conditions and across training seeds?
Reviewer_iGzVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The manuscript proposes a unified, few-step model for both text-to-image generation and editing, called Qwen-Image-Flash. The manuscript points out that effective few-step distillation hinges on both the distillation objective and the training recipe. In addition, the manuscript introduces two benchmarks: T2I-Bench and Editing-Bench.

Critical strengths

Good performance in both qualitative results and quantitative metrics.

The experiments are very substantial.

Critical weaknesses

There appears to be no innovation in methodology, only extensive analysis and research into the training data.

Only GPT-related metrics—specifically Gemini 3.1 Pro and GPT-5.5—are reported, traditional image evaluation metrics, such as FID, GenEva, and DPG, are not included.

Questions for authors

The manuscript provides an extensive analysis of training data related to distillation but lacks any methodological innovation. To my knowledge, this reads more like an experimental report than an academic paper. I am not certain whether it is a good fit for ICLR.

Reviewer_oX6tOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates training recipes for few-step distillation, focusing on data composition, teacher guidance, and joint text-to-image generation and image editing. The authors identify several interesting empirical findings and develop Qwen-Image-Flash, which achieves competitive generation and editing performance with only four NFEs.

Critical strengths

The paper explores practical aspects of distillation that are often overlooked. The empirical findings are interesting and can be important for practitioners. The final model also demonstrates promising efficiency and performance for text-to-image and editing tasks.

Critical weaknesses

1. Limited generalizability. My main concern is that the paper reads more like a technical report documenting effective recipes for distilling a particular model rather than establishing generalizable insights into few-step distillation. All experiments are based on the Qwen-Image-2.0 family, DMD, and four-step generation.

The authors should validate their findings across more:

  • Model families: For example, include models such as FLUX and BAGEL.
  • Datasets: In Section 4.2, the authors only consider synthetic data generated by the teacher. It would be important to also distill the model using real data.
  • Distillation techniques: Compare against other techniques within the same distillation family, such as ADD. It would also be interesting to include trajectory-based distillation methods, such as consistency distillation, as I think the results could be different in that case.
  • Sampling budgets.

2. Limited evaluation. The recipe ablations rely primarily on LLM-based evaluators. Human evaluation is not included in the paper, despite its importance for assessing tasks such as text-to-image generation and image editing. Moreover, additional metrics and statistical uncertainty estimates would strengthen the conclusions.

3. Incomplete investigation of editing data proportions. The joint-distillation experiments consider only T2I ratios of 9:1, 7:3, and 5:5. Since editing performance generally improves as the editing proportion increases, why not investigate editing-heavy settings such as 3:7, 1:9, or even editing-only (0:10) distillation? Without these experiments, it is unclear whether 5:5 is actually optimal.


The paper provides useful empirical observations and an effective model-specific distillation recipe. However, the experimental evidence is insufficient to support the broader generalizability of its findings. More controlled ablations and validation across different models and settings are needed to establish the contribution beyond a technical report.

Questions for authors

see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.