acceptodds
Under review as a conference paper at ICLR 2027

Z-Image Ultra-Turbo: Toward High-Fidelity Two-Step Image Generation

Abstract

Few-step diffusion distillation has become increasingly mature for 4–8-step generation, yet pushing further to 2 steps remains challenging. In this work, we introduce Z-Image Ultra-Turbo, a high-quality 2-step image generation model distilled from the 8-step Z-Image Turbo teacher. Our method addresses the central bottlenecks of the increased task difficulty and limited model capacity in 2-step generation through three simple but effective design choices tailored to this regime. First, we propose Distribution-Aligned Adversarial Learning, which uses teacher-generated images rather than external real images as real samples for GAN training, providing a more attainable and informative adversarial target. Second, we adopt Step-Decoupled Parameterization, assigning independent model parameters to the two denoising steps to better match their distinct capacity demands. Third, we perform End-to-End Training with Iterative Regularization, allowing the first step to receive gradients from final image quality while preserving a meaningful intermediate generation through an explicit step-1 loss. Together, these designs substantially narrow the quality gap between 2-step and 8-step generation in both qualitative and quantitative evaluations, highlighting the potential of carefully tailored distillation strategies for improving the quality–efficiency trade-off in few-step generation.

Primary area:
generative models
Keywords:
diffusion model, few-step distillation, DMD

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_MBa4OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper distills the 8-step Z-Image-Turbo (6B) into a 2-NFE generator trained with LGAN+λLDMD\mathcal L_{\text{GAN}}+\lambda\mathcal L_{\text{DMD}} (Eq. 6). Three design choices are proposed: teacher-generated images as the discriminator's real samples; separate 6B parameters for each of the two steps; and end-to-end gradients into the first step together with an explicit step-1 loss.

Validation: OneIG, GenEval, DPG-Bench and LongText-Bench against TwinFlow, a DMD2 reimplementation and the teacher run at 2 and 8 steps, component ablations.

Critical strengths
  1. A practically useful 2-step model. Text rendering close to the 8-step teacher , clearly ahead of the public 2-step baselines (e.g., TwinFlow at 71.99 on LongText-EN).

  2. The overall presentations is clear and easy to understand.

Critical weaknesses
  1. The three designs have direct, uncited precedents.
  • LADD (Sauer et al., arXiv 2403.12015), the successor of ADD, trains adversarial distillation on teacher-generated data and already reports that synthetic data significantly outperforms real data (Sec. 4.2);
  • Phased DMD (arXiv 2510.27684) combines phase-wise few-step DMD with a mixture of experts, compared at equal parameter budgets. It also reports that missing constraints on intermediate generator steps hurt stability, which is the "contrary to this intuition" finding of Sec. 4.4.1. These hurt the contribution of this paper.
  1. The proposed innovations may not transfer to other models.
  • All experiments use one teacher, Z-Image-Turbo, which is already distilled and reward-tuned (l. 211–215). Yet there is no result on another base model, from a non-distilled teacher, or at 3–4 steps. Without this, the paper offers a strong checkpoint rather than a reusable recipe.
  • For an even larger model (e.g., video diffusion model), end-to-end training can become infeasible again at 2 steps.
  1. With equivalent double the model size, it is unfair to compare with the other methods at two steps.
Questions for authors
  1. How are Baseline ① and the DMD2 reimplementation defined, what is the difference between them, only the real data?
  2. As a step-distilled model, how do you make sure that the z-image turbo can be used as a real score?
  3. With real images simulated from the teacher model, the GAN target distribution and the DMD target distribution become exactly the same. What would be the performance of GAN-only training?
  4. For GAN discriminator, is it in image space or latent space? Do you re-noise the samples, then send to the discriminator? With a properly chosen denoising range for the latent discriminator, the discriminator will not easily distinguish the true and the fake. Hence, the training will be more stable.
6 comments
AuthorsOpenReview ↗

Response to Reviewer MBa4 — Part 1/4

We sincerely thank you for the time and effort you have devoted to reviewing our paper, and for your constructive comments.

W1: Direct, uncited precedents for the three designs in LADD and Phased DMD

Thank you for raising this point. We agree that these two works are closely related to ours, and not discussing them in the submission was an oversight on our part. We have added a discussion of both to the related work in the revision. Below we explain how they relate to and differ from our work:

  • LADD is indeed closely related to our work: through a direct comparison, it concludes that synthetic samples outperform real samples in adversarial distillation. We acknowledge that this conclusion closely mirrors what we observe in 2-step generation, and we now emphasize this in the revision. However, we believe that before our work it was unclear whether this conclusion extends to the 2-step generation task we focus on. First, in LADD the adversarial loss is the only training objective, and the paper explicitly states that adding a distillation loss on top of the adversarial loss did not help; in our experiments, by contrast, the DMD loss is indispensable for quality (the Krea 2 experiments added in this rebuttal revision provide corresponding qualitative and quantitative results). Yet DMD2, the most influential work combining the DMD loss with a GAN loss, uses real samples as the GAN target. These practical complexities lead us to believe that our experimental results remain an informative and non-trivial finding.

Regarding Phased DMD

First, on the point that intermediate constraints help stability, we believe that we and Phased DMD are addressing entirely different questions.

  • Phased DMD never trains end-to-end; intermediate supervision is its only training signal. The authors do argue in their introduction that the lack of explicit constraints on intermediate steps hurts training stability, but they do not verify this experimentally; instead, they use it as the basis for abandoning end-to-end training in favor of conventional phase-wise training.
  • Our question is: when the whole chain is already end-to-end differentiable and supervision on the final output can propagate directly through the entire process, are additional intermediate constraints still necessary? This question never arises in the actual setting of Phased DMD.

Regarding parameter decoupling, Phased DMD does also propose decoupling parameters, so it is highly relevant to our work, and we have accordingly added a discussion of it in the rebuttal revision. Phased DMD itself recognizes the task interference and capacity bottleneck that models face under a limited step budget, and proposes parameter decoupling on that basis. We believe our contribution lies in successfully applying this insight where it is needed most: compared with general few-step generation (e.g., 4–8 steps), the inter-step task interference and capacity bottleneck truly reach their extreme at 2 steps, so the demand for capacity is most pressing; moreover, precisely because there are only two steps, we can fully decouple per step (an independent set of parameters for each step) while only doubling the parameter count. At 4–8 steps, the same approach would mean 4–8× the parameters.

Finally, we would like to note that the core of this paper is a complete and effective distillation pipeline for 2-step generation. We therefore believe that, beyond the academic novelty of each individual component—stable adversarial training on teacher-generated samples, parameter decoupling, and end-to-end gradient propagation—successfully incorporating them to achieve state-of-the-art 2-step distillation is a contribution in its own right.

AuthorsOpenReview ↗

Response to Reviewer MBa4 — Part 2/4

W2: Transferability beyond one distilled, reward-tuned teacher to other models, non-distilled teachers, and 3–4 steps

Thank you for raising this point. We agree that validation on a single model is insufficient to show that the paper offers a reusable recipe. To address this, we have added Appendix C in the rebuttal revision, which transfers the same method to Krea 2.

We distill the 8-step Krea 2 Turbo into a 2-step generator, using exactly the same training recipe and optimizer/loss settings as for Z-Image. We compare with two baselines, pure DMD and TwinFlow, and add a GAN-only ablation that removes the DMD loss. On LongText-Bench (Table 7, reproduced below; averaged over 4 seeds), our method achieves the highest scores among all 2-step models, and recovers most of the gap between directly sampling Krea 2 Turbo with 2 steps and the 8-step reference:

MethodNFELongText-CNLongText-EN
Krea 2 Turbo (8-step reference)856.1396.08
Krea 2 Turbo234.4967.01
DMD250.1787.00
TwinFlow246.4983.40
GAN-only (ablation)246.1782.59
Ours251.4889.63

Qualitatively (Fig. 6), methods such as pure DMD and TwinFlow commonly exhibit systematic issues such as a rigid appearance, lack of detail, or color shifts, whereas our method shows a consistent advantage on both models. Notably, Z-Image-Turbo is trained with Decoupled DMD, whereas Krea 2 Turbo is trained with TDM. We believe this experiment demonstrates the transferability of our approach.

  • Regarding non-distilled teachers and 3–4 steps: our setting by design starts from an existing few-step model and further compresses it to 2 steps for deployment. As stated in the first sentence of the abstract, 4–8-step distillation is already relatively mature; the difficulty lies in pushing further to 2 steps.

Q3: Performance of GAN-only training when GAN and DMD share the same target distribution

Thank you for the question. Alongside the Krea 2 experiments, we have added a GAN-only ablation that removes the DMD loss from our method (Appendix C). In our experience, GAN-only performs well at the non-semantic level in our setting: using the GAN alone effectively avoids the rigid appearance, lack of detail, or color shifts commonly seen with the pure DMD loss; however, it tends to produce samples with clearly compromised semantics, such as structural collapse and incorrect counts. Qualitative examples are shown in Fig. 6 (last row), and quantitative results on LongText-Bench are given in Table 7 (see the table in our response to W2 above): GAN-only scores 82.59 on LongText-EN, lower than the 89.63 of our full method.

Overall, we find that DMD still plays a clear role in keeping the basic structure stable.

AuthorsOpenReview ↗

Response to Reviewer MBa4 — Part 3/4

W3: Fairness of comparisons with other two-step methods when model size is doubled

Thank you for pointing this out. We acknowledge that step-decoupled parameterization doubles the parameter count. However:

  1. In the 2-step setting, parameter capacity is indeed a bottleneck, and relieving this bottleneck brings tangible gains.
  2. Inference does not become more expensive in multi-GPU serving. As described in our response to Reviewer qzvQ (W2), we measured a 2-GPU pipelined deployment on the same hardware with the same number of GPUs: per-request latency is the same as that of the shared-weight model (325 ms for both), peak per-GPU memory is the same (12.7 GiB for both), and the total time for 100 requests is only 1.5% higher. We also explicitly acknowledge that single-GPU deployment requires twice the memory.
  3. Compared with pretraining or the full model production pipeline, few-step distillation is cheap: in our practice, the 12B Krea 2 can be trained in about 24 hours on 16 H200 GPUs. As long as inference cost stays the same and generation quality is maximized, a moderate increase in training cost is acceptable in many practical scenarios.

Comparison at the same parameter count: Our advantage does not come solely from the doubled parameters. Table 1 ② uses shared weights (the same parameter count as the baselines) and adds only Distribution-Aligned Adversarial Learning, without decoupling or end-to-end training. It already reaches LongText-CN/EN of 87.16/86.03, above DMD2 (⑨, 85.30/80.99), TwinFlow (⑧, 78.65/71.99) and 2-step Z-Image-Turbo (⑩, 78.48/72.89), and its OneIG score (51.89) is also higher than all three. Decoupling and end-to-end training then bring further gains (④: 91.62/89.88).

Feasibility of end-to-end training for larger models: Let us analyze the overhead of our method in detail. A counterintuitive point is that, with a proper implementation (Appendix A), our training does not actually require more computation than non-end-to-end training. The key reason is that the gradient flowing from the second step back to the first step is backpropagated together with the contribution from the loss on the first step's own output; computationally, our cost is therefore the same as updating the first and second steps separately without end-to-end training. In other words, given sufficient memory, training throughput does not decrease. The extra cost comes from the parameters, gradients and optimizer states of the doubled parameter count (i.e., static memory overhead), and from the memory needed to keep the first step's activations during the second step's forward and backward passes. In practice, schemes such as FSDP can shard the former static overhead across GPUs, reducing it to 1/(number of GPUs) per GPU and usually making it very small, and FSDP's communication can usually be overlapped with computation. For the latter, first, (selective) activation checkpointing is essentially standard practice in training, so the actual footprint is already greatly reduced; we can further offload some activations, or use context parallelism to split the activations' tokens across devices, to relieve this pressure.

Reviewer_PL7kOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors introduce a method to post-train a text-to-image diffusion model into a two-step image generator. This method is introduced as a second distillation stage following a first few-steps distillation stage. While the first stage uses DMD2 and real examples, the second takes advantage of the few-steps teacher to distill it using its own generation, avoiding the distribution gap between model output and training examples distribution that causes instabilities. The method uses DMD combined with an additional GAN loss. The authors show how the model can be trained end-to-end for two-step generation, taking advantage of the low depth, and additionally duplicating the model weights to specialize one copy for each step. This allows the model to directly optimize both iteration for the specific generation regime they are trained to achieve.

Critical strengths

The method introduces a clearly-explained second-stage distillation process fitting a spot where standard distillation methods still struggle. The method takes advantage of both the few-stage teacher and the two-stage target to optimize the generated distribution against the teacher, and achieves results close to the teacher and superior to other methods on controlled experiments on the Z-Image model.

Critical weaknesses

None

Questions for authors
  • Can the same method be used to distill the model into a single-step image generator, either by dropping a stage, or treating the single model as two stages by adding an intermediate decoding head?
  • Sample diversity is even lower than 2-stages DMD2. Could the first stage be used to introduce stochasticity and increase diversity?
2 comments
AuthorsOpenReview ↗

Response to Reviewer PL7k

We sincerely thank you for the time and effort you have devoted to reviewing our paper, and for your positive recognition of our work.

Q1: Can the method produce a one-step generator, by dropping a stage or treating a single model as two stages with an intermediate decoding head?

Thank you for the question. Our method can be applied to 1 step directly: the design of using teacher-generated samples as the GAN's real samples is independent of the number of steps, and one can simply drop the second step.

However, we believe it is difficult for 1-step generation to reach practical quality in the current setting. At 2 steps, model capacity is already a clear bottleneck; even with parameter decoupling, a gap remains between our 2-step model and the 8-step teacher, and 1 step would only face a more severe capacity limitation.

Your idea of treating a single model as two stages with an intermediate decoding head is thought-provoking. However, in light of our per-step LoRA experiment (Table 1, ⑥), such shared-backbone schemes may not resolve the capacity bottleneck of 2-step generation. We will explore this in future work.

Q2: Sample diversity is lower than 2-step DMD2; could the first step introduce stochasticity to increase diversity?

Thank you for the question. We acknowledge that the sample diversity of our model is lower than that of DMD2 (OneIG Diversity 11.75 vs. 12.75, Table 2), as stated in the Limitations. The ablation shows that the drop occurs mainly when end-to-end training is added (Table 2, ③ → ④: 13.05 → 11.75). We conjecture that end-to-end optimization makes the two steps fit the teacher distribution in tighter coordination, thereby amplifying the mode-seeking tendency inherent in the training itself. More broadly, we note that most of the diversity is already lost once the model is distilled to 4–8 steps; addressing this problem fundamentally calls for more dedicated research.

Using the first step to introduce stochasticity is a natural idea, and DP-DMD (Diversity-Preserved Distribution Matching Distillation, Wu et al., 2026) pursues exactly this direction. Thank you for the suggestion; we will explore it in future work.

Thank you for the detailed answer which clarify the design and the limitation of the method. I maintain clear acceptance rating.

Reviewer_qzvQOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents Z-Image Ultra-Turbo, a two-step generator distilled from the eight-step Z-Image-Turbo. The method combines teacher-generated images as adversarial positives, separate parameters for the two generation steps, and end-to-end optimization with an auxiliary objective on the first step’s clean-image prediction. The experiments evaluate generation quality and investigate how these training choices affect performance under a two-step sampling budget.

Critical strengths
  1. The method demonstrates substantial improvements in text rendering over the evaluated two-step alternatives. LongText-CN/EN scores reach 91.62/89.88, compared with 78.65/71.99 for TwinFlow and 85.30/80.99 for the DMD2 reimplementation. These results approach the eight-step teacher’s 92.56/91.74, while the overall OneIG score also remains close to the teacher, at 52.50 versus 52.84.
  2. The ablations provide useful evidence about the interaction between parameterization and optimization. With separate step models, end-to-end training improves LongText-EN from 85.81 to 89.88; with shared parameters, the corresponding score decreases from 86.03 to 81.14. Removing the first-step auxiliary loss also reduces performance across all benchmarks in Table 1. These findings support the combined training design, although the proposed explanations concerning capacity and knowledge preservation remain interpretations of the evidence.
  3. The evaluation complements semantic alignment metrics with text-sensitive benchmarks, qualitative comparisons, and human evaluation. The direct two-step inference baseline illustrates that reasonable GenEval and DPG-Bench scores can coexist with visibly degraded outputs. The reported blind study, with preference rates of 82% over TwinFlow and 74% over DMD2, provides additional support for the practical quality improvements.
Critical weaknesses
  1. The novelty of Distribution-Aligned Adversarial Learning needs clearer qualification. LADD [1] already uses teacher-generated samples for adversarial distillation and compares real versus synthetic training data in Section 4.2. NitroFusion [2] likewise uses outputs from four-step DMD2 or eight-step Hyper-SDXL teachers as adversarial positives. The additional two-step stability finding should be distinguished from these precedents. Differences in image–text alignment also warrant clarification when attributing the observed advantage to distributional attainability.
  2. Practical efficiency is characterized mainly through NFE. Two evaluations of a 6B model support reduced denoising computation, but independent step models double parameter storage. Although this trade-off is acknowledged, the main paper lacks latency, peak memory, and throughput measurements to substantiate the deployment claims, particularly the claim that pipelining introduces little additional overhead. Matched hardware and device counts are needed for a meaningful comparison.
Questions for authors

Refer to weakness.

4 comments
AuthorsOpenReview ↗

Response to Reviewer qzvQ — Part 1/2

We sincerely thank you for the time and effort you have devoted to reviewing our paper, and for your constructive comments.

W1: Novelty of Distribution-Aligned Adversarial Learning relative to LADD and NitroFusion, and the role of image–text alignment

Thank you for raising this point. We agree that these two works are closely related to ours, and not discussing them in the submission was an oversight on our part. We have added a discussion of both to the related work in the revision. Below we explain how they relate to and differ from our work:

  • NitroFusion makes the simple design "choice" of using synthetic samples as the GAN's real samples, without comparing this against using real samples. In this respect it is similar to Adv-KD, which our paper already discusses.
  • LADD is more closely related: through a direct comparison, it concludes that synthetic samples outperform real samples in adversarial distillation. We acknowledge that this conclusion closely mirrors what we observe in 2-step generation, and we now emphasize this in the revision. However, we believe that before our work it was unclear whether this conclusion extends to the 2-step generation task we focus on. First, in LADD the adversarial loss is the only training objective, and the paper explicitly states that adding a distillation loss on top of the adversarial loss did not help; in our experiments, by contrast, the DMD loss is indispensable for quality (the Krea 2 experiments added in this rebuttal revision provide corresponding qualitative and quantitative results). Yet DMD2, the most influential work combining the DMD loss with a GAN loss, uses real samples as the GAN target. These practical complexities lead us to believe that our experimental results remain an informative and non-trivial finding.
  • Both works motivate the use of synthetic samples by text–image alignment. However, in all experiments we present, the discriminator is text-unconditional by default. As mentioned in our response to Reviewer MBa4, during early exploration we made extensive attempts to stabilize real-sample GAN training by weakening the discriminator, and making it unconditional was one of them. In this context, an explanation based on text–image alignment cannot account for the stability difference we still encountered. We favor the explanation given in our paper, based on the distributional discrepancy.

Finally, we would like to note that the core of this paper is a complete and effective distillation pipeline for 2-step generation. We therefore believe that achieving state-of-the-art 2-step distillation through stable adversarial training on teacher-generated samples is a contribution in its own right.

AuthorsOpenReview ↗

Response to Reviewer qzvQ — Part 2/2

W2: Efficiency is characterized mainly by NFE; matched-hardware latency, peak memory and throughput are needed to support the deployment claims

Thank you for pointing this out. We have additionally measured latency, peak memory and throughput on matched hardware.

Setup: Z-Image (6B), 1024×1024, 2× H200 (NVLink), batch size 1 per request. Both deployments use the same number of GPUs:

  • A: a standard 2-step model (both steps share weights); each GPU holds a full copy of the model, and requests are processed with data parallelism;
  • B (Ours): the two per-step models are placed on the two GPUs and requests are processed as a pipeline, i.e., GPU 1 runs step 1 and GPU 2 runs step 2.

Results:

A: shared-weight 2-step model (data parallel)B: Ours (2-GPU pipeline)
Per-request latency325 ms325 ms
Peak memory per GPU12.7 GiB12.7 GiB
Total time (N=32 requests)5.27 s5.47 s (+3.7%)
Total time (N=100 requests)16.50 s16.74 s (+1.5%)

Why latency is unchanged: The only extra operation in B compared with A is transferring the intermediate result (latent and text conditioning, 2.25 MiB in total) from GPU 1 to GPU 2 after step 1. One transfer takes about 0.05 ms, which is negligible relative to the 325 ms of two-step denoising, so the per-request latency is the same for both.

Why total time is slightly higher: Each forward pass takes about 162.5 ms. For N requests (N even), each GPU in A handles N/2 requests and performs N forward passes; both GPUs start and finish together, so the total time is about that of N forward passes. In B, GPU 1 runs step 1 for the N requests back to back; GPU 2 can only start after step 1 of the first request has finished, and must run one more step 2 after GPU 1 has finished the last request, so the total time is about that of N+1 forward passes. This extra forward pass is the pipeline fill-and-drain overhead. It is fixed, so the relative overhead is about 1/N: in theory 3.1% at N=32 and 1.0% at N=100, consistent with the measured 3.7% and 1.5%, and the gap shrinks as the number of requests grows. The transfer is issued asynchronously by GPU 1 and overlaps with the computation of the next request, so transfers do not accumulate additional overhead.

Therefore, in serving deployments with 2 or more GPUs, our per-request latency and per-GPU memory are the same as those of the shared-weight model, and throughput differs only by a fixed overhead that is amortized over the number of requests. This supports the claim in our paper that pipelining introduces almost no additional overhead.

We also explicitly acknowledge that this conclusion is conditional: in single-GPU deployment, the two sets of weights require 24.9 GiB of memory, about 2× that of the shared-weight model, and total storage is always 2×. This is consistent with the Limitations section of our paper.

Reviewer_qzvQOpenReview ↗

Response to Authors’ Rebuttal

Thank you for the detailed response and the additional matched-hardware measurements. The efficiency analysis addresses my concern for multi-GPU deployment, and I appreciate the added discussion of LADD and NitroFusion. I will keep my positive assessment.

One follow-up question on the real-vs-teacher comparison. Were the teacher-generated images produced from the same captions as the real images used in the comparison? If the two sets come from different prompt sources, their content and semantic distributions may differ. Even with a text-unconditional discriminator, this difference could contribute to the observed stability gap, independently of distributional attainability. Could you also clarify which high-quality real-image dataset was used, and its source? If this information is not already in the paper, I would suggest adding it, since it is important for interpreting this ablation.

Thank you for the follow-up question and for keeping your positive assessment.

Prompt sources: Our setup corresponds to the first case you describe. We start from a high-quality real-image dataset, and in actual training the prompts are drawn from this same dataset, regardless of whether the GAN's real samples are teacher-generated images or real images. The two settings therefore share the same prompt set, and the comparison is not confounded by different prompt sources or semantic distributions. We have explicitly stated this in the revision (Sec. 4.2 and Appendix B.1).

Dataset: Our main experiments use an internal, curated high-quality real-image dataset. During development, we have experimented with a variety of real-image datasets, including datasets that contain synthetic images generated by third-party models (i.e., not by our teacher). In all of these experiments, we followed the same protocol: the teacher first generates samples from the prompts of the selected dataset, so that the teacher-generated images and the dataset images always share the same prompt set. The conclusion was consistent across datasets: training with teacher-generated samples is more stable than training with real images, and also more stable than training with third-party synthetic images. We therefore believe this phenomenon is highly robust, and that it is not merely a matter of real versus synthetic data: even among synthetic images, samples from the teacher itself (same source) and samples from other models (different sources) behave differently.

Thank you again for your continued positive assessment of our work.

Comments on the paper

1 comment

Hi Reviewer qzvQ and MBa4,

The authors have posted their rebuttal along with new results. Please review their responses, engage with the authors to discuss any remaining concerns.

Thank you reviewer PL7k for engaging in the discussion.

Best, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.