- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper distills the 8-step Z-Image-Turbo (6B) into a 2-NFE generator trained with (Eq. 6). Three design choices are proposed: teacher-generated images as the discriminator's real samples; separate 6B parameters for each of the two steps; and end-to-end gradients into the first step together with an explicit step-1 loss.
Validation: OneIG, GenEval, DPG-Bench and LongText-Bench against TwinFlow, a DMD2 reimplementation and the teacher run at 2 and 8 steps, component ablations.
Critical strengths
-
A practically useful 2-step model. Text rendering close to the 8-step teacher , clearly ahead of the public 2-step baselines (e.g., TwinFlow at 71.99 on LongText-EN).
-
The overall presentations is clear and easy to understand.
Critical weaknesses
- The three designs have direct, uncited precedents.
- LADD (Sauer et al., arXiv 2403.12015), the successor of ADD, trains adversarial distillation on teacher-generated data and already reports that synthetic data significantly outperforms real data (Sec. 4.2);
- Phased DMD (arXiv 2510.27684) combines phase-wise few-step DMD with a mixture of experts, compared at equal parameter budgets. It also reports that missing constraints on intermediate generator steps hurt stability, which is the "contrary to this intuition" finding of Sec. 4.4.1. These hurt the contribution of this paper.
- The proposed innovations may not transfer to other models.
- All experiments use one teacher, Z-Image-Turbo, which is already distilled and reward-tuned (l. 211–215). Yet there is no result on another base model, from a non-distilled teacher, or at 3–4 steps. Without this, the paper offers a strong checkpoint rather than a reusable recipe.
- For an even larger model (e.g., video diffusion model), end-to-end training can become infeasible again at 2 steps.
- With equivalent double the model size, it is unfair to compare with the other methods at two steps.
Questions for authors
- How are Baseline ① and the DMD2 reimplementation defined, what is the difference between them, only the real data?
- As a step-distilled model, how do you make sure that the z-image turbo can be used as a real score?
- With real images simulated from the teacher model, the GAN target distribution and the DMD target distribution become exactly the same. What would be the performance of GAN-only training?
- For GAN discriminator, is it in image space or latent space? Do you re-noise the samples, then send to the discriminator? With a properly chosen denoising range for the latent discriminator, the discriminator will not easily distinguish the true and the fake. Hence, the training will be more stable.
6 comments
Response to Reviewer MBa4 — Part 2/4
W2: Transferability beyond one distilled, reward-tuned teacher to other models, non-distilled teachers, and 3–4 steps
Thank you for raising this point. We agree that validation on a single model is insufficient to show that the paper offers a reusable recipe. To address this, we have added Appendix C in the rebuttal revision, which transfers the same method to Krea 2.
We distill the 8-step Krea 2 Turbo into a 2-step generator, using exactly the same training recipe and optimizer/loss settings as for Z-Image. We compare with two baselines, pure DMD and TwinFlow, and add a GAN-only ablation that removes the DMD loss. On LongText-Bench (Table 7, reproduced below; averaged over 4 seeds), our method achieves the highest scores among all 2-step models, and recovers most of the gap between directly sampling Krea 2 Turbo with 2 steps and the 8-step reference:
| Method | NFE | LongText-CN | LongText-EN |
|---|---|---|---|
| Krea 2 Turbo (8-step reference) | 8 | 56.13 | 96.08 |
| Krea 2 Turbo | 2 | 34.49 | 67.01 |
| DMD | 2 | 50.17 | 87.00 |
| TwinFlow | 2 | 46.49 | 83.40 |
| GAN-only (ablation) | 2 | 46.17 | 82.59 |
| Ours | 2 | 51.48 | 89.63 |
Qualitatively (Fig. 6), methods such as pure DMD and TwinFlow commonly exhibit systematic issues such as a rigid appearance, lack of detail, or color shifts, whereas our method shows a consistent advantage on both models. Notably, Z-Image-Turbo is trained with Decoupled DMD, whereas Krea 2 Turbo is trained with TDM. We believe this experiment demonstrates the transferability of our approach.
- Regarding non-distilled teachers and 3–4 steps: our setting by design starts from an existing few-step model and further compresses it to 2 steps for deployment. As stated in the first sentence of the abstract, 4–8-step distillation is already relatively mature; the difficulty lies in pushing further to 2 steps.
Q3: Performance of GAN-only training when GAN and DMD share the same target distribution
Thank you for the question. Alongside the Krea 2 experiments, we have added a GAN-only ablation that removes the DMD loss from our method (Appendix C). In our experience, GAN-only performs well at the non-semantic level in our setting: using the GAN alone effectively avoids the rigid appearance, lack of detail, or color shifts commonly seen with the pure DMD loss; however, it tends to produce samples with clearly compromised semantics, such as structural collapse and incorrect counts. Qualitative examples are shown in Fig. 6 (last row), and quantitative results on LongText-Bench are given in Table 7 (see the table in our response to W2 above): GAN-only scores 82.59 on LongText-EN, lower than the 89.63 of our full method.
Overall, we find that DMD still plays a clear role in keeping the basic structure stable.
Response to Reviewer MBa4 — Part 3/4
W3: Fairness of comparisons with other two-step methods when model size is doubled
Thank you for pointing this out. We acknowledge that step-decoupled parameterization doubles the parameter count. However:
- In the 2-step setting, parameter capacity is indeed a bottleneck, and relieving this bottleneck brings tangible gains.
- Inference does not become more expensive in multi-GPU serving. As described in our response to Reviewer qzvQ (W2), we measured a 2-GPU pipelined deployment on the same hardware with the same number of GPUs: per-request latency is the same as that of the shared-weight model (325 ms for both), peak per-GPU memory is the same (12.7 GiB for both), and the total time for 100 requests is only 1.5% higher. We also explicitly acknowledge that single-GPU deployment requires twice the memory.
- Compared with pretraining or the full model production pipeline, few-step distillation is cheap: in our practice, the 12B Krea 2 can be trained in about 24 hours on 16 H200 GPUs. As long as inference cost stays the same and generation quality is maximized, a moderate increase in training cost is acceptable in many practical scenarios.
Comparison at the same parameter count: Our advantage does not come solely from the doubled parameters. Table 1 ② uses shared weights (the same parameter count as the baselines) and adds only Distribution-Aligned Adversarial Learning, without decoupling or end-to-end training. It already reaches LongText-CN/EN of 87.16/86.03, above DMD2 (⑨, 85.30/80.99), TwinFlow (⑧, 78.65/71.99) and 2-step Z-Image-Turbo (⑩, 78.48/72.89), and its OneIG score (51.89) is also higher than all three. Decoupling and end-to-end training then bring further gains (④: 91.62/89.88).
Feasibility of end-to-end training for larger models: Let us analyze the overhead of our method in detail. A counterintuitive point is that, with a proper implementation (Appendix A), our training does not actually require more computation than non-end-to-end training. The key reason is that the gradient flowing from the second step back to the first step is backpropagated together with the contribution from the loss on the first step's own output; computationally, our cost is therefore the same as updating the first and second steps separately without end-to-end training. In other words, given sufficient memory, training throughput does not decrease. The extra cost comes from the parameters, gradients and optimizer states of the doubled parameter count (i.e., static memory overhead), and from the memory needed to keep the first step's activations during the second step's forward and backward passes. In practice, schemes such as FSDP can shard the former static overhead across GPUs, reducing it to 1/(number of GPUs) per GPU and usually making it very small, and FSDP's communication can usually be overlapped with computation. For the latter, first, (selective) activation checkpointing is essentially standard practice in training, so the actual footprint is already greatly reduced; we can further offload some activations, or use context parallelism to split the activations' tokens across devices, to relieve this pressure.
Response to Reviewer MBa4 — Part 1/4
We sincerely thank you for the time and effort you have devoted to reviewing our paper, and for your constructive comments.
W1: Direct, uncited precedents for the three designs in LADD and Phased DMD
Thank you for raising this point. We agree that these two works are closely related to ours, and not discussing them in the submission was an oversight on our part. We have added a discussion of both to the related work in the revision. Below we explain how they relate to and differ from our work:
Regarding Phased DMD
First, on the point that intermediate constraints help stability, we believe that we and Phased DMD are addressing entirely different questions.
Regarding parameter decoupling, Phased DMD does also propose decoupling parameters, so it is highly relevant to our work, and we have accordingly added a discussion of it in the rebuttal revision. Phased DMD itself recognizes the task interference and capacity bottleneck that models face under a limited step budget, and proposes parameter decoupling on that basis. We believe our contribution lies in successfully applying this insight where it is needed most: compared with general few-step generation (e.g., 4–8 steps), the inter-step task interference and capacity bottleneck truly reach their extreme at 2 steps, so the demand for capacity is most pressing; moreover, precisely because there are only two steps, we can fully decouple per step (an independent set of parameters for each step) while only doubling the parameter count. At 4–8 steps, the same approach would mean 4–8× the parameters.
Finally, we would like to note that the core of this paper is a complete and effective distillation pipeline for 2-step generation. We therefore believe that, beyond the academic novelty of each individual component—stable adversarial training on teacher-generated samples, parameter decoupling, and end-to-end gradient propagation—successfully incorporating them to achieve state-of-the-art 2-step distillation is a contribution in its own right.