- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes Fixed-Point Distillation (FPD) for one-step discrete diffusion image generation. The student’s draft is partially re-masked and refined by a frozen teacher, providing targets for a drift loss in continuous feature space. A straight-through estimator enables hard-token decoding during training, with an optional adversarial objective. Experiments on MaskGIT, MaskGen, and Lumina-DiMOO demonstrate competitive one-step generation without training an auxiliary score network.
Critical strengths
1.Using teacher corrections of the student’s own drafts avoids precomputed multi-step trajectories and an auxiliary score model. Combined with feature-space optimization and hard-token gradient routing, this provides a compact approach to one-step discrete generation.
2.The method works across class-conditional and text-to-image generation, and Table 4 supports the contributions of student-conditioned targets, feature-space losses, and the straight-through estimator. Notably, the GAN-free MaskGen variant already reaches a GenEval score of 0.43, supporting the effectiveness of the core distillation objective.
Critical weaknesses
1.Eq. (3) requires distributional invariance, whereas Eqs. (6)–(7) attract each draft toward its own teacher correction and repel it from samples of other prompts. It is unclear whether the equilibrium of this objective corresponds to the stated fixed point. The decreasing paired DINO distance in Fig. 4 provides useful evidence but does not establish distributional invariance or preservation of diversity. This leaves the central interpretation of the method insufficiently supported.
2.The quality–training-cost advantage is not fully established. Soft-DiMO is discussed but omitted from quantitative comparisons, despite reporting a one-step FID of 6.40 with the same MaskGIT teacher, compared with 6.75 here. Meanwhile, the main MaskGIT/MaskGen results include GAN training, whereas Table 3 evaluates GAN-free Lumina-DiMOO distillation and compares against analytically estimated DiMO FLOPs because DiMO runs out of memory. These comparisons do not establish lower total training cost at comparable quality, which is important for assessing the practical contribution.
Questions for authors
please see weakness.
3 comments
I thank the authors for their additional experiments addressing my concerns. The comparisons of KID across different training iterations support the refinement consistency in Eq. 3. The training cost analysis and the comparison with Soft-DiMO also resolve my concerns regarding this aspect of the paper. With these clarifications and supplementary results, I have no further concerns. I believe including these discussed experiments in the final revision would strengthen the paper, so I am raising my score from 2 to 3.
Thank you for your constructive comments and positive feedback. We will include the discussed experiments in the final revision.
Thank you for recognizing our strengths: 1. compact approach; 2. experiments supporting core distillation objective. We provide more clarifications below:
Q1: Distributional invariance of Eq. 3
A1: Fig. 4b reports sample-level DINO distances. We additionally measure distribution-level KID between 10,000 student outputs and their one-step teacher refinements at each checkpoint.
KID decreases by 82.3%, providing empirical support for the refinement consistency in Eq. 3. The full curve is provided in Fig.8.
Q2: Training cost issue
A2: We apply aggressive memory optimization to accommodate DiMO training, using DeepSpeed ZeRO-3 and bf16 on 8 H100 GPUs. We compare both methods on Lumina-DiMOO under this setup.
In this setting, FPD reduces step time by 42.9%, peak VRAM by 77.1%, and FLOPs by 56.0%.
Q3: Comparison with Soft-DiMO
A3: Soft-DiMO builds on DiMO with additional reward-model post-training for T2I. Typically, such post-training can further improve generation performance. We apply the same reward models, CLIP and ImageReward, to FPD, yielding FPD+.
Reward-model post-training was outside our original scope. We have added Soft-DiMO results and discussion to the revision.