acceptodds
Under review as a conference paper at ICLR 2027

One-Step Distillation of Discrete Diffusion Image Generators via Fixed-Point Iteration

Abstract

Discrete diffusion models excel at visual synthesis but rely on slow, iterative decoding. Existing single-step distillation methods attempt to bypass this bottleneck, either by training auxiliary score networks that effectively double compute, or by introducing specialized parameterizations and multi-stage pipelines that fragment optimization. In this paper, we introduce Fixed-Point Distillation (FPD), an end-to-end framework that constructs local correction targets by partially corrupting the student's one-step draft and refining it with a single teacher step. To compute the training objective in a semantically meaningful space, we lift discrete tokens into continuous features and apply a multi-bandwidth drift loss that pulls the student toward these corrections. To backpropagate through the discrete bottleneck, we employ a straight-through estimator that decodes the exact hard-sampled tokens in the forward pass, ensuring that training and inference operate on the same codebook manifold, while routing continuous gradients back to the student logits. This fully differentiable pathway additionally accommodates an optional unconditional adversarial objective to enhance perceptual realism. Evaluations on both class-conditional and text-to-image generation validate the effectiveness of our framework. FPD achieves competitive visual fidelity and structural alignment against existing discrete distillation methods within only a single inference step.

Primary area:
generative models
Keywords:
discrete diffusion model, distillation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_PrgCOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes Fixed-Point Distillation (FPD) for one-step discrete diffusion image generation. The student’s draft is partially re-masked and refined by a frozen teacher, providing targets for a drift loss in continuous feature space. A straight-through estimator enables hard-token decoding during training, with an optional adversarial objective. Experiments on MaskGIT, MaskGen, and Lumina-DiMOO demonstrate competitive one-step generation without training an auxiliary score network.

Critical strengths

1.Using teacher corrections of the student’s own drafts avoids precomputed multi-step trajectories and an auxiliary score model. Combined with feature-space optimization and hard-token gradient routing, this provides a compact approach to one-step discrete generation.

2.The method works across class-conditional and text-to-image generation, and Table 4 supports the contributions of student-conditioned targets, feature-space losses, and the straight-through estimator. Notably, the GAN-free MaskGen variant already reaches a GenEval score of 0.43, supporting the effectiveness of the core distillation objective.

Critical weaknesses

1.Eq. (3) requires distributional invariance, whereas Eqs. (6)–(7) attract each draft toward its own teacher correction and repel it from samples of other prompts. It is unclear whether the equilibrium of this objective corresponds to the stated fixed point. The decreasing paired DINO distance in Fig. 4 provides useful evidence but does not establish distributional invariance or preservation of diversity. This leaves the central interpretation of the method insufficiently supported.

2.The quality–training-cost advantage is not fully established. Soft-DiMO is discussed but omitted from quantitative comparisons, despite reporting a one-step FID of 6.40 with the same MaskGIT teacher, compared with 6.75 here. Meanwhile, the main MaskGIT/MaskGen results include GAN training, whereas Table 3 evaluates GAN-free Lumina-DiMOO distillation and compares against analytically estimated DiMO FLOPs because DiMO runs out of memory. These comparisons do not establish lower total training cost at comparable quality, which is important for assessing the practical contribution.

Questions for authors

please see weakness.

3 comments

Thank you for recognizing our strengths: 1. compact approach; 2. experiments supporting core distillation objective. We provide more clarifications below:

Q1: Distributional invariance of Eq. 3

A1: Fig. 4b reports sample-level DINO distances. We additionally measure distribution-level KID between 10,000 student outputs and their one-step teacher refinements at each checkpoint.

Training iterations1k10k
KID ↓4.46×10−34.46\times10^{-3}0.79×10−30.79\times10^{-3}

KID decreases by 82.3%, providing empirical support for the refinement consistency in Eq. 3. The full curve is provided in Fig.8.

Q2: Training cost issue

A2: We apply aggressive memory optimization to accommodate DiMO training, using DeepSpeed ZeRO-3 and bf16 on 8 H100 GPUs. We compare both methods on Lumina-DiMOO under this setup.

MethodStep time (s) ↓Peak VRAM (GiB) ↓FLOPs/GPU/step (T) ↓
DiMO3.6138.450.9
FPD w/o GAN2.068.822.4

In this setting, FPD reduces step time by 42.9%, peak VRAM by 77.1%, and FLOPs by 56.0%.

Q3: Comparison with Soft-DiMO

A3: Soft-DiMO builds on DiMO with additional reward-model post-training for T2I. Typically, such post-training can further improve generation performance. We apply the same reward models, CLIP and ImageReward, to FPD, yielding FPD+.

MethodReward-model post-trainingGenEval ↑
DiMO✗0.42
Soft-DiMO✓0.51
FPD✗0.45
FPD+✓0.52

Reward-model post-training was outside our original scope. We have added Soft-DiMO results and discussion to the revision.

I thank the authors for their additional experiments addressing my concerns. The comparisons of KID across different training iterations support the refinement consistency in Eq. 3. The training cost analysis and the comparison with Soft-DiMO also resolve my concerns regarding this aspect of the paper. With these clarifications and supplementary results, I have no further concerns. I believe including these discussed experiments in the final revision would strengthen the paper, so I am raising my score from 2 to 3.

Thank you for your constructive comments and positive feedback. We will include the discussed experiments in the final revision.

Reviewer_bM85OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a method, FPD (Fixed-Point Distillation), for distilling a teacher diffusion model into a student. It does so by taking a student output, masking and re-generating a portion with a teacher, and pulling the student towards the teacher results and away from other random samples, through a pretrained DINO representation. A GAN discriminator is added as well. The method tests on ImageNet-256.

Critical strengths

The method is a simple recipe that does not require a teacher-sized auxiliary network, which enables it to scale to a model Lumina-DiMOO, showing that larger models can be practically trained within a training cost and memory budget. The method is evaluated across both ImageNet and Text-to-Image.

Critical weaknesses

The presentation of the paper was difficult to understand.

  • In particular, the primary explanatory figure is rather vague and high level and only shows labeled boxes, omitting any illustration of what the actual blocks do. The method would be far easier to grasp if the diagram actually showed the objects involved -- showing the token grid before and after remasking, example teacher and student images, and actually illustrating the "backbone", "inbatch rolling", "drift loss" would be much clearer.
  • In addition, the GAN was omitted from the diagram, though it is added in the method.
  • The method closely follows the distillation loss from ADD (Sauer, 2024) of corrupting the student's output, using the teacher to clean the output, and using this as a target, but does not mention this connection.

A second primary concern is the empirical results, which appear small and confounded. Notably, the results use a discriminator, whereas the main baseline does not use a GAN. Though the discriminator is not emphasized (e.g., not even shown in the main figure diagram) and the paper's contribution is about the FPD distillation technique, the FPD distillation technique is not tested separately in the results without the GAN in the ImageNet comparison. Adding a GAN is known to improve FID, potentially without actual perceptual improvement (Kynkaanniemi et al., ICLR 2023). Given this context, the results are relatively minor. In Table 2, the FPD method has a slightly lower FID than DiMo (6.75 vs 6.91), while having worse IS, Precision, and Recall.

There is also a missing comparison in Table 2 to Soft-DIMO, which was cited in the paper and published in ICLR 2026. Looking in that paper, the results outperform this submission across FID, IS, Precision, and Recall when using the same MaskGIT teacher (Table 2 in the Soft-DIMO paper).

Questions for authors

What is the novelty with respect to ADD, and please clarify the missing comparison to Soft-DIMO.

What are the results using FPD without the GAN?

Were other remasking strategies and ratios tried, and what effect did they have on the results? I see in Figure 4 that remasking at different ratios for a single training run, but not that different types of remasking strategies were tried at training time.

Is there a quantitative ablation of the repulsion term, in terms of the number of negatives?

5 comments

Thank you for recognizing our strength: scalability to larger models. We provide more clarifications below.

Q1: Fig. 2 with more detailed explanation

A1: We updated the manuscript. The revised Fig. 2 shows token grids, decoding outputs, and the DINO backbone. The drift-loss inputs are explicitly labeled, with batch rolling and their roles explained in the caption. The optional GAN loss is now included.

Q2: ADD issue

A2: Our FPD differs fundamentally from ADD in three main aspects:

MethodAdversarial-free distillationUnpaired trainingDiscrete support
ADD✗✗✗
FPD✓✓✓
  1. Distillation alone. FPD works without GAN, as demonstrated in Fig. 3 and Tab. 4a. In contrast, ADD’s own ablation reports failure with distillation alone, as shown below.
Loss in ADDFID ↓CS ↑
Distillation only315.60.076
Adversarial + distillation20.60.319
  1. Training data. ADD constructs student inputs from corrupted real images paired with text prompts. FPD learns from its own generated drafts using prompts only. Its optional unconditional GAN can use unpaired real images.

  2. Discrete support. ADD’s original formulation does not provide a mechanism for passing gradients through discrete sampling, while our FPD does.

Q3: Comparison with Soft-DiMO

A3: Soft-DiMO builds on DiMO with additional reward-model post-training for T2I. Typically, such post-training can further improve generation performance. We apply the same reward models, CLIP and ImageReward, to FPD, yielding FPD+.

MethodReward-model post-trainingGenEval ↑
DiMO✗0.42
Soft-DiMO✓0.51
FPD✗0.45
FPD+✓0.52

Reward-model post-training was outside our original scope. We have added Soft-DiMO results and discussion to the revision.

Q4: FPD without GAN

A4: Fig. 3 and Tab. 4a provide qualitative and quantitative comparisons with and without GAN, respectively. FPD works without GAN. Adding GAN further enhances visual details.

Q5: Ablations on remasking strategies and ratios

A5: We ablate the remasking rule r(u)r(u) specified in Eq. 12 and fixed remasking ratios as below. FPD works with different remasking choices.

r(u)r(u)cos⁡(πu/2)\cos(\pi u/2)uuu2u^2arccos⁡(u)/(π/2)\arccos(u)/(\pi/2)
GenEval0.390.420.410.43
Fixed ratio0.250.500.750.90
GenEval0.310.400.430.38

Q6: Ablation on the number of negatives

A6: We vary the number of negatives on MaskGen-L. Two negative samples achieve competitive result.

Negatives124
GenEval0.400.430.42
AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer,

As the discussion period draws to a close, we would greatly appreciate any further feedback on our responses. Please let us know if you have any additional questions, and we would be happy to address them.

Thank you again for your time and consideration.

AuthorsOpenReview ↗

Follow-up Before the Discussion Closes

Dear Reviewer,

Thank you again for your careful review and constructive comments. With less than 12 hours left in the discussion period, we would like to ask whether you have any remaining questions or concerns we could address.

Reviewer_LkTEOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

FPD distills masked discrete diffusion models into one-step generators. The student’s one-step draft is partially re-masked and refined by one teacher step, giving a local correction target. The student is trained toward it with a drift loss in DINOv3 feature space, using a straight-through estimator to pass gradients through token sampling, plus an optional GAN loss. FPD reaches a one-step FID of 6.75 on ImageNet and a GenEval score of 0.45 on MaskGen-L, without an auxiliary network.

Critical strengths
  • Simple and efficient. No auxiliary network, architectural changes, or multi-stage training. It scales to Lumina-DiMOO, where DiMO runs out of memory (Tab. 3).
  • Strong results with clear ablations. Matches or beats DiMO (ImageNet FID 6.75 vs. 6.91; GenEval 0.45 vs. 0.42), and Tab. 4 / Fig. 5 show each component matters.
Critical weaknesses
  • Incomplete comparisons. Soft-DiMO is missing from Tables 1–2, the efficiency claim relies on an analytic FLOP estimate for DiMO, and text-to-image is evaluated only with GenEval at 256×256.
Questions for authors
  • How does FPD compare with Soft-DiMO?
  • Can you report wall-clock time and memory versus DiMO on a model where both fit?
  • How does FPD relate to ADD’s distillation loss (Sauer et al., 2023), which also uses teacher-denoised corruptions of the student’s output as targets?
4 comments

Thank you for recognizing our strengths: 1. simplicity and efficiency; 2. scalability to larger models; 3. strong results with clear ablations. We provide more clarifications below.

Q1: Comparison with Soft-DiMO

A1: Soft-DiMO builds on DiMO with additional reward-model post-training for T2I. Typically, such post-training can further improve generation performance. We apply the same reward models, CLIP and ImageReward, to FPD, yielding FPD+.

MethodReward-model post-trainingGenEval ↑
DiMO✗0.42
Soft-DiMO✓0.51
FPD✗0.45
FPD+✓0.52

Reward-model post-training was outside our original scope. We have added Soft-DiMO results and discussion to the revision.

Q2: Training cost comparison

A2: We apply aggressive memory optimization to accommodate DiMO training, using DeepSpeed ZeRO-3 and bf16 on 8 H100 GPUs. We compare both methods on Lumina-DiMOO under this setup.

MethodStep time (s) ↓Peak VRAM (GiB) ↓FLOPs/GPU/step (T) ↓
DiMO3.6138.450.9
FPD w/o GAN2.068.822.4

In this setting, FPD reduces step time by 42.9%, peak VRAM by 77.1%, and FLOPs by 56.0%.

Q3: Relation to ADD

A3: Our FPD differs fundamentally from ADD in three main aspects:

MethodAdversarial-free distillationUnpaired trainingDiscrete support
ADD✗✗✗
FPD✓✓✓
  1. Distillation alone. FPD works without GAN, as demonstrated in Fig. 3 and Tab. 4a. In contrast, ADD’s own ablation reports failure with distillation alone, as shown below.
Loss in ADDFID ↓CS ↑
Distillation only315.60.076
Adversarial + distillation20.60.319
  1. Training data. ADD constructs student inputs from corrupted real images paired with text prompts. FPD learns from its own generated drafts using prompts only. Its optional unconditional GAN can use unpaired real images.

  2. Discrete support. ADD’s original formulation does not provide a mechanism for passing gradients through discrete sampling, while our FPD does.

Q4: Additional benchmark results

A4: We additionally evaluate FPD and DiMO on COCO val2017 using the same MaskGen-L teacher and evaluation protocol. FPD achieves both lower FID and higher CLIP score.

MethodFID ↓CS ↑
DiMO35.20.277
FPD30.10.282
AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer,

As the discussion period draws to a close, we would greatly appreciate any further feedback on our responses. Please let us know if you have any additional questions, and we would be happy to address them.

Thank you again for your time and consideration.

Thank author for their feedback. To be honest, I'm not really experienced on the literature of this discrete iterative model so I really dont have any else feedback or concern aside the raised one in my initial review. But the results of the model seem not too strong to guarantee a score of 4 so I retain my score of 3

Thank you for your feedback and for the positive rating. We would be happy to provide any further clarification if needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.