- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes on-policy few-step distillation by matching student and teacher endpoints on student-generated rollouts, combined with adversarial and perceptual objectives. The results show competitive performance on image and video generation.
Critical strengths
The proposed method is simple and seems competitive in practice.
Critical weaknesses
My main concern is that the method offers limited methodological novelty and does not demonstrate a clear advantage over prior approaches.
- The on-policy idea for few-step distillation is well established. A particularly close precedent is Imagine Flash [1], whose backward distillation also trains the student using teacher-provided regression targets from student-visited states. This work is neither cited nor discussed, and no empirical comparison is provided.
AnyFlow [2] also uses flow-map backward simulation, though with a different supervision objective. DMD2 [3] similarly uses backward simulation with distributional supervision rather than paired regression but similarly uses adversarial objective.
Basically, OPTD appears to combine an established backward-simulation strategy with naive knowledge distillation. I believe that this contribution is largely incremental and needs strong arguments to support it.
-
Adversarial and perceptual objectives are also established in step distillation. CTM [4] already combines perceptual supervision with adversarial training, while PCM [5] and AYF [6] also incorporate adversarial objectives. There are also many other works that combine these objectives. Thus, the novelty of adding these objectives is limited.
-
The results are mixed and do not show a consistent advantage over existing methods. It is not entirely clear if the comparison setups are fully controlled and tuned. The image evaluation would benefit from newer preference metrics, e.g., HPSv3 and VisionReward, and human evaluation.
[1] Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillation, 2024.
[2] AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation, 2026.
[3] Improved Distribution Matching Distillation for Fast Image Synthesis, 2024.
[4] Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion, 2023.
[5] Phased Consistency Models, 2024.
[6] Align Your Flow: Scaling Continuous-Time Flow Map Distillation, 2025.
Questions for authors
I highly encourage the authors to justify why the proposed objective combination should be preferred over dozens of possible combinations of various flow map and distributional objectives.
Could the authors provide broader evaluation of most popular flow map and distributional objectives under the similar controlled setup?
Otherwise, it is difficult to see what OPTD contributes beyond combining well-established step-distillation techniques without clear understanding why this combination should be preferred.