- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
CDMD distills a flow teacher into a MeanFlow-style average-velocity student without data. Training states come from the student itself (re-noised one-step samples, or its own intermediate states, mixed with ratio ; Eq. 6, Alg. 1). The method is presented as distribution matching in which one network acts as both generator and fake score. Validation: ImageNet-256 with three teachers (1-NFE FID 3.47 / 2.93 / 2.05, 1.37 at 4 NFE), a 2D benchmark, and a 4-step SANA-1.6B student.
Critical strengths
- A simple pipeline. One frozen teacher, one student and one loss, with no fake-score network or discriminator, and 20 GB peak memory vs. 33/37 GB for DMD2/FreeFlow (Fig. 1), reaching 2.05 FID at 1 NFE without data. Simplicity and computation savings are the main practical value of the paper.
- A useful diagnostic. The 2D experiment shows that endpoint matching does not imply transport matching: equal endpoint , but much lower average-velocity error than FreeFlow (Table 6).
Critical weaknesses
- The method is not the distribution-matching objective it is presented as, and not new.
- Eq. (7) is just the Eulerian flow-map distillation loss of Flow Map Matching (arXiv 2406.07507) and AYF-EMD (arXiv 2506.14603), evaluated on student-generated states.
- TwinFlow (arXiv 2512.05150) and APEX (arXiv 2604.12322), which do share one network between generator and fake score, is not cited.
While it is totally possible to provide new understanding and perspective for existing loss (on-policy meanflow distillation), the authors failed to present the paper in this way and make it solid.
- The state-of-the-art claim does not hold against published numbers.
- FreeFlow (arXiv 2511.19428) reports 1-NFE FID 2.24 with the same SiT-XL/2 teacher after about 20 epochs (1.45 with REPA after 300 epochs). Table 1 lists FreeFlow at 4.13 / 3.72 / 4.28 without marking these as re-runs, and CDMD at 3.47 with SiT-XL/2.
- App. E (l. 1329–1331) also says the SiT-XL/2 settings and results "are left empty until stable training outcomes are obtained", yet Table 1 reports them, and the headline LightningDiT run alone uses the DMF architecture, post-hoc EMA and a guidance interval (Table 5).
Questions for authors
- Can you derive and explain in what sense along the teacher tangent estimates the velocity of (l. 256)?
- What's the main question of the paper? can we distill a diffusion or flow teacher into a few-step student without external data, teacher rollouts, or auxiliary fake score models? or Our method aligns the teacher and student marginal probability paths without data or simulation., the later one is already addressed in many previous works
Thank the authors for their detailed response. I still have some questions regarding the mathematical aspects, so I will maintain my current score. I appreciate the value of this work and hope the authors will consider incorporating the points discussed into a revised version for resubmission.