- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes sDMD, a diffusion distillation method that augments DMD with Sinkhorn transport guidance. It interprets DMD as fixed-point regression under the Wasserstein gradient flow of an integral reverse-KL objective, and motivates transport regularization through two limitations: unreliable learned teacher scores in low-density regions and weak guidance toward underrepresented modes. The proposed method combines the original score-based update with a Sinkhorn velocity estimated from batches of student and target samples. To limit interference from finite-sample transport errors, it reuses the DMD2 discriminator to selectively activate this correction for samples with relatively weak target support. The paper provides convergence results for idealized, ungated flows under regularity and approximation-error assumptions. Experiments on class-conditional image, text-to-image, and text-to-video generation report improved sample diversity and generally competitive or improved quality relative to distillation baselines.
Critical strengths
-
Theory and motivation. The paper identifies relevant limitations of DMD and presents a coherent rationale for combining score-based guidance with transport-based correction. Its connection between DMD, integral KL, and Wasserstein gradient flow provides a useful interpretation of the update fields. The theoretical analysis also distinguishes exact OT from its Sinkhorn approximation and explicitly states the assumptions required for the convergence results.
-
Empirical evaluation. The evaluation spans class-conditional image generation, text-to-image generation, and text-to-video generation, including large models such as Qwen-Image and Wan2.1. The consistent improvements across the reported diversity metrics are encouraging. Ablations on the energy functional, transport space, gating mechanism, and batch size provide useful information about the method’s practical behavior.
Critical weaknesses
-
Practical overhead. The reviewer is concerned about the additional requirements of generating multiple student and target samples per condition and computing both cross-distribution and self-transport. Although teacher samples can be cached, the end-to-end costs of sample generation, storage, and batching remain insufficiently characterized for large image and video models.
-
Recovery from student mode collapse. The reviewer is unsure whether Sinkhorn updates can restore multiple modes when reverse-KL-based training has already concentrated outputs from different seeds around the same mode, rather than merely move these samples together. Clarifying the student sampler’s stochasticity would help assess this concern. As a possible extension, could controlled SDE-based exploration, inspired by flow-model RL methods such as Flow-GRPO, complement the transport correction?
-
Limited evidence for gating. The reviewer finds the gains over ungated transport in Table 4 modest, with their robustness unclear in the absence of uncertainty estimates. The current comparisons also do not distinguish effective sample selection from simply reducing the strength or frequency of transport updates.
-
The role of the WGF analysis in motivating the method. The reviewer recognizes that the WGF formulation provides a unified interpretation of the update fields, but finds it unclear how this perspective directly motivates the choice of Sinkhorn divergence or provides additional guidance for the method’s design. The motivation for adding transport appears to follow already from the mode-seeking behavior and unreliable local guidance of IKL-based matching. The constructive insight added by the WGF derivation could therefore be articulated more clearly.
Questions for authors
Please refer to the concerns raised in the Weaknesses section, with particular attention to points 3 and 4 regarding the gating mechanism and the role of the WGF analysis. The reviewer would be strongly inclined to raise the score if these concerns are convincingly addressed through clear explanations or supporting evidence.
after rebuttal
Thanks for response and the added experiments.
Some concerns are partly addressed. The gating control at a matched activation rate (R6) are useful evidence. Table R1 is also helpful.
However, my main concerns remain.
The coupling measurement I asked for is not provided. In Table R2, the cross-transport term alone is worse than DMD2 (FID 3.72 vs. 3.14), while the self-transport term alone already reaches 3.00. This is hard to reconcile with the claim that the benefit comes from transporting outliers toward the target. The MMD baselines in R3 are also weaker than the IKL+MMD result in Table 3 of the paper (3.01). I still believe the so-called OT complement may be just the merit from the reconstruction. THIS is my main concern. To address this concern, the authors should compare an OT alternative which is not reconstruction. Objectively, I think what the sinkhorn coupling does is just reconstruction and this is unlikely to find the OT alternative.
The new evidence is a 2D toy example with ungated sDMD-U. No teacher coverage metric is reported, and R8 does not have DMD2 reference on the held-out samples.
3 The extension to learned scores is informal and still doesn't cover gating nor adversarial training.
4 The revised PDF adds only Appendix E.6. The new tables and the changed threshold are not incorporated
I appreciate the idea to incorporate OT with DMD, but this paper does not pave a way novel enough. I encourage the author to revise and resubmit to other conference.