- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes W-Flow, a one-step generative model trained with Wasserstein gradient flow and Sinkhorn divergence. It constructs transport updates in pretrained feature spaces and uses them as regression targets for the generator. The method also uses two-batch estimation and velocity guidance. Since WGF-based generation and feature-space distribution matching already have precedents, I see the main contribution as its specific Sinkhorn-based training design and strong results on large-scale image generation.
Critical strengths
-
The paper gives a clear formulation of how to use Sinkhorn-based WGF to train a one-step generator.
-
The method achieves strong ImageNet results across different model sizes and improves over Drifting Models. The additional experiments on distribution coverage and domain transfer are also useful.
-
The paper provides useful ablations on the divergence, two-batch estimation, and guidance. These experiments help explain which choices matter.
Critical weaknesses
-
The reason for choosing Sinkhorn over KL needs more support. Table 10 shows that with velocity guidance, KL achieves 7.68 FID, compared with 7.08 for Sinkhorn. This shows an improvement in the tested setting, but it is still unclear what causes the gap and whether it supports the motivation for choosing Sinkhorn. Appendix A.1 describes the KL estimator, but I could not find the kernel bandwidth used or how it was selected. It would be helpful to know whether the gap remains after tuning this parameter.
-
The efficiency comparison with KL is unclear. The same number of epochs does not necessarily mean the same FLOPs. Although the paper reports the cost of the Sinkhorn solver, I would like to see KL and Sinkhorn compared under the same total training budget, including online sample generation and feature extraction. Does Sinkhorn still give better results at the same computational cost?
-
The dependence on feature space needs more study. Feature space is known to be very important for this kind of kernel-based distribution matching. It would be helpful to show how W-Flow works in different feature spaces and compare it with Drifting Models. Is it more robust or more sensitive to the choice of feature space? Comparing KL and Sinkhorn in these settings would also help us understand whether Sinkhorn’s advantage depends on the representation.
-
The related-work discussion could also mention VGrow [1], which trains a generator to fit gradient-flow particle updates, and DMMD [2], which uses learned kernels for WGF-based image sampling.
Questions for authors
-
What bandwidth is used for the KL baseline, and how was it selected? How sensitive are the results to this choice?
-
How do KL and Sinkhorn compare under the same total FLOPs or wall-clock budget?
-
How does changing the feature space affect W-Flow compared with Drifting Models? Does Sinkhorn still outperform KL across different feature spaces?
9 comments
Response to Reviewer CPdm (2/2)
W4. The related-work discussion could also mention VGrow, which trains a generator to fit gradient-flow particle updates, and DMMD, which uses learned kernels for WGF-based image sampling.
Thank you for both references and we have added them to Sec. 2 in the updated manuscript, with the following discussion:
- VGrow [2] trains a generator to fit particle updates of a gradient flow using f-divergences. Its velocity needs a density ratio, which is estimated with a discriminator trained alongside the generator. W-Flow computes its transport velocities from minibatch samples using Sinkhorn iterations, without training an auxiliary density-ratio discriminator.
- DMMD [3] learns a noise-adaptive MMD kernel and simulates the MMD gradient flow at sampling time, which takes many steps. W-Flow instead compresses the flow into a one-step generator during training, and uses Sinkhorn rather than MMD.
[2] Gao et al. Deep Generative Learning via Variational Gradient Flow. ICML, 2019.
[3] Galashov et al. Deep MMD Gradient Flow without adversarial training. ICLR, 2025.
Q1. What bandwidth is used for the KL baseline, and how was it selected? How sensitive are the results to this choice?
Please refer to our response to W1.
Q2. How do KL and Sinkhorn compare under the same total FLOPs or wall-clock budget?
Please refer to our response to W2.
Q3. How does changing the feature space affect W-Flow compared with Drifting Models? Does Sinkhorn still outperform KL across different feature spaces?
Please refer to our response to W3.
We have updated the manuscript to account for these revisions and marked them in red for better readability. We sincerely hope our response could address your concerns!
Look forward to your feedback
Dear Reviewer CPdm,
We sincerely thank you for the detailed review and advice.
As the discussion period is coming to the end soon, we are eager to hear your feedbacks on our rebuttal, where we have provided a point-to-point response with additional results. We have also revised the manuscript accordingly. We are more than willing to clarify or discuss more if needed.
Thank you again!
Best,
Authors
Response to Reviewer CPdm (1/2)
Thank you for the detailed review and suggestions! We provide a point-by-point response to your comments below.
Thank you for the advice. To clarify, for KL results reported in the paper, we did perform a hyperparameter sweep for the bandwidth for KL under the same budget as the entropic regularization parameter ε for Sinkhorn. More specifically, we swept over 2σ2=0.01,0.025,0.05,0.075,0.1 and reported the best performance for it.
It is clearly demonstrated that Sinkhorn consistently outperforms KL under both distribution guidance (dist.) and velocity guidance (velo.) over a budget-controlled sweep of the kernel bandwidth for KL.
Thank you for the question. We would like to emphasize that the KL and Sinkhorn ablations use identical training settings, including the feature encoder, batch size, number of training epochs, and generator architecture. Therefore, the ablation study in Table 2 is already under nearly the same total training TFLOPs. To make it clear, we augment our training compute profiling (TFLOPs per GPU per training step) in Table 13 in Appendix with the KL estimator statistics:
Notably, KL/Sinkhorn estimators only account for less than 1% of the total training compute across all model scales. Therefore, given the same number of training epochs in our ablation study, KL and Sinkhorn consume almost the same training compute.
To make the point even stronger, we also compare Sinkhorn with the KL variant that is trained on 20% more epochs.
We observe that Sinkhorn still outperforms KL even when KL was given 20% more training compute. The results support our design of using Sinkhorn-divergence-based gradient flow for the velocity field.
Thank you for the advice. Besides the default MAE feature encoder, here we provide the results on two additional feature encoders that were ablated in Drifting Models [1], namely SimCLR and MoCo-v2. We adopt the same training setting as the ablation study in Table 2 (also same as the ablation default setting in [1]).
Notably, Sinkhorn is much more robust than Drifting Models and the KL variant across different feature spaces, which, again, verifies the superiority of our approach.
[1] Deng et al. Generative Modeling via Drifting. NeurIPS, 2026.