acceptodds
Under review as a conference paper at ICLR 2027

One-Step Generative Modeling via Wasserstein Gradient Flows

37 views

Abstract

Diffusion models and flow-based methods have shown impressive generative capability, especially for images, but their sampling is expensive because it requires many iterative updates. We introduce W-Flow, a framework for training a generator that transforms samples from a simple reference distribution into samples from a target data distribution in a single step. This is achieved in two steps: we first define an evolution from the reference distribution to the target distribution through a Wasserstein gradient flow that minimizes an energy functional; second, we train a static neural generator to compress this evolution into one-step generation. We instantiate the energy functional with the Sinkhorn divergence, which yields an efficient optimal-transport-based update rule that captures global distributional discrepancy and improves coverage of the target distribution. We further prove that the finite-sample training dynamics converge to the continuous-time distributional dynamics under suitable assumptions. Empirically, W-Flow sets a new state of the art for one-step ImageNet 256×256 generation, achieving 1.29 FID, with improved mode coverage and domain transfer. Compared to multi-step diffusion models with similar FID scores, our method yields approximately 100× faster sampling. These results show that Wasserstein gradient flows provide a principled and effective foundation for fast and high-fidelity generative modeling.

Primary area:
generative models
Keywords:
one-step generative modeling, Wasserstein gradient flows

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_CPdmOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes W-Flow, a one-step generative model trained with Wasserstein gradient flow and Sinkhorn divergence. It constructs transport updates in pretrained feature spaces and uses them as regression targets for the generator. The method also uses two-batch estimation and velocity guidance. Since WGF-based generation and feature-space distribution matching already have precedents, I see the main contribution as its specific Sinkhorn-based training design and strong results on large-scale image generation.

Critical strengths
  1. The paper gives a clear formulation of how to use Sinkhorn-based WGF to train a one-step generator.

  2. The method achieves strong ImageNet results across different model sizes and improves over Drifting Models. The additional experiments on distribution coverage and domain transfer are also useful.

  3. The paper provides useful ablations on the divergence, two-batch estimation, and guidance. These experiments help explain which choices matter.

Critical weaknesses
  1. The reason for choosing Sinkhorn over KL needs more support. Table 10 shows that with velocity guidance, KL achieves 7.68 FID, compared with 7.08 for Sinkhorn. This shows an improvement in the tested setting, but it is still unclear what causes the gap and whether it supports the motivation for choosing Sinkhorn. Appendix A.1 describes the KL estimator, but I could not find the kernel bandwidth used or how it was selected. It would be helpful to know whether the gap remains after tuning this parameter.

  2. The efficiency comparison with KL is unclear. The same number of epochs does not necessarily mean the same FLOPs. Although the paper reports the cost of the Sinkhorn solver, I would like to see KL and Sinkhorn compared under the same total training budget, including online sample generation and feature extraction. Does Sinkhorn still give better results at the same computational cost?

  3. The dependence on feature space needs more study. Feature space is known to be very important for this kind of kernel-based distribution matching. It would be helpful to show how W-Flow works in different feature spaces and compare it with Drifting Models. Is it more robust or more sensitive to the choice of feature space? Comparing KL and Sinkhorn in these settings would also help us understand whether Sinkhorn’s advantage depends on the representation.

  4. The related-work discussion could also mention VGrow [1], which trains a generator to fit gradient-flow particle updates, and DMMD [2], which uses learned kernels for WGF-based image sampling.

Questions for authors
  1. What bandwidth is used for the KL baseline, and how was it selected? How sensitive are the results to this choice?

  2. How do KL and Sinkhorn compare under the same total FLOPs or wall-clock budget?

  3. How does changing the feature space affect W-Flow compared with Drifting Models? Does Sinkhorn still outperform KL across different feature spaces?

9 comments
AuthorsOpenReview ↗

Response to Reviewer CPdm (1/2)

Thank you for the detailed review and suggestions! We provide a point-by-point response to your comments below.

W1. The reason for choosing Sinkhorn over KL needs more support. Appendix A.1 describes the KL estimator, but I could not find the kernel bandwidth used or how it was selected. It would be helpful to know whether the gap remains after tuning this parameter.

Thank you for the advice. To clarify, for KL results reported in the paper, we did perform a hyperparameter sweep for the bandwidth for KL under the same budget as the entropic regularization parameter ε\varepsilon for Sinkhorn. More specifically, we swept over 2σ2=0.01,0.025,0.05,0.075,0.12\sigma^2=0.01,0.025,0.05,0.075,0.1 and reported the best performance for it.

FID (velo.)FID (dist.)
KL (2σ2=0.012\sigma^2=0.01)7.8910.38
KL (2σ2=0.0252\sigma^2=0.025)7.7410.17
KL (2σ2=0.052\sigma^2=0.05)7.6810.32
KL (2σ2=0.0752\sigma^2=0.075)7.8510.53
KL (2σ2=0.12\sigma^2=0.1)8.1310.81
Sinkhorn (ε=0.05\varepsilon=0.05)7.087.29

It is clearly demonstrated that Sinkhorn consistently outperforms KL under both distribution guidance (dist.) and velocity guidance (velo.) over a budget-controlled sweep of the kernel bandwidth for KL.

W2. The efficiency comparison with KL is unclear. The same number of epochs does not necessarily mean the same FLOPs. Although the paper reports the cost of the Sinkhorn solver, I would like to see KL and Sinkhorn compared under the same total training budget, including online sample generation and feature extraction. Does Sinkhorn still give better results at the same computational cost?

Thank you for the question. We would like to emphasize that the KL and Sinkhorn ablations use identical training settings, including the feature encoder, batch size, number of training epochs, and generator architecture. Therefore, the ablation study in Table 2 is already under nearly the same total training TFLOPs. To make it clear, we augment our training compute profiling (TFLOPs per GPU per training step) in Table 13 in Appendix with the KL estimator statistics:

StageB/2L/2XL/2
model forward/backward (shared by KL and Sinkhorn)158.9244.6296.0
Sinkhorn solver + OT loss0.820.820.82
KL estimator + KL loss0.240.240.24

Notably, KL/Sinkhorn estimators only account for less than 1% of the total training compute across all model scales. Therefore, given the same number of training epochs in our ablation study, KL and Sinkhorn consume almost the same training compute.

To make the point even stronger, we also compare Sinkhorn with the KL variant that is trained on 20% more epochs.

KL 100 epochKL 120 epochSinkhorn 100 epoch
FID (dist.)10.179.217.29
FID (velo.)7.687.357.08

We observe that Sinkhorn still outperforms KL even when KL was given 20% more training compute. The results support our design of using Sinkhorn-divergence-based gradient flow for the velocity field.

W3. The dependence on feature space needs more study. It would be helpful to show how W-Flow works in different feature spaces and compare it with Drifting Models. Is it more robust or more sensitive to the choice of feature space? Comparing KL and Sinkhorn in these settings would also help us understand whether Sinkhorn’s advantage depends on the representation.

Thank you for the advice. Besides the default MAE feature encoder, here we provide the results on two additional feature encoders that were ablated in Drifting Models [1], namely SimCLR and MoCo-v2. We adopt the same training setting as the ablation study in Table 2 (also same as the ablation default setting in [1]).

Feature EncoderVelocity FieldFID
SimCLRDrifting11.05
SimCLRKL11.88
SimCLRSinkhorn9.32
MoCo-v2Drifting8.41
MoCo-v2KL8.83
MoCo-v2Sinkhorn7.17

Notably, Sinkhorn is much more robust than Drifting Models and the KL variant across different feature spaces, which, again, verifies the superiority of our approach.

[1] Deng et al. Generative Modeling via Drifting. NeurIPS, 2026.

AuthorsOpenReview ↗

Response to Reviewer CPdm (2/2)

W4. The related-work discussion could also mention VGrow, which trains a generator to fit gradient-flow particle updates, and DMMD, which uses learned kernels for WGF-based image sampling.

Thank you for both references and we have added them to Sec. 2 in the updated manuscript, with the following discussion:

  • VGrow [2] trains a generator to fit particle updates of a gradient flow using f-divergences. Its velocity needs a density ratio, which is estimated with a discriminator trained alongside the generator. W-Flow computes its transport velocities from minibatch samples using Sinkhorn iterations, without training an auxiliary density-ratio discriminator.
  • DMMD [3] learns a noise-adaptive MMD kernel and simulates the MMD gradient flow at sampling time, which takes many steps. W-Flow instead compresses the flow into a one-step generator during training, and uses Sinkhorn rather than MMD.

[2] Gao et al. Deep Generative Learning via Variational Gradient Flow. ICML, 2019.

[3] Galashov et al. Deep MMD Gradient Flow without adversarial training. ICLR, 2025.

Q1. What bandwidth is used for the KL baseline, and how was it selected? How sensitive are the results to this choice?

Please refer to our response to W1.

Q2. How do KL and Sinkhorn compare under the same total FLOPs or wall-clock budget?

Please refer to our response to W2.

Q3. How does changing the feature space affect W-Flow compared with Drifting Models? Does Sinkhorn still outperform KL across different feature spaces?

Please refer to our response to W3.

We have updated the manuscript to account for these revisions and marked them in red for better readability. We sincerely hope our response could address your concerns!

AuthorsOpenReview ↗

Look forward to your feedback

Dear Reviewer CPdm,

We sincerely thank you for the detailed review and advice.

As the discussion period is coming to the end soon, we are eager to hear your feedbacks on our rebuttal, where we have provided a point-to-point response with additional results. We have also revised the manuscript accordingly. We are more than willing to clarify or discuss more if needed.

Thank you again!

Best,

Authors

Reviewer_12pROpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes W-Flow, a one-step generator. It is trained by distilling an explicit-Euler Wasserstein gradient flow (WGF) of the Sinkhorn divergence Sε(q,p)S_\varepsilon(q,p) into a static network through a stop-gradient regression: $\mathcal{L}(\theta) = \mathbb{E}{z\sim p{\mathrm{ref}}}\big| f_\theta(z) - \mathrm{sg}\big(x + \eta V(x)\big)\big|^2, \qquad x = f_\theta(z). $

The particle velocity is the population Sinkhorn WGF field, the difference of two entropic barycentric projections:

Vq,pε(x)=Tq,pε(x)−Tq,qε(x).V^\varepsilon_{q,p}(x) = T^\varepsilon_{q,p}(x) - T^\varepsilon_{q,q}(x).

It is estimated on minibatches with Sinkhorn–Knopp.

The paper claims four contributions over Drifting Models (Deng et al., 2026) and the concurrent Sinkhorn-Drifting (He et al., 2026):

A two-batch estimator for the self-transport term Tq,qεT^\varepsilon_{q,q}. It replaces Drifting's diagonal mask. A velocity-level classifier-free guidance. Under a KL energy, it is shown to recover the exponentially tilted CFG target. A finite-horizon W2W_2 stability bound for the empirical Euler particle system (Thm. 3.1/B.4). ImageNet 256×256256\times256 results of FID 1.29 (XL/2) and 1.35 (L/2) at 1 NFE, plus FFHQ experiments on domain transfer and mode coverage.

All ImageNet experiments reuse Drifting's infrastructure. The generator works in SD-VAE latent space (32×32×432\times32\times4). The Sinkhorn velocity is computed in the feature space of a frozen ResNet ϕ\phi: it was MAE-pretrained on SD-VAE latents and, for the main results, classification-fine-tuned on ImageNet labels. Losses are summed over multiple feature maps ϕk\phi_k.

Critical strengths

Strong empirical results. Training also converges faster than Drifting.

Controlled comparison with Drifting. The comparison uses the same generator, encoder and hyperparameters, so the improvement over Drifting is credible within this pipeline.

The two-batch self-transport estimator is simple and effective.

Velocity guidance is well motivated. Prop. A.1 is clean under KL, and the 2D illustration (Fig. 8) shows the difference from distribution guidance clearly.

Small sample sizes. The method holds up better than Drifting when the number of particles per class is smal. Negligible OT overhead. The Sinkhorn solver costs less than 0.6%0.6\% of FLOPs (Table 13).

Partly transparent positioning. The paper is open about its overlap with Sinkhorn-Drifting and NSGF, and it states limitations (A.5). It does, however, miss Gretton et al. (2026); see W2.

Critical weaknesses

W1. The headline models use a single Sinkhorn iteration.

  • Table 6 sets L=10L=10 iterations for the ablations and for B/2, but L=1L=1 for L/2 and XL/2. XL/2 also uses ε=0.01\varepsilon = 0.01.

  • With v(0)=1v^{(0)} = \mathbf{1}, one iteration of Alg. 2 is a row scaling followed by a column scaling of the Gibbs kernel. This is structurally close to Drifting's row/column-softmax normalization.

  • The narrative of a "globally coordinated, mass-conserving OT coupling" and the "plateau at $\sim$10 iterations" argument therefore do not describe the configurations behind the state-of-the-art numbers.

  • Gretton et al. (2026) show that a related one-shot approximation of the Sinkhorn flow (their “Sinkhorn Proxy”) is not conservative, i.e. not the Wasserstein gradient of any energy (their Prop. 4). The authors should discuss whether the single-iteration field used at L=1L=1 is the gradient of any functional; if not, the WGF interpretation does not apply to the headline models.

W2. Missing prior work. The paper does not cite Gretton et al. (2026, arXiv:2605.05118), which already interprets Drifting as a fixed point of the Sinkhorn-divergence WGF. That undercuts the "Drifting is heuristic" motivation, and its fixed-point view sits uneasily with the "compressed Euler discretization" story here.

W3. Theory vs. practice. As the authors acknowledge in A.5, the theorem does not characterize the training of the network itself.

W4. Table 4's "from scratch" framing does not reflect the classification-fine-tuned MAE encoder used in training. That encoder is not counted in #Params. No experiment runs without the encoder, or toggles its fine-tuning at a fixed scale. Sec. 3.4 also does not specify which features are used or how they are normalized.

W5. The KL variant relies on KDE score estimates in a high-dimensional feature space. Neither the KDE bandwidth nor the MMD kernel choice is reported. The poor FIDs in Table 2a may therefore reflect the estimators rather than the choice of energy.

Questions for authors
  • What do L/2 and XL/2 achieve with L=10L = 10 (or with converged) Sinkhorn iterations? Is Sinkhorn run in the log domain at ε=0.01\varepsilon = 0.01?

  • Can you report Drifting + two-batch self-term and Drifting + velocity CFG under the Table 2 setup, so the gain attributable to the OT normalization is isolated?

  • Please specify ϕ\phi fully

  • Can the convergence result be extended to the two-batch estimator Tq^,q^′εT^\varepsilon_{\hat q,\hat q'} actually used?

  • For age transfer, can you report a face-identity metric (e.g., ArcFace cosine similarity) and compare against a minibatch-OT regression or Schrödinger-bridge baseline?

  • How does W-Flow relate to Gretton et al. (2026, arXiv:2605.05118)?

7 comments
AuthorsOpenReview ↗

Response to Reviewer 12pR (1/4)

Thank you for the detailed review and suggestions! We provide a point-to-point response to your comments below.

W1.1. Table 6 sets L=10L=10 iterations for the ablations and for B/2, but L=1L=1 for L/2 and XL/2. XL/2 also uses ε=0.01\varepsilon=0.01.

Thank you for the question. First, we would like to refer the reviewer to the Sinkhorn iteration convergence analysis we have provided in Appendix E with the numerical results in Table 12. It is observed that empirically the iteration converges very quickly. For example, the marginal violation drops to 0.019 within only 5 iterations.

Built on this observation, we further investigate the effect of L=1L=1 and L=10L=10 on the headline models, with results as follows:

L=1L=1L=10L=10
W-Flow, B/21.601.52
W-Flow, L/21.351.34
W-Flow, XL/21.291.32

We observe that Sinkhorn iteration LL in fact has less impact on the generation performance on the L and XL-size models. These results suggest that computing the steepest descent direction exactly at every update may be unnecessary for effective particle-based incremental learning. This is consistent with the broader principle of inexact gradient methods, which can retain convergence guarantees when gradients are inexact with errors [1].

Furthermore, this is also computationally beneficial since with a small LL the Sinkhorn solver becomes even more lightweight while the generation performance remains nearly unaffected. We are therefore in favor of L=1L=1 for the L and XL-scale model training.

For ε\varepsilon on the XL model, it obtains FID 1.33 with ε=0.05\varepsilon=0.05 and FID 1.29 with ε=0.01\varepsilon=0.01. The performance is relatively stable while ε=0.01\varepsilon=0.01 yields slightly better result at XL size.

[1] Bertsekas, D. P., and Tsitsiklis, J. N. Gradient Convergence in Gradient Methods with Errors. SIAM Journal on Optimization, 10(3):627–642, 2000.

W1.2. With v(0)=1v^{(0)}=\mathbf{1}, one iteration of Alg. 2 is a row scaling followed by a column scaling of the Gibbs kernel. This is structurally close to Drifting's row/column-softmax normalization.

We agree that both constructions involve row and column normalization. The key distinction is in which matrices are normalized and whether the two normalizations are applied sequentially or independently.

W-Flow applies Sinkhorn scaling separately to the data and self-transport Gibbs kernels. Even at L=1L=1, the column scaling is computed after the row scaling, using the updated row factors. Drifting (Alg. 2 in [2]) instead concatenates the data and self-interaction logits into a single matrix S=[Sq,p,Sq,q]S=[S_{q,p},S_{q,q}], computes Arow=softmax⁡row(S)A_{\mathrm{row}}=\operatorname{softmax}_{\mathrm{row}}(S) and Acol=softmax⁡col(S)A_{\mathrm{col}}=\operatorname{softmax}_{\mathrm{col}}(S) independently from that same matrix, and combines them as A=Arow⊙AcolA=\sqrt{A_{\mathrm{row}}\odot A_{\mathrm{col}}}. This is different from Sinkhorn iteration that computes the row and column normalization sequentially.

We kindly refer the reviewer to Figure 10 in Appendix, where we have provided a detailed illustration on the different constructions of the drifting model normalization and our Sinkhorn iteration-based normalization.

[2] Deng et al. Generative Modeling via Drifting. NeurIPS, 2026.

W1.3. The narrative of a "globally coordinated, mass-conserving OT coupling" and the "plateau at ~10 iterations" argument therefore do not describe the configurations behind the state-of-the-art numbers.

Please see our response to W1.1.

W1.4. Gretton et al. (2026) show that a related one-shot approximation of the Sinkhorn flow (their “Sinkhorn Proxy”) is not conservative, i.e. not the Wasserstein gradient of any energy (their Prop. 4). The authors should discuss whether the single-iteration field used at L=1L=1 is the gradient of any functional.

Thank you for raising the point. In fact, we do not aim to claim that the L=1L=1 field is the exact gradient of any functional. Rather, we view it as an approximation to the Sinkhorn gradient field, where the algorithms parameterized by different LL constitute a family of reasonable approximations. In practice, our convergence analysis on Sinkhorn iterations has revealed that a very small LL can already approximate the final solution pretty well, and the results from the large-scale experiments also agree with this choice (see our response to W1.1).

Per your advice, we have revised the manuscript to make this point explicit. Specifically, we added:

Empirically, we run Sinkhorn iteration at a finite LL as an approximation of the Sinkhorn coupling, where the resulting velocity field may no longer correspond to the exact gradient flow of any functional.

AuthorsOpenReview ↗

Response to Reviewer 12pR (2/4)

W2. Missing prior work. The paper does not cite Gretton et al. (2026, arXiv:2605.05118), which already interprets Drifting as a fixed point of the Sinkhorn-divergence WGF. That undercuts the "Drifting is heuristic" motivation, and its fixed-point view sits uneasily with the "compressed Euler discretization" story here.

Thank you for bringing this relevant work to our attention. We have cited Gretton et al. [3] and revised our discussion to acknowledge their interpretation of Drifting.

We would, however, like to clarify an important distinction. Gretton et al. describe a construction related to implemented Drifting as a Sinkhorn Proxy, rather than the exact Sinkhorn-divergence WGF. Their Section 3 distinguishes its one-shot normalized affinities from converged OT plans, and Proposition 4 shows that the proxy field is generally not a Wasserstein gradient field. Thus, this connection does not establish that their implemented Drifting follows the Sinkhorn-divergence WGF.

Our goal, distinct from that of Gretton et al., is to make one-step generation effective by following the Sinkhorn variational formulation, using OT-derived velocities via Sinkhorn iteration together with two novel designs: a two-batch self-transport estimator and velocity guidance.

Besides, we would also like to point out that the "diagonal mask" method applied in Drifting does appear to be a heuristic design and is also acknowledged in Gretton et al. (their footnote 4 on Page 8) but remains unexplored. Our two-batch self-transport estimator provides an alternative and empirically achieves much better performance.

Regarding the fixed-point perspective, identifying a stationary condition does not establish the intermediate transport dynamics. Our compressed-Euler formulation specifies these incremental updates, and our particle-consistency analysis establishes finite-horizon consistency of the empirical Euler dynamics with the population continuity equation under stated assumptions.

[3] Gretton et al. On the Wasserstein Gradient Flow Interpretation of Drifting Models. arXiv:2605.05118, 2026.

W3. Theory vs. practice. As the authors acknowledge in A.5, the theorem does not characterize the training of the network itself.

Thank you for highlighting this distinction. Our goal is not to study neural network approximation or optimization, but to provide theoretical support for the particle-based approximation of the underlying continuity equation. This connection is central to our framework, which incorporates distributional flows into generative modeling through particle-based implementations. Establishing that these particle dynamics consistently approximate the intended flow provides a principled foundation for the training updates.

Specifically, Theorem 3.1 (formally, Theorem B.4) establishes finite-horizon consistency of the empirical Euler particle dynamics with the population continuity equation under Assumption B.1. Its bound separates initial-particle sampling, target sampling, and time-discretization errors. For the Sinkhorn velocity, the required regularity is verified under the bounded-support conditions in Proposition B.7. The theorem does not establish convergence of finite-network optimization or convergence to the target distribution as training time tends to infinity.

The connection to network training is through the transport targets in Eqs. (3) and (14). With the exact population field, write q_k=(f_{\theta_k})_\\#p_{\mathrm{ref}} and define

\qquad r_k=\|f_{\theta_{k+1}}-T_k\|_{L^2(p_{\mathrm{ref}})}.$$ Coupling both outputs using the same input $z$ gives $$W_2\left(q_{k+1},(\mathrm{Id}+\eta V_{q_k,p})_{\\#q_k}\right)\le r_k.$$ Thus, the regression approximates an Euler pushforward, while $r_k$ captures the additional fitting error. Characterizing this neural network fitting error, while interesting, is beyond the scope of the theorem. We have clarified this scope and the connection to the training objective, consistently with Appendix A.5.
AuthorsOpenReview ↗

Response to Reviewer 12pR (3/4)

W4. Table 4's "from scratch" framing does not reflect the classification-fine-tuned MAE encoder used in training. That encoder is not counted in #Params. No experiment runs without the encoder, or toggles its fine-tuning at a fixed scale. Sec. 3.4 also does not specify which features are used or how they are normalized.

We would like to clarify that we exactly followed the convention in drifting models (see Table 6 in [2]) when presenting the results in Table 4. Specifically, "from scratch" is adopted from drifting models (see the caption of Table 6 in [2]), which refers to the fact that the generator was not pretrained in a separate stage, in contrast to diffusion distillation methods. Similarly, we also follow [2] when reporting #Params in Table 4, which refers to the inference-related parameter count. For the latent-space models, #Params refers to the generator parameters and the VAE decoder parameters, since these are the only modules that are retained at inference time. The feature encoders are only applied during training and remain frozen, while being totally discarded at inference time.

That said, we agree that it would be helpful to further clarify these points in the paper. We have revised the caption of Table 4 to explicitly specify #Params as the inference parameter count.

W5. The KL variant relies on KDE score estimates in a high-dimensional feature space. Neither the KDE bandwidth nor the MMD kernel choice is reported. The poor FIDs in Table 2a may therefore reflect the estimators rather than the choice of energy.

Thank you for the advice. To clarify, for KL results reported in the paper, we did perform a hyperparameter sweep for the bandwidth for KL under the same budget as the entropic regularization parameter ε\varepsilon for Sinkhorn. More specifically, we swept over 2σ2=0.01,0.025,0.05,0.075,0.12\sigma^2=0.01,0.025,0.05,0.075,0.1 and reported the best performance for it.

FID (velo.)FID (dist.)
KL (2σ2=0.012\sigma^2=0.01)7.8910.38
KL (2σ2=0.0252\sigma^2=0.025)7.7410.17
KL (2σ2=0.052\sigma^2=0.05)7.6810.32
KL (2σ2=0.0752\sigma^2=0.075)7.8510.53
KL (2σ2=0.12\sigma^2=0.1)8.1310.81
Sinkhorn (ε=0.05\varepsilon=0.05)7.087.29

It is clearly demonstrated that Sinkhorn consistently outperforms KL under both distribution guidance (dist.) and velocity guidance (velo.) over a budget-controlled sweep of the kernel bandwidth for KL. For the MMD experiments reported in the paper, we used a Gaussian (RBF) kernel, a standard choice in MMD-based distribution matching [4, 5].

[4] Gretton et al. A Kernel Two-Sample Test. JMLR, 2012.

[5] Li, Swersky, and Zemel. Generative Moment Matching Networks. ICML, 2015.

Q1. What do L/2 and XL/2 achieve with L=10L=10 (or with converged) Sinkhorn iterations? Is Sinkhorn run in the log domain at ε=0.01\varepsilon=0.01?

Please refer to our response to W1.1 for the results on L=10L=10.

For the second question: Yes, Sinkhorn runs in the log domain using PyTorch logsumexp updates at every ε\varepsilon.

Q2. Can you report Drifting + two-batch self-term and Drifting + velocity CFG under the Table 2 setup, so the gain attributable to the OT normalization is isolated?

Thank you for the question. We report the results below.

two-batchvelocity CFGFID
Drifting--8.46
Drifting✓\checkmark-8.03
Drifting✓\checkmark✓\checkmark7.79
W-Flow✓\checkmark✓\checkmark7.08

It is observed that both two-batch and velocity CFG improve Drifting models, while the Drifting field still yields notably inferior performance than the Sinkhorn OT field. This again verifies our design.

Reviewer_WLQROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces W-Flow, a framework for training one-step generative models using Wasserstein gradient flows. The idea is to iteratively move generated samples toward the target data distribution using a velocity field derived from the Sinkhorn divergence, then train a neural network to approximate these updates. At inference time, generation requires only a single forward pass. The experiments suggest that optimal-transport-based training dynamics can improve both sample quality and mode coverage while retaining the speed advantages of one-step generation. On ImageNet256 it reports 1.29 FID, and it also reports experiments on FFHQ for domain transfer and mode coverage.

Critical strengths
  • Model choices are justified with the sinkhorn divergence (table 2a), the entropy (table 2b), the CFG (table 2b), the mask/batch and cost (table 2c), and the two-batch vs one-batch (Fig 3)
  • Strong results in ImageNet256, especially for W-Flow B/2 and L/2 that outperform the drifting and single-step diffusion/flow models even when the W-Flow model is smaller.
  • It can achieve equivalent performance to drifting models with many fewer epochs (384 vs 1280) and greater performance with the same number of epochs.
  • Can generalize better than drifting models with improved domain transfer (Fig 5b) and improved mode coverage (Fig. 6a, where drifting models drop the minority class entirely while W-Flows can capture it more accurately).
Critical weaknesses
  • The abstract mentions that the method yields 100x faster sampling, but I couldn’t find this mentioned anywhere in the main paper. Table 6 in the appendix does show some comparisons to multistep diffusion methods, but it would be interesting to see how the method compares to one-step solvers as well, and whether the additional computation of the Sinkhorn solver slows down training in any significant way.

  • Lastly, the paper mainly relies on results from ImageNet for the core results, it would have been interesting to see other datasets (or perhaps multiple seeds), the FFHD was only used for the transfer problem and no other publicly available dataset is used to compare performance against the drifting models.

Questions for authors
  • Table 5 in the appendix that the number of iterations L varies between the B/2 (L=10), L/2 (L=1) and XL/2 (L=1) models. Could the authors clarify why this decision was made? It may affect performance, as shown in Table 7, where higher iteration counts achieved a lower FID.
4 comments
AuthorsOpenReview ↗

Response to Reviewer WLQR (1/2)

Thank you for the detailed review and suggestions! We provide a point-to-point response to your comments below.

W1 (a). The abstract mentions that the method yields 100x faster sampling, but I couldn’t find this mentioned anywhere in the main paper. Table 6 in the appendix does show some comparisons to multistep diffusion methods, but it would be interesting to see how the method compares to one-step solvers as well.

Thank you for the question! Our statement about 100$\times$ faster sampling compared with multi-step diffusion models with similar FID scores is supported in both the title figure (Figure 1, right) and quantitatively in Table 7 in the appendix due to space limit.

In Figure 1, the red dashed line clearly shows the ~100$\times$ sampling compute reduction of W-Flow compared against SiT and LightningDiT. It also includes the comparison with many state-of-the-art one-step solvers, including Drifting, iMeanFlow, AdvFlow, and the GAN-based approaches. W-Flow clearly lies on the Pareto frontier of sampling efficiency and FID.

For better clarity, we repeat the results in Table 7 below while also adding the throughput comparisons with the one-step solvers:

ModelNFEFIDThroughput (img/s)
SiT-XL/2250×22.060.93
LightningDiT-XL/2250×21.351.14
MeanFlow-XL/213.4379.15
iMeanFlow-XL/211.7251.86
Drifting B/211.75107.76
Drifting L/211.5484.88
W-Flow B/211.52107.76
W-Flow L/211.3584.88
W-Flow XL/211.2977.88

Among one-step generators, W-Flow gives a better FID at the same or even higher inference speed. For example, W-Flow B/2 beats iMeanFlow-XL/2 in FID while running 2.1$\times$ faster.

W1 (b). Whether the additional computation of the Sinkhorn solver slows down training in any significant way.

Thank you for raising the point! In fact, we would like to point out that we have provided a comprehensive training time profiling in Table 13 of Appendix E. The Sinkhorn solver plus OT loss costs 0.82 TFLOPs/GPU, out of 159.7 / 245.4 / 296.8 TFLOPs in total for B/2 / L/2 / XL/2, i.e., 0.51% / 0.33% / 0.28% of the total training compute. Since the coupling is computed under no_grad on detached features, the Sinkhorn iterations do not add nodes to the autograd graph. The dominant cost lies in the model forward/backward pass, which is identical to Drifting Models.

W2. Lastly, the paper mainly relies on results from ImageNet for the core results, it would have been interesting to see other datasets (or perhaps multiple seeds).

Thank you for the advice. We first provide the results of 3 independent runs with different seeds.

seed42 (paper result)1001000
W-Flow, B/21.521.531.50
W-Flow, L/21.351.331.33
W-Flow, XL/21.291.271.30

We also add an unconditional generation experiment on FFHQ dataset. Both methods use the same MLP generator with 512-dimensional input and output and three hidden layers of width 1,024, trained for 100,000 updates with a batch size of 64.

DatasetSettingDriftingW-Flow FID
FFHQ (ALAE latent; no age filtering)Unconditional, 1-NFE, evaluated at 256×25617.0215.66

FID is computed from 50,000 generated images against the 60,000 original training images using the same Inception feature extractor. The result shows a lower FID for W-Flow at the same architecture and update budget.

AuthorsOpenReview ↗

Response to Reviewer WLQR (2/2)

Q1. Table 5 in the appendix that the number of iterations L varies between the B/2 (L=10), L/2 (L=1) and XL/2 (L=1) models. Could the authors clarify why this decision was made? It may affect performance, as shown in Table 7, where higher iteration counts achieved a lower FID.

Thank you for the question. First, we would like to refer the reviewer to the Sinkhorn iteration convergence analysis we have provided in Appendix E with the numerical results in Table 12. It is observed that empirically the iteration converges very quickly. For example, the marginal violation drops to 0.019 within only 5 iterations.

Built on this observation, we further investigate the effect of L=1L=1 and L=10L=10 on the headline models, with results as follows:

L=1L=1L=10L=10
W-Flow, B/21.601.52
W-Flow, L/21.351.34
W-Flow, XL/21.291.32

We observe that Sinkhorn iteration LL in fact has less impact on the generation performance on the L and XL-size models. These results suggest that computing the steepest descent direction exactly at every update may be unnecessary for effective particle-based incremental learning. This is consistent with the broader principle of inexact gradient methods, which can retain convergence guarantees when gradients are inexact with errors [1].

Furthermore, this is also computationally beneficial since with a small LL the Sinkhorn solver becomes even more lightweight while the generation performance remains nearly unaffected. We are therefore in favor of L=1L=1 for the L and XL-scale model training.

We have updated the manuscript to account for these revisions and marked them in red for better readability. We sincerely hope our response could address your concerns!

[1] Bertsekas, D. P., and Tsitsiklis, J. N. Gradient Convergence in Gradient Methods with Errors. SIAM Journal on Optimization, 10(3):627–642, 2000.

AuthorsOpenReview ↗

Look forward to your feedback

Dear Reviewer WLQR,

We sincerely thank you for the detailed review and advice.

As the discussion period is coming to the end soon, we are eager to hear your feedbacks on our rebuttal, where we have provided a point-to-point response with additional results. We have also revised the manuscript accordingly. We are more than willing to clarify or discuss more if needed.

Thank you again!

Best,

Authors

Thank you for the additional experiments, and for the effort you put into answering my questions and adding all these results to the revised manuscript.

These experiments address my concerns, and I believe the work is valuable. I will maintain my positive assessment and my current score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.