acceptodds
Under review as a conference paper at ICLR 2027

Gradient Flow Drifting: One-Step Generative Modeling with KDE Divergence Flows

14 views

Abstract

We introduce Gradient Flow Drifting(GFD), a unified, directly estimable energy-defined framework for one-step implicit generative modeling. GFD treats state-space geometry, kernel representation, and divergence energy as explicit modeling choices and maps them to particle dynamics and generator training through a common variational construction. The central principle is to take the Wasserstein gradient of the KDE-defined energy with respect to the clean model distribution and amortize the resulting particle updates into a neural generator. For admissible kernel-density -divergences, we derive the first variation, clean-particle fields, energy dissipation along classical flows, and distributional identification under injective smoothing. The framework also incorporates a quadratic distance between kernel densities as a density-ratio-free energy and supports nonnegative energy mixtures through the corresponding linear combination of particle fields. Gaussian and von Mises-Fisher kernels give explicit Euclidean and spherical instances. Sample-based field estimation and stop-gradient regression provide a common training procedure while preserving single-pass inference. Experiments on encoder-free synthetic distributions, spherical directional data, and high-dimensional DNA sequences, alongside encoder-based CIFAR-10 and ImageNet-256 generation, demonstrate the effectiveness and versatility of representative GFD instances and analyze the behavior of different divergence choices.

Primary area:
generative models
Keywords:
One-Step Generative Modeling, Wasserstein Gradient Flows, Kernel Density Estimation, Implicit Generative Models, Particle-Based Generative Modeling

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_dPbZOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper "GRADIENT FLOW DRIFTING: ONE-STEP GENERATIVE MODELING WITH KDE DIVERGENCE FLOWS" proposes a novel framework for one-step generative modeling. Building upon KDE-defined Energy functions and Wasserstein gradients, particle dynamics for effective one-step generations are defined through a variational construction. The authors test their new method in various experiments with the largest dataset considered being ImageNet 256.

Critical strengths
  • The authors present present the mathematical background for their new method which is then supported by experiments.
  • One-step generation is a in general important field of research given the comparable high cost of many modern generative modeling approaches.
  • The authors show strong results for family-balanced distribution matching on ImageNet 256
  • The Appendix is very detailed and all important hyperparameters for the experiments are mentioned.
Critical weaknesses
  • In the Reviewers opinion, the writing of the paper can be improved. For instance, an overview of the algorithm in the main part of the paper would be very helpful.
  • In the experimental section it is difficult to see a clear improvement when using the new method. While the new approach performs best in some cases, the performance is not clearly better than existing approaches.
  • The authors perform experiments on CIFAR 10 and ImageNet 256. Given CIFAR 10 is a very small dataset a larger second image dataset would be helpful.
  • Computational cost of the new method is not sufficiently discussed.
  • The related work section is very short and should be expanded and could be use to center the new method more general with respect to other generative modeling approaches and not just the ones closest related.
  • No code is provided which reduces reproducibility.
Questions for authors

See weaknesses. and:

  • In the Appendix H the authors mention a special manifold retraction technique. Where and why is this used exactly ?
  • What do the authors mean by the effects of "finite neural parameter updates remain open" in the limitation section. Why is this a limitation specifically for the new method ?
  • Why are both Gaussian and von Mises–Fisher kernels used and was the influence of the different kernels investigated in an ablation ?
3 comments
AuthorsOpenReview ↗

Response to Reviewer dPbZ

Thank you for your review and for highlighting the importance of one-step generation and our family-balanced results on ImageNet-256. Please see the "Author Response to All Reviewers" (Parts 1-2, C1-C7) and the revised manuscript (changes in blue) for full details and tables.

W1. Algorithm overview. In the main text, Eq.1 (Sec.1) diagrams the 4-stage pipeline (G(q;p)→u[q]→u^→sg[x+]G(q;p)\to u[q]\to\hat{u}\to\mathrm{sg}[x^+]); Def.3.1 (Sec.3.1) specifies a GFD instance; and Sec.3.4 (Eqs.14-17) details the step-by-step training loop: (i) empirical KDEs and ratio r^(y)\hat{r}(y) (Eq.14), (ii) the LL-query field estimator u^(x)\hat{u}(x) (Eq.15), and (iii) the retracted target x+x^+ and stopped-gradient regression loss (Eq.17). App.H,Table 9 summarizes this 5-step procedure (C7).

W2. Experimental improvements. Across benchmarks, GFD demonstrates consistent gains: (1) lowest FID at all checkpoints on Controlled CIFAR-10 (C2, Fig.1); (2) 2.04 FID at 1 NFE on Scaled CIFAR-10, outperforming iCT-deep (2.51) and MeanFlow (2.92) (Sec.4.3); (3) best rank on all 5 family-balanced discrepancies on ImageNet-256 among all 6 direct models (Table 7; C5); (4) lowest mean SW2\mathrm{SW}_2 (0.0099) and smallest seed SD (0.0008) on 5-seed Swiss roll (Table 3); (5) best Euclidean and Spherical Swiss-roll SW2\mathrm{SW}_2 (Table 2) and Volcano/Flood scores (Table 4); and (6) 0.96 6-mer correlation at 1 NFE for SOTA on length-1024 DNA, outperforming Drifting (0.88) and KGD (0.91) (Table 5).

W3. Dataset scale. Alongside CIFAR-10, Secs.4.3-4.4 (Tables 6-7) evaluate on ImageNet-256 (1.28M images across 1,000 classes at 256×256256\times256 resolution, as you said the larger second image dataset), complemented by 4 Earth-event datasets and length-1024 Promoter DNA.

W4. Computational cost. Sec.4.3 (Tables 5-6) reports 1 NFE single-pass inference, a 15.5×15.5\times smaller frozen encoder (173.2M parameters vs. 2.69B in Drifting-L and W-Flow-L), and fewer training epochs (384 vs. 640 and 1,280). In the revision (C6;Sec.4.3;App.L.7.3,Table 26), under a unified sample size, GFD-L uses 70.70 TFLOP/update vs. 151.77 for W-Flow-L (53.4% reduction), 18.01 GB peak memory, 2.52 s/update, and 5,376 RTX 5090 GPU-hours for GFD-L.

W5 & W6. Related work and code. Secs.1-2 discuss GANs, Diffusion, Flow Matching, MMD-GAN, Sinkhorn, Consistency Models, Generator Matching, and drifting flows. Besides, the extended comparisons is in App.M (Tables 27-28;C1). Anonymous code is provided below (and in C7).

Q1. Where and why manifold retraction is used. Retraction is introduced in Sec.3.3 and Sec.3.4 (Eq.17). For a unit vector xx on the sphere Sd−1S^{d-1} (∥x∥=1\Vert x\Vert=1) and any nonzero tangent velocity uu (x⊤u=0x^\top u=0), ∥x+ηu∥2=1+η2∥u∥2>1\Vert x+\eta u\Vert^2=1+\eta^2\Vert u\Vert^2>1, so Euclidean addition leaves the sphere. A retraction maps a tangent step back onto the manifold and is used in all spherical tasks (Spherical toys, Earth events, DNA, and normalized image features) via the exact spherical exponential map.

Q2. "Finite neural parameter updates remain open" in Limitations. Thm.3.3 (Sec.3.2,Eq.7) proves energy dissipation dG(q;p)dt=−∫∥u[q](x)∥2dq(x)≤0\frac{dG(q;p)}{dt}=-\int\Vert u{[q]}(x)\Vert^2\mathrm{d}q(x)\le0 along a continuous-time, non-parametric population PDE. Updating finite neural parameters via discrete SGD steps on Eq.17 (Sec.3.4) projects the velocity onto the finite-dimensional generator Jacobian tangent space. This is not specific to GFD; it is a shared theoretical property of all amortized particle-flow, flow-matching, and drifting methods.

Q3. Why Gaussian and vMF kernels are used, and kernel ablation. (Sec.3.3,Corollaries 3.4-3.5,Eqs.10-12) A GFD kernel ϕ(x,y)\phi(x,y) integrates to 1 over the manifold volume measure to define the query law Y∼ϕ(x,⋅)Y\sim\phi(x,\cdot). On Rd\mathbb R^d, the Gaussian kernel induces additive queries Y=x+hϵY=x+h\epsilon (Eq.10). On the unit sphere Sd−1S^{d-1}, ∥x−y∥2=2(1−x⊤y)\Vert x-y\Vert^2=2(1-x^\top y), so restricting a Gaussian affinity to Sd−1S^{d-1} and normalizing over spherical area yields the vMF kernel (κ=h−2\kappa=h^{-2}), whose queries stay on Sd−1S^{d-1} with tangent score κ(Y−(x⊤Y)x)\kappa(Y-(x^\top Y)x) (Eq.12). In our controlled CIFAR-10 ablation (C2;Fig.1a;App.K.5,Table 18), GFD Gaussian forward KL achieves 7.37 FID at 50k vs. 7.22 FID for spherical vMF GFD forward KL (and 6.84 FID for GFD+mix). They cannot be investigated in an ablation if you want to maintain the geometry.

Anonymous code: https://anonymous.4open.science/r/gfd_demo-D085/README.md

We hope these clarifications and additions address all of your concerns, and we respectfully ask you to reconsider your rating. If you have any further questions, we would be very happy to discuss them.

The Reviewer thanks the authors for addressing parts of the concerns and has thus raised their score. Some concern remains: while the authors have given a list where the new method performs strongly in the rebuttal, especially for the large image data set (Table 6) the performance is worse than some competing methods. Moreover, the Reviewer is still surprised why for some experiments the authors used a mix of Gaussian and vmf-kernel and this even has the strongest performance for one setting. If the kernel is based on the geometry this is surprising.

Thank you for reconsidering our paper and raising your score. We appreciate the opportunity to clarify the ImageNet comparison and what are combined in GFD.

  1. ImageNet FID (Tables 6–7).

We agree that Drifting-L and W-Flow-L achieve lower Inception FID than GFD-L in Table 6. One relevant difference is the feature encoder used to train each generator. GFD-L uses frozen self-supervised features. Drifting-L uses MAE features fine-tuned for ImageNet classification, and W-Flow-L adopts the same released encoder weights. In Drifting’s B/2 ablation, classification fine-tuning improves FID from 4.28 to 3.36 with the same MAE architecture and pretraining budget. See Drifting, Table 3, the authors’ response to Q2/W2.

This matters because matching Inception’s predicted ImageNet class frequencies can lower FID without actually improving the quality of results (Kynkäänniemi et al., ICLR 2023). We therefore consider alignment with the ImageNet classification task a plausible contributor to the FID gap. Table 7 gives a complementary comparison: across six encoder families held out from GFD training, GFD-L ranks first on all metrics among the directly trained models.

  1. What are combined in GFD.

We never combine Gaussian and vMF kernels in a single run. Table 18 places two ablations side by side, which may have made this unclear:

  • Same divergence, different geometries. Two separate runs use forward KL. One uses Gaussian smoothing with Euclidean steps; the other uses vMF smoothing with spherical steps.
  • Same geometry and kernel, different divergences. All runs in this ablation use spherical geometry and the same vMF kernel settings. GFD+mix combines the gradient fields of reverse KL and half-scaled Pearson χ2\chi^2 (Proposition 3.6) in Sphere.

The geometry guides how we smooth the distributions and move particles. The divergence is a separate choice that determines how we compare those smoothed distributions and how we transport the particles. The strongest result in this ablation therefore comes from combining two divergence fields while keeping the geometry and kernel fixed.

The current revision was formatted for the rebuttal, with additions marked in blue so reviewers could distinguish them from the original text. We are sorry for some cost to readability. In the final version, we will annotate the ablation tables to show what each comparison holds fixed and add a unified overview explaining how geometry, kernel, and divergence together specify a GFD update.

Reviewer_jN7COpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper develops Gradient Flow Drifting, a framework for one-step generation based on divergences between kernel-smoothed distributions. It derives gradient fields of the underlying distributions, extends them to manifold-valued representations, and estimates them using samples. Experiments cover synthetic distributions, Earth events, DNA, and image generation.

Critical strengths
  1. The framework clearly distinguishes gradients with respect to the underlying distribution from pointwise KDE score differences. How it deals with multiple divergences and manifold kernels is useful for designing distribution-matching updates.

  2. The experiments are diverse and thorough, and the ImageNet evaluation includes multiple held-out representations and discrepancy measures.

Critical weaknesses
  1. The gains over closely related methods need clarification. The benefit of the correction over the pointwise KDE score-difference formulation in arXiv:2603.10592 is not discussed in experiments. The W-Flow toy baseline also uses non-quadratic cost rather than the quadratic cost in its main published configuration, which seems to be inconsistent. On ImageNet, GFD has worse FID than Drifting and W-Flow, while gains on other metrics are due to different training representations. These comparisons make it unclear how much benefit comes from the proposed gradient field.

  2. The reliability of the high-dimensional density-ratio estimates is unclear. Image experiments use only 64 references per distribution and two queries per particle and kernel. The estimation error and numerical stability may be an issue here. Without sensitivity studies or comparison against higher-budget estimates, it is unclear how closely training follows the proposed flow.

Questions for authors

Could the authors address the weaknesses above, particularly the benefit of the adjoint correction under matched training settings, the W-Flow baseline configuration, and the reliability of the density-ratio estimates across different reference and query budgets?

5 comments
AuthorsOpenReview ↗

Response to Reviewer jN7C

Thank you for your constructive review. Please see the "Author Response to All Reviewers" (Parts 1-2, C1-C7) and the revised manuscript (changes in blue) for full details. Here is a summary of the answers to your questions.

Firstly, our contribution is theoretical. GFD establishes a unified, critic-free variational framework (general estimator) across geometries and divergences which is a breakthrough. We derive the clean-measure Wasserstein gradient of KDE ff-divergences and prove energy dissipation along classical population flows (Thm. 3.3), and then test the resulting updates on synthetic data, Earth events, sequences, CIFAR-10, and ImageNet-256 (C1-C7).

W1(a). Why KLclean degrades.

Write r=sq/spr=s_q/s_p for the smoothed density ratio. The pointwise baseline KLclean (L=0L=0) moves each clean particle along −∇log⁡r-\nabla\log r. The descent field of KL(sq∥sp)\mathrm{KL}(s_q\Vert s_p) with respect to the clean law qq is −∇Ak[log⁡r]-\nabla A_k[\log r], where AkA_k averages over kernel queries. Omitting AkA_k gives the wrong flow direction for this objective in general, even with exact densities (App. M, Eq. 137).

With DINOv3, DriftDiT, and the spherical settings fixed, the adjoint update (L=2L=2) lowers CIFAR-10 FID at all five checkpoints, from 20.78 to 15.09 at 10k and from 7.83 to 7.22 at 50k (Table 18) to pairwise score-difference(KGD). On the 768-dimensional product sphere at n=64n=64, the same change lowers five-seed mean MMD from 0.0957 to 0.0487 (C4 in "Author Response to All Reviewers", Table 12). With these experiments, we can find that pairwise score-difference is not stable on high-dimensional space.

W1(b). W-Flow cost (C2, Tables 2, 18 and 19).

We included quadratic cost in the reruns. Under the same toy protocol, switching from non-squared distance to the best tested cost(quadratic cost) per target lowers W-Flow's SW2\mathrm{SW}_2 from 0.0247 to 0.0131 on Checkerboard (Drifting 0.0148, GFD+mix 0.0199) and from 0.0355 to 0.0186 on Swiss roll (GFD+mix 0.0097). On CIFAR-10, W-Flow's 50k FID improves from 11.64 to 8.39, while it's still worse than Gaussian GFD forward KL, spherical GFD forward KL, and GFD+mix which reach 7.37, 7.22, and 6.84, respectively.

W1(c). MAE class-tuning and ImageNet FID.

Drifting-L and W-Flow-L train their generators with frozen MAE encoders fine-tuned on ImageNet. Drifting's authors acknowledge that "classification leakage could affect FID." Their paper and their rebuttal reports the ablation.

Drifting, Table 3,

Author rebuttal, Q2/W2

Removing class-tuning worsens FID by 0.92 for B/2. Kynkäänniemi et al. showed that matching Inception's predicted ImageNet class histograms can lower FID while perceptual quality stays the same. We interpret a part of Drifting's and W-Flow's advantage in FID as artifact of leakage.

The Role of ImageNet Classes in Fréchet Inception Distance, ICLR 2023

To compare update rules fairly, we recommend controlled CIFAR-10 FID with the same encoder, generator, and budget. GFD+mix leads at all checkpoints(C2, Table 18). On ImageNet-256 we recommend metrics in self-supervised features held out from generator training. With these features, GFD-L leads all five family-balanced metrics among the models (Table 7).

W2. High-dimensional stability and field direction.

On (S3)M(S^3)^M, we fix the training budget at n=64n=64 references per law and L=2L=2 queries. C4 (Table 12) reports final MMD over five training seeds.

Intrinsic dimension 3M3M31248192768
MMD (mean ±\pm SD)0.0253±0.00370.0253\pm0.00370.0423±0.00450.0423\pm0.00450.0505±0.00210.0505\pm0.00210.0475±0.00300.0475\pm0.00300.0487±0.00420.0487\pm0.0042

Mean MMD stays near 0.05 over a 16-fold increase from 48 to 768 dimensions, indicating stable training with this estimator.

CIFAR-10. C3 (Table 10) compares directions to the reference field at (n,L)=(32,768,256)(n,L)=(32{,}768,256).

The estimate is aligned even at n=16n=16. Increasing nn from 64 to 1,024 improves cosine by only 0.0129. The larger gain from extra queries points to Monte Carlo variance as the main reducible source of directional error here. Given the references, the variance of the mean of LL independent queries scales as 1/L1/L (App. I.2). At n=64n=64, raising LL from 2 to 8 quarters that variance and raises cosine from 0.6926 to 0.8626 (+0.1700). Fresh draws across batches also average down this centered noise. For BB independent batches at a fixed generator, its variance contribution to the mean scales as 1/(BL)1/(BL), so we increase batch size in practice.

We hope our detailed responses and clarifications have adequately addressed all your concerns. If you find our response satisfactory, we respectfully ask if you would consider re-evaluating our work, and let us know if you have any further questions.

Dear Reviewer jN7C,

Thank you again for your constructive feedback. With the discussion period closing soon, we wanted to briefly check whether our new experiments have resolved your two main concerns:

  1. Adjoint correction vs. pointwise score-difference (KLclean) & matched baselines:
    • Under identical DINOv3/DriftDiT/spherical settings (Table 18), the adjoint update (L=2L=2) improves CIFAR-10 FID over pointwise KLclean (L=0L=0) at every checkpoint (15.09 vs. 20.78 at 10k; 7.22 vs. 7.83 at 50k), and avoids weight collapse (two-class ESS 43.04 vs. 1.32, App. I.3).
    • Including the quadratic cost for W-Flow (Tables 2, 18, 19) improves its toy SW2\mathrm{SW}_2 (0.01310.0131 / 0.01860.0186, vs. GFD+mix 0.00970.0097 on Swiss roll) and CIFAR-10 50k FID (8.39), which still trails Euclidean GFD Gaussian forward KL (7.37), spherical GFD forward KL (7.22), and GFD+mix (6.84).
    • On ImageNet-256, accounting for MAE class-tuning leakage (Drifting Table 3) and evaluating across held-out self-supervised families (Tables 7, 22, 25–26), GFD-L leads all direct models across all five discrepancies and all six leave-one-family-out checks, while using 53.4% fewer TFLOPs/update than W-Flow-L (70.70 vs. 151.77 at a unified batch sample size).
  2. High-dimensional stability (App. I.3, Table 12): At n=64,L=2n=64, L=2 on (S3)M(S^3)^M, 5-seed MMD stays stable near 0.05 from 48D (0.05050.0505) to 768D (0.04870.0487), whereas KLclean (L=0L=0) degrades to 0.09570.0957.

Please let us know if any points remain unclear, and we would appreciate your reconsideration if these results address your concerns.

Thank you for the response and the additional experiments. Could you clarify whether the samples in Figure 3 of the appendix correspond to the updated results in Table 2, or to the original baseline configuration?

Reviewer_eHuPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Gradient Flow Drifting (GFD), which trains a one-step generator using particle updates derived from divergences between kernel-smoothed distributions. It gives direct estimators for general f-divergences and a quadratic KDE energy, in both Euclidean and spherical settings. The experiments cover synthetic data, images, and DNA sequences.

Critical strengths

I like that the method allows different divergence choices without training a separate critic. The distinction between the gradient with respect to the original distribution and a score difference between smoothed densities is useful.

Critical weaknesses
  1. My main concern is how much is new. As Appendix M notes, clean-measure KDE-KL, MMD flows, and energy-based stopped-target training are already known. The main addition is the direct KDE-ratio/query estimator for general f-divergences and its use in different geometries. This seems fairly incremental. Figure 1 shows that the mixture improves FID over KL, but does not show whether mixing helps beyond using reverse KL or χ2\chi^2 alone.

  2. The theoretical contribution is also limited in depth. The main results give the population field, energy dissipation, and identification under injective smoothing. These are useful consistency results, but do not establish convergence or characterize possible non-target stationary distributions. In particular, the global zero-field condition in Theorem 3.3 is stronger than stationarity on the model's support. The analysis therefore gives limited insight into the optimization behavior of the proposed flows.

  3. The estimator analysis also leaves an important gap. Appendix I gives a fixed-kernel bias expansion and discusses query variance, but does not bound the finite-sample error in the estimated field. The batch-dependent kernel scales in Appendix K.1 are not covered by that expansion either. With 64 real and 64 generated references and two queries in the image experiments, the analysis does not tell us how accurate the update directions are.

Questions for authors
  1. Figure 1 shows gains over KL. How does the mixture compare with reverse KL and (\chi^2) used separately under the same setup? This would help distinguish a benefit from mixing responses from a benefit of simply changing the divergence.

  2. In a simple setting, can you give a condition under which stationary particle distributions must match the target? Theorem 3.3 assumes the field vanishes everywhere, while particle dynamics only require it to vanish on the current support.

  3. Can you quantify the field-estimation error with batch-dependent kernel scales, even in a simple setting, or compare the estimated field with a high-accuracy reference? This would help assess how well the practical estimator follows the population field, beyond the fixed-kernel bias expansion.

4 comments
AuthorsOpenReview ↗

Response to Reviewer eHuP

Thank you for your constructive review. Please see the "Author Response to All Reviewers" (Parts 1-2, C1-C7) and the revised manuscript (changes in blue) for full details and tables.

W1 / Q1. Core contributions of GFD, and mixture vs. Reverse KL and χ2\chi^2 alone (C1-C2).

  • What GFD contributes: While Euclidean KDE-KL, MMD flows, and stopped-target have precedents, GFD establishes a unified, critic-free variational framework (general estimator) across geometries and divergences which is a breakthrough:
    1. Direct general-ff adjoint estimator with clean-measure dissipation (Thm.3.3): Gretton et al. (2026) require dual/critic forms for general ff-divergences, while KGD applies a pointwise score difference to clean particles. The resulting clean-measure energy derivative has no general negative-square form. GFD gives a direct query estimator (Eq.15) without training a critic. The corresponding classical population flow satisfies the exact clean-measure dissipation.
    2. Compositional and geometric generality (Prop.C.1,Prop.3.6,Table R1 in C1): GFD unifies general ff-divergences, ratio-free KDE-L2L^2, and nonnegative mixtures across Rd\mathbb R^d, spheres Sd−1S^{d-1}, product spheres (Sd−1)m(S^{d-1})^m, and other closed manifolds (Td,SO(3)T^d,\mathrm{SO}(3), etc.).
  • Mixture vs. single components on CIFAR-10 (C2,Table 18;Fig.1a): With features, backbone, spherical geometry, vMF scale rule, L=2L=2 queries, and 2∘2^\circ geodesic cap fixed, GFD+mix outperforms both components used alone at all checkpoints (50k FID: 6.84 for GFD+mix vs. 7.24 for Reverse KL and 7.51 for Half-scaled Pearson; KL is 7.22). On 5-seed Swiss roll (Table 3), KDE-L2 + KL achieves the lowest reported mean SW2\mathrm{SW}_2 (0.0099) and has the smallest across-seed SD (0.0008).

W2 / Q2. When stationarity on the model support identifies the target (C1;App.B.5,Prop.B.1).

Let qq be the law of G(Z)G(Z), where Z∼N(0,Im)Z\sim\mathcal N(0,I_m) and G:Rm→RdG:\mathbb R^m\to\mathbb R^d is locally Lipschitz. Assume supp(p)⊆S=supp(q)\mathrm{supp}(p)\subseteq S=\mathrm{supp}(q), the regularity of Thm.3.3, positive smoothed densities, continuous u[q]u[q], strictly convex differentiable ff, and injective smoothing.

  1. Zero field on the support. Set sq=Skqs_q=S_kq, sp=Skps_p=S_kp, r=sq/spr=s_q/s_p, and a=Ak[f′(r)]a=A_k[f'(r)]. At stationarity the energy is constant. With u[q]=−∇au[q]=-\nabla a, Eq.7 gives ∫∥∇a∥2dq=0\int\Vert\nabla a\Vert^2\mathrm{d}q=0. Thus ∇a=0\nabla a=0 almost everywhere under qq. By continuity it vanishes on SS, since every neighborhood of a support point has positive qq-mass.
  2. Constancy on SS. Each open neighborhood of G(z)G(z) has a nonempty open preimage of positive Gaussian mass. Since qq is carried by the image closure, S=G(Rm)‾S=\overline{G(\mathbb R^m)}. Locally Lipschitz GG maps each latent segment to an absolutely continuous path cc in SS. The chain rule gives (a∘c)′=∇a(c)⋅c′=0(a\circ c)'=\nabla a(c)\cdot c'=0 almost everywhere. Hence aa is constant on the image of GG and, by continuity, on its closure SS.
  3. Identification. Both laws have mass one on SS, so ∫ad(q−p)=0\int a\mathrm{d}(q-p)=0. Adjointness and ∫(sq−sp)dy=0\int(s_q-s_p)\mathrm{d}y=0 give
=\int s_p(r-1)(f'(r)-f'(1))\mathrm{d}y.$$ Strict convexity gives $(r-1)(f'(r)-f'(1))>0$ for $r\ne1$. Since $s_p>0$, a zero integral forces $r=1$ almost everywhere. Hence $s_q=s_p$ and injectivity gives **$q=p$**. **W3 / Q3. Field-estimation error with batch-dependent kernel scales and reference comparison (C3-C4;App.I.3,Tables 10-12,Fig.2).** - **Batch-calibrated bandwidth sensitivity (C3):** Across 10 paired CIFAR-10 batches, calibrating kernel scales on $n=64$ vs. 4096 target samples yields mean directional cosine **$0.999950\pm0.000062$** (fixed queries) and **$0.999953\pm0.000059$** (coupled draws), showing little change in field direction from batch scale calibration on these paired batches. - **High-accuracy reference comparison (C3,Table 10):** On CIFAR-10, cosine against the $(32768,256)$ reference is **0.6926** at $(n,L)=(64,2)$. Increasing only $n$ to 1,024 gives **0.7055** (+0.0129), and increasing only $L$ to 8 gives **0.8626** (+0.1700). A 16-fold reference increase thus adds little directional alignment here. Given the particle and references, averaging $L$ independent queries reduces Monte Carlo variance as $1/L$ (App.I.2). Fresh draws across batches also average down this centered noise. For $B$ independent batches at a fixed generator, its variance contribution to the mean scales as $1/(BL)$, so we decide to increase batch size instead of increasing independent queries. Reference-sample bias depends separately on $n$. We hope our detailed responses and clarifications have adequately addressed all your concerns. If you find our response satisfactory, we respectfully ask if you would consider re-evaluating our work. Please let us know if you have any further questions, and we would be very happy to discuss them.

Dear Reviewer eHuP,

Thank you again for your thoughtful review. As the discussion period is nearing its close, we wanted to briefly follow up to ask whether our revisions and responses have adequately addressed your three main concerns:

  1. Mixture vs. individual divergences (Table 18, Fig. 1a): Under identical spherical geometry, vMF scale calibration, and training settings, GFD+mix (50k FID 6.84) outperforms Reverse KL (7.24), Half-scaled Pearson χ2\chi^2 (7.51), and Forward KL (7.22) at all five checkpoints.
  2. Support-level stationarity (App. B.5, Prop. B.1): We proved that for qq the pushforward of N(0,Im)\mathcal{N}(0, I_m) under a locally Lipschitz GG with supp⁡(p)⊆supp⁡(q)\operatorname{supp}(p) \subseteq \operatorname{supp}(q), stationarity on supp⁡(q)\operatorname{supp}(q) alone (∫∥u[q]∥2dq=0\int \|u[q]\|^2 \mathrm{d}q = 0) implies q=pq = p.
  3. Estimator accuracy & batch-calibrated scales (App. I.3, Tables 10–11): Scale calibration on n=64n=64 vs. 40964096 yields a field cosine of 0.999950±0.0000620.999950 \pm 0.000062, the (64,2)(64,2) field aligns with the (32768,256)(32768, 256) reference at cosine 0.69260.6926 (0.86260.8626 at L=8L=8), and 16×1616\times 16 patch groups maintain healthy ESS (44.56/6444.56/64 real, 46.01/6346.01/63 model at τ=0.05\tau=0.05).

We would be grateful to know if any questions remain, or if these additions help support reconsidering your assessment.

Thank you for the new experiments and theory results, they have helped address some of my questions. However, I still have concerns of the size of the advance over earlier work and how much the theory explains the actual training process. The new result gives a useful condition for stationary laws to match the target, but the theory still offers limited insight into convergence. Therefore, I will keep my initial score.

Thank you for acknowledging the additional results. We would like to clarify two points.

  1. Our additional contributions include support-level identification and a directly implementable framework.

Proposition B.1 establishes distributional equality from a field that vanishes on the model support. Under general assumptions, this gives �0��0� even when the model support is a proper subset of �1��1�. Thus, identification does not require assuming a zero field throughout the whole space.

We additionally derive the clean-particle adjoint field and its exact energy-dissipation identity for admissible �2��2�-divergences across Euclidean and Riemannian geometries, together with a direct critic-free estimator, quadratic KDE−L2KDE-L^2 fields, and energy composition (Theorem 3.3, Eq. 15, and Propositions C.1 and 3.6). These results turn choices of geometry, kernel, and energy into computable training updates.

  1. Long-time finite-particle convergence has never been the objective of this paper.

Our objective is to correct and broaden gradient-flow formulations for one-step generation, provide simple, practical estimators, and evaluate the resulting framework across downstream tasks. The experiments demonstrate the practical benefits of these constructions and estimators. Long-time convergence is explicitly acknowledged as an open problem beyond the paper's scope.

We respectfully ask that these specific contributions and their empirical validation be reflected in the assessment rather than a convergence proofs in related fields.

Reviewer_pV3uOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Gradient Flow Drifting (GFD), a framework that constructs particle updates by taking the Wasserstein gradient of a KDE-defined divergence with respect to the clean model distribution. These updates are amortized into a one-step neural generator through stop-gradient target regression. The framework covers general (f)-divergences, a density-ratio-free KDE-(L^2) energy, nonnegative energy combinations, and Euclidean and spherical geometries. The analysis establishes first variations, energy dissipation along classical flows, and distributional identification under injective smoothing. Experiments cover synthetic distributions, spherical data, Earth events, DNA sequences, CIFAR-10, and ImageNet-256.

Critical strengths

1.The variational construction is clear and carefully scoped. Section 3.2 explicitly includes the adjoint smoothing operator when differentiating the smoothed energy with respect to the clean distribution. Appendix B.3 also distinguishes a globally vanishing field from a field that vanishes only at the current particles. These distinctions improve the theoretical clarity.

2.The framework provides a useful interface for method design. Divergence responses, kernels, geometry, and particle targets follow a common construction. The KDE-(L^2) component avoids density ratios, while spherical and product-spherical instances support directional data and sequence representations. This should interest researchers working on one-step generation, kernel methods, and distributional optimization.

3.The experiments demonstrate breadth and include controlled comparisons. The CIFAR-10 comparison shares the feature extractor, backbone, and training schedule, with GFD+mix achieving the lowest FID at every reported checkpoint. Table 3 compares energy choices across five training seeds. GFD-L reaches ImageNet FID 2.34 and leads the listed directly trained models under the family-balanced evaluation in Table 7.

4.The manuscript states important limitations explicitly. It acknowledges the gap between population flows and finite neural updates, the effect of numerical normalization, and the limits of identification through non-injective encoders. It also reports that GFD does not achieve the best conventional ImageNet FID and provides per-representation results and leave-one-family-out analyses.

Critical weaknesses

1.Estimator reliability needs further validation. Image and DNA experiments use only 64 references per distribution and two queries per component. For high-dimensional product kernels, a sensitivity study varying these counts would help establish whether the estimated fields are sufficiently stable.

2.The source of the empirical gains is unclear. The controlled CIFAR-10 comparison changes geometry and target construction alongside the divergence. An ablation with fixed geometry and step settings would help isolate the contribution of the divergence response.

3.The ImageNet aggregate needs additional robustness checks. Table 7 reports normalized ratios without raw distances or uncertainty estimates. Reporting these quantities and checking sensitivity to sampling and aggregation would strengthen the claimed ranking.

Questions for authors

1.How do increasing the reference count and query count affect field directions and generation quality on image or DNA tasks? Do product-kernel weights become dominated by a small number of references?

2.Does the CIFAR-10 advantage persist when geometry, kernel scales, and target-step construction are fixed and only the divergence response changes?

3.What are the raw distances, normalization denominators, and evaluation uncertainties underlying Table 7? How are empty histogram bins handled in the sliced KL estimates?

4.What are the ImageNet training GPU-hours, peak memory usage, and time per update? Do the smaller frozen encoders and fewer training epochs translate into lower measured training cost?

1 comment
AuthorsOpenReview ↗

Response to Reviewer pV3u

Thank you for your positive evaluation of our variational construction, method-design interface, and experimental breadth. Please see the "Author Response to All Reviewers" (Parts 1-2, C1-C7) and the revised manuscript (changes in blue) for full details. Here is a summary of the answers to your questions.

W1 / Q1. Field direction, weight concentration, and generation quality (C3-C4;App.I.3,Tables 10-12,Fig.2).

  • Field direction (C3,Table 10): Against a high-budget reference (n,L)=(32768,256)(n,L)=(32768,256) on CIFAR-10, the default (64,2)(64,2) estimator achieves cosine 0.6926 (grid minimum: 0.6191 at (16,1)(16,1)). Raising LL from 2 to 8 at n=64n=64 gives 0.8626 (+0.1700); raising nn from 64 to 1,024 at L=2L=2 gives 0.7055 (+0.0129). A 16-fold increase in references beyond the default thus gives only a small gain in CIFAR-10 field alignment at L=2L=2. For a fixed particle and reference batch, averaging LL independent queries reduces conditional Monte Carlo variance in proportion to 1/L1/L (App. I.2). At a fixed generator, their contribution to the variance of the mean over BB independent batches scales as 1/(BL)1/(BL). With fresh draws, these centered query errors also average down across batches, so we decide to increase batch size instead of increasing independent queries. Calibrating kernel scales on n=64n=64 vs. 4096 across 10 paired batches gives cosine 0.999950±0.0000620.999950\pm0.000062.

  • Weight concentration (C3,Table 11): For 16×1616\times16 CIFAR-10 patch groups across 100 batches, mean ESS at τ=0.05\tau=0.05 is 44.56/64 (real) and 46.01/63 (model after leave-one-out).

  • Generation quality (C2,C4,Tables 12,18,Fig.2): KLclean (L=0L=0) is a separate pointwise clean-anchor baseline(KGD). Our GFD estimator uses adjoint queries starting at L=1L=1. On CIFAR-10, GFD forward KL at the training budget (64,2)(64,2) reaches 7.22 FID after 50k updates. C4 (Table 12) varies the query budget on synthetic product spheres. At n=64n=64, the spread in five-seed mean MMD across L=1,2,4,8L=1,2,4,8 is at most 0.0076 within each of the tested dimensions 3, 12, 48 and 192. At dimension 768, the corresponding means are 0.0534, 0.0487, 0.0431 and 0.0392. Each doubling lowers mean MMD by 0.0039 to 0.0056, for a total reduction of 0.0142 from one to eight queries. At L=2L=2, raising nn from 32 to 128 lowers 768-D mean MMD from 0.0657 to 0.0359, and n=64n=64, L=2L=2 is a good default set.

W2 / Q2. Geometry and the matched divergence comparison on CIFAR-10 (C2;Sec.4.1;Fig.1a;App.K.5,Table 18).

With spherical geometry, vMF scale calibration, temperatures {0.03,0.05,0.30}\lbrace0.03,0.05,0.30\rbrace, L=2L=2, and 2∘2^\circ geodesic cap fixed, 50k FID is 6.84 (GFD+mix), 7.22 (Forward KL), 7.24 (Reverse KL), and 7.51 (Half-scaled Pearson). GFD+mix leads at all five checkpoints and reflects the change in divergence. For the geometry comparison, GFD Gaussian forward KL uses Gaussian queries and RMS-normalized steps in Euclidean space and reaches 7.37 FID at 50k. Spherical forward KL uses vMF queries and capped geodesic steps on normalized features and reaches 7.22. The 0.15 FID difference reflects the change in geometry.

W3 / Q3. ImageNet denominators, empty bins, and uncertainty (C5;App.L.7.1-L.7.2,Tables 24-25).

C6 reports all 60 validation-training reference distances. Sliced KLs use 1,000 projections and 100 equal-width bins with 10−810^{-8} smoothing for empty bins. Across 5 evaluation draws (C6 part 2), GFD-L leads all direct models on every metric and all leave-one-family-out checks significantly.

Q4. ImageNet training cost (C6;Sec.4.3;App.L.7.3,Table 26).

On 64 RTX 5090 GPUs (120k updates), GFD-B/L used 3,840 / 5,376 GPU-hours, 1.80 / 2.52 s/update, and 14.36 / 18.01 GB peak memory. Under a unified sample size, GFD-L uses 70.70 TFLOP/update vs. 151.77 for W-Flow-L (53.4% reduction).

We hope our detailed responses and clarifications have adequately addressed all your concerns. If you find our response satisfactory, we respectfully ask if you would consider re-evaluating our work. Please let us know if you have any further questions, and we would be very happy to discuss them.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to all Reviewers (Part 1)

Response to All Reviewers (Part 1/2: C1-C3)

Section, equation, figure, and table indices refer to the revised PDF (changes marked in blue).

C1. Variational Scope (Table R1) and Support Coverage (Prop.B.1)

Clean-measure flow (Sec.3.2,Thm.3.3,Eqs.5-7;App.C,Prop.C.1). Differentiating the KDE ff-divergence energy G(q;p)G(q;p) with respect to the clean model law qq yields the adjoint velocity field u[q](x)u{[q]}(x) and exact energy dissipation dG(q;p)dt=−∫∥u[q](x)∥2dq(x)≤0\frac{dG(q;p)}{dt}=-\int\Vert u{[q]}(x)\Vert^2\mathrm{d}q(x)\le0. Eq.15 estimates u[q](x)u{[q]}(x) directly from queries Y∼ϕ(x,⋅)Y\sim\phi(x,\cdot) without a critic. Prop.C.1 gives the ratio-free KDE-L2L^2 field, and Prop.3.6 combines nonnegative energy mixtures.

Table R1. Clean-measure energy dissipation by geometry and update (App.M,Tables 27-28).

GeometryGFD KLGFD Rev.KLGFD χ2\chi^2GFD L2L^2GFD other ffSinkhorn OTPointwise KDE KL (c)OT proxy
Euclidean Rd\mathbb R^dYes (a)YesYesYes (b)YesW-FlowKGDDrifting
Sphere Sd−1S^{d-1} & (Sd−1)m(S^{d-1})^mYesYesYesYesYes-KGD-
Closed manifolds (Td,SO(3)T^d,\mathrm{SO}(3))YesYesYesYesYes---

Yes denotes negative-squared-velocity dissipation (Thm.3.3,Prop.C.1) with GFD's theorem. (a) Euclidean clean-measure KDE-KL also appears in Gretton et al. (2026); their general-ff route uses dual/critic forms. (b) Recovers MMD flow for the induced kernel (Arbel et al.,2019). (c) Pointwise KDE score difference (KGD) yields the derivative in App.M,Eq.137, which lacks a general negative-square form.

Prop.B.1 (Stationarity on the model support,App.B.5). Let qq be the Gaussian pushforward on Rd\mathbb R^d under a locally Lipschitz generator G:Rm→RdG:\mathbb R^m\to\mathbb R^d with supp(p)⊆supp(q)\mathrm{supp}(p)\subseteq\mathrm{supp}(q). Under the regularity conditions of Thm.3.3, positive smoothed densities, continuous u[q]u[q], strictly convex differentiable ff, and injective smoothing, stationarity on supp(q)\mathrm{supp}(q) (∫∥u[q]∥2dq=0\int\Vert u[q]\Vert^2\mathrm{d}q=0) implies q=pq=p.

C2. Matched CIFAR-10 and Toy Comparisons (Sec.4.1,Table 2,Fig.1a;App.K.5,Tables 18-19)

Controlled CIFAR-10 FID (App.K.5,Table 18). All rows share frozen DINOv3 features, DriftDiT backbone, optimizer, and budget. Rows 1-4 fix spherical geometry, vMF scale calibration, temperatures {0.03,0.05,0.30}\lbrace0.03,0.05,0.30\rbrace, L=2L=2, and 2∘2^\circ geodesic cap, varying only the divergence response:

Update rule10k20k30k40k50k
GFD+mix (spherical)12.849.467.977.166.84
GFD forward KL (spherical)15.0910.379.208.077.22
GFD reverse KL (spherical)16.3110.048.517.537.24
GFD half-scaled Pearson χ2\chi^2 (spherical)18.0511.409.188.097.51
KLclean (L=0L=0, spherical, pointwise KDE score-difference)20.7812.4610.018.517.83
GFD Gaussian forward KL18.1411.198.968.117.37
Drifting14.0311.329.859.098.35
W-Flow (original)17.8814.2612.1911.3111.64
W-Flow (best tested cost)19.4112.5110.438.898.39

Euclidean toy SW2\mathrm{SW}_2 (Tables 2,19): Sweeping W-Flow costs (including quadratic) improves Checkerboard from 0.0247 to 0.0131 (Drifting 0.0148, GFD+mix 0.0199) and Swiss roll from 0.0355 to 0.0186 (GFD+mix 0.0097, Drifting 0.0112, GFD-KL 0.0159).

Five-seed Swiss-roll SW2\mathrm{SW}_2 (Table 3;App.K.2,Table 16): Forward KL 0.0100±0.00250.0100\pm0.0025; Reverse KL 0.0107±0.00150.0107\pm0.0015; Pearson χ2\chi^2 0.0099 ±0.0043\pm0.0043 (best 0.0063); KDE-L2L^2 0.0107±0.00160.0107\pm0.0016; Reverse KL+χ2\chi^2 0.0102±0.00220.0102\pm0.0022; KDE-L2L^2+forward KL 0.0099±0.00080.0099\pm0.0008 (worst 0.0107).

C3. Field Direction, Batch Calibration, and Weight Concentration (App.I.3,Tables 10-11)

CIFAR-10 field cosine vs. reference (n,L)=(32768,256)(n,L)=(32768,256) (Table 10):

References per law nnL=1L=1L=2L=2L=4L=4L=8L=8
160.61910.66750.73290.8205
640.63800.69260.76640.8626
2560.64540.70330.78000.8799
1,0240.64710.70550.78300.8834

Batch scale calibration (App.I.3): On 10 paired CIFAR-10 batches, scales calibrated on n=64n=64 vs. 4096 give cosine 0.999950±0.0000620.999950\pm0.000062 (fixed queries) and 0.999953±0.0000590.999953\pm0.000059 (redrawn queries).

Effective Sample Size (ESS=1/∑jwj2\mathrm{ESS}=1/\sum_j w_j^2) on 16×1616\times16 patch groups (Table 11): Real (of 64) / Model (of 63) ESS is 26.83±7.0726.83\pm7.07 / 29.27±7.0229.27\pm7.02 (τ=0.03\tau=0.03), 44.56±4.8844.56\pm4.88 / 46.01±4.5646.01\pm4.56 (τ=0.05\tau=0.05), and 63.32±0.1863.32\pm0.18 / 62.42±0.1662.42\pm0.16 (τ=0.30\tau=0.30). In a two-class diagnostic, KLclean (L=0L=0, pointwise KDE score-difference) gave real-side ESS 1.32 (top-5 weight share 99.9%), vs. 43.04 (21.6%) for L=2L=2 queries.

AuthorsOpenReview ↗

Response to all Reviewers (Part 2)

Author Response to All Reviewers (Part 2/2: C4-C7)

C4. Generation Quality on Product Spheres (S3)M(S^3)^M (App.I.3,Table 12,Fig.2)

Final MMD after 2,000 updates on (S3)M(S^3)^M by intrinsic dimension 3M3M (mean ±\pm SD over 5 training seeds):

nnQueries LL3M=33M=3 (M=1M=1)3M=123M=12 (M=4M=4)3M=483M=48 (M=16M=16)3M=1923M=192 (M=64M=64)3M=7683M=768 (M=256M=256)
64KLclean (L=0L=0)0.0292±0.00780.0292\pm0.00780.0505±0.00600.0505\pm0.00600.0609±0.00250.0609\pm0.00250.0841±0.01080.0841\pm0.01080.0957±0.00350.0957\pm0.0035
64L=1L=10.0297±0.00410.0297\pm0.00410.0438±0.00440.0438\pm0.00440.0527±0.00310.0527\pm0.00310.0522±0.00400.0522\pm0.00400.0534±0.00360.0534\pm0.0036
64L=2L=20.0253±0.00370.0253\pm0.00370.0423±0.00450.0423\pm0.00450.0505±0.00210.0505\pm0.00210.0475±0.00300.0475\pm0.00300.0487±0.00420.0487\pm0.0042
64L=4L=40.0245±0.00240.0245\pm0.00240.0417±0.00300.0417\pm0.00300.0491±0.00100.0491\pm0.00100.0485±0.00530.0485\pm0.00530.0431±0.00670.0431\pm0.0067
64L=8L=80.0272±0.00340.0272\pm0.00340.0420±0.00150.0420\pm0.00150.0486±0.00190.0486\pm0.00190.0446±0.00290.0446\pm0.00290.0392±0.00320.0392\pm0.0032
32L=2L=20.0273±0.00870.0273\pm0.00870.0457±0.00320.0457\pm0.00320.0509±0.00250.0509\pm0.00250.0541±0.00580.0541\pm0.00580.0657±0.00370.0657\pm0.0037
128L=2L=20.0242±0.00270.0242\pm0.00270.0367±0.00100.0367\pm0.00100.0472±0.00110.0472\pm0.00110.0420±0.00210.0420\pm0.00210.0359±0.00180.0359\pm0.0018
256L=2L=20.0226±0.00380.0226\pm0.00380.0325±0.00310.0325\pm0.00310.0414±0.00080.0414\pm0.00080.0376±0.00110.0376\pm0.00110.0371±0.00910.0371\pm0.0091
-Real-real0.0199±0.01100.0199\pm0.01100.0199±0.00240.0199\pm0.00240.0178±0.00280.0178\pm0.00280.0176±0.00130.0176\pm0.00130.0212±0.00350.0212\pm0.0035

C5. ImageNet-256 Reference Distances and Evaluation Sensitivity (Sec.4.4;App.L,Tables 21-25)

Validation-training reference distances (App.L.7.1,Table 24). These normalize the ratios in Table 21 via Eq.18. Sliced KLs use 5k samples, 1,000 projections, and 100 shared equal-width bins with 10−810^{-8} empty-bin smoothing:

ReadoutFDSWDMMDKL(T∥V)\mathrm{KL}(T\Vert V)KL(V∥T)\mathrm{KL}(V\Vert T)
DINOv3 ViT-B CLS (train)2.21340.0096500.0004470.0370750.037121
DINOv3 ViT-B patch (train)0.29070.0022190.0006710.0460110.035638
DINOv3 ConvNeXt-B (train)0.59770.0033050.0011680.0523720.040197
SigLIP2 CLS0.72300.0048500.0009010.0404030.037932
SigLIP2 patch1.90370.0130000.0005580.0392590.035860
ConvNeXtV2-FCMAE69.81680.3800490.0010370.0447060.032870
Inception-v31.33450.0046060.0006640.0422530.038943
DreamSim0.00900.0000160.0005880.0428220.041037
EUPE CLS1.41770.0109080.0007250.0375970.036299
EUPE patch0.74120.0062750.0008760.0411510.035381
Pixio global1.14060.0164180.0007790.0396860.035174
Pixio patch0.65500.0090990.0007570.0390700.034187

Five evaluation draws on held-out families (App.L.7.2,Table 25; 50k FD in Table 7):

Direct modelSWDMMDKL(T∥G)\mathrm{KL}(T\Vert G)KL(G∥T)\mathrm{KL}(G\Vert T)
W-Flow-B9.690±1.3029.690\pm1.30212.012±1.11812.012\pm1.1181.598±0.0281.598\pm0.0281.334±0.0361.334\pm0.036
Drifting-L8.867±1.0978.867\pm1.09711.451±1.00411.451\pm1.0041.573±0.0251.573\pm0.0251.280±0.0301.280\pm0.030
W-Flow-L7.938±1.1467.938\pm1.14610.113±0.94110.113\pm0.9411.477±0.0301.477\pm0.0301.261±0.0321.261\pm0.032
pMF-L6.607±0.8546.607\pm0.8548.127±0.7668.127\pm0.7661.413±0.0241.413\pm0.0241.191±0.0341.191\pm0.034
GFD-B4.804±0.7234.804\pm0.7235.385±0.6695.385\pm0.6691.245±0.0201.245\pm0.0201.136±0.0241.136\pm0.024
GFD-L3.381±0.4883.381\pm0.4883.808±0.3993.808\pm0.3991.159±0.0181.159\pm0.0181.067±0.0261.067\pm0.026

GFD-L leads all direct models across all six leave-one-family-out omissions rather than sampling error(App.L.5,Table 22).

C6. ImageNet-256 Training Cost (App.L.7.3,Table 26)

GFD uses 173.2M frozen encoder parameters (vs. 2.69B in Drifting-L and W-Flow-L). Measured at a unified batch sample size, per-update compute and memory alongside the 64 RTX 5090 run totals (120k updates / 384 epochs) are:

ModelTFLOP/updatePeak GB/GPUDayss/updateRTX 5090 GPU-hours
GFD-B37.9314.362.51.803,840
GFD-L70.7018.013.52.525,376
W-Flow-L151.7718.28---

GFD-L uses 53.4% fewer TFLOPs/update than W-Flow-L.

C7. Shared Training Procedure and Code (Sec.3.4;App.H,Table 9)

Training follows Table 9: (1) sample real references and noise; (2) generate particles x=G(ϵ)x=G(\epsilon); (3) estimate field u^(x)\hat{u}(x) (Eqs.14-15); (4) form retracted targets x+x^+ (for spheres, exact exponential map capped at 2∘2^\circ, Eq.129); (5) update generator via stopped-target regression (Eq.17).

Anonymous code: https://anonymous.4open.science/r/gfd_demo-D085/README.md

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.