- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper "GRADIENT FLOW DRIFTING: ONE-STEP GENERATIVE MODELING WITH KDE DIVERGENCE FLOWS" proposes a novel framework for one-step generative modeling. Building upon KDE-defined Energy functions and Wasserstein gradients, particle dynamics for effective one-step generations are defined through a variational construction. The authors test their new method in various experiments with the largest dataset considered being ImageNet 256.
Critical strengths
- The authors present present the mathematical background for their new method which is then supported by experiments.
- One-step generation is a in general important field of research given the comparable high cost of many modern generative modeling approaches.
- The authors show strong results for family-balanced distribution matching on ImageNet 256
- The Appendix is very detailed and all important hyperparameters for the experiments are mentioned.
Critical weaknesses
- In the Reviewers opinion, the writing of the paper can be improved. For instance, an overview of the algorithm in the main part of the paper would be very helpful.
- In the experimental section it is difficult to see a clear improvement when using the new method. While the new approach performs best in some cases, the performance is not clearly better than existing approaches.
- The authors perform experiments on CIFAR 10 and ImageNet 256. Given CIFAR 10 is a very small dataset a larger second image dataset would be helpful.
- Computational cost of the new method is not sufficiently discussed.
- The related work section is very short and should be expanded and could be use to center the new method more general with respect to other generative modeling approaches and not just the ones closest related.
- No code is provided which reduces reproducibility.
Questions for authors
See weaknesses. and:
- In the Appendix H the authors mention a special manifold retraction technique. Where and why is this used exactly ?
- What do the authors mean by the effects of "finite neural parameter updates remain open" in the limitation section. Why is this a limitation specifically for the new method ?
- Why are both Gaussian and von Mises–Fisher kernels used and was the influence of the different kernels investigated in an ablation ?
3 comments
The Reviewer thanks the authors for addressing parts of the concerns and has thus raised their score. Some concern remains: while the authors have given a list where the new method performs strongly in the rebuttal, especially for the large image data set (Table 6) the performance is worse than some competing methods. Moreover, the Reviewer is still surprised why for some experiments the authors used a mix of Gaussian and vmf-kernel and this even has the strongest performance for one setting. If the kernel is based on the geometry this is surprising.
Thank you for reconsidering our paper and raising your score. We appreciate the opportunity to clarify the ImageNet comparison and what are combined in GFD.
- ImageNet FID (Tables 6–7).
We agree that Drifting-L and W-Flow-L achieve lower Inception FID than GFD-L in Table 6. One relevant difference is the feature encoder used to train each generator. GFD-L uses frozen self-supervised features. Drifting-L uses MAE features fine-tuned for ImageNet classification, and W-Flow-L adopts the same released encoder weights. In Drifting’s B/2 ablation, classification fine-tuning improves FID from 4.28 to 3.36 with the same MAE architecture and pretraining budget. See Drifting, Table 3, the authors’ response to Q2/W2.
This matters because matching Inception’s predicted ImageNet class frequencies can lower FID without actually improving the quality of results (Kynkäänniemi et al., ICLR 2023). We therefore consider alignment with the ImageNet classification task a plausible contributor to the FID gap. Table 7 gives a complementary comparison: across six encoder families held out from GFD training, GFD-L ranks first on all metrics among the directly trained models.
- What are combined in GFD.
We never combine Gaussian and vMF kernels in a single run. Table 18 places two ablations side by side, which may have made this unclear:
- Same divergence, different geometries. Two separate runs use forward KL. One uses Gaussian smoothing with Euclidean steps; the other uses vMF smoothing with spherical steps.
- Same geometry and kernel, different divergences. All runs in this ablation use spherical geometry and the same vMF kernel settings.
GFD+mixcombines the gradient fields of reverse KL and half-scaled Pearson (Proposition 3.6) in Sphere.
The geometry guides how we smooth the distributions and move particles. The divergence is a separate choice that determines how we compare those smoothed distributions and how we transport the particles. The strongest result in this ablation therefore comes from combining two divergence fields while keeping the geometry and kernel fixed.
The current revision was formatted for the rebuttal, with additions marked in blue so reviewers could distinguish them from the original text. We are sorry for some cost to readability. In the final version, we will annotate the ablation tables to show what each comparison holds fixed and add a unified overview explaining how geometry, kernel, and divergence together specify a GFD update.
Response to Reviewer dPbZ
Thank you for your review and for highlighting the importance of one-step generation and our family-balanced results on ImageNet-256. Please see the "Author Response to All Reviewers" (Parts 1-2, C1-C7) and the revised manuscript (changes in blue) for full details and tables.
W1. Algorithm overview. In the main text, Eq.1 (Sec.1) diagrams the 4-stage pipeline (G(q;p)→u[q]→u^→sg[x+]); Def.3.1 (Sec.3.1) specifies a GFD instance; and Sec.3.4 (Eqs.14-17) details the step-by-step training loop: (i) empirical KDEs and ratio r^(y) (Eq.14), (ii) the L-query field estimator u^(x) (Eq.15), and (iii) the retracted target x+ and stopped-gradient regression loss (Eq.17). App.H,Table 9 summarizes this 5-step procedure (C7).
W2. Experimental improvements. Across benchmarks, GFD demonstrates consistent gains: (1) lowest FID at all checkpoints on Controlled CIFAR-10 (C2, Fig.1); (2) 2.04 FID at 1 NFE on Scaled CIFAR-10, outperforming iCT-deep (2.51) and MeanFlow (2.92) (Sec.4.3); (3) best rank on all 5 family-balanced discrepancies on ImageNet-256 among all 6 direct models (Table 7; C5); (4) lowest mean SW2 (0.0099) and smallest seed SD (0.0008) on 5-seed Swiss roll (Table 3); (5) best Euclidean and Spherical Swiss-roll SW2 (Table 2) and Volcano/Flood scores (Table 4); and (6) 0.96 6-mer correlation at 1 NFE for SOTA on length-1024 DNA, outperforming Drifting (0.88) and KGD (0.91) (Table 5).
W3. Dataset scale. Alongside CIFAR-10, Secs.4.3-4.4 (Tables 6-7) evaluate on ImageNet-256 (1.28M images across 1,000 classes at 256×256 resolution, as you said the larger second image dataset), complemented by 4 Earth-event datasets and length-1024 Promoter DNA.
W4. Computational cost. Sec.4.3 (Tables 5-6) reports 1 NFE single-pass inference, a 15.5× smaller frozen encoder (173.2M parameters vs. 2.69B in Drifting-L and W-Flow-L), and fewer training epochs (384 vs. 640 and 1,280). In the revision (C6;Sec.4.3;App.L.7.3,Table 26), under a unified sample size,
GFD-Luses 70.70 TFLOP/update vs. 151.77 for W-Flow-L (53.4% reduction), 18.01 GB peak memory, 2.52 s/update, and 5,376 RTX 5090 GPU-hours for GFD-L.W5 & W6. Related work and code. Secs.1-2 discuss GANs, Diffusion, Flow Matching, MMD-GAN, Sinkhorn, Consistency Models, Generator Matching, and drifting flows. Besides, the extended comparisons is in App.M (Tables 27-28;C1). Anonymous code is provided below (and in C7).
Q1. Where and why manifold retraction is used. Retraction is introduced in Sec.3.3 and Sec.3.4 (Eq.17). For a unit vector x on the sphere Sd−1 (∥x∥=1) and any nonzero tangent velocity u (x⊤u=0), ∥x+ηu∥2=1+η2∥u∥2>1, so Euclidean addition leaves the sphere. A retraction maps a tangent step back onto the manifold and is used in all spherical tasks (Spherical toys, Earth events, DNA, and normalized image features) via the exact spherical exponential map.
Q2. "Finite neural parameter updates remain open" in Limitations. Thm.3.3 (Sec.3.2,Eq.7) proves energy dissipation dtdG(q;p)=−∫∥u[q](x)∥2dq(x)≤0 along a continuous-time, non-parametric population PDE. Updating finite neural parameters via discrete SGD steps on Eq.17 (Sec.3.4) projects the velocity onto the finite-dimensional generator Jacobian tangent space. This is not specific to GFD; it is a shared theoretical property of all amortized particle-flow, flow-matching, and drifting methods.
Q3. Why Gaussian and vMF kernels are used, and kernel ablation. (Sec.3.3,Corollaries 3.4-3.5,Eqs.10-12) A GFD kernel ϕ(x,y) integrates to 1 over the manifold volume measure to define the query law Y∼ϕ(x,⋅). On Rd, the Gaussian kernel induces additive queries Y=x+hϵ (Eq.10). On the unit sphere Sd−1, ∥x−y∥2=2(1−x⊤y), so restricting a Gaussian affinity to Sd−1 and normalizing over spherical area yields the vMF kernel (κ=h−2), whose queries stay on Sd−1 with tangent score κ(Y−(x⊤Y)x) (Eq.12). In our controlled CIFAR-10 ablation (C2;Fig.1a;App.K.5,Table 18),
GFD Gaussian forward KLachieves 7.37 FID at 50k vs. 7.22 FID for spherical vMFGFD forward KL(and 6.84 FID forGFD+mix). They cannot be investigated in an ablation if you want to maintain the geometry.Anonymous code: https://anonymous.4open.science/r/gfd_demo-D085/README.md
We hope these clarifications and additions address all of your concerns, and we respectfully ask you to reconsider your rating. If you have any further questions, we would be very happy to discuss them.