acceptodds
Under review as a conference paper at ICLR 2027

Tangent-Space Preconditioning of Wasserstein Gradient Flows for Particle Sampling

Abstract

We propose a simple geometric perspective on Wasserstein gradient flows for designing efficient particle-based sampling algorithms in high-dimensional settings. The central idea is to precondition the Wasserstein gradient flow in its tangent space, compensating for slowly relaxing directions that can impede sampling. Remarkably, we show that using the 2-Wasserstein Hessian of the chi-squared flow at the target as a preconditioner recovers the idealized population dynamics underlying Laplacian-adjusted Wasserstein gradient descent (LAWGD), providing a geometric interpretation of LAWGD as Wasserstein preconditioning. This perspective further identifies a scalar-to-vector transport operator as the key object to approximate, leading to two principled finite-sample realizations based on kernels and neural networks. Experiments demonstrate improved sampling efficiency and reveal complementary regimes for the two realizations, with kernels performing well in low dimensions and neural approximations becoming increasingly advantageous as dimension grows.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Wasserstein gradient flows, particle-based sampling, LAWGD, neural operator learning, spectral learning, Langevin dynamics, kernel methods

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6LKEOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates Wasserstein gradient flows for particle sampling and provides a new interpretation of LAWGD's preconditioning mechanism from the perspective of second-order Wasserstein geometry. This paper further proposes a framework for directly learning the transport operator with two finite-sample implementations based on kernels and neural networks. The convergence properties of the approximate transport dynamics and the representational limitations of fixed kernels in high-dimensional settings are analyzed in the paper. Experiments on synthetic distributions, stochastic physical systems, and molecular dynamics data demonstrates the effectiveness of the proposed methods.

Critical strengths
  1. This paper provides a geometric interpretation of LAWGD, which is insightful. Besides, the theoretical analysis is generally well-motivated and clearly presented.
  2. This paper develops both kernel and neural implementations of the proposed approach, supported by theoretical analysis and experiments across different settings. Experimental results demonstrate some promising improvements.
Critical weaknesses
  1. The proposed method requires access to samples from the target distribution, which raises questions about its practical necessity as sampling algorithms. While this assumption may be reasonable in certain generative modeling settings, this paper does not sufficiently justify why learning a transport operator and performing iterative particle updates would be preferable to directly training generative models on the available samples. Moreover, the improvements in sampling quality are relatively modest outside the synthetic Gaussian experiments, and comparisons with existing generative models are missing. These limitations make the practical advantages of the proposed approach unclear.
  2. The convergence analysis does not directly cover the practical finite-sample algorithms. Besides, experiments provide limited evidence of how these theoretical results translate into actual convergence behavior and sampling performance.
Questions for authors
  1. How sensitive is the proposed method to the number of target samples, the transport rank, and particle initialization?
  2. How do the assumptions underlying Theorem 3 relate to the practical finite-sample learning algorithms?
  3. What are the computational complexities of the proposed kernel and neural methods, considering both operator learning and iterative particle sampling?
1 comment

We thank the reviewer for their constructive comments.

1. Why learn from target samples, rather than train a generative model directly?

As the reviewer pointed out, our setting is generative modeling from a finite dataset: the goal is to transport fresh source particles toward the target distribution represented by those data. This follows the sample-based particle-flow setting of MMD flow (Arbel et al., 2019), KALE (Glaser et al., 2021), and DrMMD (Chen et al., 2025). Our contribution provides a principled way to learn inverse-generator preconditioning and study how its approximation affects particle motion. The operator is fitted once and reused during sampling.

To test sampling beyond Gaussian targets, we added a smoothed version of DrMMD’s three-ring experiment, where particles start in a compact cloud and must spread across all three rings while recovering their masses and thickness. Across five random seeds, K-T achieves a mean energy distance of 3.3×10−43.3\times10^{-4}, comapred with 2.5×10−42.5\times10^{-4} for DrMMD. The two methods also produce rings of similar thickness, approximately 0.050.05, matching the target’s radial spread..

2. Relationship between Theorem 3 and the practical algorithms.

Theorem 3 provides sufficient conditions for contraction of the population flow. Its purpose is to identify the properties an approximate transport operator must satisfy; we are not claiming that it certifies the particular operators obtained by our finite-sample training procedure. The revised manuscript makes this distinction explicit in Section 3.3 and clarifies the connection between the population learning objective and the ideal truncated operator in Section 4.2.

3. Sensitivity to sample size, rank, and initialization.

  • Target samples: The reported comparisons use fixed sample banks: 200,000 target samples per dimension in the Gaussian experiment and 10,000 in the rings experiment. These data determine both the learned functions and their target averages; the relevant consideration is coverage as well as sample count.

  • Transport rank: The Gaussian benchmark uses rank five to approximate its two linear and three quadratic slow modes. The rings experiment uses higher ranks, including a maximum requested rank of 128 for K-T. We specify the kernel settings and actual retained ranks, since increasing the requested rank is useful only when additional functions can be estimated reliably.

  • Initialization: A distant initialization can encounter very small kernel gradients or inaccurate neural extrapolation. The rings comparison uses the same compact initial cloud for every method.

4. Computational complexity and total cost.

For nn target samples, mm dictionary features, and dimension dd, forming the full covariance and Dirichlet matrices and solving the generalized eigenvalue problem costs O(ndm2+m3)O(ndm^2+m^3), assuming each kernel value and gradient costs O(d)O(d) to evaluate. Once a rank-rr model is fitted, one velocity evaluation on PP particles costs O(P(dm+mr)+mr)O(P(dm+mr)+mr).

For neural fitting, let UU denote the number of outer updates, bb the minibatch size, rr the rank, and KHK_H and KAK_A the numbers of forward and adjoint response updates per outer update. The training cost is

O(Ub[CF+KHCH+KACA+dr2]).O\left(Ub\left[C_F+K_HC_H+K_AC_A+dr^2\right]\right).

Here, CFC_F, CHC_H, and CAC_A are the per-sample network-computation costs for factor and response updates, including the required automatic differentiation and frozen-network evaluations. The dr2dr^2 term accounts for the factor Gram matrices. These costs depend on network architecture and rank.

For PP particles and SS velocity evaluations, neural sampling costs

O(SP[Cg+Cb+dr]).O\left(SP\left[C_g+C_b+dr\right]\right).

Here, CgC_g and CbC_b include evaluation of the calibrated scalar and vector factors. Auxiliary response optimization occurs during training only. Calibration contributes an additional one-time cost.

Once fitted, both methods evaluate particle velocities at a cost linear in the number of particles. We can reuse the fitted operator for further sampling runs with the same target, without repeating the training. We have added these complexity expressions to Appendix D.

Reviewer_uUiSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper interprets LAWGD as the chi-squared Wasserstein gradient preconditioned by the Hessian frozen at the target distribution. It develops kernel and neural approximations of the resulting transport operator, with the neural method learning scalar features and vector fields through auxiliary forward and adjoint Poisson problems. The method is evaluated on spiked Gaussian targets, Allen–Cahn fields after latent compression, and pentapeptide slow-feature recovery.

Critical strengths
  1. The geometric interpretation is precise. Theorem 1 identifies the equilibrium Wasserstein Hessian with the Witten Laplacian on tangent fields and recovers the LAWGD direction through the scalar–vector intertwining identity (§3.1, Eqs. (2)–(4)). This explains which operator determines the preconditioned particle motion.

  2. The neural construction gives a simple sampler after training. The forward and adjoint weak Poisson problems provide the operator actions needed to train the scalar/vector factors using target samples, and the stopped-response surrogate has the correct factor derivatives at exact responses (§4.2, Proposition 5; Appendix C). Sampling then requires feature averages and vector-network evaluations, without target scores, density-ratio estimates, or input differentiation of a learned potential (Eq. (5)).

  3. The controlled experiment supports the representation argument. Appendix B establishes an obstruction for the specified Gaussian kernel features, and the OU experiment tests increasing ambient dimension while keeping the relevant structure fixed (§6, Figure 1). This is useful evidence for adaptive features in that setting, although it does not establish general sampling scalability.

Critical weaknesses

Overall, I am not convinced that the paper demonstrates a sampling advantage that justifies the additional operator-learning machinery. The main issues are whether the comparisons isolate the proposed mechanism, whether the retained modes are sufficient for sampling, and whether the accuracy gain is worth the complete training and sampling cost.

  1. The direct-transport comparison changes the generator estimator. In the kernel construction, direct transport is exactly the gradient of the scalar Galerkin inverse built from the same covariance/Dirichlet eigenpairs, so holding the estimator, rank, centering, and regularization fixed gives the same velocity (§4.1). The Allen–Cahn K-IG baseline instead uses short-lag Koopman estimation, so its instability may come from the spectral estimator rather than from representing the inverse generator (§6; Appendix E.2). The reported comparison therefore does not isolate the benefit of direct transport, and N-IG is not specified well enough to determine whether the neural comparison resolves this issue.

  2. The low-rank dynamics do not determine the full target distribution. Equation (5) gives �0��0�, so non-target distributions matching the retained feature means are stationary; Theorem 3 assumes control of the unresolved discrepancy and does not establish that the learned sampler controls it. The OU study keeps the signal dimension at two and the default rank at five while evaluating sampling only on the signal marginal, Allen–Cahn runs in eight latent dimensions, and pentapeptide evaluates features rather than generated samples (§6; Appendix E). These experiments do not show what happens when more transport modes are needed, leaving the practical consequence of the rank restriction and the claimed sampling benefit unresolved.

  3. The computational benefit is not established at comparable accuracy. Neural training uses five forward-response and five adjoint-response updates per factor update, yet the paper provides no corresponding time or memory scaling as the required rank grows (Appendix C.3). Table 1 does show a substantial training speedup over N-IG, but the displayed N-T fit-plus-sampling time is 30.4 s versus 1.0 s for K-T, while latent W2W_2 changes from 1.911 to 1.904. With only seed 0 displayed, no aggregate uncertainty from the four pipelines described in Appendix E.2, and no matched-quality amortization study, I cannot tell when the extra neural training cost is justified.

Questions for authors
  1. What remains of the direct-transport advantage when the generator estimator is held fixed? Please specify the N-IG objective and information used, and provide one neural comparison with the same target data, generator, rank, and comparable training resources; the corresponding kernel equality should serve as an equivalence check.

  2. What does the sampler correct outside the retained slow modes? Please state the OU particle initialization and report full-distribution or complementary-coordinate diagnostics; a small extension of the existing experiment that increases the number of relevant directions and the transport rank would directly test the unresolved scalability issue.

  3. At what accuracy and reuse level does the neural construction become preferable? Please report the aggregate errors, failure counts, and timings from the four Allen–Cahn pipelines, together with accuracy versus total cost that includes fitting and the intended number of sampling runs.

1 comment

We thank the reviewer for their constructive comments.

1. What the direct-transport comparison establishes.

As the reviewer noted, the equivalence follows directly from our kernel construction: with the same functions, weights and settings, taking the gradient of the scalar inverse-generator approximation gives exactly the direct-transport update. We checked the two velocities at the 1,000 initial particles, using five seeds, ranks 16 and 32, and the median-rule kernel width. The maximum absolute difference over particles and coordinates, divided by the largest absolute component of the direct-transport velocity, was below 6×10−116\times10^{-11}.

We acknowledge, however, that our K-IG and K-T implementations learn the functions and weights differently, and any improvement in sampling could come from that difference, rather than from the direct-transport formula itself. We have specified each method’s estimator and data requirements in Section 6 and Appendix E, and stated the kernel equivalence in Section 4.1.

We also have clarified the information available to each method in Section 6. K-IG and N_IG estimate Koopman modes from lagged pairs, while K-T and N-T use target samples alone. We expanded Appendix E to specify N-IG’s training objective, rate estimation, normalization, and particle update. We describe N-T’s objective and response optimization in Section 4.2 and Appendix C. We also added comparisons with MMD (Arbel et al., 2019) and DrMMD (Chen et al., 2025) because both use target samples alone. In Allen–Cahn, we compare the methods with their different operator estimators.

2. Rank and recovery of the full distribution.

We acknowledge that matching finitely many feature means does not generally determine the full target distribution. Theorem 3 provides sufficient conditions for contraction, including control of the unresolved discrepancy. The revised text makes this scope explicit after Equation (5) and in the discussion of Theorem 3.

As an additional benchmark, we added the new smoothed three-ring experiment from DrMMD (Chen et al., 2025); it evaluates the full two-dimensional particle distribution, including ring masses, thickness, and mass in the holes. With the kernel settings and retained ranks specified in the appendix, K-T achieves sampling accuracy comparable to DrMMD. K-T retained 128 modes in four runs and 32 modes in one run. Across these five seeds, K-T’s mean energy distance is 3.3×10−43.3\times10^{-4}, compared with 2.5×10−42.5\times10^{-4} for DrMMD; both methods produce similar ring thickness and little mass in the holes.

3. Complete computational cost and reuse.

In the revised appendix, we include the learning and sampling costs of both constructions, including all forward- and adjoint-response updates (see also our response to Reviewer 6LKE, point 4, for a more detailed discussion). We report fitting and sampling times separately in the Allen–Cahn table: K-T has the lowest total cost, while N-T is substantially faster overall than N-IG for the reported configurations. Reusing a fitted operator avoids repeating the fitting cost as shown in this experiment, but we acknowledge that the current experiments do not establish a reuse threshold at matched sampling accuracy.

Reviewer_NEzYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper’s main contribution is a geometric interpretation of ideal LAWGD, together with kernel and neural methods for approximating its transport operator. Its strongest practical evidence concerns relative performance among four spectral methods; its guarantees concern population operators and continuous-time flows, rather than the implemented finite-particle samplers.

Critical strengths

The paper contains a substantive learning proposal, not just a reinterpretation of LAWGD. Its result establish a principled approximation target and provide useful proof-of-concept evidence. I think the strongest positive is the sample-only neural construction: it gives a way to learn and deploy the transport field without estimating density ratios.

Critical weaknesses
  1. their training loss measures operator approximation, while convergence depends on derivatives of that operator. Their contraction theorem assumes that stronger control rather than deriving it.

  2. the experiments are too narrow to compensate fully for the missing connection between training accuracy and sampling behavior.

    a. There is also no direct SVGD comparison

    b. the comparison with Li & Marzouk's DMPS and Xu et al.'s KSWGD should be clear in the experiment

    c. Spiked OU provides meaningful positive evidence. Better operator recovery translates into better sampling. However, the informative structure remains two-dimensional, the sampling metric examines those coordinates, and performance deteriorates at the largest tested dimension.

    d. Allen–Cahn shows a useful stability improvement, but the comparison also changes how the operator is estimated. Sampling improvements are modest, all methods operate in an eight-dimensional latent space, and the displayed table reports one seed. At least, It would be helpful to clarify which differences between the implementations produce the reported gains, and whether the neural direct-transport approach retains an advantage when compared under comparable data, operator information, and computational budgets.

    e. I am also curious whether the method delivers useful samples more efficiently overall.. Does its advantage survive comparisons of total training-plus-sampling cost at matched distributional accuracy? A method can generate particles quickly after expensive training, so sampling time alone is insufficient.

In general, i think the theoretical contribution is decent but not strong enough if the practical advantage cannot be fully justified.

Questions for authors

See critical weaknesses.

1 comment

We thank the reviewer for their effort in reviewing our manuscript and providing feedback. We provide answers to the reviewer’s questions below, and have updated the manuscript to address the concerns.

Significance of the theoretical and algorithmic contribution.

Before we provide responses to the questions, we would like to clarify our contribution. We provide a principled route to learning inverse-generator transport from target samples alone, supported by an equilibrium geometric characterization and low-rank approximation principle. This supplies both a mathematical foundation and a new learning procedure, including the sample-only neural construction highlighted by the reviewer.

This perspective is consistent with the development of this literature. LAWGD (Chewi et al., 2020) explicitly presents its low-dimensional implementation as proof of concept while leaving efficient high-dimensional implementation open. MMD flow (Arbel et al., 2019) likewise centers its contribution on constructing and analyzing transport dynamics, supported by targeted numerical demonstrations. Our work advances this program by developing a framework for learning the transport operator. In the revised manuscript, we will clarify these points to better posit our paper with respect to the existing literature.

1. Connection between the training objective and convergence.

As the reviewer pointed out, our theory does not establish that small training error alone implies the derivative control required by Theorem 3. Theorem 3 gives sufficient conditions for an approximate transport operator to preserve contraction. Theorem 4 identifies the truncated operator TrT_r as a global optimum of the population objective over rank-rr operators. At this optimum, Dπ∗Tr=ΠrD_\pi^*T_r=\Pi_r, so the derivative condition in Theorem 3 holds with zero approximation error. This connects the ideal learning target to the contraction analysis, while establishing the corresponding control for an approximately trained operator remains a further step. We have clarified this distinction in Sections 3.3 and 4.2 of the revised manuscript.

2. Broader experimental evidence.

We have added a non-Gaussian sampling experiment based on DrMMD’s (Chen et al., 2025) three-ring example. The target consists of three rings with small Gaussian noise, and particles start in a compact cloud. We compare K-IG, N-IG, K-T, SVGD, MMD, DrMMD, and N-T+kernel, a variant that combines learned neural scalar functions with kernel-computed responses and transport fields.

Across five random seeds, K-T achieves a mean energy distance of 3.3×10−43.3\times10^{-4}, compared with 2.5×10−42.5\times10^{-4} for DrMMD, with similar ring thickness and similarly little mass in the ring holes. We include the generated particles and reports distributional distances, ring masses, and radial accuracy in an appendix table of the revised manuscript.

3. Learning and sampling costs.

We have added computational complexity expressions for kernel and neural learning and sampling, including the auxiliary response updates used in neural training. The Allen–Cahn table reports fitting and sampling times separately. Summing these costs gives approximately 29.7 seconds for N-T, 179.2 seconds for N-IG, and 0.91 seconds for K-T. Thus, N-T is substantially faster than N-IG in this configuration, while K-T has the lowest total cost.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.