acceptodds
Under review as a conference paper at ICLR 2027

Accelerated-JKO: Discretizing Wasserstein Accelerated-Information Gradient Flow

Abstract

Wasserstein gradient flows (WGF), discretized by the JKO scheme, are widely adopted in optimization, sampling and generative model. JKO inherits the convergence rate of WGF, which degrades for densities that are not strongly log-concave. Accelerated Information Gradient Flow augments the density with a momentum for improved convergence rate, but existing realizations integrate the accelerated particle ODE explicitly which does not fully leverage the well-posed benefit of the JKO proximal step. We introduce A-JKO and its folding-free variant IA-JKO, momentum-augmented JKO schemes that discretize the AIGF momentum dynamics while keeping each step a well-posed variational problem, and prove that, as the step size vanishes, both schemes converge to the Wasserstein AIGF. Building on this scheme, we construct a neural sampler that transports a prior to the target through a flow-based velocity field, trained with a off-policy matching loss derived from the JKO optimality condition. On synthetic (DW-4, LJ-13, LJ-55) and molecular (alanine dipeptide) benchmarks, our sampler outperforms conventional JKO-based neural samplers and is competitive with several recent neural samplers.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Wasserstein Gradient Flow, Probabilistic Methods, Sampling, Optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_dFGuOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This work introduces a novel sampling method for Wasserstein gradient flows. The existing standard discretization scheme, JKO, has a slow convergence rate for non-strongly log-concave density functions. AIGF can improve this convergence rate using a Nesterov-style momentum on the density. However, this method can not leverage the well-posedness of the JKO proximal step. In this paper, the authors discretize the AIGF momentum dynamics while keeping the well-posed benefit. The authors implement two variants, A-JKO, including extrapolation, proximal, and momentum steps, and IA-JKO, an implicit version of A-JKO by optimizing a single proximal update and incorporating momentum. Then, the authors train a flow-based neural sampler with an offline matching loss, derived from the proposed discretization schemes. The authors test their approaches on both synthetic and real-world problems, which show promising results.

Critical strengths
  • The paper is nicely written. The authors make a significant effort to present relevant literature, background, and their research question. Their contributions are clearly presented. In addition, the paper is quite theoretical; however, the authors successfully highlight their main results. Their proofs are complete. Overall, I really enjoyed reading the paper.
Critical weaknesses
  • I would suggest that the authors visualize sampled distributions in the peptides experiment to better illustrate their results and comparisons with baselines.

  • In lines 170-175, the authors might need to motivate why we need discretization schemes for gradient flows.

  • In Section 5, there might be no need for a subsection since there is only one.

Questions for authors
  1. Could the authors explain more clearly the well-posed benefit of the standard JKO scheme?

  2. Could the authors clearly explain their off-policy matching loss objective? It seems that the neural sampler is trained with an off-policy matching loss using the original JKO optimality condition. Is it the optimality of the standard JKO? How does the neural sampler relate to accelerated JKO schemes?

  3. In Preliminaries, could the authors show how the Wasserstein-2 distance (line 129) is used to construct the accelerated gradient flow?

  4. In Table 2, why does flow sampling perform better than the proposed JKO schemes?

  5. For a single proximal update, the method requires a learned velocity vector field vθv_{\theta}; does the method require multiple velocity models vθv_{\theta} for the whole trajectory of multiple steps? Would the proposed sampling approach be expensive compared to conditional flow sampling?

I do not have strong expertise in this area, so my technical comments remain limited. But I hope the authors can make their work more accessible to novice readers.

Reviewer_KaZJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies sampling from an unnormalized target distribution using a sequence of continuous normalizing flows that approximate accelerated JKO updates. Standard JKO balances minimizing an energy functional against the Wasserstein transport cost from the previous distribution. The proposed A-JKO scheme incorporates momentum by extrapolating the current distribution before the proximal update, while IA-JKO incorporates the momentum displacement directly into the transport penalty. The authors present well-posedness and convergence results under stated assumptions and propose a matching objective derived from the IA-JKO optimality condition.

The methods are evaluated on DW-4, LJ-13, LJ-55, and alanine dipeptide. The reported results show substantial improvements over the JKO-IC baseline on the Lennard-Jones benchmarks and competitive performance with recent neural samplers on several metrics, although the improvements are not uniform across all benchmarks and metrics.

Critical strengths

The paper addresses an interesting problem, and combining accelerated distribution dynamics with variational JKO updates is a promising approach. The evaluation includes a useful range of benchmark dimensions and several relevant baselines. The results are encouraging, particularly the improvements over JKO-IC on LJ-13 and LJ-55.

Critical weaknesses

My main concerns are the clarity of the presentation and the completeness of the algorithmic description.

  1. The exposition assumes substantial background and omits several connections needed to understand the method. The preliminaries introduce information metrics, Hamiltonian dynamics, and their particle formulations in rapid succession, without sufficiently explaining how these formulations relate. For example, Φ\Phi appears on page 3, line 137, before its role is clearly introduced. Explaining its interpretation as a momentum potential and its relationship to particle velocity would help. Similarly, Equation (3) would benefit from a derivation, or a precise reference to one in the appendix, showing how the JKO objective becomes a CNF training loss. The transition from the variational objective to the matching loss also needs a clearer explanation of displacement fields, instantaneous velocities, and the projection onto gradient fields.

    I appreciate that the underlying mathematics is nontrivial. Nevertheless, a more gradual explanation would make the paper substantially more accessible to a broader machine-learning audience.

  2. Notation and apparent inconsistencies further impede understanding. In particular:

    • Momentum update: page 5, line 232, describes the updated momentum as xk+1−ykx_{k+1}-y_k, whereas the inverse-map update in Equation (4), page 4, line 210, records the total displacement xk+1−xkx_{k+1}-x_k. These expressions should be reconciled.
    • Velocity versus displacement: vθv_\theta denotes an instantaneous CNF velocity below Equation (3), page 4, line 190. However, the regression on page 7, lines 348–349, identifies the fitted field with the endpoint displacement Mk−idM_k-\mathrm{id}. Please explain the connection, including the role of the geodesic constraint on page 6, lines 316–318, and how that constraint is enforced in the neural implementation.
    • Force notation: On page 7, line 345, Wρk+1W_{\rho_{k+1}} is defined as the vector field ∇(log⁡ρk+1+V)\nabla(\log\rho_{k+1}+V), but line 351 applies an additional gradient to it, creating an apparent scalar/vector inconsistency. Moreover, the notation WρW_\rho is easily confused with the Wasserstein distance W2W_2. Distinct symbols for the scalar potential, its gradient, and the Wasserstein distance would improve clarity.
    • Argument in Equation (6): page 7, line 362, uses Mk(s)M_k(s), although the expectation is over xx and the preceding expressions use Mk(x)M_k(x). This appears to be a typographical error.
    • Normalization of Equation (3): the relationship between the JKO step size sks_k in page 4, line 177, and the transport-cost coefficient in Equation (3), line 187, is unclear. With ω˙θ=∣vθ∣2\dot\omega_\theta=\\|v_\theta\\|^2 as defined in line 190 and an integration interval [0,τ][0,\tau], the usual coefficient is τ/(2sk)\tau/(2s_k). Please explain the normalization or time-rescaling convention that produces the displayed coefficient of 11.
  3. The complete training and sampling procedures are difficult to reconstruct. Page 7, line 364, states that the full algorithm is summarized in the appendix, but I could not find pseudocode specifying the sequence of operations. An algorithm box should describe how blocks are trained, how momentum is represented and propagated, how new samples and replay-buffer entries are generated, how score estimation and matching updates are interleaved, and when correction stages are applied. It should also clarify which parameters remain fixed at each stage.

Questions for authors
  1. Computational cost. Which ODE solver and integration settings are used for each CNF block? Please report the number of function evaluations per block and across the complete sampler, separately for training and sampling where appropriate. The number of JKO blocks and optimizer updates does not determine the total integration cost, and Table 2 lists the proposed method’s training NFE as “N/A.”

  2. Training order and refinement. Is the method trained strictly sequentially over kk, with earlier blocks fixed? Have you considered a second training sweep initialized from the previously learned sequence of networks, in which blocks are refined and downstream samples, momentum fields, and replay targets are refreshed as necessary? Could such refinement reduce the approximation errors accumulated across blocks?

  3. Visualizing acceleration and mode coverage. Could you visualize the evolving distribution on a two-dimensional target, such as a Gaussian mixture with 40 modes? In particular, it would be informative to compare the proposed method with a version whose momentum displacement is set to zero, while keeping the architecture, proximal step sizes, and training procedure unchanged. Showing generated samples or density estimates alongside the target at several iterations kk, together with the number of modes covered, would help explain how momentum affects transport, mode discovery, and convergence.

  4. Stability and initialization. How sensitive is the practical algorithm to the step-size schedule, damping or momentum coefficients, and score-estimation accuracy? Are there settings in which training becomes unstable or fails to approach the target? How does the location and scale of the initial distribution affect mode coverage? The conditional guarantees for the idealized scheme do not by themselves establish robustness of the finite neural implementation.

  5. Scope of the matching-loss contribution. Could you distinguish more explicitly between the established JKO optimality condition, the IA-JKO-specific optimality condition, and the proposed off-policy regression procedure? Which part constitutes the new contribution, and how does it relate to existing methods that construct learning objectives from JKO optimality conditions?

Reviewer_EV2tOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes two momentum-augmented variants of the JKO scheme, A-JKO and IA-JKO, for discretizing Wasserstein Accelerated Information Gradient Flow. The authors establish theoretical results on well-posedness and convergence to the continuous-time accelerated flow. They further develop neural samplers based on these formulations, including an off-policy matching objective for IA-JKO. Experiments on synthetic distributions and molecular sampling benchmarks demonstrate improvements over a conventional JKO-based sampler and competitive performance against several recent methods.

Critical strengths

The paper addresses an interesting problem by combining momentum-based acceleration with the variational structure of JKO. The proposed IA-JKO formulation is particularly meaningful because it aims to avoid the folding issue associated with explicit momentum extrapolation.

Both theoretical analysis and empirical validation are provided in the paper. The convergence results offer a foundation for the proposed discretization schemes, while the experiments demonstrate substantial improvements over the conventional JKO baseline, particularly on higher-dimensional sampling problems.

Critical weaknesses

The experimental results are somewhat mixed. The proposed methods substantially outperform the JKO baseline on several benchmarks, but stronger fixed-edge samplers still achieve better results on some metrics. More controlled comparisons and computational cost measurements would help clarify the practical advantages of the proposed approach.

Questions for authors

Could the authors clarify how the theoretical well-posedness and convergence guarantees relate to the approximate neural implementation, particularly when the individual proximal steps are not solved exactly?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.