- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper introduces Wasserstein Saddle-Free Newton (WSFN), extending saddle-free Newton methods to optimization over probability measures. The authors establish a saddle-escape complexity bound of , improving the saddle-dependent contribution of previous first-order methods. They also characterize the local spectral convergence of the linearized dynamics. Numerical experiments illustrate the proposed method on several Wasserstein optimization problems.
Critical strengths
The extension of saddle-free Newton methods to Wasserstein space is mathematically interesting. In particular, the analysis of Hessian-dependent operators along nonlinear transport trajectories is technically nontrivial, and the improved saddle-escape bound represents a theoretical contribution under the stated assumptions.
Critical weaknesses
1. Restrictive assumptions and questionable applicability.
My main concern is Assumption 4.5, which appears substantially stronger than standard Hessian Lipschitz regularity.
In particular, if I am correct, its first inequality seems to force to be spatially constant -almost everywhere. To see this, take and consider the coupling
+\frac{t}{2}(\delta_{(a,b)}+\delta_{(b,a)}).$$ Assumption 4.5 then implies $$|\nabla\nabla_\mu F(\mu,a)-\nabla\nabla_\mu F(\mu,b)| \le L_M\sqrt{t}|a-b|.$$ Letting $t\to0$ forces equality of the two Hessians. The same argument extends to general measures. This excludes many standard Wasserstein objectives, including simple potential energies with nonconstant spatial Hessians, and apparently the regularized Coulomb MMD used in the main experiments. The examples provided to justify the assumptions essentially reduce to optimization over the mean of the distribution, with a vanishing multiplication Hessian. This raises serious doubts about the applicability and significance of the main theoretical guarantees. **2. Limited practical motivation and computational guarantees.** The motivation for using a computationally expensive second-order method is not sufficiently convincing. First, the complexity analysis counts exact Wasserstein transport steps without accounting for the cost of Hessian preconditioning or its numerical approximation. Consequently, the claimed improvement does not establish an actual computational advantage over first-order methods. Second, although the saddle-dependent contribution improves from $\widetilde O(\delta^{-4})$ to $\widetilde O(\delta^{-3})$, the total bound remains dominated by $\widetilde O(\varepsilon^{-2})$. Under the required regime $\varepsilon\lesssim\delta^2$, the leading complexity remains $\widetilde O(\delta^{-4})$. This substantially limits the significance of the improvement emphasized in the abstract. The numerical experiments also provide mixed evidence regarding wall-clock performance. Overall, it remains unclear when or why WSFN should be preferred to simpler first-order approaches. **3. Presentation** The manuscript is particularly lengthy and difficult to follow. Given that the underlying saddle-free Newton principle and much of the spectral analysis are already well established, the presentation could more clearly distinguish genuinely new Wasserstein-specific contributions from adaptations of existing theory. Moreover, I question whether the manuscript, in its current form, is well suited to a conference submission. Since the contributions are primarily theoretical, a careful verification of the mathematical arguments is particularly important. However, given the extensive appendices and the limited reviewing period, it seems difficult for reviewers to thoroughly assess the correctness of all the proofs. This raises concerns about whether the manuscript can receive the level of mathematical scrutiny it requires within the conference review process.Questions for authors
Q1. Is the implication identified above for Assumption 4.5 correct? If so, can the authors provide meaningful Wasserstein optimization problems with nonconstant spatial curvature satisfying the assumptions, and clarify whether the theory applies to their numerical examples?
Q2. Can the authors provide an end-to-end computational complexity analysis accounting for the Hessian preconditioning step, or otherwise demonstrate a convincing computational advantage over first-order Wasserstein methods?
3 comments
Complexity and computational evidence (Weakness 2 and Question 2)
We agree that counting exact Wasserstein transport steps does not provide an end-to-end computational advantage, and that the large gradient contribution still determines the leading order under . This type of condition is typical for saddle escape analysis (see Theorem 5.2 in Yamamoto et al. (2025) and the observation they make in footnote 2).
Thus our improvement is in the cost of saddle episodes. With and a stepsize independent of , WSFN reduces the number of transport steps per episode from to , while keeping an objective decrease of order .
Appendix I describes this fact. Proposition I.1 shows that the cubic available objective decrease scale is sharp, not that is an iteration complexity lower bound. Proposition I.2 constructs examples requiring steps in our large gradient phase under its stated stepsize restriction, with uniform regularity constants and a bounded initial objective gap. Thus this contribution cannot be uniformly improved for that phase under the present assumptions.
Note, however, that this is not an impossibility statement about other algorithms. As discussed in Remark 4.7, cubic regularization may provide a route to a total bound of transport steps. This approach would define each update through a regularized optimization subproblem, instead of the explicit update (1), and would require addressing how that subproblem is solved. Proving such a guarantee in Wasserstein space remains a challenging direction for future work.
To test the saddle escape mechanism directly, the new Appendix C.4 and Figure 6 vary the negative curvature while keeping a positive curvature direction fixed. Across WSFN-PWGF paired trials, the two transports receive identical Gaussian perturbations and objective decrease thresholds. As decreases, median WSFN acceptance times increase very slightly, whereas the matched PWGF crossing times increase very sharply (Panel A). Controls with fixed or a stepsize proportional to recover the slowdown observed for PWGF. These results support the saddle episode mechanism we proved theoretically. However, they do not establish a global complexity bound or a wall-clock advantage.
The revised Section 5 also uses an exact representation for applying the full Hessian preconditioner to the gradient. It reports earlier WSFN progress in updates and favorable elapsed time behavior in this particular model. This identifies one setting in which preconditioning is inexpensive, but it is not evidence of general computational superiority.
The new experiments provide evidence of faster saddle escape in a controlled setting (Appendix C.4) and shorter elapsed times to reach small losses in the example of Section 5. These results motivate investigating conditions under which WSFN offers a provable computational advantage. Establishing an end-to-end complexity theorem that accounts for particle approximation, preconditioner application costs, and the accuracy required to preserve our convergence guarantees remains an important direction for future work.
Presentation and Wasserstein contribution (Weakness 3)
We agree that the central argument should be assessable without navigating an extensive appendix. Thus we have removed several background appendices, which do not affect the main paper, and reorganized the remaining material, reducing the manuscript from 107 to 99 pages while adding new assumption verifications and targeted experiments. The essential reading route is now: the operator estimates in Section 3.2 and Lemmas G.8, G.11, G.12, the localization and escape argument in Lemma H.3 and Propositions H.5 and H.7, and the iteration count in Theorem 4.6. Appendix H provides a proof roadmap.
The saddle-free Newton principle, the Gaussian perturbation construction, and the spectral calculus tools are established ingredients but we would like to highlight two important points.
(1) The saddle-free Newton principle is from Dauphin et al. (2014), who derive the method in finite dimensions and demonstrate its numerical benefits. Our contribution is to formulate and analyze regularized saddle-free Newton dynamics on Wasserstein space, where applying the inverse absolute Hessian requires additional operator level control.
Although is well defined, it need not have a bounded inverse. Moreover, scalar Lipschitz continuity of the absolute value function does not imply operator Lipschitz continuity of . This matters because the escape analysis must compare preconditioners along nonlinear transport trajectories, instead of only characterizing a fixed Hessian.
Therefore, we use , which is bounded for every , and establish -dependent stability estimates and a controlled expansion of the preconditioned gradient on a common reference space (Lemmas G.11-G.12). These estimates support the nonlinear saddle escape guarantee. The distinction from ideal saddle-free Newton is summarized above Table 1 and developed in Appendix G.4.
(2) Although spectral tools are classical, their application here characterizes the local behavior of regularized second-order Wasserstein dynamics without assuming a uniform positive Hessian gap. This analysis explains which convergence rates can be expected from WSFN and why its second-order construction alone does not imply classical quadratic Newton convergence. Classical quadratic Newton guarantees rely on suitable regularity and a boundedly invertible local Hessian. In our setting, at an absolutely continuous minimizer, is compact on an infinite-dimensional space, so bounded invertibility on the full space is unavailable. Moreover, fixed-regularization WSFN has a different linearization from an exact Newton step as Table 1 shows.
Theorem 4.9 shows that the linearized iterates converge strongly to the kernel projection, while their operator norm remains one. Kernel components are unchanged by the linearized dynamics. If there are infinitely many positive eigenvalues, they accumulate at zero and their contraction factors approach one, precluding a common geometric contraction factor across all positive modes. For a finite-rank Hessian, however, the positive spectral component does contract geometrically. The theorem accordingly provides geometric contraction on positive spectral bands bounded away from zero and polynomial decay under spectral source conditions. These conclusions identify how the infinite-dimensional Hessian geometry governs the effectiveness and limitations of curvature preconditioning.
Appendix G.6 further clarifies the connection with Newton’s method, including quadratic convergence under additional assumptions for a spectrally projected Newton iteration on a fixed positive spectral subspace. The rates in Theorem 4.9 concern the linearization and transferring them to nonlinear WSFN requires the additional structural control discussed in Remark 4.10.
We would appreciate a reassessment in light of these explicit assumption verifications, the targeted saddle escape evidence, and the clarified scope of the theoretical contribution.
Assumption 4.5 and applicability (Weakness 1 and Question 1)
We thank the reviewer for identifying an important restriction in Assumption 4.5 and for asking us to clarify the significance and scope of the results. The spatial constancy implication is correct. We address it directly below, together with the revised examples, additional experiments, and a clearer account of what the complexity result shows.
Because Assumption 4.5 is over every coupling, its first inequality forces a substantive restriction. Hence, an example with a nonconstant spatial multiplication Hessian cannot satisfy the assumption as stated. As suggested by the reviewer, we stated this implication explicitly in the revised paper.
Spatial constancy of Mμ does not imply convexity of the objective or eliminate the need for saddle escape. Indeed, under Assumption 4.2, Mμ=0 at critical measures in the relevant sublevel set, so Hμ=Kμ and any negative curvature at such points comes entirely from Kμ. The Gaussian Coulomb MMD example in Appendix B.1 explicitly exhibits this mechanism. Its nonglobal critical measures have a strictly negative eigenvalue of Kμ, while Mμ need not vanish away from criticality.
We would also like to highlight that these Hessian coupling conditions also appear in Yamamoto et al. (2025, Assumption 3, (13)-(14)): equation (14) is the same multiplication Hessian bound, and equation (13) is the corresponding integral kernel bound. Therefore, our improved saddle contribution is obtained without strengthening these Hessian coupling conditions relative to PWGF. This places the comparison within an existing regularity setting, while leaving the applicability restriction to be acknowledged explicitly.
Replacing the L2 transport displacement by its L∞ counterpart would allow spatially varying curvature, provided measure dependence is controlled as well. However, this would give operator norm continuity in L∞ displacement, whereas our localization and Gaussian perturbation estimates control L2 displacement. In particular, the Gaussian perturbation tool may fail. For instance, with μ=N(0,1) \begin{equation} \xi(x)=Zx,\quad Z\sim N(0,1), \end{equation} satisfies ∥ξ∥Lμ2=∣Z∣<∞, whereas ∥ξ∥Lμ∞=∞ almost surely. We also highlight the role of Lemma D.5, established by Kim and Suzuki (2024), which applies to L2-valued Gaussian random elements. Therefore, a direct assumption substitution in the present proof would require both different perturbation tools and analysis. Relaxing these conditions remains an important direction.
The revision demonstrates that this restricted class nevertheless contains nonconvex problems with a nonzero multiplication Hessian and dependence on more than the mean. Appendix B.1 considers Coulomb MMD between Gaussian laws whose mean and variance depend on a latent law μ. In particular, Mμ is spatially constant but need not vanish. The model has strict saddles because at a nonglobal critical point, the transport field is a negative curvature eigenvector (Proposition B.12). Proposition B.9 verifies Assumptions 4.4, 4.5 globally, while Propositions B.11-B.14 verify gradient and Hessian control, and benignity on sublevels with confined variance.
In the revised manuscript, Section 5 presents the Gaussian Coulomb MMD experiment analyzed in Appendix B.1, replacing the original Coulomb MMD experiment. We verify Assumptions 4.2–4.5 for this model, but also for the matrix decomposition example in Appendix B.4, whose numerical results are presented in Appendix C.2. For both examples, the reported comparisons show that WSFN’s reduction in transport updates is accompanied by shorter elapsed times to reach small losses.