- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies minimizing V-energy + entropy H(\mu) over Wasserstein space, subject to a linear moment constraint. It proposes W-LALM, a Wasserstein linearized augmented Lagrangian method. Each iteration consists of 3 steps:
- an explicit pushforward step on V + linearized multiplier and penalty term;
- a JKO step on H alone;
- a dual ascent on multipliers.
The theory has 4 parts:
- a plan-based Lyapunov descent inequality (Theorem 4.1);
- convergence of the whole sequence to a saddle point in weak topology (Themrem 4.2)
- a characterization of saddle points by a plan-valued Wasserstein KKT system (Theorem 4.3)
- an iteration complexity for a plan-valued KKT residual (Theorem 4.4)
Experiments on two generalized-Bayes calibration problems demonstrate that W-LALM reaches the same objectives as constrained JKO baseline faster. The speedup comes from cheaper inner iteration, not fewer outer iterations.
Critical strengths
- Keeping the JKO step for H unchanged while moving the constraint into the explicit step is a good design (commonly used in ADMM for Euclidean).
- The analysis is careful and rigorous. Optimal maps need not exist, and the proof handles this throughout.
- the first order theory is useful. The plan-valued KKT system proved equivalent to the saddle-point condition gives a checkable stationary notion.
- Experimental results show the efficiency of the proposed method.
Critical weaknesses
- Once the glued plan is introduced, the Lyapunov proof is almost a line-by-line transcription of the Euclidean linearized ALM proof. In particular, some Wasserstein ingredients are established, such as the gluing lemma, JKP three-point inequality, the WPG analys pattern of Salim et al (2020). Thus, the paper would be stronger if it mentioned explicitly which steps of the proof requires new ideals/techniques.
- The linear-mean constraint is very special, because the constraint depends only on the mean, and the Lagrangian is unconstrained composite problem with a slightly modified potential. Would it be able to extend more general settings?
- The theory assumes exact JKO steps on continuous measures but the implementation differs in the following ways: 1) particle supports are fixed during JKO step with only reweighing, 2) the entropy is replaced by the discrete surrogate, 3) the weight problem is solved with entropic Sinkhorn which is inexact and regularized. These are not covered by the theory developed in this paper.
- The experiment evaluation is narrow. There are only two datasets, and fewer particles than dimensions, leading to very coarse representation of posterior.
- No study of how results depends on gamma, rho, beta or Sinkhorn regularization is shown.
- The only rate is for the KKT residual. For linearized ALM in Euclidean space, one would expect the rates for the objective gap and the feasibility violation.
- Strong convexity is not used. In Euclidean space, when V is strongly convex, linear convergence would be natural result.
Questions for authors
See the weaknesses.
3 comments
Response to Reviewer 67M3 (Part 2/2)
Q5 (Comment 5. Sensitivity to algorithm and solver parameters).
Study gamma, rho, beta, and Sinkhorn regularization.
A5. On exact-Gaussian Wine, we have tested 14 configurations with five repetitions each. All 70 runs meet feasibility , KKT residual , and the motion–feasibility stopping threshold .
- Step size . Increasing the admissible fraction from 0.25 to 0.99 reduces outer steps from 381 to 93.
- Penalty . At a common admissible step size, outer steps decrease from 298 at to 136 at both and .
- Entropy coefficient . All five tested values terminate in 93 steps in this instance. This is consistent with termination being governed mainly by the mean/multiplier dynamics here; it does not imply general insensitivity to .
The revised continuous experiments use no Sinkhorn regularization. Table 6 and Appendix D.2 report the sensitivity configurations and terminal residuals. Appendix D.5 describes the retained particle baselines separately, including their empirical moments and the weight entropy used by Proximal-ALM.
Q6 (Comment 6. Objective-gap and feasibility rates).
Provide objective and feasibility rates in addition to KKT-residual complexity.
A6. The existing Lyapunov estimate also yields ergodic objective and feasibility bounds. These can be stated for a recursively averaged measure under the current assumptions.
Fix a base measure , such as . Let . For , choose a generalized geodesic , with base , from to along which is convex, and set . Generalized-geodesic convexity ensures that this recursive construction has finite objective. The construction may depend on the admissible couplings chosen.
First moments interpolate affinely, and recursive convexity gives
Write . Summing the Lyapunov inequality yields
Meanwhile, the multiplier update and the Lyapunov bound give
With the constant used in Theorem C.13,
Combining recursive convexity with the saddle-gap estimate, and using the saddle inequality for the lower bound, gives
\Phi(\bar\mu_N)-\Phi(\mu^\star)&\ge-\Vert \lambda^\star\Vert \,\Vert \mathcal C(\bar\mu_N)\Vert ,\\ \Phi(\bar\mu_N)-\Phi(\mu^\star)&\le\frac{\widetilde E_0^\star}{2\gamma N}+\Vert \lambda^\star\Vert \,\Vert \mathcal C(\bar\mu_N)\Vert . \end{aligned}$$ Thus, $$\Vert \mathcal C(\bar\mu_N)\Vert =O(1/N),\qquad |\Phi(\bar\mu_N)-\Phi(\mu^\star)|=O(1/N).$$ These results are now stated in Theorem 4.5, with explicit constants and proof in Theorem C.13. Theorem C.13 additionally gives an $O(1/N)$ bound on the average squared constraint violation. These rates concern the averaged measure; they do not assert a general last-iterate $O(1/N)$ rate. In the Dirac specialization, the construction gives $\bar\mu_N=\delta_{N^{-1}\sum_{j=1}^N x_j}$, recovering the Euclidean Cesàro average. --- ***Q7 (Comment 7. The role of strong convexity).*** *Discuss whether strong convexity of V permits a linear rate.* **A7.** Our results establish primal-dual convergence and KKT complexity under convexity. A linear primal-dual rate may require additional regularity beyond strong convexity of $V$ alone. The Conclusion identifies faster rates as future work.Thank you for the rebuttal.
Thank the authors for clarification. I'd like keep my score. Good luck.
Response to Reviewer 67M3 (Part 1/2)
We thank the reviewer for recognizing the analysis, the KKT characterization, and the computational motivation. We respond to each of the seven concerns in the order of the review.
The questions below summarize the review; the corresponding original labels are retained for reference.
Q1 (Comment 1. New analysis beyond Euclidean linearized ALM).
Identify which proof steps require new work, given the established gluing lemma, JKO inequalities, and WPG analysis.
A1. We agree that gluing, Wasserstein proximal inequalities, and Euclidean ALM identities are established tools. The result here requires combining them for the proposed primal-dual update.
The forward displacement and JKO coupling are glued through a common intermediate measure, then combined with a comparison coupling in a common multi-marginal plan. This gives the Plan EVI adapted to the two-stage update. The ensuing estimates control the primal displacement together with consecutive moment residuals, allowing the Euclidean multiplier update to enter a corrected Lyapunov inequality. The full-sequence argument also uses the Wasserstein weak topology. Lemma C.7 states the Plan EVI, and Theorem C.8 establishes the corrected Lyapunov estimate used in the convergence proof. These identify the update-specific estimates separately from the standard ingredients.
Q2 (Comment 2. The special structure of linear mean constraints).
Explain whether the analysis extends to more general constraints.
A2. The current theory concerns AEμ[X]−b=0. Nonlinear moments give a position-dependent multiplier force, while variance, quantile, and inequality constraints require additional analysis. We have stated the scope of the current guarantees and identified nonlinear constraints as future work in the Conclusion.
Q3 (Comment 3. Exact theory versus the particle implementation).
Fixed support, discrete entropy, and inexact regularized Sinkhorn solves are outside the exact continuous theory.
A3. We agree that fixed support, weight entropy, Sinkhorn regularization, and finite inner accuracy are not covered by the original exact-law theorem. The revised experiments use two entropy-only regression settings and twelve Gaussian moment-model settings.
The entropy Gaussian update is available in closed form. For product-Gaussian input, the second-moment models admit an exact finite-dimensional continuous-JKO reformulation, solved to numerical tolerance. Appendix D.1 also verifies that the diagonal forward map and mean translations preserve product-Gaussianity throughout the iteration. We distinguish this exact reduction from exact numerical solution; the exact-law theorem does not itself provide an inexact-JKO convergence guarantee. Explicit or numerically certified references permit direct checks of means, covariances, W2 distance, feasibility, and KKT residuals. Appendix D.2, Eq. (79), gives the outer Gaussian residual, and Table 5 independently checks one constrained subproblem.
Q4 (Comment 4. Breadth of the experimental evaluation).
Two datasets and a limited particle representation provide narrow empirical evidence.
A4. We have revised the two entropy-only regression experiments and added twelve Gaussian moment-model settings, with continuous Gaussian representations and direct distributional accuracy checks. Table 4 reports the complete twelve-setting screening results, including cases where CWPG is faster. W-LALM has a median paired runtime advantage in eight of the twelve settings; we do not claim a universal advantage for coupled models.
Controlled constraint-count study. On NOAA East linear hinge, we fix d=2560, the objective, initial law, and algorithm parameters. Constraints are nested. CWPG uses adaptive Newton–CG throughout.
Times are medians of four per-instance timing medians. Ratios are medians of paired per-instance ratios; outer steps are ranges across instances. Each instance has one untimed warm-up and five timing repetitions. All 160 runs meet W2 error ≤10−5, feasibility ≤10−8, and outer KKT residual ≤10−5. Setup and reference computation are excluded from timing. These results locate the advantage in the cost of the coupled backward solve in this controlled study, despite CWPG taking fewer outer steps. Table 3 and Appendix D.6 report the results and protocol.