acceptodds
Under review as a conference paper at ICLR 2027

A Note on Wasserstein-2 Error of Langevin Monte Carlo Under Strong Log-Concavity

Abstract

This note refines a recent nonasymptotic bound on the Wasserstein error of Langevin Monte Carlo for strongly log-concave target distributions. The refinement preserves the linear dependence on the step size while accounting more precisely for the spectrum of the Hessian. For a representative spiked Gaussian model, the resulting bound improves the dimension dependence of the previous linear-in-step-size estimate from order to order .

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Langevin Monte Carlo, Sampling guarantees

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ePCTOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper refines an existing bound on the Wasserstein error of LMC for strongly log-concave target distribution. It preserves the linear dependence on the step size, but it also considers the spectrum of the Hessian. For the example of spiked Gaussian model, the bound has a clear improvement.

Critical strengths

I do not find any error in the proof, and the final bound has an improvement over prior work.

Critical weaknesses
  1. The technical novelty of the paper is limited.

  2. A comprehensive discussion of the literature has not been included, thus it’s unclear whether the improvement is significant or not.

  3. The insight of the paper is limited. It only provides the proof, rather than the insightful reasoning behind the mathematical derivations.

  4. (Minor) The writing style of the paper (especially the introduction) is more like a note, rather than an academic paper.

Questions for authors

See weaknesses

3 comments
AuthorsOpenReview ↗

Answer to the review

Thank you for reading and evaluating my paper. As noted in my general response above, the main criticisms in your review largely stem from the stylistic choices I deliberately made for this manuscript. Below are some additional responses.

  1. Discussion of prior work: I believe that all relevant work that may help contextualize our result is cited and discussed. There are hundreds of papers on Langevin-related sampling algorithms, and it would be impossible to be exhaustive. If some papers are not cited, it is because we found that mentioning them was of limited relevance. As in the example of the paper by Freund et al. suggested by Reviewer 1oRw, discussed in my answer above, its main bound is looser than the bound in Durmus et al. (2019). Since that paper was published later and its main result directly relevant to our work is looser, I felt this was a valid reason not to feel obliged to cite it. That said, if you feel that some papers not currently discussed deserve discussion, I would be glad to add them in the revised version.

  2. Limited insight: The proof of the main result is so short and elegant that I felt it would be of limited use to provide a lengthy discussion of it. I wanted to keep the focus squarely on that elegant proof. I hope that many readers will draw useful insights from it.

I politely disagree with the authors on the response. As the ability of AI has been mentioned in the general response, I'd like also to share my opinions on the paper writing of AI era.

In my understanding, a good theoretical paper should have (not limited to) some good properties

  1. Solving a important or difficult research problem that the community cares about and is working on
  2. Inventing some new proof techniques that can be applied in many other problems.
  3. Providing insights for new problems or practice

Since the authors already claim that what is important is "human-written papers that offer clear, accessible, and easy-to-understand insights", this contradicts with the current sloppy version with only the "elegant" proof. Since AI can already generate lengthy research paper, proof is getting cheaper. You cannot write something randomly and "hope" readers can learn something from it.

Thus, it's necessary to show why the problem matters, and how you managed to prove the results. This does not require citing all of the "hundreds of papers on Langevin-related sampling", but the current version, even is not clear what is the literature, without a section of related work. Without those, it's even hard to understand why you study that upper bound.

Finally, I apologize for critizing on the "note" style. As I wrote, it is only a minor issue. A paper can have lots of writing styles, including what you want. However, the writing issues persist. Therefore, I decide to keep my score.

Thank you for your reply and for sharing your views on this important question: what makes a good article in the age of LLMs? I agree with the three criteria you mentioned for defining a good theoretical article, although I wouldn’t necessarily emphasise the difficulty aspect in the first criterion. These three criteria define a good article not only in the age of LLMs, but they were also valid before this.

As regards the sloppy version of my article and the fact that it contains random elements, I shall take your comments on board. I hope not all readers will share this view, but if I receive further comments of a similar nature, I shall rewrite the article. I have just reread the introduction to my article, and I believe it clearly explains why we are studying the upper bound. We have not devoted a separate section to the links with previous work, since they are detailed in the introduction and in Table 1.

Reviewer_bMmJOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proved an improved nonasymptotic bound on the Wasserstein error of Langevin Monte Carlo for strongly log-concave distributions. Paralle to existing work, the condition number used in bound proved in a recent work by Francesco Pedrotti & Peter Whalley has been replaced by an imrpoved average condition number ( the worst-case smoothness MM vs. average smoothness MavM_{\rm av}).

This improvement also allows the improvement of the bound the special spiked Gaussian distributions from $hp^{3/2} $ to hphp

The proof techniques follow closely some existing work but with a new reuslts that bounds the temporal increments of square-integrable observable of a stationary reversible Langevin diffusion, The lemma itself can be of use for other contexts.

Critical strengths
  1. Provide improved bounds for the Wasserstein error of Langevin Monte Carlo for strongly log-concave distributions. This is a genuine technical improvement over previous bounds under the same strong-convexity/smoothness setting.

  2. Key lemma proved during the process can be of independent interest: Lemma 1 controls temporal increments of observables of a stationary reversible Langevin diffusion through Dirichlet energy, without requiring strong convexity or a global smoothness constant. This may be useful for other discretization, regularity, or covariance analyses.

  3. The paper is self-contained: assumptions are stated, the key lemma is proved, the Gaussian calculation is included, and the reproducibility statement is appropriate for a theoretical paper.

Critical weaknesses
  1. I felt that the imrpovement seems to be incremental and may not fit for prime time at ICLR (the improvement is real but moderate): The improvement mainly comes from combining an existing step-size refinement with the new average-smoothness quantity. The core proof strategy follows prior coupling arguments; the main novelty is the refinement and the key lemma, not a completely new analysis framework.

(Disclaimer: I do not follow closely works in this area, and my assessment of the significance of the improved results should take this ignorance into account. )

  1. No empirical or numerical illustration. This is O.K. for a theoretical note, but it means the practical tightness of the bound, especially the role of MavM_{\rm av} is not illustrated.

  2. Limited discussion of algorithmic implications. The paper does not translate the improved bound into practical step-size selection, warm-start, or complexity guarantees for downstream sampling tasks.

Questions for authors

N/A

2 comments
AuthorsOpenReview ↗

Answer to weaknesses 2 and 3

  1. Regarding the absence of empirical results. This is a theoretical note. The main interest is the dependence of the bound on dimension and Hessian curvature, which is difficult to illustrate numerically in high dimensions because accurately evaluating W2W_2 is computationally intractable. I will clarify this point and retain the analytical Gaussian example, where the Wasserstein error is explicit.

  2. Regarding algorithmic implications. The main message is that vanilla LMC adapts to average curvature. I believe that this message is stronger than it can seem at first sight. This suggests that alternative methods designed to address worst-case conditioning when average curvature is much smaller may be of limited interest.

I read the response of the authors, and I maintain my rating

Reviewer_1oRwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper refines the Wasserstein-2 discretization bound for Langevin Monte Carlo from (O(h\sqrt{\kappa Mp})) to (O(h\sqrt{\kappa_{\mathrm{av}}Mp})), where (\kappa_{\mathrm{av}}=\mathbb{E}_{\pi}[\operatorname{tr}(\nabla^2f)]/(pm)). The proof combines the martingale decomposition of Pedrotti and Whalley [2] with a reversible-diffusion increment estimate, preserving linear step-size dependence under standard strong-convexity and smoothness assumptions.

Critical strengths
  • The average-curvature refinement is meaningful for high-dimensional statistical targets with heterogeneous curvature, where worst-case smoothness can be overly pessimistic.
  • The result requires no Lipschitz-Hessian assumption. The curvature parameter is interpretable and invariant under orthogonal transformations, and the theoretical presentation is clear.
Critical weaknesses
  • Incomplete positioning relative to prior work. Trace-based curvature improvements have precedents. Freund, Ma, and Zhang [1] established dimension-independent guarantees for structured sampling problems with Gaussian priors, including non-Gaussian targets. Although their algorithms and assumptions differ, a comparison is needed to clarify the contribution specific to standard LMC.

  • Limited technical novelty. The martingale decomposition and the framework yielding linear step-size dependence come from [2]. Lemma 1 follows from a standard reversible-semigroup inequality, and the proof in [2] already retains the expected Hessian trace before bounding it by MpMp. The main technical increment therefore appears to be a refined gradient-increment estimate within an existing proof framework.

  • Limited development of the contribution. The only detailed example is Gaussian, where the new bound is O(hp)O(hp), while the asymptotic bias is of order hph\sqrt p. A non-Gaussian example with Mav≪MM_{\mathrm{av}}\ll M, together with an iteration-complexity comparison, would strengthen the result’s significance. Currently, the contribution seems closer to a focused theoretical note than a fully developed conference paper.

References

[1] Yoav Freund, Yi-An Ma, and Tong Zhang. When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpoint. JMLR, 2022.

[2] Francesco Pedrotti and Peter A. Whalley. Wasserstein Mixing Time of the Unadjusted Langevin Algorithm. 2026.

Questions for authors

How does the proposed guarantee compare with the trace-based bounds in [1] on overlapping classes of non-Gaussian targets? What additional argument, beyond the framework of [2], is essential to obtain the improved curvature dependence?

2 comments
AuthorsOpenReview ↗

Answer to the weakness: Incomplete positioning relative to prior work

We thank the reviewer for pointing out this relevant work. We agree that Freund, Ma, and Zhang should be included in the discussion, and we will add an explicit comparison. Their work provides important precedent for trace-sensitive convergence guarantees, and their prior-diffusion algorithm is closely related to LMC. As explained below, our result is both sharper and holds under weaker conditions. In addition , the result in Freund et al. is weaker than the one obtained in Durmus et al. 2019 that we discussed in our manuscript The discussion below will be added to the revised version of the manuscript.

The comparison also clarifies the improvement provided by our result in Wasserstein-2 distance. Freund et al.’s Theorem 12 establishes a KL-divergence bound for a weighted mixture of iterate distributions. Writing TT for the iteration count (which is roughly equivalent to 1/(mh)1/(mh) in LMC) and taking temperature equal to one, its leading term is

\lesssim h \left( \frac{L\operatorname{tr}H}{m} \right),$$ with an additional term of order $h^2$. Strong log-concavity then implies $$W_2(\overline\nu_T,\pi) \lesssim \left[ \frac{hL\operatorname{tr}H}{m^2} \right]^{1/2}.$$ To compare the curvature dependence, let us first regard their $L$ and $\operatorname{tr}H$ as comparable to our $M$ and $dM_{\mathrm{av}}$. Even under this identification, the resulting leading Wasserstein bound is (a consequence of the transportation-cost inequality) $$W_2(\overline\nu_T,\pi) \lesssim \sqrt{\frac{MM_{\mathrm{av}}}{Tm^2}}\sqrt{\frac{d}{m}} = \sqrt{\frac{ \kappa \kappa_{\mathrm{av}} }{T}}\sqrt{\frac{d}{m}} ,$$ apart from the initialization-dependent term. By comparison, our Theorem 1 gives (assuming the relation $T=C/(mh)$) $$W_2(\nu_T^{LMC},\pi) \le (1-mh)^T W_2(\nu_0,\pi) +\frac{3.6C\sqrt{MM_{\mathrm{av}}}}{Tm}\sqrt{\frac{d}{m}} = (1-mh)^T W_2(\nu_0,\pi) + 3.6C\frac{\sqrt{\kappa \kappa_{\mathrm{av}} }}{T}\sqrt{\frac{d}{m}} .$$ Choosing the constant step size for a prescribed accuracy therefore improves the accuracy dependence from $\kappa \kappa_{\mathrm{av}} /\varepsilon^{2}$ to $\sqrt{\kappa \kappa_{\mathrm{av}} }/\varepsilon$, up to logarithmic factors and the step-size restriction. The distinction is particularly transparent for our spiked Gaussian example, $U(x)=\frac12x^\top(I_d+\mathbf1_d\mathbf1_d^\top)x.$ For our theorem, $$m=1,\qquad M=d+1,\qquad dM_{\mathrm{av}}=2d,$$ giving a discretization term of order $hd$. In the notation used by Freund et al., we have $L=d$, $H=\mathbf1_d\mathbf1_d^\top$, $\operatorname{tr}H=d.$ Their theorem consequently implies $O(d^2\varepsilon^{-2})$ iterations, whereas our bound gives $\widetilde O(d\varepsilon^{-1})$, for initialization with logarithmically controlled contribution. As mentioned in the first paragraph, our condition on $\nabla^2 f$ is weaker than that in Freund et al in the sense that the trace of $H$ where $H$ satisfies $\nabla^2 f(x)\preceq H$ might be much larger than $d M_{av}$. The example is provided in the next reply. Finally, our improvement concerns the resulting Wasserstein guarantee. Freund et al. directly control KL divergence, which implies a Wasserstein bound under strong log-concavity. Our Wasserstein bound does not imply a corresponding KL bound. Their result therefore provides a guarantee in a stronger metric, and our result improves the available Wasserstein accuracy bound.
AuthorsOpenReview ↗

Answer to the weakness: Limited development of the contribution

The Gaussian example provided in the paper was given to show that even in that simple case, the improvement we get is substantial. Below is another example illustrating two features. First, the improvement over uniform-Hessian-based Wasserstein bounds is not limited to the Gaussian example: for this non-Gaussian, strongly log-concave target, the averaged curvature reduces the curvature-dependent term in our bound by a factor of order d\sqrt d. Second, it shows that MavM_{\mathrm{av}} can be much smaller than Tr⁡(H)/d\operatorname{Tr}(H)/d, the normalized uniform likelihood-Hessian trace appearing in Freund et al.’s bound.

Consider fd(x)=1+∥x∥2+12d∥x∥2,x∈Rd.f_d(x)=\sqrt{1+\|x\|^2}+\frac{1}{2d}\|x\|^2, \qquad x\in\mathbb R^d. This potential is 1/d1/d-strongly convex, smooth, and has a well-defined Gibbs distribution πd(x)∝e−fd(x)\pi_d(x)\propto e^{-f_d(x)}. The Hessian of its pseudo-Huber term is

\left(I_d-\frac{xx^\top}{1+\|x\|^2}\right),$$ which is bounded above by $I_d$, with equality at $x=0$. Hence $$\frac1d I_d\preceq\nabla^2 f_d(x) \preceq\left(1+\frac1d\right)I_d.$$ The smallest uniform matrix upper bound for the full Hessian is therefore $$H_{\mathrm{full}}=\left(1+\frac1d\right)I_d, \qquad \operatorname{Tr}(H_{\mathrm{full}})=d+1.$$ By contrast, the expected Hessian trace is bounded independently of dimension. For $X\sim\pi_d$, integration by parts gives $$\mathbb E_{\pi_d} \left[\operatorname{Tr}(\nabla^2 f_d(X))\right] = \mathbb E_{\pi_d} \left[\|\nabla f_d(X)\|^2\right].$$ Writing $r=\|X\|$, radial integration by parts also gives $$\mathbb E_{\pi_d}\left[ \frac{r^2}{\sqrt{1+r^2}}+\frac{r^2}{d} \right]=d,$$ and thus $\mathbb E_{\pi_d} [r^2]\le d^2$. Since $$\|\nabla f_d(X)\|^2 \le 2\frac{r^2}{1+r^2}+2\frac{r^2}{d^2},$$ we obtain $$\mathbb E_{\pi_d} \left[\operatorname{Tr}(\nabla^2 f_d(X))\right]\le4.$$ Therefore the uniform full-Hessian trace is $d+1$, whereas the target-averaged trace is at most $4$. In our notation, $M_{\mathrm{av}}\le4/d$. To compare this with Freund et al.’s curvature quantity, write $f_d=F+G$, where $$F(x)=\sqrt{1+\|x\|^2}, \qquad G(x)=\frac{1}{2d}\|x\|^2.$$ Here $G$ is the Gaussian-prior potential with curvature $m=1/d$, and the likelihood term $F$ satisfies $\nabla^2F(x)\preceq I_d$. Thus the uniform likelihood-Hessian bound is $H=I_d$, giving $\operatorname{Tr}(H)/d=1$. In this example, $M_{\mathrm{av}}\le4/d$, so it is smaller than $\operatorname{Tr}(H)/d$ by a factor of order $d$. Since this curvature quantity enters our Wasserstein bound under a square root, this gives a factor of order $\sqrt d$ in the curvature-dependent term.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Framing and Philosophy of the Submission

I thank the reviewers for their thoughtful feedback on my paper. Despite the critical evaluation, I appreciate that the reviewers recognized the value in several aspects of the work.

Before addressing the specific points raised in the reviews, I would like to briefly reflect on the framing of this submission. While I agree that the technical contribution can be viewed as incremental, I strongly echo Reviewer bMmJ’s observation that the improvement is real. For instance, even in the basic case of a simple Gaussian target distribution, our result yields a significant improvement on the order of the square root of the dimension O(d)\mathcal{O}(\sqrt{d}). At the same time, I think we can all agree that some papers published in ML conferences feature far more involved technical machinery while delivering more modest improvements.

In the current era of AI, where LLMs can generate dense, overly complex manuscripts with hundred-page proofs that often yield minimal practical advancement, I believe there is growing value in concise, human-written papers that offer clear, accessible, and easy-to-understand insights.

This philosophy is the core motivation behind my submission, which is why I deliberately framed it as a "Note." As a frequent participant in machine learning conferences, I recognize that this format breaks from recent convention, and it would likely be difficult to find a similar short note accepted over the past five years. However, I believe now is the right moment for the community to rethink how we write and evaluate research, placing greater value on concise papers that deliver clear, meaningful improvements.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.