acceptodds
Under review as a conference paper at ICLR 2027

Speeding Up Nonsmooth Bayesian MCMC Sampling via Inexact Proximal Unadjusted Langevin Algorithm

Abstract

We study sampling from posterior distributions with nonsmooth composite potentials, a setting in which proximal-based Langevin methods are theoretically appealing but in practice limited to simple functions with closed-form proximal operators. We introduce iPULA for composite potentials, an inexact proximal unadjusted Langevin algorithm that replaces exact proximal steps with controlled approximations. Our approach leverages the Moreau envelope to smooth the potential, while allowing inexact evaluation of its gradient through inexact proximal computations. We establish non-asymptotic convergence guarantees for iPULA, explicitly characterizing the impact of inexactness on the sampling error and showing that the inexactness preserves convergence rates up to a quantifiable bias. We demonstrate the practical relevance of iPULA on a medical image reconstruction task, where proximal operators cannot be computed exactly. Experiments demonstrate the effectiveness of iPULA and support our theoretical results.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Nonsmooth Bayesian MCMC, Inexact Proximal Method, Unadjusted Langevin Algorithm

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Hjg4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies sampling from posteriors with nonsmooth convex composite potentials U=f+gU=f+g. It applies Moreau smoothing to the full potential and proposes iPULA, which replaces exact proximal evaluations in the resulting Langevin update with controlled approximations. The main result gives a non-asymptotic KL bound for averaged iterate laws with biased, potentially history-dependent errors. Further results concern objective gaps under a Polyak–Łojasiewicz condition and final-iterate Wasserstein bounds under strong convexity. Experiments consider Gaussian MRI deblurring and Poisson fluorescence-microscopy deconvolution with total variation regularization.

Critical strengths
  1. Analysis accommodating biased and history-dependent errors. The general-convex KL analysis addresses an error model relevant to iterative proximal solvers without requiring unbiasedness. The explicit dependence on the error sequence, and the square-summability result, provide useful information about how inner approximation errors propagate through the outer sampling procedure.

  2. Computable certificates linking optimization and sampling accuracy. The residual-to-gradient and gap-to-gradient bounds translate available inner-solver certificates into control of the Moreau-gradient error. This is a useful practical component: it makes the theoretical accuracy requirement more actionable than a condition involving the unknown exact proximal point alone.

Critical weaknesses
  1. Insufficiently substantiated novelty claim for full-potential Moreau sampling. Section 1.2 claims priority for combining an inexact full-objective oracle with Moreau-based ULA. However, Diffusion at Absolute Zero: Langevin Sampling Using Successive Moreau Envelopes already explicitly considers full-potential smoothing and uses iterative proximal computations in its experiments. Proximal-Based Generative Modeling for Bayesian Inverse Problems also studies joint-posterior proximal approximations and analyzes splitting error. These connections are not discussed.

  2. Inaccurate and incomplete positioning against proximal-MCMC theory. Section 1.1 describes Ehrhardt et al. (2024) as replacing gg with its Moreau envelope while retaining ff. Their analyzed inexact PGLA instead uses a forward–backward update with Gaussian noise inside the proximal step, rather than the described MYULA construction. The paper also omits From stability of Langevin diffusion to convergence of proximal MCMC for non-log-concave sampling, which establishes ULA stability to drift approximations and PSGLA convergence for nonconvex potentials under strong convexity at infinity. A precise comparison of update rules, approximation models, assumptions, and convergence metrics is needed to assess the theoretical increment.

  3. Restricted theoretical scope and an unresolved link to the implemented inner solver. All main results retain convexity, limiting their applicability to nonconvex posteriors; a discussion of weakly convex extensions would clarify this scope. More importantly, the different mean-square, conditional-mean-square, and almost-sure oracle conditions are not connected to the experimental implementation. Although residual and gap certificates are provided, the actual tolerance schedules, stopping rules, and inner iteration costs are insufficiently specified. The authors should identify which guarantees apply to the experiments and explain how the corresponding conditions are enforced.

  4. Unquantified original-target error and insufficient justification of acceleration. Equation (23) acknowledges TV(πγ,π)TV(\pi_\gamma,\pi), but leaves this smoothing bias unquantified. The analysis therefore does not provide a joint choice of smoothing parameter, step size, chain length, and inner tolerances guaranteeing prescribed accuracy for the original posterior. Moreover, the convergence results add perturbation terms to standard ULA guarantees without establishing an improved convergence order or lower total computational complexity.

  5. Limited empirical support for posterior accuracy and practical efficiency. The evaluation covers only two 128×128128\times128 TV imaging settings, including real-source microscopy images. The reported ESS improvements are not normalized by total runtime, while time to peak PSNR is not a sampling-efficiency measure. Comparisons with directly related inexact and accelerated proximal samplers, including ESS per second at matched accuracy are needed. Broader inverse-problem evaluations and diffusion-based comparisons could strengthen generality.

Questions for authors

See Critical Weaknesses

Reviewer_XgNROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on analysing an inexact Proximal Langevin algorithm. The convergence analysis is made in the convex setting (and under PL condition and strong convexity). The algorithm is shown to be faster than some existing baseline for posterior sampling.

Critical strengths

The question of accelerating Langevin MCMC for posterior sampling is major to make these methods tractable.

Critical weaknesses

Major :

  • All the paper results are included and weaker than the result of a previous published paper [1]. In [1] Theorem 4, the authors show for potential strongly convex at infinity (outside a ball) that the inexact proximal Langevin algorithm converges to the target law up to the discretization error and to the proximal approximation (that is not assume to be zero at infinity). The only difference that I see between [1] and the current paper is the metric that is Wasserstein distance in [1] and Kullback-Leibler in this paper.
  • Line 104-107 and Line 135-139 : The paper claims that it is more general than Ehrhardt et al., but the iPULA is a particular case of the Langevin dynamic study by Ehrhardt et al. (with F=0F = 0). The convex study made in this paper is a particular case of the study of Ehrhardt et al.
  • Theorem 3.1 implies only the convergence to the target law if the approximations of the proximal operator are exact at infinity, this is very restrictive and not mentioned in the paper.
  • The composite structure of the potential is not used in the method. The paper should be written outside this restrictive composite setting.

Minor :

  • The paper needs to be re-written. In particular, the two first paragraph of the introduction are very repetitive.
  • Line 38-42 : A random-walk and ULA are not solving the same sampling problem. Moreover, the ULA is not faster than a random walk. This idea should be precise.
  • Line 51 : The curvature information is not used in ULA where only the first-order information on the potential is used.
  • Line 71 : The proximal version of ULA have not been proved to be faster than standard ULA, could you give references for this claim ?

[1] Renaud, M., De Bortoli, V., Leclaire, A., & Papadakis, N. (2026). From stability of Langevin diffusion to convergence of proximal MCMC for non-log-concave sampling. Advances in Neural Information Processing Systems, 38, 115709-115773.

Questions for authors
  • Could you explain the novelty of this paper with existing litterature, especially Renaud et al. and Ehrhardt et al. ?
  • Could you extend your results outside the convex case ?
Reviewer_rndwOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The authors introduce an unadjusted Langevin algorithm that targets the Moreau envelope of a nonsmooth convex composite potential and computes its proximal map approximately at each step, and give computable sufficient conditions on the resulting gradient error. They compare against the Moreau-Yosida unadjusted Langevin algorithm and two subgradient schemes on 128x128 total-variation deblurring.

Critical strengths

I appreciate that the authors bound the gradient error by a computable residual or duality gap, which could make the inexact oracle verifiable in practice. The analysis also accounts for biased, history-dependent errors.

Critical weaknesses
  • I disagree with the authors that proximal Langevin methods are in practice limited to closed-form proximal maps, since the Moreau-Yosida algorithm has been used with total-variation priors whose proximal map is computed iteratively.

  • While the authors argue that enveloping the full potential preserves the mode, the relevant quantity for a sampler is the bias of its target distribution, which they do not compare. In particular, the paper's own example the Moreau-Yosida algorithm reaches nearly the same image quality as the proposed method.

  • The title claims a speed-up and the introduction presents smoothing-based Langevin methods as scalable, but the experiments do not show either. The only timing reported is the time to the best image, selected with the ground truth, and the Moreau-Yosida algorithm is faster on this measure in both experiments.

  • The theory does not apply to the experiments. The asymptotic Kullback-Leibler floor of the main theorem grows linearly with the dimension, so the bound is uninformative.

  • The authors do not use the samples for uncertainty quantification. The experiments evaluate only two point estimates, the best iterate and the posterior mean, by image-quality metrics.

Questions for authors
  • How does the proposed method compare to the Moreau-Yosida algorithm in effective sample size per second, including the cost of the inner proximal solves?

  • Could the authors report pixel-wise credible intervals, validated against a long reference chain?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.