acceptodds
Under review as a conference paper at ICLR 2027

One-Step Sampling via MCMC Distillation

Abstract

Markov chain Monte Carlo (MCMC) enables sampling from unnormalized distributions, but obtaining well-mixed samples often requires many sequential transitions. We study MCMC distillation, which amortizes these iterative refinements into an implicit neural sampler that generates samples in a single forward pass. Starting from the current generator, we apply short MCMC refinements and train the generator to match the resulting distribution using sample-based discrepancies, without requiring target samples or evaluating the generator density. We establish a unified weak convergence framework based on a finite-window error decomposition that separates reference mixing, accumulated distillation error, and transition bias. We investigate sliced Wasserstein distance, MMD with Riesz and Matérn kernel, and Sinkhorn divergence as potential sample-based divergences in this theoretical framework. For unadjusted Langevin algorithm (ULA) teachers, we derive explicit sufficient conditions for weak convergence to the target and conditional rates for expectation errors. Experiments on synthetic and high-dimensional Bayesian inference tasks achieves comparable results to ULA teachers, demonstrating the effectiveness of repeated distillation across different sample-based discrepancies.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
MCMC distillation, implicit neural samplers, amortized inference, weak convergence

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ESBtOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies MCMC distillation for training an implicit neural sampler from an unnormalized target density. Training alternates between short MCMC refinements initialized from the current generator and fitting the resulting teacher distribution using sample-based discrepancies. The paper develops a conditional weak-convergence framework that separates reference mixing, fitting error, and transition bias, with specific results for Langevin teachers. Experiments evaluate MMD and sliced-Wasserstein fitting on four two-dimensional targets, a logistic-regression posterior, and a conditioned-diffusion posterior.

Critical strengths
  1. The approach avoids evaluating the generator density and produces independent samples in a single forward pass after training.

  2. The theoretical framework explicitly distinguishes fitting error from teacher transition bias. The paper also acknowledges the mismatch between its theoretical and practical step-size schedules.

Critical weaknesses
  1. Limited algorithmic novelty and no competing neural-sampler baselines. The refinement-and-fitting procedure follows established amortized MCMC [1], as the paper acknowledges. Although the convergence analysis is a distinct contribution, the experiments do not establish the practical value of the proposed fitting objectives relative to existing learned samplers. ULA is the only external sampling baseline; the MMD and sliced-Wasserstein variants are both instances of the proposed method. Relevant comparisons include the amortized-MCMC approach and kernel semi-implicit variational inference [2], both already cited by the submission. Diffusion-based methods for unnormalized targets, such as Denoising Diffusion Samplers [3], also provide relevant context.

  2. Limited experimental evidence for efficiency and scalability. The experiments cover four two-dimensional targets and two Bayesian posteriors, but provide no training-time, memory, sampling-throughput, or amortization measurements. Matching teacher transition counts does not account for generator optimization and discrepancy evaluation. Pairwise MMD scales quadratically with batch size, whereas sliced Wasserstein has projection and sorting costs; their practical trade-offs should be measured. Sinkhorn fitting is analyzed but not evaluated experimentally. Moreover, the conditioned-diffusion ELM diagnostic measures coordinatewise densities and explicitly omits temporal dependence (Appendix B.5), limiting the evidence for accurate joint sampling.

  3. Recovery of global mode weights is not demonstrated. Correct mode probabilities are essential for multimodal sampling, beyond locating modes and matching their local geometry [4]. If short teacher chains rarely move between modes, their output can retain incorrect mode proportions, which the generator is then trained to reproduce. Repeated refinement and the annealing used in some experiments may help, but the paper does not demonstrate that they recover correct weights within the training budget. The eight-Gaussian example uses equal weights and reports no explicit mode-weight errors. An unequal-weight mixture with controlled separation and deliberately imbalanced initialization would directly test this concern.

  4. The conditional guarantees do not establish the accuracy of practical training. Theorem 4.6 requires mixing and stability assumptions, vanishing ULA step sizes, and population fitting errors satisfying εn=o(hn)\varepsilon_n=o(h_n). The paper does not establish that its finite-batch optimization achieves this error condition, and its practical step-size schedules differ from those required by the theory. The result therefore does not resolve the finite-budget mixing and mode-weight concerns above.


[1] Yingzhen Li, Richard E. Turner, and Qiang Liu. Approximate Inference with Amortised MCMC. arXiv:1702.08343, 2017. Paper.

[2] Ziheng Cheng, Longlin Yu, Tianyu Xie, Shiyue Zhang, and Cheng Zhang. Kernel Semi-Implicit Variational Inference. ICML, 2024. Paper.

[3] Francisco Vargas, Will Grathwohl, and Arnaud Doucet. Denoising Diffusion Samplers. ICLR, 2023. Paper.

[4] Louis Grenioux, Maxence Noble, and Marylou Gabrié. Improving the Evaluation of Samplers on Multi-Modal Targets. ICLR, 2025. Paper.

Questions for authors
  1. What benefits do the proposed fitting objectives provide over closely related learned samplers when training cost and sampling accuracy are both considered?

  2. Can the method recover unequal mode weights from an imbalanced initialization, and how does this depend on mode separation and the teacher-refinement budget?

  3. What evidence connects the population fitting-error condition in Theorem 4.6 to the implemented finite-batch optimization and step-size schedules?

Reviewer_BT46OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a novel amortised sampling method for learning a one-step generator GθG_\theta. The proposed method minimises a divergence DD between the pushforward (Gθ)♯λ(G_\theta)_\sharp \lambda of the base measure λ\lambda and the result of its short-time evolution under a π\pi-invariant MCMC kernel (e.g. ULA). Here, π\pi denotes the target distribution, whose density is known only up to a normalising constant. The authors discuss several choices of DD, including MMD, the sliced Wasserstein distance, and the Sinkhorn divergence. The paper also provides a convergence analysis of the proposed method.

Critical strengths

Identified strengths

  1. The method enables one-step sampling from distributions, whose densities are known only up to a normalising constant. The proposed method does not require modelling of the log-density and allows for a different choice of an MCMC kernel and a divergence.

  2. establishes a convergence result for the algorithm, relating it to the convergence of the underlying MCMC dynamics (the proof is given for ULA)

  3. I appreciated the discussion of prior work, which includes Li et al., 2017, among others, and the connection to drifting models, as well as the mention of the concurrent work (Pauline et al., 2026). However, the discussion of related work could be improved, as I explain below.

Critical weaknesses

Presentation:

  • The paper lacks a discussion of alternative amortised sampling algorithms, examples of which include FAB (Midgley et al., 2022; can also be used as a one-step sampler), PIS (Zhang & Chen, 2021), DIS (Berner et al., 2022), ASBS (Liu et al., 2025), AS (Havens et al., 2025), and off-policy diffusion samplers (Sendera et al., 2024). These works represent alternative approaches to amortised sampling that are, in my opinion, worth discussing.

Experiments:

  • The numerical experiments in the paper are relatively low-dimensional. The 2D experiments are fairly simple, three of the four targets are unimodal. It is hard to judge the scalability of the proposed algorithm from the logistic regression and conditioned diffusion experiments alone. To strengthen the paper, I would suggest testing the algorithm on more challenging benchmarks, such as those in Blessing et al., 2024.
  • Currently, the paper does not compare against any baselines. The following experiments would strengthen the paper. The two most obvious are the comparison against ULA (with a matched number of function evaluations) and against rejection sampling. Second, it would be interesting to compare the algorithm to the baselines mentioned above (FAB, PIS, DIS, etc.). Finally, I suggest running a long MCMC chain with restarts, gathering its samples, and training a diffusion model, a flow matching model, or a GAN on the gathered samples. This would be a reasonable way to check that distilling MCMC offers any advantage at all.
  • There is a lack of discussion about the possibility of using other MCMC algorithms, e.g. MALA or HMC. ULA itself is biased, so amortising its iterations cannot yield a generator GθG_\theta whose pushforward targets π\pi exactly.

Theory:

  1. I would suggest improving the notation, as it is currently somewhat confusing. Here are some examples:
  • KK is defined in the Background section as a π\pi-invariant kernel, and Kk,nK_{k,n} as a product of the T~i\tilde T_i. Meanwhile, TT denotes the transition kernel of the algorithm, and T~\tilde T the idealised Langevin dynamics.” I would suggest using KK for kernels that are truly π\pi-invariant, and TT for transitioned induced by the algorithm.
  • μn+1\mu_{n+1} is not properly defined. The only explanation I managed to find in the manuscript was in line 215: "Fitting the teacher law ... return μn+1\mu_{n + 1} ... ". This seems to imply μn+1=arg⁡min⁡μD(μ,T♯μn)\mu_{n+1} = \arg\min_{\mu} D(\mu, T_\sharp \mu_n). I would kindly ask the authors to clarify this matter.
  1. The assumptions are given without a sufficient discussion. For instance, Assumption 4.2 is satisfied if π\pi is strongly log-concave or it satisfies log-Sobolev inequality. Assumption 4.4 is not discussed at all. What does it mean? Moreover, Assumption 4.4 is directly related to Term 3, which seems to control the error between the ideal Langevin dynamics and its discretisation. The proof of Theorem 4.5 claims that the errors go to zero. However, this does not seem clear to me, as it seems to imply that the proposed algorithm does not inherit the discretisation error of ULA.
  2. Equation 25 in the appendix has the form ...+(A+...)ewA+...+...... + (A + ...) e^{wA + ...} + ..., and line 809 states that " ... letting A→∞A\to\infty proves convergence ... ". Yet, with w>0w>0, AewAAe^{wA} diverges as A→∞A\to\infty. This may just be a typo, but I would ask the authors to clarify.

Overall, I would strongly suggest revisiting the proofs and definition.

Questions for authors
  1. Why is the analysis restricted to ULA? It seems possible to consider MALA (which shares the convergence analysis with ULA), as well as other MCMC kernels.

  2. How does the algorithm scale with dimension? I suggest evaluating the algorithm on the benchmarks from Blessing et al. (2024).

  3. What is the main difference between Pauline et al. (2026) and your work?

Reviewer_f56POpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses an important problem and offers a useful theoretical treatment of error propagation in MCMC distillation, together with empirical evidence that implicit generators can approximate several target distributions.

Critical strengths
  1. The problem is important and well-motivated. Learning a reusable sampler for an unnormalized distribution is valuable for Bayesian inference and EBMs, especially when many independent samples are needed from the same target.
  2. The method avoids explicit generator density evaluation. Using MMD or SWD matching permits flexible implicit generators and only requires samples from the student and short MCMC transitions.
Critical weaknesses
  1. My understanding is that the student is trained through sample-based distribution matching and does not use the Markov transition structure beyond obtaining teacher samples. What is the role of MCMC in the proposed distillation framework?
  2. The proposed generator refinement procedure closely resembles earlier amortized MCMC and cooperative learning approaches, while its convergence proof relies on small projection errors and regularity assumptions. What new learning principle does this work establish beyond the existing projected MCMC formulation? Otherwise, the algorithmic novelty is limited. The central mechanism of initializing short MCMC chains from a generator and teaching the generator using the refined samples was already explored in amortized MCMC.
Questions for authors

See Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.