acceptodds
Under review as a conference paper at ICLR 2027

Learning to Sample from Multimodal Distributions

Abstract

Markov chain Monte Carlo (MCMC) is an established method for sampling from complex probability distributions by building Markov chains whose state transitions occur by a proposal-acceptance mechanism. MCMC however tends to fail on distributions whose modes are separated by wide low-probability regions. Neural networks have been integrated into MCMC to try and address this challenge, but existing approaches cannot guarantee that the resulting chain asymptotically produces samples from the target distribution, or even that they cover all modes. In this work, we use constrained optimization to put forward a new way of training neural MCMC that reduces samples correlation, ensures the coverage of all modes, and lower bounds the proposal acceptance rate. We prove that, under mild conditions, any solution of this constrained problem results in an ergodic Markov chain whose empirical measure converges to the target distribution. We then propose an effective algorithm to tackle this problem and illustrate its performance on a variety of applications from mixture models to physical systems, and reliability engineering showing that it matches or outperforms existing baselines.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Sampling, Markov Chain Monte Carlo, Neural MCMC, Probabilistic Inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_2TuFOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Constrained Involutive MCMC (CI-MCMC), which trains neural MCMC proposals with constraints on mode coverage, local fidelity within each mode, and acceptance. It provides a conditional uniform ergodicity guarantee, and empirically evaluates CI-MCMC on synthetic datasets and multiple applications.

Critical strengths
  • The constraints are well-motivated to address complementary sampling failures and have a clear connection to the convergence bound.
  • Empirical evaluation metrics provide meaningful evidence of improved cross-mode mixing across diverse tasks.
Critical weaknesses
  • The setup of numerical experiments seems not to satisfy the assumptions in the theoretical results. For example, Appendix B.6 assumes δ=0.25\delta=0.25, which results in ρ≥2(1−δ)=1.5\rho \ge 2(1-\delta) = 1.5 in equation 12 and thus contradicts with the assumption ρ<1\rho < 1 in Theorem 1. Furthermore, Appendix B.6.1 reports only 4.7% acceptance, which also violates the required acceptance rate δ=0.25\delta=0.25.
  • There is no ablation study on the effect of each component in equation 14.
Questions for authors
  • Can you explain the mismatch between the theoretical assumptions and numerical results?
Reviewer_nBGpOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a Markov Chain Monte Carlo (MCMC) sampler, similar to Hamiltonian Monte Carlo (HMC), that uses a neural network to model the transitions (to generate proposals). The network is trained to make large transitions while constrained by factors such as Fisher divergence between model and true distribution, the KL divergence between model and a reference distribution (that covers the support of true distribution), and acceptance rate. This constrained optimisation problem is solved by a primal-dual optimisation scheme. Experiments are performed on mixture models, lattice field theory, failure-state sampling, and conditioning generative models, where the proposed method outperforms some state of the art samplers.

Critical strengths

The presentation is very clear and easy to follow, while being rigorous at the same time.

The use of ESJD and other terms to construct a differentiable loss function is impressive.

Experiments are conducted across diverse problems to demonstrate the effectiveness of the proposed method.

The method does not depend on the availability of true samples, making it useful for practical problems where such samples are unavailable.

Critical weaknesses

Scalability is an issue. High-dimensional systems may pose computational challenges for this method. The current experiments are done in very low dimensions (<=64). The involutive network needs to generate latent variables as well, further increasing the computational load.

Reliably sampling all modes is still challenging. The requirement of knowing a reference distribution (made of components centred at samples that estimate the modes and intended to cover the support of the target distribution) may be difficult to fulfil in many practical problems. If the initial samples miss a mode, it may not be possible to recover it later.

The experimental results do not include a discussion about computational costs during both training and inference.

These issues limit the utility of the proposed method for many high-dimensional problems.

Questions for authors

I have pointed out some major issues in the Weaknesses section. Please address them.

There are frequent typos in the paper, which could be easily fixed. E.g., line 129: optimization, line 162: setting since, line 172: not not.

Regarding evaluation: NLL and R-NLL are good metrics to evaluate mode collapse and mode coverage behaviour [FUND: Density Flow for Sampling Unnormalised Distributions, TMLR 2026], and could be included in the results; likewise, ESS could be reported. These are the standard metrics commonly reported by papers on this problem.

Reviewer_Y5vcOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The method combines an expected squared jump-distance objective with Fisher-divergence, reference-based KL, and acceptance constraints, implemented through an involutive neural proposal and primal–dual optimization. The manuscript then goes on to provide some synthetic experiments together with a Stanford drone dataset experiment. It then provides theoretical guarantees with hard-to-satisfy assumptions.

Critical strengths

The paper tries to establish both experimental evidence and theoretical guarantees for the proposed method. The overall approach will provide proofpoints and better understanding, if done correctly.

Critical weaknesses

The manuscript does not convincingly establish a substantial methodological advance beyond integrating existing ingredients. See the following discussions on experiments and theory for details. The experimental evidence is limited in scope. The claimed theoretical contribution is also limited.

Questions for authors

See the review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.