Improving Multimodal MCMC with Learned Proposal Mixtures

Local Markov chain Monte Carlo (MCMC) samplers can remain trapped in one mode. We train a shared normalizing flow to propose states from different regions across targets that share the state's conditional distribution given a known statistic. We study a reversible rule that controls how often the sampler switches proposal components, with a Metropolis–Hastings (MH) correction. More frequent switching can help cross modes, but can also increase rejection. For non-atomic targets, we identify when switching cannot increase covariance between successive values of any square-integrable observable. This holds exactly when the target-to-mixture density ratio has the same distribution under every component. Acceptance is then unchanged, and long-run estimation variance cannot increase. We also bound the additional variance of bounded observables caused by component and weight errors, without assuming a spectral gap. Across four asymmetric inverse designs at $d=200$, proposal mixtures double the per-cycle efficiency of matched target-only flows for estimating mode probabilities; switching adds a further $13\%$. On nonzero-field lattices, mixtures pass all 40 mixing checks, compared with 8/40 for target-only flows. Corrected flips and parallel tempering remain more efficient. Code is available at https://github.com/Alphachain123/Improving_multimodal_MCMC_with_learned_proposal_mixtures.