- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper propose a general MCMC framework to collect samples from multimodal distributions. Ideally, these samples are used for downstream inference tasks.(e.g. estimating for some function )
Motivated by linear inverse problem, this paper assume that the posterior distribution can be determined by some statistics of state , e.g. , where is the state, is the observation, is the prior, is some known function. Given this structure, the authors aim to resolve one hardness of the task: If the sampler is trapped within one mode, even though the estimation has low empirical variance, the inference could still be significantly biased; while if one propose to switch mode, two aspects may show up and be harmful: (a) variance increase; (b)high rejection rate.
To address this hardness, the authors propose the following procedure:
- Partition the state space in to components and introduce an auxiliary label variable.
- Train a black-box model (Normalizing flow) to approximate the conditional distribution given observation and component label .
- Use mixture of conditional distributions per component as proposal to run a MH sampler.
- Within the sampling process, they propose a whitebox routing rule to switch the component.
On the theory side, they proved that if a calibration condition holds, the proposed routing rule would not increase the lag-one covariance. They then give some case studies for this calibration assumption and discuss the robustness in the Appendix.
On the experiment side, they test their method's performance on Gaussian mixtures and nonzero-field lattices.
Critical strengths
The main novelty of this paper is sustained by their Theorem 1, where they show the lag-one covariance is not increasing if and only if the calibration condition holds.
Critical weaknesses
- Writing is unacceptable. The abstract and intro are either ai-generated or heavily ai-assisted. The ICLR 2027 policy did not forbid this but in this specific case, the readability of those parts is very restricted. I suggest the authors put more human effort to polish the writing and try to make the article readable. I would like to make two concrete examples to justify my statement:
(a). The core contribution of this paper is the analysis of the routing rule in Theorem 1. In the intro, the authors define and discuss the condition required for inverse problem or posterior in the very begining and followed the notation afterwards, which is unnecessary and redundant. The whole statement still holds for general multimodal distribution and all the discussion about posterior could be first deleted and push back to later sections if they would like to incorporate case studies.
(b). Their conclusions on hard routing and indepent mixing is straightforward and should serve as intuitive arguments instead of a notable contribution. Intuitively, the blocks discussed in this paper are disjoint so if the learned component is exact, a hard routing would never touch other components. The exact version could be relaxed, where small KL/chisq error implies small probability mass on support mismatch. The proof to quantify this and give downstream arguments are straightforward and standard. The authors should simplify their argument on this part and convey the essential idea. Ideally, the saved space could used to show more content about the calibration assumption.
- The contribution of this paper remains unclear and the applicability is somehow restricted, at least in this version. I would like to justify my arguments in the following aspects:
(a). Restricted to posterior sampling, the authors did not discuss and compare with the diffusion and other flow based sampler, which is also an important line of research e.g. [1-3]. In particular, [1,2] studied the linear inverse problem; [3] targets on more general distributions. The settings of these papers may be different with this one, but a proper dicussion would make the scope of this paper clearer. In particular, because of this, the applicability of this work to posterior sampling problem remains unclear: Given the empirical success of diffusion based samplers and the theoretical results/understandings of them, the author should justify the advantage or contribution of their framwork. I think the comparsion is fair and necessary as the author also trained a blackbox model to approximate the conditional distribution per component.
(b). The proposed framework in this paper requires learning components. However, the advantage of this approach remains unclear to me. Are there any principled way for us to partition the state space? Could we get more refined sample complexity bound by using this framework? How could we establish and test the calibration condition with different partition designs? Some of my questions are reflected in the experiments, but I did not see a well-orqganized discussion.
(c) The experiment is not convincing. The advantage of the proposed method is not clear compared with modified HMC on the lattice experiment. And the scale of the Gaussian experiment is rather restricted. Given the fact that linear inverse problem is extensively discussed in image generation field, experiments on real-world data would be much more convincing.
[1] Bruna J, Han J. Posterior sampling with denoising oracles via tilted transport[J]. arXiv preprint arXiv:2407.00745, 2024.
[2] Xun Z, Gupta S, Price E. Posterior sampling by combining diffusion models with annealed langevin dynamics[J]. Advances in Neural Information Processing Systems, 2026, 38: 76108-76165.
[3] Parulekar A, Rout L, Shanmugam K, et al. Efficient approximate posterior sampling with annealed langevin monte carlo[C]//International Conference on Learning Representations. 2026, 2026: 40513-40543.
Questions for authors
See weakness.