- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes a learning-based method for one-step sampling.
The key idea is the target distribution is a fixed point for its MCMC kernel. Therefore, given a kernel K, one can optimize the distributional distance between and .
Based on the insight, the authors test different choices of the kernel and the loss function, and compare their results with existing methods on both simulated examples and real datasets.
Critical strengths
-
I found the idea of treating target distribution as a fixed point very elegant. The corresponding learning algorithm is also clean and simple.
-
Some of the empirical results seem promising.
-
The paper is very well-written.
Critical weaknesses
-
I do not agree with some of the claims, such as "OSIS performs similarly – higher mean reward, lower FID – to the diffusion-based amortised baseline (TB+MALA) while requiring only one function evaluation per sample" (Table 4 shows TB + MALA is clearly better). Also, Table 1 shows the OSIS is clearly much worse than many samplers in terms of sampling accuracy.
-
OSIS has systematic bias (as training is never perfect). Therefore I think the paper is better positioned as a very fast sampler while having bias. The method should also compare with other inexact samplers such as ULA.
-
I suspect there is a tradeoff between speed vs the complexity of the target distribution. For very complex multimodal distribution, I suspect the current simple training mechanism will not help escaping local modes. Therefore annealing or diffusion-based method might be unavoidable. I did not see much discussion on these potential failure cases.
Questions for authors
-
Can you discuss more on the initialization strategy? I think it's important for multimodal targets.
-
Can you comment on what concrete applications that can benefit from your method? I suppose the systematic bias will preclude most Monte Carlo estimation tasks. Also the simplicity (which is overall a good thing!) might also make the method not competitive for very complex distributions. I'd like to know the sweetspot of the method.
I'd be happy to increase the score if the weaknesses and questions are sufficiently discussed by the authors during the rebuttal.
3 comments
thanks for the additional results and careful response!
I understand (and appreciate) it's one-step, very fast, and may be used as high-accuracy algorithm such as Metropolis-Hastings's initialization (though one can argue other neural samplers can also do these). At the same time, it has all the issues just like any other neural samplers, such as intrinsic bias, and mode coverage, and so on.
Therefore, to me, the main evaluation criterion is whether there's a clear and practically meaningful sweetspot, instead of outperforming other methods on all tasks. After reading the rebuttal, I found it still somewhat unclear about the claimed advantage rather than being another neural sampler.
I still find the framework elegant. Nevertheless the concerns seem substantial and therefore I'll maintain my score.
We sincerely appreciate your response and your engagement in the discussion.
We would first like to emphasise that OSIS inherits some limitations of the MCMC kernel it distils. This is discussed in the manuscript, and we will make it more explicit.
The practical sweet spot, which we will also emphasise more clearly, is fast sampling without energy evaluations at sampling time. OSIS is therefore relevant to applications that require fast inference and can tolerate some approximation error.
We further emphasise that OSIS is the first sampler of its kind to be evaluated on large-scale targets. We also develop a theoretical perspective based on Wasserstein gradient flows and establish connections to existing generative models, such as drifting models. We believe these contributions represent a meaningful advance towards new and more efficient approaches to sampling.
We would also appreciate clarification on which of your original concerns were addressed by our response and which remain unresolved, beyond the need to clarify this practical advantage. This would help us address your remaining concerns more precisely.
We would have welcomed further discussion of these points, and we sincerely thank you for your thoughtful feedback.
Sincerely,
The authors
Response to Reviewer TfVR
We thank the reviewer for the constructive review and for offering to reconsider the score.
(W1) OSIS performance
Please find some comments regarding the claims and efficiency comparison of OSIS with the baselines in Tables G1 and G2 of the general response. Also, please see the updated Table 4 and Table 1 for more recent baselines and updated results. Table 4 shows that the difference between OSIS and TB+MALA lies in the trade-off between average log-reward and FID scores. The higher FID scores of OSIS can be attributed to slightly worse mode coverage; however, OSIS is capable of sampling images with a much higher log-reward. At the same time, TB+MALA is quite underwhelming in terms of log-reward, and OSIS FID can be improved with a short MCMC refinement.
It is worth pointing out that TB+MALA is a much more complex algorithm that requires simulating trajectories and calculating trajectory-level divergences, while OSIS is much simpler. Please see the discussion on cost comparison in the general response.
(W2) Systematic bias
Regarding the bias, any learnt component, be it a diffusion sampler or a learnt one-step amortised sampler, introduces some approximation error. However, because OSIS is implicit, we cannot apply an importance-sampling correction to debias its samples. An alternative approach is to use the generated samples to initialise π-invariant MALA or PT-MALA chains. To reduce this systemic bias, we also used MALA rather than ULA in the main experiment to avoid introducing further bias during training. This discussion will be added to the manuscript revision.
(W3-Q1) Speed / complexity tradeoff
We note that the speed versus complexity trade-off holds true for virtually any sampling algorithm, be it a classic MCMC or an amortised sampler. Training samplers for high-dimensional distributions with complex geometries is difficult and requires additional techniques such as replay buffers or tempering. However, when done right, it is possible to train an efficient and scalable amortised sampler for high-dimensional distributions. This is exactly why we discuss several techniques in Section 5 and Appendix C.1 (MALA step-size adaptation), as well as Appendix A.2 and Algorithm 3, which help us stabilise the training and scale the algorithm. We also provide here an extended discussion on practical considerations below, which we will include in the camera-ready version.
Replay buffer. A replay buffer is a standard technique in reinforcement learning and diffusion-based sampling (e.g., Sendera et al. (2024)) that stores samples from previous iterations to help prevent mode collapse. In this work, we use a replay buffer to store samples generated by the model at the previous outer step. These samples are then resampled with replacement during the inner iterations.
Inner and outer iterations. We found empirically that performing several gradient updates on samples obtained from a short MCMC run at each step stabilises training and improves convergence, while also significantly reducing the number of energy evaluations. This keeps the generator's target distribution fixed during the inner loop and lets the generator more accurately approximate the distribution obtained after each MCMC update.
Initialisation strategy. We do not use any sophisticated initialisation strategy beyond matching the output scale of the generator to the scale of the target distribution. This prevents MCMC from immediately collapsing to a single mode. This is a common practice when sampling with MCMC algorithms.
Choice of neural network. Modelling a multimodal distribution with a single forward pass through a neural network is complicated, as the network may struggle to separate the modes, cf. the famous ‘string’ behaviour seen in normalising flows. Although we use a simple ResNet-based architecture for all our experiments, using a better network may help improve the quality of generated samples.
(Q2) OSIS applicability
We want to highlight two key benefits of OSIS: (1) it is a one-step sampler and (2) it does not require energy evaluations at sampling time, which makes sampling very cheap. We therefore believe OSIS has the potential to be applied to any problem that requires fast sampling. For example, it can serve as an initial distribution for any MCMC method. In practice, MCMC often requires a long burn-in before it samples from the target distribution, and this burn-in must be repeated every time the chain is run from scratch. OSIS can amortise this cost: it is trained once, and its samples are then used to initialise the MCMC chains, which considerably shortens convergence time. Finally, we highlight that OSIS is a much simpler method that surprisingly performs on par with much more sophisticated methods like diffusion samplers or amortised SMC methods. The simplicity and flexibility of our method pave the way for improvement and scaling of similar methods in the future.