- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Quenched Ensemble Sampling (QES), a sequential Monte Carlo method that aims to extend nested-sampling-style inference to high-dimensional problems. QES replaces the hard energy constraints of traditional nested sampling with softened constraints, allowing gradient-based MCMC kernels while preserving the quenched energy path. It uses SMC importance weights to estimate marginal likelihoods and reconstruct posterior samples.
The authors evaluate QES on synthetic distributions, a high-dimensional lattice model, and Bayesian neural networks on real datasets. QES successfully traverses first-order phase transitions where the tested tempered SMC methods fail, while achieving comparable predictive performance and marginal likelihood estimates on real-world BNN tasks. The paper's contributions are primarily methodological and empirical, with limited new theoretical performance guarantees.
Critical strengths
1. Well-motivated and mathematically grounded methodology: The paper addresses a clear limitation of traditional nested sampling: the difficulty of using gradient-based MCMC kernels under hard energy constraints. QES combines softened energy constraints with SMC-based volume estimation, providing a coherent framework for posterior sampling and marginal likelihood estimation.
2. Promising results on challenging phase-transition problems: QES demonstrates substantial improvements over tempered SMC on the phase-transition tasks. In particular, it accurately estimates the marginal likelihood in high-dimensional spike-and-slab problems where tempered SMC misses an important mode. These results suggest that QES could be valuable for sampling problems where conventional tempering struggles.
Critical weaknesses
-
The softened energy constraints are closely related to existing microcanonical and demon-based nested sampling methods, while SMC-based volume estimation has also been studied previously. Although QES combines these components into a practical framework, the paper does not sufficiently establish what fundamentally new capabilities this combination provides, neither theoretically nor practically. In particular, the choice of a dimension-independent softness parameter appears to be a design choice rather than a substantial methodological innovation.
-
A central motivation of this paper is to make nested sampling scalable to high-dimensional problems. However, the scalability comparison is mainly conducted on a simple Gaussian target, while the more challenging high-dimensional phase-transition experiments omit competitive NS baselines. Although the authors attribute these omissions to computational cost, they could report the performance of NS methods under a fixed computational budget rather than omitting the comparisons entirely. Furthermore, comparing gradient evaluations for QES with likelihood evaluations for NS does not establish actual computational efficiency. These limitations weaken the evidence for QES's main claimed advantage over existing NS methods.
Questions for authors
-
Could the authors clarify the key methodological differences between QES and existing microcanonical or demon-based nested sampling methods, particularly Habeck (2015)? Beyond the dimension-independent softness parameter and SMC-based volume estimation, what specific advantages does QES offer that these approaches do not already provide?
-
The paper omits NS results on the more challenging high-dimensional benchmarks due to computational cost. Could the authors clarify what computational budgets were attempted and how the NS baselines performed before termination? Even incomplete runs under a fixed budget could help assess whether QES provides a meaningful computational advantage over existing NS methods.
-
The paper uses substantially deeper, problem-specific stopping thresholds for the phase-transition experiments to ensure that QES reaches the low-energy phase. In practical applications where the location of a phase transition is unknown, how should users determine an appropriate stopping criterion? Could the default criterion terminate before discovering an important low-energy phase, potentially leading to inaccurate evidence estimates?
I would be happy to raise my score if the authors can satisfactorily address these concerns.
2 comments
Thank you for the clarifications and additional experiments. The new comparison addresses my main concern about computational performance. Although the comparison remains limited to one NS implementation, it strengthens the evidence for QES’s practical contribution. Overall, while some concerns about novelty and the scope of the empirical claims remain, the additional evidence has improved my assessment. I will raise my score.
We thank the reviewer for their thorough reading of our paper. We think the questions raised are very useful, and appreciate the opportunity to provide clarifications.
1. Implications of prior work
The key point is that Habeck does not concretely demonstrate an advantage over classical gradient-free nested sampling. In the 32×32 Potts comparison, “Demonic nested sampling took approximately one third longer than standard nested sampling”, with similar evidence errors, leading to the conclusion that “there seems to be no advantage for the particular case of the Potts model…”. We think our results help isolate why the exponent matters: we demonstrate that this is a family of paths interpolating between annealing and nested sampling, and empirically identify when the quenched advantage is retained (Appendix B, Fig. 5). We think this is useful pedagogy alongside the empirical and implementation novelty.
Secondly, we focus on continuous targets, rather than the main emphasis of demonic NS, and give a simple identity integrating out the soft boundary term, which admits a wider range of MCMC kernels. Compared with Habeck’s bespoke HMC, we found MALA more effective: HMC trajectories still have to respect the energy boundary, so long trajectories can be rejected and wasted. Langevin sampling was also easier to tune.
Beyond softness, we emphasise improved SMC accounting (important for consistent results at scale), a parallel implementation suited to modern accelerators, and a framing connecting to modern MCMC variants. We think the overall package delivers a substantial practical advance over previous methods, demonstrating efficient sampling and evidence estimation on challenging high-dimensional problems where classical NS is prohibitively expensive and tempered SMC fails. This is supported by experiments that isolate the important components, alongside a synthesis of research threads less familiar at venues like ICLR.
2. Demonstration of scaling of classical nested sampling
We thank the reviewer for this suggestion, which we have explored. The NS baselines stop at D=102 primarily for cost reasons. With the standard 2D slice steps, versus approximately D MALA steps, QES has a more favourable mutation budget, and NSS already shows significantly more bias on the 100D Gaussian.
Beyond wall time, linear scaling of slice steps need not maintain mixing at scale. Theoretical arguments give spectral-gap bounds with dimension dependence as unfavourable as cubic for an idealised hybrid slice sampler (Power et al.), suggesting that the usual linear heuristic is not necessarily sufficient. Even gradient-guided NS, such as GGNS (Lemos et al., 2024), using reflective Hamiltonian moves, demonstrates unimodal Gaussian examples up to 256D. We are therefore unaware of convincing general-purpose NS demonstrations at the dimensions reached by QES.
We have now tested NSS on the 40×40 lattice, using our other NS baseline configuration: 1,000 live points, 10% replacement per iteration, and 2D=3200 slice steps per replacement. We stopped after one hour on an RTX 5090, whereas QES converges in approximately seven minutes on the same GPU. NSS failed to converge: slice moves repeatedly hit a conservative bracket-shrinkage cap, mixing deteriorated, and after an hour the particles had degenerated to a single point. The NSS run consumed a similar order of target evaluations to QES (∼109, albeit likelihood versus gradient evaluations), yet reported an ordered share of zero, failing on the primary metric of interest.
We will include this theoretical context and practical test in the revision.
Weak Poincaré inequality comparisons for ideal and hybrid slice sampling, Power et al.
3. Termination
We agree that the draft gives too little guidance on termination and that a run can stop before reaching a low-energy phase, especially near first-order transitions. There are useful practical mitigations.
Firstly, runs can be resumed and extended until logZ and relevant observables settle. In materials or molecular applications, independent experimental or simulation information often indicates the stable energy, so a quench ending above it should continue. A known global lower bound on energy can also replace the lowest observed energy in the criterion, making it more conservative.
Because QES is a quench, one can also continue until walkers collapse onto a single energy. This is expensive, but may be a natural endpoint when the ground state itself is of interest, although it is not a guarantee of global convergence. For Bayesian benchmarks we use the nested-sampling criterion for a fairer comparison with tempered SMC, with deeper follow-up runs where warranted.
A possible future extension is curvature-based termination, inspired by the use of L-BFGS to quench molecular configurations. This could provide a useful convergence diagnostic, though not a guarantee against undiscovered phases.