- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper extends Boltzmann generators (NVT ensembles) to Grand Canonical generators GCG (muVT ensembles) which allow for a varying number of particles. The authors propose two different ways for realizing GCGs. First, they adapt the jump processes with generator matching, and second, they factorize the density in to conditional NVT ensembles and a categorical distribution over particle numbers. Furthermore, the authors show that their model can be used to do self-normalized importance weighting, allowing for debiasing the estimation of observables. They test their method on LJ fluids and methane adsorption in a zeolite.
Critical strengths
- The paper is well-written and tackles an interesting problem, i.e., learning GC ensembles with is a direct extension of Boltzmann generators. Moreover, the paper is honest in reporting negative results for GC-SNIS. The particle number density p(N) seems to be learned well across tasks.
- The factorized is simple and principled and allows to use methods for learning standard BGs, which makes it easy to adapt existing methods to GC ensembles.
Critical weaknesses
- As presented, the paper shows the joint likelihood exists but not that it is useful. Moreover, computing likelihoods is expensive (as already pointed out by the authors)
- The cost comparison to GCMC does not seem fair to me. To my understanding, the evaluations for generating the training data are not counted. Moreover, I’m not an expert in GCMC but I would assume that there exist better baselines than Metropolis GCMC? Moreover, the comparison in Fig 4 at the bottom can be misleading as it is almost trivial that GCMC initialized from model samples converge faster than GCMC from starting from gas.
Questions for authors
- It would be interesting to see if the model can accurately estimate the grand potential? On that note, can the authors also report the ESS of the learned model across different values for ?
- If I understand it correctly then the full GC-SNIS weights are never used. Could the authors clarify this? Also, why was GC-SNIS not used for the zeolite system?
- Does the cost comparison to GCMC also count the NFE to generate the training data?
- In Fig. 4, how is an "energy evaluation" counted for GCMC (one trial move = one evaluation?), and how does the comparison look in wall-clock time including the CNF likelihood cost? Can you provide a wall clock comparison? I would suggest that it would highly favor GCMC over the learned model
- Which mu values are used in Fig. 4? Are they coming from the training set or test set?
We thank the reviewer for their careful reading and for highlighting the principled factorization and the importance of reporting the negative result for grand canonical reweighting.
Reweighting
We used the full grand canonical weights in Fig. 4. We drew 400 configurations at each particle number from N=10 to 50, divided the target density by the product of the allocation probability and the conditional model density, and normalized the weights across all 16,400 configurations. This changes the particle-number probabilities and increased their W1 error from 0.08 to 0.20. Conditional SNIS preserves the learned particle-number probabilities and reduces the energy W1 from 0.60 to 0.30 and the pair-distance W1 from 1.6×10−3 to 9×10−4. We did not run SNIS or measure ESS for zeolite adsorption because of the large cost of evaluating CNF likelihoods at the larger particle numbers. In canonical systems the cost of likelihoods is usually alleviated by using likelihood flow maps or autoregressive models, which we leave for future work.
Effective sample size and grand potential
The pooled normalized ESS in Fig. 4 was 0.13 for conditional SNIS and 0.12 for stratified GC-SNIS. In a proof of concept of 100 joint proposals per state, the normalized joint ESS was 0.171, 0.328, 0.185, 0.405, and 0.308 at βμ=−3.75,−3.50,−3.25,−3.00, and −2.75, respectively. The mean of the unnormalized joint importance weights estimates the grand partition function, from which we can compute the grand potential. Our preliminary LJ estimates of changes in grand potential followed the broad trend from WHAM.
Cost and nearby-state GCMC initialization
Fig. 4 uses βμ=−2.75. The separate interpolation experiment uses βμ=−3.50. We calculate the speedup under the assumption that the generator is reused and target evaluations dominate inference time. The cost comparison excludes GCMC training-data generation, model training, sample generation, and CNF likelihood evaluation.
We count one trial move for GCMC and one full configuration energy for SNIS. The CNF took about 61 seconds to generate and score one N=20 configuration on an A100. FEASST ran about 158,000 trials per second on one AMD EPYC 7532 core. These timings are specific to LJ, where GCMC moves can be computed using a cheap local energy difference. We view our experiment as a proof of concept of likelihood-based correction in grand canonical systems. In realistic systems, the cost of accurate energies can often dominate, especially when fast likelihoods via flow maps or autoregressive models are available.
We agree that a nearby equilibrium state is a strong baseline when configurations are available. We ran 16 chains per initialization at βμ=−2.75 with the same GCMC kernel.
More advanced move sets remain to be tested, and could also be applied to configurations from the model.
Chemical potential in Fig. 4
Fig. 4 uses βμ=−2.75. The separate interpolation experiment uses βμ=−3.50. We will clarify this in the manuscript.