- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes DIG-IN, a neural sampler for discrete unnormalized distributions. It uses exact single-site Gibbs conditionals as soft supervision for a masked predictor, together with tempered importance weighting and damping. The paper establishes population-level consistency and convergence results under specified assumptions, and evaluates the method on Ising/Potts models, amortized sampling across thermodynamic conditions, and alloy phase diagrams. The controlled lattice experiments are encouraging.
Critical strengths
The Gibbs-supervision construction has a principled foundation. Proposition 3.1 establishes consistency and per-site variance reduction under target sampling, and the subsequent analysis distinguishes preserving the target as a fixed point from convergence toward it.
The controlled lattice experiments provide meaningful evidence. Tables 1 and 8 evaluate complementary metrics and include controls for Gibbs labels, importance weighting, and damping. These results support the usefulness of the combined method and should be distinguished from the less conclusive alloy comparison.
Critical weaknesses
N/A
Questions for authors
Clarification of theory versus implementation: What range of tempering exponents is selected during ordered-phase training? What exactly is the multi-site reveal procedure in Table 4, and which fixed-point or convergence statements apply to it? Does the damping-dependent behavior in Table 8 remain when generation uses a single fixed order?
Clarification of reference accuracy: How were the reference log-partition values obtained for the nonzero-field amortized lattices and the fixed-composition alloy tables? Equations (134)–(135) define and transform the alloy quantities but do not describe how the sums were computed.
2 comments
Thanks for the response
Thanks the authors for the response, my questions are addressed. I keep my positive score of the paper.
Reply to Reviewer E24B
We thank the reviewer for their careful reading and constructive feedback. Below, we address each point in turn and indicate the corresponding changes in the revised manuscript.
To avoid any confusion, we are not selecting the tempering exponents manually; it is automatically selected using the ESS control in in Eq. (13). Nevertheless, we added an additional experiment in the appendix (App. F.4, Tempering exponent during training) that shows the tempering exponent, which the ESS control sets automatically over the course of training. In the beginning ν is small and therefore smooths the importance weights a lot whereas in later stages when the model approaches the target ν goes to 1 (no smoothing).
The 50×50 amortized sampler generates a configuration in 512 instead of 2,500 network calls: we first reveal one by one for the first 64 sites, and then randomly reveal 5-6 sites together. The probability of the sample is accumulated exactly, so the importance weights remain exact for this procedure, such that the lnZ estimates stay unbiased and reweighted observables stay consistent.
This can be viewed as a coarser discretization for the underlying CTMC process. Our Theorem 3.2 does not consider the discretization error, and hence it holds up to the discretization error.
To quantify the effect, we sampled the trained 50×50 models both with the multi-site reveal and with exact single-site reveal. The two procedures give the same mode balance in every setting. The block reveal therefore does not cause or hide mode collapse. It lowers the effective sample size, most strongly in the disordered phase, in exchange for roughly five times faster generation. We added the procedure and this comparison to the appendix (App. F.6, Multi-site reveal in the amortized Ising sampler).
We added an experiment in which DIG-IN is trained and evaluated with a single fixed reveal order, with and without damping (App. F.5, Damping under a fixed reveal order). Damping helps under a fixed order as well. More seeds keep both modes of the ordered Ising model with damping than without. The random-order model nevertheless performs better and is less prone to mode collapse. With damping, it keeps both modes in every seed and reaches a higher effective sample size than the fixed-order model.
Amortized lattices: We noticed a minor error: The column header shows that the metrics are computed in expectation over inverse temperature β and external field h. However, the logZ error was only computed in expectation over β for zero-field ( h=0), while other metrics were computed correctly in expectation over both β and h. To avoid any confusion, we removed the logZ error from the table of the amortized runs.
Fixed-composition alloys. The reviewer is right that the appendix did not describe how the sums are computed. The fixed-composition lnZn(T) are computed with a standard high-precision Monte Carlo method from statistical physics, which samples all compositions evenly and therefore yields lnZn(T) for every composition at once (multicanonical sampling [1, 2] combined with replica exchange [3]). We added these details to the appendix and replaced "exact" with "reference" throughout.
We included these clarifications in the revised version of the manuscript.
[1] Berg & Neuhaus. "Multicanonical ensemble: A new approach to simulate first-order phase transitions."
[2] Wang & Landau. "Efficient, multiple-range random walk algorithm to calculate the density of states."
[3] Hukushima & Nemoto. "Exchange Monte Carlo method and application to spin glass simulations."
We hope these clarifications and additional experiments address the reviewer's concerns. We would like to thank the reviewer again for helping us improve our manuscript with the insightful comments and feedback. We are happy to answer any further questions during the rebuttal period.