- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper combines discrete Langevin samplers with parallel tempering. It shows that swap rules alone cannot correct the bias of unadjusted local updates, and it derives an edge-balance condition for reversibility in the small-step limit. Based on this condition, it proposes A-DULA and AV-DULA, which use only discrete energy differences and work with the classical Metropolis swap. It proves that swapping reduces the cold-chain asymptotic variance and gives a multimodal spectral-gap bound. Experiments cover synthetic mixtures, Ising and Potts models, RBMs, facility location, and MaxSAT EBMs.
Critical strengths
Critical Strengths
S1. The paper establishes a concrete limitation of swap-only correction. A log-quadratic counterexample shows that the history-dependent swap rule analyzed in prior work violates detailed balance with respect to both the intended product target and the claimed intermediate distribution. For the same local kernels, a symmetric-observable argument further shows that no post-proposal retain-or-swap rule can preserve the product target. This challenges the prior detailed-balance characterization and provides a clear justification for correcting the local dynamics.
S2. The edge-balance condition provides a precise diagnosis and a constructive remedy. By identifying the small-step limit of DLS as a continuous-time Markov jump process, the paper derives a necessary and sufficient condition for target reversibility along graph edges. This explains the vanishing-bias behavior in the log-quadratic setting and identifies why it need not extend to general energies. The condition directly motivates extension-free local updates based on discrete energy differences.
Critical weaknesses
Critical Weaknesses
W1. The computational benefit over gradient-based DLS remains unestablished. Gradient-based DLS constructs parallel coordinate proposals from one gradient evaluation, whereas the proposed FULL implementation requires to energy queries per score table, matching the neighborhood-query count of LB. Appendix B.1 discusses this overhead and lower-cost variants, but the reported wall-clock and ESS/s comparisons do not include gradient-based DLS baselines. Consequently, the iteration-based improvements over DLS do not establish better sampling accuracy at an equal computational budget. This matters because the additional energy evaluations could offset the gains per iteration.
W2. The evidence for gains beyond standard PT is limited and task-dependent. Appendix C.2 provides a relevant comparison with classical PT-DMALA and shows a substantial improvement on MNIST under highest-density initialization. However, many other RBM differences are small relative to the reported variability, and Table 4 shows very similar NLL values for PT-MANA and the proposed methods on MaxSAT, with more modest differences in gradient error and mode coverage. The synthetic benchmarks omit classical PT-DMALA. A broader comparison under matched computational budgets would clarify when replacing the original local kernels provides a meaningful benefit beyond adding standard PT.
Questions for authors
Q1 (experiment request). Could you compare the proposed methods with DMALA and DREAM at an equal number of energy evaluations or equal wall-clock time, at least on the Ising and RBM tasks? A neural EBM target would further show whether the energy evaluations per step remain practical.
Q2 (experiment request). Could you add Classical PT-DMALA to the synthetic and Ising/Potts experiments? This baseline would separate the effect of the corrected local dynamics from the effect of PT.
Q3 (clarification). On MaxSAT, PT-MANA stays within one standard deviation of the proposed methods in most settings. What do the corrected local dynamics add over classical PT in this task?
Thank you for the detailed rebuttal and additional experiments. The new compute-matched comparisons and classical PT-DMALA baselines address my concerns regarding W1 and W2. I appreciate the authors' efforts to strengthen the empirical evaluation, and I am inclined to raise my rating accordingly.