- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes a Gaussian cooling framework for efficient mass matrix estimation in strongly log-concave and smooth distributions. The method constructs a sequence of regularized distributions and recursively estimates their covariance matrices to precondition subsequent sampling stages. Under a structural assumption on the target Hessian, the authors establish logarithmic dependence on the original condition number, achieving first-order query complexities of in general and for translation-invariant distributions using FORS. Experiments on ill-conditioned Gaussian distributions and lattice models demonstrate effective preconditioning, with performance comparable to conventional stagewise empirical preconditioning.
Critical strengths
- The proposed Gaussian cooling scheme maintains a bounded condition number at each stage through recursive covariance estimation, independently of the original condition number . Under the stated structural assumptions, this yields an effective preconditioner in stages, providing a rigorous justification for multistage preconditioning.
- By leveraging Fourier diagonalization, the method reduces the sample complexity of covariance estimation from linear to logarithmic dependence on dimension, improving the overall query complexity from to .
Critical weaknesses
- In Figures 1–5, Gaussian cooling performs nearly identically to conventional stagewise empirical preconditioning across both Gaussian and lattice experiments, despite introducing an additional cooling schedule. While this demonstrates the effectiveness of the proposed approach, the experiments do not establish a clear practical benefit of Gaussian cooling over existing methods, particularly in terms of downstream sampling efficiency.
- In the proof of Corollary F.3 (lines 1948–1950), the authors appear to equate the error relative to the true target covariance with the sampling fluctuation around the covariance of the approximate sampling distribution, thereby omitting the approximation bias. Although earlier results on covariance stability under -divergence may help address this gap, the current proof does not establish how the two error sources are combined while preserving the claimed complexity bounds.
Questions for authors
- In Lines 1948–1950, the proof appears to equate the empirical covariance error relative to the true target distribution with the sampling error relative to the approximate distribution , omitting the discrepancy between their covariance matrices. How is this approximation bias controlled, given that the true Fourier variances are defined under in Lines 1817–1822?
- The concentration arguments in Lemmas F.8–F.9 appear to establish only one-sided deviation bounds, whereas Corollary F.3 requires control of absolute deviations. Furthermore, Lines 1911–1912 omit the negative-tail event when bounding the squared sample mean. Could the authors provide the complete two-sided concentration argument?
- Since Corollary F.3 is used to establish Theorem 3.3 and the translation-invariant complexity bound in Corollary 3.1, can the authors demonstrate that correcting the covariance approximation and concentration arguments preserves the claimed query complexity? In particular, does the correction introduce any additional dependence on the conditioning parameter or the estimation accuracy ?
4 comments
Questions
-
Approximation Bias vs. Concentration Error. As noted above, Appendices E and F have been rewritten to make the distinction between the sampler’s approximation bias (Lemma F.7) and the finite-sample concentration error (Lemmas F.8–F.9), which Corollary F.3 combines, more explicit.
-
Two-Sided Concentration Argument. The previous versions of Lemmas F.8 and F.9 did establish two-sided control of the tail event where the sample covariance fails to concentrate, but this consequence was not presented particularly clearly. We apologize again for the confusion, and the updated Equation (48) preceding Lemma F.9 makes the fact that we control the absolute deviations of explicit.
-
Asymptotic Translation-Invariant Query Complexity. The above clarifications to the proofs in Appendices E and F preserve the claimed query complexity in the translation-invariant setting, with no additional dependence on either or . In Appendix E, we now provide a complete proof of Theorem 3.3 that makes the (unchanged) dependence on both and explicit.
I thank the authors for the response. I will keep my initial positive rating.
We thank the reviewer for their helpful comments and suggestions, which improved the clarity of our work.
We thank the reviewer for their time and helpful comments. Below, we respond to the weaknesses and questions from the review.
Weaknesses
Gaussian Cooling vs. Stagewise Preconditioning. Our work does not claim to outperform stagewise empirical preconditioning, which performs remarkably well across a variety of ill-conditioned sampling tasks. In fact, this project began as an attempt to understand the impressive empirical performance of stagewise preconditioning, which is more or less the gold standard method for learning linear preconditioners [1]. After concluding that we currently do not have the necessary mathematical tools to satisfactorily characterize empirical preconditioning's performance, we instead chose to develop a new algorithm that is amenable to theoretical analysis and matches the performance of empirical preconditioning. When it comes to downstream sampling efficiency, Figure 8 shows that Gaussian cooling can be used to efficiently estimate two-point correlators for the lattice ϕ4 model, for example. We view the main contributions of our work as (1) a better theoretical foundation for mass matrix estimation and (2) a practical method for mass matrix estimation that matches the impressive performance of empirical preconditioning while maintaining strong theoretical guarantees. For empirical preconditioning, no such guarantees currently exist, so even though the method tends to perform well, it is not possible to systematically improve its performance for difficult sampling problems that may be of interest to practitioners (e.g., sampling from lattice field theories such as the lattice ϕ4 model).
Translation-Invariant Query Complexity Proofs. We apologize for the previous lack of clarity in the analysis of the translation-invariant case in Appendix F. We have made several substantial changes to both Appendices E and F to better clarify how the analysis in the general case transfers to the translation-invariant setting. Appendix E now makes the overall strategy more concrete, including the approximation bias between the target distribution and the distribution that the sampler produces at each stage. As the reviewer anticipated, this is handled by the χ2-divergence stability guarantee in Corollary F.2. We now provide an explicit translation-invariant counterpart to this corollary, found in the new Lemma F.7. The remainder of Appendix F.4 then handles the concentration of the estimate of the sample covariance in the Fourier basis, and the revised version of Corollary F.3 ties both steps together. We hope that these changes make the translation-invariant analysis easier to parse, and none of the claimed complexity bounds has changed upon cleaning up the proof.
[1] Carpenter et al. "Stan: A Probabilistic Programming Language." Journal of Statistical Software 76.1 (2017). ISSN: 1548-7660. DOI: 10.18637/jss.v076.i01.