acceptodds
Under review as a conference paper at ICLR 2027

Efficient Mass Matrix Estimation with Gaussian Cooling

Abstract

We introduce an algorithm for efficiently preconditioning log-concave and log-smooth distributions that scales logarithmically with the condition number of the underlying distribution. Based on Gaussian cooling, our multistage method approximately samples from a sequence of well-conditioned distributions to construct a preconditioner for each subsequent stage of the cooling schedule. This method is motivated by existing practical approaches for mass matrix estimation in Markov chain Monte Carlo (MCMC), in which an effective preconditioner for the target distribution is estimated from samples produced during a warm-up period. However, our approach is conceptually distinct from existing approaches and moreover permits theoretical analysis. Our algorithmic framework is flexible, allowing essentially any log-concave sampling algorithm to act as the subroutine within each cooling stage. For instance, using first-order rejection sampling (FORS) as the sampler, the first-order query complexity of our method is for a large class of log-concave and log-smooth distributions with condition number and for translation-invariant distributions. In the translation-invariant setting, we provide numerical experiments for the lattice model, a simple lattice field theory.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
sampling, log-concave sampling, query complexity

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_zctDOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a Gaussian cooling framework for efficient mass matrix estimation in strongly log-concave and smooth distributions. The method constructs a sequence of regularized distributions and recursively estimates their covariance matrices to precondition subsequent sampling stages. Under a structural assumption on the target Hessian, the authors establish logarithmic dependence on the original condition number, achieving first-order query complexities of O~(d4/3log⁡κ)\widetilde O(d^{4/3}\log\kappa) in general and O~(d1/3log⁡κ)\widetilde O(d^{1/3}\log\kappa) for translation-invariant distributions using FORS. Experiments on ill-conditioned Gaussian distributions and lattice ϕ4\phi^4 models demonstrate effective preconditioning, with performance comparable to conventional stagewise empirical preconditioning.

Critical strengths
  1. The proposed Gaussian cooling scheme maintains a bounded condition number at each stage through recursive covariance estimation, independently of the original condition number κ\kappa. Under the stated structural assumptions, this yields an effective preconditioner in O(log⁡κ)O(\log\kappa) stages, providing a rigorous justification for multistage preconditioning.
  2. By leveraging Fourier diagonalization, the method reduces the sample complexity of covariance estimation from linear to logarithmic dependence on dimension, improving the overall query complexity from O~(d4/3log⁡κ)\widetilde O(d^{4/3}\log\kappa) to O~(d1/3log⁡κ)\widetilde O(d^{1/3}\log\kappa).
Critical weaknesses
  1. In Figures 1–5, Gaussian cooling performs nearly identically to conventional stagewise empirical preconditioning across both Gaussian and lattice ϕ4\phi^4 experiments, despite introducing an additional cooling schedule. While this demonstrates the effectiveness of the proposed approach, the experiments do not establish a clear practical benefit of Gaussian cooling over existing methods, particularly in terms of downstream sampling efficiency.
  2. In the proof of Corollary F.3 (lines 1948–1950), the authors appear to equate the error relative to the true target covariance with the sampling fluctuation around the covariance of the approximate sampling distribution, thereby omitting the approximation bias. Although earlier results on covariance stability under χ2\chi^2-divergence may help address this gap, the current proof does not establish how the two error sources are combined while preserving the claimed complexity bounds.
Questions for authors
  1. In Lines 1948–1950, the proof appears to equate the empirical covariance error relative to the true target distribution μ\mu with the sampling error relative to the approximate distribution μ^\widehat\mu, omitting the discrepancy between their covariance matrices. How is this approximation bias controlled, given that the true Fourier variances are defined under μ\mu in Lines 1817–1822?
  2. The concentration arguments in Lemmas F.8–F.9 appear to establish only one-sided deviation bounds, whereas Corollary F.3 requires control of absolute deviations. Furthermore, Lines 1911–1912 omit the negative-tail event when bounding the squared sample mean. Could the authors provide the complete two-sided concentration argument?
  3. Since Corollary F.3 is used to establish Theorem 3.3 and the translation-invariant complexity bound in Corollary 3.1, can the authors demonstrate that correcting the covariance approximation and concentration arguments preserves the claimed O~(d1/3log⁡κ)\widetilde O(d^{1/3}\log\kappa) query complexity? In particular, does the correction introduce any additional dependence on the conditioning parameter α\alpha or the estimation accuracy δ\delta?
4 comments

We thank the reviewer for their time and helpful comments. Below, we respond to the weaknesses and questions from the review.

Weaknesses

  1. Gaussian Cooling vs. Stagewise Preconditioning. Our work does not claim to outperform stagewise empirical preconditioning, which performs remarkably well across a variety of ill-conditioned sampling tasks. In fact, this project began as an attempt to understand the impressive empirical performance of stagewise preconditioning, which is more or less the gold standard method for learning linear preconditioners [1]. After concluding that we currently do not have the necessary mathematical tools to satisfactorily characterize empirical preconditioning's performance, we instead chose to develop a new algorithm that is amenable to theoretical analysis and matches the performance of empirical preconditioning. When it comes to downstream sampling efficiency, Figure 8 shows that Gaussian cooling can be used to efficiently estimate two-point correlators for the lattice ϕ4\phi^4 model, for example. We view the main contributions of our work as (1) a better theoretical foundation for mass matrix estimation and (2) a practical method for mass matrix estimation that matches the impressive performance of empirical preconditioning while maintaining strong theoretical guarantees. For empirical preconditioning, no such guarantees currently exist, so even though the method tends to perform well, it is not possible to systematically improve its performance for difficult sampling problems that may be of interest to practitioners (e.g., sampling from lattice field theories such as the lattice ϕ4\phi^4 model).

  2. Translation-Invariant Query Complexity Proofs. We apologize for the previous lack of clarity in the analysis of the translation-invariant case in Appendix F. We have made several substantial changes to both Appendices E and F to better clarify how the analysis in the general case transfers to the translation-invariant setting. Appendix E now makes the overall strategy more concrete, including the approximation bias between the target distribution and the distribution that the sampler produces at each stage. As the reviewer anticipated, this is handled by the χ2\chi^2-divergence stability guarantee in Corollary F.2. We now provide an explicit translation-invariant counterpart to this corollary, found in the new Lemma F.7. The remainder of Appendix F.4 then handles the concentration of the estimate of the sample covariance in the Fourier basis, and the revised version of Corollary F.3 ties both steps together. We hope that these changes make the translation-invariant analysis easier to parse, and none of the claimed complexity bounds has changed upon cleaning up the proof.

[1] Carpenter et al. "Stan: A Probabilistic Programming Language." Journal of Statistical Software 76.1 (2017). ISSN: 1548-7660. DOI: 10.18637/jss.v076.i01.

Questions

  1. Approximation Bias vs. Concentration Error. As noted above, Appendices E and F have been rewritten to make the distinction between the sampler’s approximation bias (Lemma F.7) and the finite-sample concentration error (Lemmas F.8–F.9), which Corollary F.3 combines, more explicit.

  2. Two-Sided Concentration Argument. The previous versions of Lemmas F.8 and F.9 did establish two-sided control of the tail event where the sample covariance fails to concentrate, but this consequence was not presented particularly clearly. We apologize again for the confusion, and the updated Equation (48) preceding Lemma F.9 makes the fact that we control the absolute deviations of Z^n\widehat{Z}_n explicit.

  3. Asymptotic Translation-Invariant Query Complexity. The above clarifications to the proofs in Appendices E and F preserve the claimed O~(d1/3log⁡(κ))\widetilde{O}(d^{1/3} \log(\kappa)) query complexity in the translation-invariant setting, with no additional dependence on either α\alpha or δ\delta. In Appendix E, we now provide a complete proof of Theorem 3.3 that makes the (unchanged) dependence on both α\alpha and δ\delta explicit.

I thank the authors for the response. I will keep my initial positive rating.

We thank the reviewer for their helpful comments and suggestions, which improved the clarity of our work.

Reviewer_F5WSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes an annealing scheme that learns a linear preconditioner (mass matrix) for a strongly log-concave, smooth target π∝e−U\pi\propto e^{-U}. It samples from the potentials Uk(x)=U(x)+γkL2∥x−x∗∥2U_k(x)=U(x)+\gamma^k\frac{L}{2}\|x-x^*\|^2, k=0,…,Kk=0,\dots,K. At each stage, it draws nn approximate samples from the current target, preconditioned by the previous covariance estimate, and uses them to estimate the next covariance. The key assumption is αB⪯∇2U⪯B\alpha B\preceq\nabla^2U\preceq B for an unknown matrix BB. Under this assumption every stage is well conditioned and K=O(log⁡κ)K=O(\log\kappa) stages suffice. With FORS as the inner sampler, the total cost is O~(d4/3log⁡κ)\tilde O(d^{4/3}\log\kappa) gradient queries for constant α\alpha, and O~(d1/3log⁡κ)\tilde O(d^{1/3}\log\kappa) when the covariance is translation-invariant.

Critical strengths
  • Warm-up mass-matrix adaptation is used in essentially every HMC package (e.g., Stan), yet, as the paper notes, no analysis captures its benefit. The paper gives a staged procedure in the same spirit, with a non-asymptotic guarantee whose dependence on κ\kappa is only logarithmic. In the paper's first-order oracle model, this improves on the closest analysis of learned preconditioners (Hird et al., 2026), which needs polynomial dependence on κ\kappa.

  • The introduction explains Stan's success by mixing along different eigendirections at different time scales. The cooling path makes this idea precise.

  • Some of the tools are useful beyond this paper. Appendix F shows that if approximate samples are close to a log-concave target in chi-squared distance, their covariance is close to the true one. This is exactly what one needs whenever covariances are estimated from MCMC output. The procedure also works with any inner sampler, so it benefits automatically from better samplers.

  • The method scales well. For translation-invariant targets, estimating the covariance in Fourier space cuts the number of samples per stage from about dd to about log⁡d\log d, and the method runs on lattices with about 260,000 variables.

Critical weaknesses
  • I might be wrong but it seems the theory does not cover the main application. With the lattice parameters used in the experiments, α\alpha is roughly 1/κ1/\kappa. Since the bounds grow like a high power of 1/α1/\alpha, they are much worse here than simply running the sampler without a preconditioner. So the ϕ4\phi^4 experiments, which motivate the paper seem not supported by the theory. This matters because if this the case it makes the application outside of the theory's domain.

  • Rounding along an annealing schedule is classical in log-concave sampling and volume computation (Lovász & Vempala, 2006; Jia, Laddha, Lee & Vempala, 2021). There, samples from each phase put the next phase in near-isotropic position, and the number of phases is logarithmic in the initial roundness. The paper does not discuss this line of work, which weakens its novelty claim.

Questions for authors
  1. Since α≈1/κ\alpha\approx1/\kappa in the lattice experiments, which of your results applies there?
5 comments

We thank the reviewer for their time and helpful comments. Below, we respond to the weaknesses and questions from the review.

Weaknesses

  1. Application Beyond Theory. We agree that the lattice ϕ4\phi^4 experiments in the first draft of the paper did not quite convey the utility of our theoretical framework. We have included a new set of lattice ϕ4\phi^4 experiments in Section 4.2, with parameters m=0.01m = 0.01, λ=0.5\lambda = 0.5, and β\beta variable. Under this new choice of lattice parameters (alongside a fixed truncation radius R=0.1R = 0.1), we have that α=0.4\alpha = 0.4, while increasing β\beta increases the condition number of the target distribution linearly. In other words, the new set of experiments better captures the regime in which our theoretical results predict the query complexity of translation-invariant Gaussian cooling to scale well with the underlying dimension and target condition number, as α\alpha is a fixed constant. The newly included Figures 5 and 6 show exactly this, with translation-invariant Gaussian cooling and empirical preconditioning continuing to obtain O(1)O(1) relative condition number as β\beta grows. We have left the original ϕ4\phi^4 experiments in the paper because we believe that demonstrating our method's robust empirical performance across several different lattice parameter choices may still be of interest.

  2. Prior Literature on Rounding Along an Annealing Schedule. In our revised submission, we have added this additional background to our related work section (Section 1.2). We thank the reviewer for pointing out the additional connection to the log-concave sampling and volume estimation literature. The main novelty of our method is not the high-level idea behind Gaussian cooling, which we acknowledge is inspired by a long line of related annealing algorithms, but rather the broad theoretical framework that we develop for its application to affine-invariant sampling and mass matrix estimation. In our setting, the fact that we require K=O(log⁡κ/log⁡(1/γ))K = O(\log \kappa / \log(1/\gamma)) stages of Gaussian cooling is related to the results that the reviewer points out, but beyond this parallel, our theoretical framework is entirely distinct.

Question

As noted above, we have included new lattice ϕ4\phi^4 experiments to which our theoretical results directly apply: α=Θ(1)\alpha = \Theta(1) as κ\kappa increases, and translation-invariant Gaussian cooling continues to perform well even when κ≈108\kappa \approx 10^8. In the paper's conclusion, we have added a remark on the limitations of our theoretical framework in explaining the success of Gaussian cooling as m→0m \to 0, warranting future work in this direction (as is also the case for empirical preconditioning).

I thank the authors for clarification.

While the regime in the experiments seems to be cover the outside of theory, I don't think it changes the papers contributions significantly because these are experiments to show the effectiveness of gaussian cooling outside of theory. One weakness could be that the theory does not cover strong quartic term in the energy (interaction tern), that is, the settings mostly cover when the distribution is close to Gaussian because α\alpha could be tiny since Hessian changes too much from one point to another (please correct me if I am wrong ).

I think making this study in the context of lattice field theory is a strong reason why I would want to see this paper in the conference. Previously, these problems have been considered in the context of volume estimation, optimization, bayesian estimation, etc. While of course these techniques are known applied in high energy or condensed matter literature, this study opens up questions that can be studied rigorously for the log-concave sampling within this context and bridges these fields.

We thank the reviewer for the follow-up response, and we also hope that our work will spur additional mathematical study of the difficult sampling problems that arise in high energy and condensed matter physics.

The quartic interaction term in the lattice ϕ4\phi^4 potential poses an issue, but not simply because the resulting distribution is non-Gaussian. The reviewer is correct that the condition that αB⪯∇2U(x)⪯B\alpha B \preceq \nabla^2 U(x) \preceq B with α=Θ(1)\alpha = \Theta(1) effectively captures the notion that the Hessian cannot fluctuate too much globally, and a quartic term (with truncation R=∞R = \infty) ensures that the Hessian is unbounded.

However, it is possible to have a distribution that is decidedly non-Gaussian but where α=Θ(1)\alpha = \Theta(1). Our new set of experiments in the updated Figures 5 and 6 is meant to give a relevant example of this setting. To show how, consider the magnetization, a physically interesting order parameter given by

M=1d2∑x∈Λϕx,M = \frac{1}{d^{2}} \sum_{x \in \Lambda} \phi_x,

for a fixed lattice sidelength dd, fixed mass m=0.01m = 0.01, fixed truncation radius R=0.1R = 0.1, and fixed self-interaction λ=0.5\lambda = 0.5. As β→∞\beta \to \infty, the fact that only the constant fields lie in the kernel of the Laplacian Δβ\Delta_\beta allows one to show that the limiting distribution is concentrated on constant fields with ϕx=M\phi_x = M for all x∈Λx \in \Lambda. The limiting magnetization MM is then distributed as

p(M)∝exp⁡(−d2wR(M)),p(M) \propto \exp\left(-d^2 w_R(M)\right),

where wRw_R is the truncated potential defined in Equation (50) of Appendix G. Consequently, the limiting distribution of the magnetization (and hence of the spins) is non-Gaussian, even though α=0.4\alpha = 0.4 is fixed as β→∞\beta \to \infty in our new set of experiments. One empirical test of non-Gaussianity is the excess kurtosis, given by

K(M):=E[M4]E[M2]2−3.K(M) := \frac{\mathbb{E}[M^4]}{\mathbb{E}[M^2]^2} - 3.

If the magnetization MM followed a centered Gaussian distribution, then we would have K(M)=0K(M) = 0, regardless of the variance of MM. As the following table demonstrates using samples produced with Gaussian cooling followed by preconditioned ULMC, for fixed dd, the magnetization's distribution is non-Gaussian as β→∞\beta \to \infty (with the non-Gaussianity shrinking as dd grows). Moreover, as β→∞\beta \to \infty, our sampled estimates agree with the analytical value of the excess kurtosis, estimated from the density above via quadrature.

Excess kurtosis of MM (standard error)

β\betad=128d = 128d=512d = 512
11−0.010-0.010 (0.0060.006)−0.003-0.003 (0.0060.006)
1010−0.009-0.009 (0.0060.006)0.0000.000 (0.0060.006)
10210^2−0.078-0.078 (0.0050.005)−0.009-0.009 (0.0070.007)
10310^3−0.270-0.270 (0.0050.005)−0.061-0.061 (0.0070.007)
10410^4−0.315-0.315 (0.0040.004)−0.090-0.090 (0.0060.006)
10510^5−0.322-0.322 (0.0050.005)−0.094-0.094 (0.0060.006)
Exact (β→∞\beta \to \infty)−0.331-0.331−0.090-0.090

We hope that our new set of experiments effectively demonstrates this genuine application of our theory to a non-Gaussian distribution.

Reviewer_ykifOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes Gaussian cooling for learning a preconditioner from samples. Each stage uses the previous stage’s covariance to precondition a less regularized target. Under Eq. 2, the method achieves logarithmic dependence on the original condition number. Experiments consider transformed Gaussians and lattice field models.

Critical strengths

The central insight is that learning a useful preconditioner does not require first sampling efficiently from the original target. A constant-factor covariance estimate at each cooling stage keeps the next stage well-conditioned, without compounding approximation errors (Proposition 2.1). This gives a clean, sampler-agnostic first-order guarantee. The construction is informative even though much of the concentration analysis uses standard tools.

Critical weaknesses
  1. The guarantee relies on strong global structure. Eq. 2 assumes that a fixed linear transformation can make the target well-conditioned, and the complexity depends heavily on the remaining conditioning parameter. For the lattice example, Eq. 48 deteriorates in the small-mass regime, so the theory does not fully explain the empirical success there.

  2. The algorithm’s parameter requirements limit its practical interpretation. It assumes access to the exact minimizer, whose computation is excluded from the query complexity. Theorem 3.2 also chooses sampling budgets using the conditioning parameter, although the method is presented as not requiring its value. A guaranteed procedure without this information remains unclear.

Questions for authors
  1. Can the exact minimizer be replaced by an approximate one? What accuracy is needed, and how would obtaining it affect the overall first-order complexity?

  2. How can the sampling budgets in Theorem 3.2 be selected with guarantees when the conditioning parameter is unknown?

4 comments

We thank the reviewer for their time and helpful comments. Below, we respond to the weaknesses and questions from the review.

Weaknesses

  1. Strong Global Structure. The primary goal of our method is to rigorously capture the intuition underlying affine-invariant samplers: there exists an (unknown) linear transformation BB such that, after applying said transformation, a sampling problem becomes well-conditioned. Of course, the difficulty is then finding this transformation. If such a transformation does exist, however, then the conditioning parameter satisfies α=Θ(1)\alpha = \Theta(1). Importantly, our method sidesteps computing the linear transformation BB entirely. In the lattice ϕ4\phi^4 example in Section 4.2, it is true that α\alpha decreases as m→0m \to 0. Explaining why both translation-invariant Gaussian cooling and translation-invariant empirical preconditioning still perform well in this regime is an interesting direction for future work. To better motivate the application of our method to the lattice ϕ4\phi^4 model, we have added an additional set of experiments to Section 4.2, where α=0.4\alpha = 0.4 is held constant but κ\kappa grows without bound as the coupling constant β\beta increases. This is the setting that our theory is well-equipped to describe, and indeed, translation-invariant Gaussian cooling performs well in this regime, obtaining O(1)O(1) relative condition number after K=12K = 12 stages of Gaussian cooling, even when the condition number of the target distribution reaches κ≈108\kappa \approx 10^8.

  2. Exact Minimizers and Parameter Requirements. We have fully removed the requirement that our algorithm has access to the exact minimizer, and we thank the reviewer for suggesting this improvement. Initially, this requirement was inherited from the corresponding query complexity guarantees for ULMC and FORS, both of which initialize based on the exact minimizer x∗x^* of the target distribution. However, with a more careful analysis of the initialization in both algorithms, it is possible to initialize with an approximate minimizer at each stage of Gaussian cooling. Our new Lemma E.1 describes how to compute this approximate minimizer at each stage of cooling, using accelerated gradient descent on the well-conditioned problem formed at each stage. Importantly, the first-order query complexity of the approximate minimization at each stage contributes only at subleading order under the mild assumption that ∣∣x∗∣∣2≤d/LeO(d)||x^*||_2 \leq \sqrt{d / L} e^{O(d)}. Beyond this new lemma, the rest of our analysis is largely unchanged. On the other hand, some dependence on α\alpha is likely unavoidable. We have made this requirement clearer in Section 1.1 of the paper, clarifying that a lower bound on α\alpha is needed to select the cooling rate. Empirically, we find that our method is quite robust as the sampling budgets presented in Theorem 3.2 vary. For instance, in Figure 9 in Appendix G.2, we even demonstrate that for the lattice ϕ4\phi^4 model with sidelength d=100d = 100 and m=0.01m = 0.01, the relative condition number produced by Gaussian cooling after K=12K = 12 stages is nearly one once the number of ULMC steps per chain NN reaches 256256.

Questions

  1. Removing the Exact Minimizer Assumption. Yes, the exact minimizer can be replaced with an approximate minimizer, and we have updated our paper accordingly, starting with the relevant changes to Assumption 3.1 and Algorithm 1. In particular, it suffices to compute an approximate minimizer at each stage of Gaussian cooling using accelerated gradient descent as in [1, Section 3.7]. As we show in the new Lemma E.1, the guarantees in [1, Theorem 3.18] then imply that at each stage of Gaussian cooling, the additional query complexity due to the approximate minimization procedure is onlyO(α~−1log⁡(ed+L∣∣x∗∣∣22α~d)).O\left(\widetilde{\alpha}^{-1} \log\left(\frac{e d + L ||x^*||_2^2}{\widetilde{\alpha} d}\right)\right).This cost is dominated by the nKNn K N sampler queries when ∣∣x∗∣∣2≤d/LeO(d)||x^*||_2 \leq \sqrt{d / L} e^{O(d)}. Finally, in the updated Lemma C.3 (for FORS) and the discussion following Theorem D.2 (for ULMC), we show that both choices of sampler satisfy Assumption 3.1 without any changes to their respective query complexities. More broadly, Assumption 3.1 now only requires that the sampler takes a point x∈Rdx \in \mathbb{R}^d that is within distance ∣∣x−x∗∣∣2≤12d/L||x - x^*||_2 \leq \frac{1}{2} \sqrt{d / L} of the true minimizer, and such a point can be computed efficiently at each stage with accelerated gradient descent. Removing the exact minimizer assumption did not impact any of the numerical experiments in Section 4 because, in both settings, the exact minimizer is x∗=0x^* = 0.

  2. Selecting Sampling Budgets. Note that the sampling budgets in Theorem 3.2 depend only on α~=min⁡(α,γ)\tilde{\alpha} = \min(\alpha, \gamma), where γ>0\gamma > 0 is the cooling rate. Consequently, choosing γ\gamma to be any lower bound on α\alpha will suffice and still yield reasonable first-order query complexity guarantees. While a trivial lower bound of α≥1/κ\alpha \geq 1 / \kappa is always available, such a choice would be highly suboptimal. Rather, computing a good lower bound for α\alpha should be done on a case-by-case basis. For example, for the lattice ϕ4\phi^4 experiments in Section 4.2, we explicitly compute a lower bound on α\alpha in terms of the lattice ϕ4\phi^4 model parameters (see Appendix G). For other applications of our method, carrying out a similar computation to determine a lower bound on α\alpha is advisable, even if the resulting bound is somewhat loose. In practice, it is also possible to estimate a good choice for the cooling rate γ\gamma adaptively. In particular, beginning from an initial value of γ\gamma, verify whether the intermediate stages of Gaussian cooling are mixing by computing the integrated autocorrelation time (IAT) for a relevant statistic. If the IATs at intermediate stages are particularly large, then reduce γ\gamma and repeat.

[1] Sébastien Bubeck. Convex Optimization: Algorithms and Complexity. Foundations and Trends in Machine Learning, 8(3–4):231–357, 2015. doi:10.1561/2200000050 (arXiv:1405.4980)

I thank the authors for their rebuttal. Most of my concerns have been addressed, and I would like to maintain my original ratings and positive assessment.

We thank the reviewer for their helpful comments, and we feel that the removal of the exact minimizer assumption, based on the reviewer's suggestion, strengthens the theoretical side of our work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised PDF Uploaded

We would like to thank all of our reviewers for the time that they spent reviewing our paper and the helpful feedback that they provided on our work. We have uploaded a revised version of our paper, taking into account the comments that we received from all of our reviewers. All major revisions to the paper are marked in red. Although the vast majority of the paper remains the same, three notable changes are:

  1. Additional Lattice ϕ4\phi^4 Experiments. As Reviewers ykif and F5WS pointed out, our initial set of lattice ϕ4\phi^4 experiments did not fully align with our theoretical framework because α≈1/κ\alpha \approx 1 / \kappa as m→0m \to 0. To address this concern, we have added a new set of experiments to Section 4.2. In the new experiments, we choose different lattice parameters so that α=Θ(1)\alpha = \Theta(1) remains fixed even as κ\kappa, the condition number of the target distribution, grows. This regime is squarely within our theoretical framework, and the new results in Figures 5 and 6 demonstrate the predicted strong performance of translation-invariant Gaussian cooling.
  2. Removal of Exact Minimizer Assumption. In response to Reviewer ykif's concern, we have entirely removed the assumption that our algorithm has access to x∗x^*, the exact minimizer of the target potential UU. The updated Assumption 3.1 now only assumes that the chosen sampling algorithm takes an approximate minimizer x∈Rdx \in \mathbb{R}^d with ∣∣x−x∗∣∣2≤12d/L||x - x^*||_2 \leq \frac{1}{2} \sqrt{d / L} as input. In Algorithm 1, we now compute such an approximate minimizer of the well-conditioned potential U~k\widetilde{U}_k at each stage of Gaussian cooling via accelerated gradient descent. Lemma E.1 shows that the additional first-order query complexity of this procedure contributes only at subleading order as long as ∣∣x∗∣∣≤d/LeO(d)||x^*|| \leq \sqrt{d / L} e^{O(d)}. We verify in Appendices C and D that FORS and ULMC, respectively, satisfy the updated Assumption 3.1.
  3. Clearer Translation-Invariant Analysis. We have clarified several portions of the analysis of translation-invariant Gaussian cooling, addressing Reviewer zctD's concerns about the proofs in Appendix F. Namely, we have added significant detail to Appendices E and F to more clearly separate the approximation bias of the sampler from finite-sample concentration error in the translation-invariant case.

We have also corrected minor ambiguities and typos throughout the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.