- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Spectrally Calibrated Score Repulsion Monte Carlo (SC-SRMC), a sampling method that modifies the Metropolis–Hastings acceptance probability using a tilt derived from score information, where the score denotes the gradient of the log target density. The approach extends Score Repellent Monte Carlo (SRMC; Hu et al., 2026) by incorporating the history of past scores into future acceptance decisions. Concretely, score information is accumulated over the trajectory and used to construct a tilt vector, while the proposal distribution itself remains the standard MALA proposal. A key design choice is the construction of the tilt vector. The authors choose it to align with the accumulated score history and introduce a schedule that adaptively controls its magnitude, including radial clipping when the norm becomes too large. As a result, the tilt can be viewed as a scaled and truncated version of the running sum of past scores. Beyond the algorithmic proposal, the paper provides theoretical analysis and empirical results on unimodal target distributions. In these experiments, SC-SRMC yields lower observed mean squared error for estimating coordinate-wise means than preconditioned MALA.
Critical strengths
The present paper turns SRMC into a stable and cheap MALA modification. SC-SRMC shows lower error of the estimated means compared to preconditioned MALA.
Critical weaknesses
The scope of the empirical and theoretical evaluation appears limited. The current evaluation does not make the contribution feel fully convincing. On the theory side, the most concrete and interpretable theoretical calibration result is derived in the Gaussian setting, while the more general theory relies on fairly strong abstract assumptions and may be harder to assess in practically relevant non-Gaussian settings. On the empirical side, the strongest empirical claims are supported primarily on Gaussian/quartic unimodal targets, and the broader posterior benchmark section seems more mixed and less conclusive.
This is at odds with how the method is motivated, namely as a general-purpose sampling approach for Bayesian inference and uncertainty quantification. In its current form, the paper does not demonstrate relevance on more challenging target distributions that better reflect modern MCMC use cases. In particular, it is unclear how the method behaves on difficult unimodal distributions with strong scale heterogeneity or heavy tails, such as Neal's funnel, or on multimodal targets such as Gaussian mixtures.
As a result, the practical significance of the method remains somewhat unclear: the paper shows gains on relatively simple unimodal problems, but does not yet establish whether those gains carry over to settings where improved sampling methods are most needed.
Minor points
-
The notation is used both for the target distribution and for its density. This is potentially confusing, especially when switching between expressions such as the pointwise density and expectations written in the form . Using separate notation for the measure and its density would improve clarity.
-
The formatting of lemmas, theorems, propositions, and related statements is unusual and makes the structure harder to parse. These results are presented more like inline paragraphs than clearly delimited theorem environments, which can make it difficult to identify where a statement begins and ends. Standard theorem-style formatting would improve readability.
Questions for authors
- Can SC-SRMC be extended to heavy-tailed and multimodal targets?
- How does this affect the scaling and clipping of the tilt?
5 comments
Response to W2: Separate notation for the target measure and density
Thank you for pointing out this source of ambiguity. We have implemented the distinction throughout the prepared revision so that density evaluations and expectations have visibly different notation.
| Object | Notation in the revision |
|---|---|
| Target probability measure | |
| Target density | |
| Score | |
| Target expectation of an observable | |
| Tilted probability measure |
Section 2 introduces these definitions before the algorithm. The MALA acceptance probability now uses the density ratio , while integrals and expectations use or . The same convention is used in the Gaussian derivation, the compact example, and the discussion of normalizability. This also makes the distinction between the original target and a tilted target easier to follow.
We appreciate this helpful suggestion. It improves the presentation at precisely the point where the algorithm switches between evaluating a density and reasoning about its associated probability distribution.
Response to W3: Standard theorem environments and clearer statement boundaries
Thank you for this practical suggestion. We have replaced the paragraph style statements with standard, explicitly delimited theorem environments in the prepared revision. Lemmas, theorems, propositions, corollaries and assumptions now have distinct labels and numbering, with statement boundaries separated from the surrounding explanation.
In particular, the covariance drift lemma, the finite horizon theorem, the compact verification proposition, and the new consistency corollary are presented as separate mathematical statements. The assumptions in the proof appendix are also clearly delimited. Explanatory paragraphs then describe the meaning of the result and direct readers to the relevant proof.
We have additionally included a compact table that pairs each level of the theory with its required conditions and conclusion. This helps readers distinguish the exact Gaussian calculation from the controlled kernel theorem and the ordinary mean consistency result without reconstructing that structure from the prose.
Thank you for helping us make the mathematical contribution easier to navigate. These changes are included in Sections 3 and 4 and the proof appendices of the prepared revision.
Response to W1: Broader evidence and a precise practical scope
Thank you for recognizing the value of a stable, inexpensive MALA modification and for asking us to establish its practical scope more clearly. We have prepared substantial additions that directly address the three parts of this concern: interpretable theory beyond Gaussian targets, difficult target geometries, and comparison with adaptive NUTS including adaptation costs.
The theory now separates mechanism, general conditions, and estimation guarantees. The Gaussian calculation explains a concrete design choice: a large tilt can magnify early errors, and spectral warmup controls this amplification. The general result is not obtained by assuming that a posterior is Gaussian. Its key identity is DS(u)=−Covπu(s), which describes how the mean score changes with the tilt. Theorem 1 then accounts for correlations between transitions through a Poisson equation. In accessible terms, this separates systematic feedback from correlated sampling noise.
The prepared revision makes each level of the analysis explicit:
The last result uses the finite radial cap and the existing second moment history bound. It does not require the fourth moment rate used for the sharper residual bound. We state the extra observable conditions separately so that the reader can identify exactly what is proved.
The new experiments directly test the requested geometries. We added rotated Student targets in dimension 10 with degrees of freedom 3 and 5 and scatter condition number 100. We also added unequal Gaussian mixtures in dimensions 2 and 10, with moderate or large separation and both dispersed and single mode initializations. The complete study has 48 configurations, each with 64 chains, evaluated at three gradient budgets up to 20,000. It reports tail events, mode occupancy and actual crossings between component cores as well as means. These choices test whether a mean improvement corresponds to useful exploration. The results distinguish admissibility from mixing: the Student and mixture tilts are normalizable, while the harder Student cases do not establish a stable mean advantage and widely separated mixtures still require attention to movement between modes. We address the centered funnel analytically in Q1 and explain its implications for clipping in Q2.
The practical comparison includes the cost of adaptation. We additionally evaluated 32 chains on two public data posteriors with 50,000 score calls per chain, including setup. Shared MALA and SC setup is amortized across the 32 chains, while NUTS adaptation is charged per chain. NUTS continues from its adaptation endpoint. The table gives absolute coordinate mean MSE for the fixed reference SC configuration.
Diabetes provides a concrete posterior mean benefit after accounting for setup, with approximately 8.6 times lower MSE than pMALA and 4.8 times lower MSE than NUTS. Digits identifies a geometry in which NUTS is more effective. This usual NUTS pipeline comparison was added after an initialization stress test and is reported alongside that original arm. Second moments and predictive quantities are reported separately, so the mean result is not presented as a guarantee for uncertainty quantification.
Together, these additions support a specific practical contribution: reusing already computed gradients can materially improve finite budget mean estimation on suitable targets, with explicit checks of the target, observable and mixing behavior. Section 5 and Appendix J of the prepared revision contain the full results. Thank you for prompting this more informative assessment of where the method is useful.