Making Gradient History Count: Spectrally Calibrated Monte Carlo
Abstract
Bayesian inference and uncertainty quantification rely on Monte Carlo estimates, whose accuracy can remain limited by correlated samples under a finite gradient budget. We ask how gradients already computed during sampling can reduce this error. Spectrally calibrated score repulsion Monte Carlo (SC-SRMC) uses the running average of target scores, the gradients of the log density, to adjust acceptance in the Metropolis adjusted Langevin algorithm (MALA). An exact Gaussian recursion identifies the largest score covariance eigenvalue as the scale for increasing tilt strength. A radial cap bounds the resulting tilt. Under uniform stability and regularity conditions, we bound the score history over finite horizons, with explicit terms for transition correlations. An exact projection transfers the bound to the score component of observable error. The method preserves MALA's rule for generating candidate states and requires no extra online gradients or second derivatives. At matched online gradient budgets, one configuration selected on five development targets gives lower observed mean squared error of coordinate means across 11 Gaussian and quartic validation targets, including six new geometries. The geometric mean reduction relative to diagonally preconditioned MALA is 5.65 fold at the same step size and 6.59 fold with separately tuned step sizes. The error decomposition ties parameter choice to both the observable and target geometry.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.