Geometry-Aware Score-Repellent Monte Carlo

The recent breakthrough Score-Repellent Monte Carlo (SRMC) method improves MCMC sampling by tilting the target density $\pi$ in ${\mathbb{R}}^d$ with the factor $\exp (-\alpha {\theta }_n^{\top }s(x))$, where the score function $s(x)={\nabla }_x\log \pi (x)$, and ${\theta }_n$ is the score average from past samples. However, a single scalar repellence strength $\alpha \ge 0$ cannot handle both the steep and the flat directions in anisotropic energy landscapes at once. We introduce Geometry-Aware SRMC (GA-SRMC), which generalizes the Euclidean alignment to ${\theta }^{\top }Ms(x)$ for any symmetric positive-definite matrix $M$. A stochastic-approximation central limit theorem under a trace constraint on $M$ identifies the trace-normalized inverse Fisher $M_{\mathrm{opt}}\propto S^{-1}$ as the unique minimizer of the worst-direction covariance bound, elevating the Fisher information $S={\text{Cov}}_{\pi }(s,s)$ from a proposal-side preconditioner (e.g., FisherMALA, natural gradient) to the optimal repulsion-side matrix, with a single online estimate ${\hat{S}}_n$ serving both roles at no additional cost. On an $8$-target simulation spanning the condition number $\kappa \in [1.3,100]$ and dimension $d\in [2,785]$, GA-SRMC dominates SRMC by $4.4\times$ on anisotropic Gaussian ($\kappa =100$) and $6.8\times$ on MNIST under the MALA baseline, by $18\text{ - }32$ on the well-conditioned Bayesian logistic-regression posteriors, and by up to $+48$ under FisherMALA; the advantage scales monotonically with the condition number $\kappa$.