- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes tamed MALA, a version of MALA where the score is tamed to avoid instabilities. Here, the score is tamed by block (or coordinate-wise for block size ) instead of a global shrinking. A high-dimensional scaling result is given under standard i.i.d. and regularity assumptions. The experiments show that tamedMALA is robust.
Critical strengths
-
It's a simple method that works. The experiments show that tamedMALA suffers negligible loss of performance compared to default MALA (unlike other robust baselines), but still recovers well outside stationary.
-
The presentation is generally readable and not dense, unlike other papers written with AI assistance.
Critical weaknesses
-
The novelty of the method seems limited, and the theoretical results are simple with no real technical innovation. In particular, no theoretical result is available for outside-stationary or light-tailed scenarios. The paper also studies exclusively MALA and leaves out unadjusted versions.
-
No description of the stress-test targets is provided, making it difficult to see what makes the algorithms work or fail in each example, and to judge whether the test set is sufficiently exhaustive.
Questions for authors
I'm inclined to accept simple papers that work. My main question is already stated in the Critical Weakness section, namely:
-
Please provide a description of the stress-test targets, in particular, specific features that make each target difficult. From your description in Section 5.5 it seems like those targets are all perturbed only in one dimension. I believe this limits the power of your conclusions.
-
What happens for the block size ? What is the rationale for ?
Minor suggestions: please avoid putting heavy formula in the abstract with unexplained terms. Additionally formulas (5) and (6) should be moved after Eq. (7) for a smoother reading.
2 comments
Acceptance
I'd like to thank the authors for the clarifications. I keep my original positive score.
Theoretical Scope, Robustness Experiments, and Blockwise Taming
We thank the reviewer for the constructive criticisms. In particular, we have addressed the following points according to your comment:
1. Theoretical scope and contributions
Our core contribution is to characterize the effect of proposal-scale-dependent taming with an MH step. The principal theoretical step is the forward/reverse Hastings expansion in Equation (8), which identifies the taming-induced terms in the MALA error. This yields the scaling regime in Theorem 4.2, the sharpness result in Corollary 4.3, and the taming penalty in Proposition 4.4.
We have expanded Section 2 to discuss TULA and related unadjusted schemes. Because our analysis concerns an MH-adjusted algorithm with exact target invariance, we focused on stationary acceptance and empirical pathological-state recovery.
2. Added details on stress-test targets
We have expanded Section 5.5 to include detailed descriptions of the two stress-test targets. For the sextic product target, V(x)=∑ixi6/6, the score grows as −xi5. An extreme tail state produces a raw Langevin drift at risk of overshooting the typical region. For the planar-singular target, V(x)=∣x∣2/2−2log∣x1∣, the pathological score instead arises near the finite-position density zero at x1=0, where g1=2/x1−x1 diverges. In both experiments, the reviewer is correct that only x1 is displaced from stationarity. However, because acceptance is determined by a single joint MH ratio, one extreme score can cause rejection of the full high-dimensional proposal.
3. Coordinate-wise taming and the case for m>1
The theoretical analysis is formulated for fixed-dimensional blocks and is not restricted to m=1. Our empirical default is m=1 because it prevents an extreme score component from unnecessarily attenuating otherwise moderate gradients.
To illustrate this mechanism, we rewrite the blockwise taming transformation from Equation (2) fixing p=2 and γ=1 as
\qquad a_b:=\left(1+\frac{\delta\\|g^{(b)}\\|^2}{c^2}\right)^{-1/2},$$ Here, $a_b$ denotes the scalar attenuation factor shared by all coordinates in block $b$. If one component $g_j$ becomes arbitrarily large while another component $g_i$ remains moderate, then $a_b\to0$, suppressing the drift of both components. Coordinate-wise taming avoids this within-block cross-attenuation because each coordinate has its own multiplier. The joint MH acceptance decision nevertheless continues to couple coordinates. This provides a mechanism-level justification for choosing $m=1$, particularly for localized score pathologies. We do not claim that $m=1$ is universally optimal. Other block sizes may be appropriate when coordinates have a meaningful physical grouping. For example, in three-dimensional electronic variational Monte Carlo, $m=3$ provides a natural particle-wise grouping that preserves rotational equivariance of the tamed drift. Our stationary experiments also include a globally tamed variant, but intermediate block sizes have not been systematically evaluated. Their empirical behaviors remain an open direction for future investigation. **4. Minor fixes** We have re-written the section around equations (5), (6), and (7). Equation (5) and (6) are now grouped under 'settings and notation' section.