How Much Coordinate Coupling Can Cautious Momentum Tolerate?

Cautious momentum suppresses coordinate updates unless the new momentum and current gradient have the same nonzero sign, while retaining the updated momentum buffer. We study the raw, unnormalized rule on time-varying quadratic objectives with a common minimizer, diagonal curvature ratio $\kappa$, and relative off-diagonal coupling strength $\rho$. For a fixed tuning and zero initial momentum, we prove that the method reaches relative Euclidean error $\epsilon$ within $O(\sqrt{\kappa }\log (2+\kappa )\log (1/\epsilon ))$ gradient evaluations under coupling $\rho =O({\kappa }^{-1/2})$. This bound is independent of dimension and counts evaluations with suppressed updates. It improves on the optimal worst-case bound of $\Theta (\kappa \log (1/\epsilon ))$ gradient evaluations for linear methods with predetermined coefficients. Conversely, for sufficiently large $\kappa$ and $\rho =\Omega ({\kappa }^{-1/2})$, we construct a predetermined Hessian sequence for which the same method diverges from a suitable initial point with zero momentum. The convergence and divergence bounds use explicit universal constants that differ by only a factor of about $2.14$. Thus we characterize, up to constant factors, the robustness threshold for accelerated convergence of cautious momentum as $\Theta ({\kappa }^{-1/2})$. The proofs use a piecewise quadratic Lyapunov argument for convergence and an explicit construction for divergence.