Restarting Heavy Ball: Sharp Intervals for Momentum Retention
Abstract
Resetting momentum too frequently can destabilize a convergent heavy-ball method. We characterize restart intervals over the complete scalar class with spectral radius at most . Building on the known repeated-root peak effect, we show that full reset is uniformly stable exactly when . Retaining a fixed fraction of momentum changes the exact answer: the optimal retention tends to , and we determine the restart interval through its constant-order asymptotic term. Exact rational exclusions and independently checked interval certificates establish exact results at finite margins: at , partial retention permits shorter optimal common intervals than full reset. A secondary gradient-only checkpoint construction preserves an accelerated exponent on fixed quadratics. The experiments separate rejected work from delays in checking the stopping target: observed-target stopping removes all gradient-query overhead when the proposal equals the fallback. A delayed-gradient handoff illustrates a distinct use for velocity control: endpoint loss can be small while a subsequent position limit is violated. Controlled optimization experiments find no general speed advantage from joint-energy certification. The main contribution is a sharp restart design rule, with the scope and costs of additional certificates made explicit.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.