acceptodds
Under review as a conference paper at ICLR 2027

Restarting Heavy Ball: Sharp Intervals for Momentum Retention

Abstract

Resetting momentum too frequently can destabilize a convergent heavy-ball method. We characterize restart intervals over the complete scalar class with spectral radius at most . Building on the known repeated-root peak effect, we show that full reset is uniformly stable exactly when . Retaining a fixed fraction of momentum changes the exact answer: the optimal retention tends to , and we determine the restart interval through its constant-order asymptotic term. Exact rational exclusions and independently checked interval certificates establish exact results at finite margins: at , partial retention permits shorter optimal common intervals than full reset. A secondary gradient-only checkpoint construction preserves an accelerated exponent on fixed quadratics. The experiments separate rejected work from delays in checking the stopping target: observed-target stopping removes all gradient-query overhead when the proposal equals the fallback. A delayed-gradient handoff illustrates a distinct use for velocity control: endpoint loss can be small while a subsequent position limit is violated. Controlled optimization experiments find no general speed advantage from joint-energy certification. The main contribution is a sharp restart design rule, with the scope and costs of additional certificates made explicit.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.