acceptodds
Under review as a conference paper at ICLR 2027

Sharpness Corrections Need Not Act Immediately in Momentum-Based SAM

Abstract

Sharpness-Aware Minimization (SAM) evaluates gradients at perturbed parameters, but under momentum the resulting sharpness correction acts through an immediate zero-lag route and a delayed route carried by the momentum state. The central question is whether a generated correction must act immediately to remain useful. Temporal Feedback SAM (TF-SAM) isolates this choice by removing the zero-lag route while preserving correction generation and its momentum-mediated influence. An exact original-iterate identity shows that this intervention replaces the fresh correction at the target update with its momentum-weighted previous counterpart. A target-matched first-order criterion then shows that the preferred route depends on the corrections' relative alignment with the current descent direction. At regular training states, the normalized-SAM correction has an explicit Jacobian, and the two routes enter different coefficients of the local closed-loop characteristic matrix polynomial. Across the evaluated image and text settings, removing zero-lag feedback does not produce a consistent loss in predictive accuracy or aggregate corruption robustness; target-matched measurements show that the relative first-order value of the routes varies across regimes. Generating sharpness information and applying it immediately are therefore separable design decisions in momentum-based SAM, while holding correction generation fixed.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.