Optimizer Switching at the Edge of Stability: The Onset of Late-Phase SAM Dynamics

Sharpness-aware minimization (SAM) is often applied throughout training, yet recent empirical evidence suggests that switching from standard gradient-based training to SAM only late in optimization can recover much of its benefit. Why can such a late optimizer switch have an immediate dynamical effect? We study this question through the interaction between optimizer switching and the edge of stability. We show that a point that remains stable under gradient descent can become unstable immediately after switching to normalized SAM: the two methods obey different local stability boundaries, and the discrepancy grows as the gradient norm decreases. Near a sharp valley, we further show that normalization converts this boundary crossing into a finite post-switch displacement of order \(O(\rho)\), rather than a perturbation that vanishes with the gradient. In the late-switch regime, the leading normal displacement is spectrally filtered by the local Hessian, preferentially concentrating motion toward dominant-curvature directions. For an exact frozen quadratic normal model, we characterize the resulting phase-alternating dynamics and its attracting principal shell. Controlled synthetic experiments verify the predicted switching boundary, finite-amplitude kick, and shell dynamics. Neural-network experiments on CIFAR-10 with ResNet-18 further reveal a structured post-switch regime boundary, with short phase-alternating transients separating rapid flat-side stabilization from sharp-side instability. Together, these results identify optimizer switching near the edge of stability as a mechanism that initiates the characteristic oscillatory dynamics of late-phase SAM.