acceptodds
Under review as a conference paper at ICLR 2027

SHARD-SAM: Signed Hessian-Amplified Radius and Direction for Sharpness-Aware Minimizatio

Abstract

Reconciling optimization and generalization remains a fundamental challenge in deep learning. Sharpness-Aware Minimization (SAM) advances this goal, yet its fixed perturbation radius and first-order geometric approximation limit its adaptability to heterogeneous loss landscapes. We propose SHARD-SAM (Signed Hessian-Amplified Radius and Direction), a sharpness-aware framework that jointly adapts perturbation magnitude and orientation through a shared curvature signal. SHARD-SAM constructs a signed Hessian-amplified spectral surrogate using zeroth-order curvature probes. A symmetric asymptotic envelope adjusts the perturbation radius relative to a running curvature baseline, while magnitude-adaptive gating combines the gradient with a spectral correction. A shared curvature-temperature parameter controls both modules, coupling radius and direction adaptation within a unified perturbation rule. Our theoretical analysis characterizes the surrogate's spectral properties, estimation error, and conditions for local directional benefits. Experiments on CIFAR-10, CIFAR-100, and ImageNet across CNN and ViT architectures demonstrate consistent generalization improvements over representative first-order and second-order geometry-adaptive SAM optimizers, with a favorable accuracy–efficiency trade-off among the evaluated curvature-aware methods. Ablations and direction-source controls support the contribution of the shared curvature signal and its two adaptive modules.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.