A Spectral Analysis of Averaged Stochastic Gradient Methods

Averaging is central to the statistical behavior of stochastic gradient methods, yet existing analyses often couple the optimization dynamics, the averaging rule, and the stochastic covariance recursion. Moreover, label noise is commonly summarized by a uniform relative-noise condition, which can obscure its distribution across spectral directions. We develop a modular spectral-readout theory for constant-stepsize stochastic gradient dynamics in least-squares regression. The framework separates an algorithm-dependent transfer kernel, a deterministic readout filter, and a second-moment feedback closure. For averaged SGD, we derive finite-time spectral bounds under a general positive trace-class label-noise covariance $\Sigma$, without requiring $\Sigma ⪯{\sigma }_{\mathrm{rel}}^{2}H$, with matching feedback lower bounds after a noise-dependent warmup. The bounds retain the directional noise profile and identify convergent power-law regimes with no finite relative-noise constant. For stochastic heavy-ball, a direction-wise block closure exposes a self-damping mechanism and controls trace-class noise over the reference stepsize region. For full averaging, we further obtain matching risk bounds after warmup and under a fixed stability margin. Synthetic experiments and frozen-feature linear prediction corroborate the predicted filters and rates.