Accelerated Stochastic Method under -Smoothness and Heavy-Tailed Noise
Abstract
We develop an accelerated stochastic method for convex -smooth optimization under heavy-tailed noise. The unbiased oracle has a finite -th noise moment, , with constant, gradient-dependent, and gap-dependent terms. Our method combines accelerated updates with clipping, projection, and phase restarts. Each iteration uses a single stochastic-gradient sample, without minibatching. We prove high-probability convergence to any desired accuracy despite clipping bias. The deterministic terms retain square-root dependence on both and . In our three-component noise model, strong convexity leaves a polynomial accuracy cost only for the constant component; without it, the dependence is logarithmic even for . More generally, for noise moments scaling as , the stochastic cost becomes logarithmic at for convex objectives and under strong convexity.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.