acceptodds
Under review as a conference paper at ICLR 2027

Accelerated Stochastic Method under -Smoothness and Heavy-Tailed Noise

Abstract

We develop an accelerated stochastic method for convex -smooth optimization under heavy-tailed noise. The unbiased oracle has a finite -th noise moment, , with constant, gradient-dependent, and gap-dependent terms. Our method combines accelerated updates with clipping, projection, and phase restarts. Each iteration uses a single stochastic-gradient sample, without minibatching. We prove high-probability convergence to any desired accuracy despite clipping bias. The deterministic terms retain square-root dependence on both and . In our three-component noise model, strong convexity leaves a polynomial accuracy cost only for the constant component; without it, the dependence is logarithmic even for . More generally, for noise moments scaling as , the stochastic cost becomes logarithmic at for convex objectives and under strong convexity.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.