Sharp Outlier Amplification in Error-Feedback Clipping

Gradient clipping limits the immediate effect of an unusually large gradient. Error feedback stores the discarded part and reintroduces it in later updates. How much of clipping's protection survives this memory? We characterize the maximum subsequent loss after an isolated outlier for residual error-feedback clipping. On a scalar quadratic, an outlier of magnitude $A$ produces a peak loss asymptotic to $\eta cA$, despite every update having norm at most $\eta c$. We obtain this sharp law from a support-function Lyapunov inequality that applies to projections onto arbitrary compact convex update sets. The same inequality shows that capping the stored residual preserves exact-gradient convergence for every step size $0<\eta <2/L$. It also gives an amplitude-independent bound after finitely many arbitrary corruptions. For a Euclidean update threshold $c$ and residual threshold $B$, the single-outlier bound is $L{\eta }^2{c}^2/2+\eta cB$; the dependence on $B$ is asymptotically sharp on quadratics. An explicit conservation identity identifies the gradient mass sacrificed by this protection. Short, reproducible computations illustrate the bounds and their limitations.