Sharp Heterogeneity Barriers for Adam: Basis-Aware Convergence and High-Dimensional Divergence

Convergence proofs for Adam on finite sums $f=n^{-1}{\sum }_i{f}_i$ often assume strong growth, ${\sum }_i\Vert \nabla f_i{\Vert }^2\le D_1\Vert \nabla f{\Vert }^2$. The constant is at least $n$, with equality exactly when all component gradients agree, yet the known cyclic counterexamples have $D_1=\Omega (n^2)$. We ask how much heterogeneity Adam tolerates at fixed memory $0\le {\beta }_1\le {\beta }_2<1$, and the answer depends on the dimension and on the coordinate system. In one dimension the threshold is $D_1=2n$: a sharp inequality for Adam's averaged normalized update gives convergence below it, for strongly convex averages with Lipschitz component gradients, and divergent instances exist at $D_1=2n+O_{{\beta }_1,{\beta }_2}(\log n)$. In higher dimensions, positive-definite quadratics with a common minimizer can make Adam diverge with $D_1/n$ arbitrarily close to one, under cyclic order, IID minibatches and random reshuffling; in the random cases the probability of divergence tends to one for distant starting points. An outward divergent ray needs $D_1/n>1+4\kappa /(\kappa +1{)}^2$ at condition number $\kappa$, and this is exact under cyclic order. Bounded conditioning gives a dimension-independent convergence region, which is nonempty exactly when $\kappa <3+2\sqrt{2}$; for ${\beta }_1={\beta }_2=0$ this limit is sharp, because at every larger $\kappa$ trajectories that keep turning can diverge as $D_1/n\downarrow 1$. The threshold $2n$ returns under coordinatewise growth, for separable averages and with a single shared second moment, also under arbitrary epoch orders and IID sampling, with non-asymptotic bounds for cyclic order. Since a quadratic average is separable in its eigenbasis, a change of basis alone can turn divergence into convergence. All results allow denominator regularization and bias correction, and interval arithmetic certifies cyclic and IID divergence at the default $(0.9,0.999)$. In a small Transformer, function-preserving rotations consistently increase test loss, and spectral bounds on Adam's frozen updates separate two bases with identical aggregate growth.