An Infinite Information Exponent that Gradient Descent Still Learns at the Optimal Rate: The Value Catalyst in Softmax Attention

The information exponent of a single-index target describes how gradient descent learns it: an exponent-$k$ target can trap online SGD for $\tilde{\Theta }(d^{k-1})$ samples, while an infinite exponent ordinarily indicates a direction that gradient descent cannot recover. We show that this interpretation fails for a coupled attention model. Consider a softmax head that selects a token through a key direction $w^{\star }$ and reads a scalar through an orthogonal value direction $u^{\star }$, $y={\sum }_i\mathrm{softmax}{\left(\beta ⟨w^{\star },x_i⟩\right)}_i⟨u^{\star },x_i⟩.$ The selection direction has infinite information exponent: the label is mean-zero conditional on the key scores, so every key-only correlational query and every single-index surrogate gradient vanishes. Nevertheless, projected gradient descent on the invariant slice of the coupled key-value loss learns $w^{\star }$ with $n=\Theta (d)$ samples per step; unrestricted random initializations exhibit the same cascade numerically. The value direction has information exponent one; once it is partially learned, the key gradient gains the explicit component ${\nabla }_w\mathcal{L}\cdot w^{\star }=-\frac{{\beta }^2\mu (1-\bar{P}{)}^2}{L-1},$ where $\mu =⟨u,u^{\star }⟩$ and $\bar{P}$ is the mean softmax collision probability. This value catalyst produces a value-to-selection cascade with dimension-free population convergence. Squaring the label exposes the same selection direction at generative exponent two and yields a one-step spectral estimator with a linear dimensional exponent up to logarithmic factors. For every noise level, an oracle lower bound that reveals all exact key scores still requires $n=\Omega (d/L)$ samples for constant projective accuracy; at fixed positive noise, a Fano bound additionally gives explicit parameter dependence. The phenomenon extends to additive and multiplicative combinations of orthogonal-subspace heads, and CPU experiments check the predicted gradients, spectra, and scaling laws.