Learning between the peaks: sharp asymptotics for kernel ridge regression under power-law anisotropy

We study Kernel Ridge Regression for polynomial inner-product kernels under anisotropic Gaussian data, where the eigenvalues of the input covariance follow a power-law decay with exponent $\alpha \ge 0$. In the polynomial high-dimensional regime $n=\Theta (d^{\kappa })$, we derive asymptotically sharp expressions for the kernel spectrum and the generalization error, revealing how anisotropy reshapes the learning curves. For weak anisotropy ($0<\alpha <1$), the problem remains effectively high-dimensional and retains some features of the isotropic case, while departing from it in others: the variance still peaks at integer sample complexities $\kappa \in \mathbb{N}$, though these peaks are progressively flattened as $\alpha \to 1^{-}$. Meanwhile, for targets strongly aligned with the data's principal directions, the bias drops at fractional sample complexities $\kappa =m(1-\alpha ),m\in \mathbb{N}$. This early transition effectively decouples the bias from the interpolation peaks. For strong anisotropy ($\alpha >1$), the effective dimension is constant. The variance stops depending on sample size, plateauing under ridgeless interpolation or vanishing at an explicit rate under fixed ridge penalty. The bias undergoes a sharp transition governed by the target's decay rate: below a threshold, learning is abrupt rather than gradual; above it, the bias decays as a power law, recovering classical source and capacity rates. We finally specialize these results to single-index targets, showing how the alignment of the index with the data's principal directions determines the effect of anisotropy on learning. Together, our results clarify how the input geometry shapes the kernel features and fundamentally impacts its generalization properties.