NTK Regression under dual power-law model: Deterministic Equivalents via SDE and PDE Methods

In the infinite-width limit, training a neural network (NN) via gradient descent reduces to kernel ridge regression (KRR) on a deterministic kernel, the Neural Tangent Kernel (NTK). While the isotropic-input case of the NTK is well understood, the spectra of real world inputs always exhibit power-law decay, and the alignment between this spectrum and the target signal has been shown to govern the generalization behavior of the model. Motivated by this, we study finite-width NTK regression under a dual power-law dataset, where data $\mathbf{x}\in {\mathbb{R}}^p$ is drawn with covariance ${\mathbf{\Sigma }}_{jj}=j^{-\alpha }$ and labels are generated from power-law truth vector ${\mathbf{\beta }}_j=j^{-r}$. We introduce a novel technique combining random matrix theory (RMT) and stochastic matrix calculus to derive deterministic equivalents for random matrix functionals that determine the prediction risk, including a new $\text{signal-weighted}$ resolvent, which yields an explicit formula for the bias-variance decomposition of NTK regression in the high-dimensional proportional regime. As an application we prove a scaling law for finite-width NTK regression, explicitly characterizing the decay rate of the optimally-tuned excess risk jointly in the data and width resources.