Beyond Kernel Dynamics: Label Influence at Feature Emergence.

Linear influence estimates the effect of training-label perturbations using the derivative of a learned predictor. We show that feature emergence can sharply restrict its range of accuracy, even when this derivative is computed exactly. We study factorized models and smooth, bias-free two-layer neural networks of fixed finite width trained from small initialization by squared-loss gradient flow and fixed-step gradient descent. Near first emergence, the label Jacobian separates into a rank-one timing term proportional to $L=\log (1/ϵ)$ and a bounded remainder. Under nondegeneracy and, for gradient descent, stability and moment conditions, relative root-mean-square linearization error under isotropic Gaussian label noise vanishes if and only if $\sigma L\to 0$. At the critical scale, the response converges to a nonlinear random shift along the nominal trajectory. Propagated Gauss–Newton sensitivity misses the growing timing term; retaining curvature along the training direction restores it under gradient flow. We construct a derivative-preserving correction from the nominal trajectory and exact label response, prove critical-scale accuracy for exact trajectory evaluation, and give a finite-perturbation error bound for gradient flow. In a prospective test on width-64 networks, predictions from $L=20$ transfer to $L=40$ across three training configurations. A rate-based correction reduces retraining-forecast error by $70$–$83\%$ at the primary emergence checkpoint and two prespecified noise levels, compared with exact linear influence. Multi-seed comparisons distinguish curvature-approximation error from finite-perturbation error and identify phase-dependent and small-noise failures.