The Statistical Cost of Redundant Width in Bayesian Quadratic Neural Networks

How much prediction error do redundant directions contribute when a quadratic network learns a low-rank target, and when does regularization suppress that contribution? We study this question through the singular geometry of the factor map $W\mapsto W{W}^T$. For a global Gaussian factor prior, fixed dimensions, and width $m\ge d+1$, we prove a noisy boundary law for expected Gibbs and posterior-mean prediction risks. At matched likelihood temperature, its unregularized mean risk is ${\sigma }^2[p+\alpha (d-r)]/n+o(n^{-1})$, where $p=d(d+1)/2$ and $\alpha =(m-d-1)/2$: each additional neuron costs ${\sigma }^2(d-r)/(2n)+o(n^{-1})$ within the same PSD function class. An Euler–Stein identity establishes beneficial vanishing regularization; a PSD-estimator theorem calibrates the risk gap. The same boundary law resolves teacher modes of order $n^{-1/2}$. At zero added decay, every finite nonzero PSD local signal lowers mean risk below the zero-signal endpoint. At $m=d+1$, the zero- and strong-signal endpoints coincide, yet the risk is strictly lower between them. The population quartic-to-quadratic crossover as a regularization gap opens explains the geometry; the empirical law retains the surviving random tilt. Finite-data experiments and limiting-law quadrature examine these distinct predictions. The broader local-prior empirical crossover remains conjectural. The proved law links the width cost to regularization and a weak-signal transition that fixed-rank coefficients alone cannot describe.