TANGENT KERNELS MEASURE THE GAUGE: NON-IDENTIFIABILITY AND FULL-SPACE AUDITS OF TRAINING-FREE NAS PROXIES

Architectures are increasingly compared without training them, by reading spectral summaries (effective rank, condition number, kernel–target alignment) off an empirical tangent kernel as architectural properties. For softmax classifiers they are not. We prove that a function- and training-preserving softmax gauge $z\mapsto z+\gamma h_{\varphi }(x)1_C$ injects arbitrary PSD contamination into the trace NTK while leaving the softmax, cross-entropy loss, gradient and training trajectory untouched; its same-architecture form $W\mapsto W+\gamma 1_C{a}^{\top }$ leaves every gradient unchanged. The class-quotient NTK, the class-projected kernel used to analyse cross-entropy dynamics, is exactly invariant under both forms; sample centering removes the injected block only when its tangent features are constant across the probe set. Auditing whole search spaces (all $15,625$ NAS-Bench-201 cells and the $4,096$-cell TransNAS-Bench-101 micro space) shows the consequence is practical. Editing one weight matrix leaves the spoofed top-10 shortlist holding 2 of its 10 architectures, and an adversary choosing the gauge strength per candidate forces the selection of a network 34.6 accuracy points worse than the proxy's choice, while predictions, loss and gradients are preserved to $10^{-3}$ in float32. On the quotient kernel the same sweep leaves every shortlist and the selected architecture fixed, with drift below $10^{-4}$. The effect persists on a $3.0\times 10^8$-parameter vision transformer and across graph architectures, where sample centering amplifies rather than repairs drift; a pre-specified control on softmax-free heads bounds the claim. Consequently, a tangent-kernel number reported for a softmax model is a claim about a gauge choice unless it is quotiented, so proxy comparisons built on one are not falsifiable as stated. The repair is cheap (one extra evaluation of the diagnostic under a random function-preserving gauge, released as an audit) but precision-critical: default reduced-precision GPU matmuls alone make an exact invariance look violated.