When Geometry Flips Architecture Rankings: An Exact Theory of Alignment and Complexity

A growing family of model-selection heuristics scores an architecture by how well its inductive bias matches the geometry of the data. We ask when such a score can reverse: given two architectures and a group acting on input coordinates, when does scrambling the geometry flip which one generalises better? Working in a Gaussian sequence idealization in which an architecture acts as diagonal shrinkage in the Fourier basis of the input geometry, we give four results. First, averaging the data over a permutation group makes the covariance distance-independent, so the population fit of any exponential-decay locality score collapses to its floor, and when the mean off-diagonal covariance vanishes the score becomes unidentifiable rather than small, a failure mode we observe in $82\%$ of scrambled fits on natural image patches. Second, the risk gap between two architectures splits exactly into a geometry-free complexity gap $M$ and an alignment gain $S$ equal to $d$ times the covariance between the architectures' squared-bias profiles and the data spectrum; preference inverts under scrambling precisely when $S$ opposes and dominates $M$. Third, in a two-band model the inversion boundary admits an exact closed form whose minimum is $2\sqrt{\rho }+O(\rho )$ at noise level $t\asymp {\rho }^{-1/2}$, so a geometry-matched architecture pays off only inside a bounded window of sample sizes. Fourth, along any chain of subgroups the alignment gain is controlled by a quantity that contracts monotonically, giving a lock-in theorem: past a certain coarseness of scrambling, the ranking can no longer change. Experiments on natural image patches, a synthetic Gaussian field and handwritten digits confirm each prediction, and parameter-matched convolutional and fully connected networks invert where the theory says they should.