WHITENING INVERTS THE HIERARCHY: WHAT THE NORM OF A WHITENED EMBEDDING MEASURES

Whitening a foundation-model embedding and reading off its squared norm has become a training-free likelihood surrogate, justified by a single observation: the whitened coordinates are close to standard normal. We show that this observation is guaranteed by the projection central limit theorem, so it says nothing about the joint law, and that the joint law is not Gaussian. Against clones with identical mean and covariance the whitened radius is over-dispersed, for every encoder we tested and across three training objectives, and the published agreement between empirical and theoretical norm statistics is an algebraic identity that in-sample whitening imposes on any distribution. The mechanism is elementary. Whitening divides every direction by its eigenvalue and so inverts the encoder's hierarchy, moving the mass of the norm off the semantic subspace and onto the near-degenerate directions the encoder treats as noise, where a single input-dependent scale sets the variance; we identify that scale from two moments and confirm it by predicting, with no free parameter, how strongly disjoint halves of the spectrum move together. The squared whitened norm is therefore a Mahalanobis estimate of semantic atypicality rather than a log-likelihood, and that identification explains both its successes and its failures: it ranks and detects well, agreeing with a nonparametric density estimate and with encoders that share no objective, training data or supervision, while its Gaussian thresholds are wrong by orders of magnitude. We prove that a contrastive objective built on cosine similarity cannot determine the radial law, derive in closed form the group size at which a coordinatewise test can detect a given kurtosis, and give the one-dimensional recalibration that restores nominal tail probabilities.