The Missing Semantic Reference: Consensus Geometry and Observable Dynamics in Language Models

Intermediate states are increasingly used to read, compare, and manipulate the internal computation of language models, yet they are measured in native residual coordinates that carry no scientific reference. An invertible change of hidden basis leaves the model's function intact but changes every Euclidean length, angle, and shrinkage penalty computed from raw activations. We call this the missing reference frame. The model itself supplies the information needed to fix it: under a hidden reparameterization $h\mapsto hT$, the input embedding and the output decoder transform reciprocally, and balancing these two lexical interfaces has a unique positive-definite solution, the matrix geometric mean of their Gram operators, which is the one reference in which both interfaces induce the same geometry. Because coordinatewise models, shrinkage, and the direction of a lexical gradient are all defined relative to a reference, this model-induced reference is indispensable for results that describe the model rather than its parameterization. builds on it in three modules. Consensus Geometry fixes this data-free, model-wide reference and reads every state through the decoder's lexical covectors as a vocabulary-indexed Semantic State. Observable Dynamics estimates, without any decoder signal, the part of the terminal state that a source layer predictably supports, which defines Semantic Transitions across depth. Information Projection passes the Riesz-lifted exact forward-KL gradient through the same frozen response operator and allocates three coefficients in a globally convex problem. Across ten models from four families, SmolLM2, MiniCPM5, Granite 4.2, and Ministral 3 (135M to 14B parameters, 322 layers), resetting the Euclidean identity after a chart change or using a lexical covector as a vector distorts the measured quantity by 14% to 92%. lowers macro held-out forward KL from 2.984 for a full-affine Tuned Lens trained on the same data to 2.033, improves every layer and every held-out sequence, and also improves NLL and next-token accuracy on every model. The decoder-free Observable Dynamics alone accounts for most of this gain, and both the lens and its Semantic States remain stable as calibration data grow, while Semantic Transitions need the largest budgets.