From Language Statistics to Semantic Control: Spectral Geometry and Group Actions

Language models represent ordered concepts in structured activation patterns that can span many directions. An intervention that looks correct in a low-dimensional projection can therefore leave other coordinates inconsistent. We connect language statistics, spectral geometry and semantic control to understand how this structure arises and how to change it consistently. Starting from a latent variable data generating process, we relate latent semantic structure to word co-occurrence and, under explicit embedding assumptions, to Fourier coordinates. Group-based month and weekday steering transfers to held-out prompts, while an open-interval extension transports a fitted neural year component bidirectionally. To steer richer concepts, we must also understand which geometries arise beyond the circle and what transformations they support. Drawing on lattice normal modes, we interpret spectral modes as semantic phonons: coordinated patterns of variation across semantic positions. Sturm–Liouville theory explains how boundary conditions shape these patterns for second-order operators, yielding a broader family of curved geometries even when the interior rule is unchanged. This work connects understanding neural geometry to reliable control: for safety-relevant manifolds such as refusal, interventions must intervene across all relevant subspaces.