Symmetry, Not Dimension: When Sparse-Coding Atoms in a Feature Plane Are Identifiable

Sparse autoencoders trained from different seeds disagree, which concurrent work attributes to feature manifolds of dimension $\ge 2$; in the sharpest analysis, from an isotropy assumption. For a feature plane, the sparse-coding objective under a rigid rotation of its atoms is a circular cross-correlation between the angular density and a configuration-dependent cost kernel, flat along the rotation iff the density's relevant Fourier coefficients vanish, else pinned with computable curvature. Under $L_1$, dimension $\ge 2$ is necessary for that flatness, not sufficient. On densities from models' own rings, twelve-seed dictionaries meet predictions the density condition makes and symmetry cannot: agreement at an installed harmonic (strongest in $28$ of $35$ Gemma-2-2B tilts at $q\ge 2$ no symmetry explains; chance $8.8$), and its loss when one Fourier coefficient is removed from a spread density, including the number circles of Llama-3.1-8B and Pythia-6.9B, but not from clustered rings. Only ReLU+$L_1$ seeds reliably sit near their landscape's minimum; TopK's in-plane optimum at $k\ge 2$ is flat on full-support densities. On released dictionaries and our own, ablating a dictionary's plane for a concept transfers across dictionaries in all $20$ cells, its most selective latent in $7$, failing where dictionaries disagree on its item. Making the data exactly symmetric in one plane removes the seeds' agreement on its atoms (matched atoms $2^{\circ }\to 16^{\circ }$ apart) and reinstalling one harmonic restores it; a plane found in natural text without probes transfers too. Claims should attach to the plane, not the atom. Code, records and some trained dictionaries are in the supplement; all will be public at https://github.com/xxx/xxx upon acceptance.