What does it take to find a feature manifold?

There is a growing interest in identifying "feature manifolds" or "multidimensional features" in neural network activations, where computational variables are encoded with some organized geometry in low-dimensional subspaces of activation space. However, the craft of discovering and characterizing these structures in practice remains nascent. In this work, we perform a deep case study of the representation of "character count" in language models, building on the work of Gurnee et al. (2025). We use this setting to explore a variety of questions about the characterization and automatic discovery of feature manifolds. In particular, we describe in detail how sparse autoencoders "tile" the character count manifold and then test whether methods that cluster SAE latents like Bhalla et al. (2026) can re-discover this structure. We find that clustering methods can correctly group together the latents that tile the character count manifold, but using these latents to then visualize raw model activations does not surface clean structure. We then apply these lessons to discover a novel representation of character count that appears on a different data distribution from Gurnee et al. (2025), finding that models represent character count modulo 8, 4, and 2 on strings of hexadecimal characters. We investigate the functional role of these representations and their universality across models. We lastly consider whether these "character count" representations are part of a higher-dimensional joint representation of other computational variables, which leads to some new ways of visualizing model computation. Overall, we hope that our work provides the community with useful lessons for the identification of feature manifolds and a calibrated picture of the difficulty of this task.