- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This study investigates how language models (LMs) encode numbers of characters in individual lines and is built upon an earlier study proposing the character counting manifolds.
Critical strengths
The authors carefully probe the earlier observation and extend the earlier finding. They outline the potential ways to identify manifolds more systematically and present alternative manifolds for character counting.
Critical weaknesses
Major
As the proposed mechanism is tested on the character counting only, the scope of this study is extremely restricted, which leaves almost no room for generalized tasks. The authors mention the character counting manifold may be ideal for testing new methods, but they did not test their new method against other manifolds. If their method could indeed discover a completely new manifold, it would be of great interest to those who study low-dimensional manifolds. Even rediscovering a previously reported a manifold (different from character counting manifold) would make this study more compelling.
Minor
-
Lindsey 2026 is a post on X, which is not an academic reference. I recommend replacing it with a more formal literature.
-
The figures from the Appendix need to be marked accordingly.
Questions for authors
-
The authors mention only intermediate layers show interesting geometry, which is spiral shape. Why do they consider it more interesting than linear encoding?
-
Only a few manifolds have been observed under the limited contexts, indicating that the observed low-dimensional manifolds could be epiphenomenon rather than the central mechanism underlying LMs. Can the authors provide their thoughts on this possibility?
2 comments
I appreciate that the authors have revised the manuscript with more details, and I do understand that the newly proposed method discovered a new type of manifold of character counting, but my major concern is that their method has been tested for a single task of “character counting” only. This does not guarantee that their method can be applied to discover new manifolds for additional different tasks/domains. As we already have an earlier study that found a manifold of character counting, I fail to see the significant value of another type of manifold of the same task. If they could provide evidence suggesting that their newly proposed method can be generally used in other domains, I would recognize the utility of their method, but so far I have not see such evidence.
Thank you, and some clarifications
Thank you for your review and your questions, and for giving us the opportunity to improve and clarify some things in the paper!
In Section 3 we indeed applied our method to discover a new manifold (perhaps more accurately a “multidimensional feature” in this case) that has not been reported yet in the literature. In particular, we found that language models represent character count modulo 8, 4, and 2 on strings of hexadecimal characters. We then investigated a variety of questions about this new structure, such as its universality across open-weights models. While this is a character count representation, it appears on a different distribution of inputs from the character count manifold of Gurnee et al. (2025) (hex instead of prose), and the character count manifold of Gurnee et al. (2025) did not exhibit this periodic structure. Do you feel that Section 3 qualifies as discovering a new manifold?
In their work first documenting the character count manifold, Gurnee et al. (2025) found character count to be represented along a spiral, and described how the model used this spiral to compare the character count with the line width. They also gave a theoretical explanation for the origin of that spiral geometry. So the fact that character count is represented along a spiral had already been established, and our finding in Section 2.1 is just that there are a narrow range of layers where that spiral can be found, which informs the experiments in the rest of Section 2.
In Appendix E, we also give our own theoretical account of why representations have the curvature they have. As we mention in Section 4, we analyze the character count spiral as part of a higher-dimensional structure that jointly encodes character count and document width. We find that we can reproduce the qualitative shape of this activation geometry from some theoretical analysis that assumes that models linearly encode the probability that each future token position contains a newline (Appendix E.3).
This is a great question but on balance we think the current evidence favors that many of these structures are causally important for model computation, and that understanding their particular geometry is important for doing intervention on model representations. The best examples of this are probably Engels et al. (2024), who show that one can steer the “days of the week” manifold along a circle to change model behavior, and also Gurnee et al. (2025), who perform causal interventions on the character count manifold and find that it affects the model’s prediction of the newline token in the way you’d expect.
Engels, J., Michaud, E., Liao, I., Gurnee, W. and Tegmark, M., 2025, May. Not all language model features are one-dimensionally linear. In International Conference on Learning Representations (Vol. 2025, pp. 84591-84622).
Gurnee, W., Ameisen, E., Kauvar, I., Tarng, J., Pearce, A., Olah, C. and Batson, J., 2026. When models manipulate manifolds: The geometry of a counting task. arXiv preprint arXiv:2601.04480.