- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The submission approaches activation steering in the spectral coordinates to support the symmetries in particular concepts such as periodic concepts (weekdays, months) and ordered concepts (year). Authors argue that although one could identify and read the concept in low-dimensional projections of corresponding activations, such readable view could only represent a part of broader picture (supported by less than 50% total varianace explained by low dimensional projection). Then they propose steering in full spectral coordinates as truncating it can potentially ignore large contributions coming from high-frequency modes. The modest experiments are concentrated on "circular" weekday shifting, and extended to year-shifting.
Critical strengths
- A nice conceptual contrast between decoding and control: The idea of high-frequency modes contribute non-negligible error term provides an interesting motivation for decoupling concept decoding and steering
Critical weaknesses
- The motivation of the methodology (other than the part stated in Strength section above) is heavily rooted in existing works with minimalistic new insights. For example, Eqns (1) and (2) are introduced as authors' contribution while they directly come from taking P(i) to be a uniform distribution in Section 4 of [1], followed by the same fourier diagonalization derivations.
- The difference between the results in [2] is not explicitly discussed other than mentioning that the authors use the derivations in [2, Appendix G] for Fourier Fourier embeddings of circulant similarity matrices and cyclic shifts on retained invariant subspaces. This makes it vague/harder for the reader to understand the significance of this work given [2, 3]. Is it the experiments on different steering tasks, transferring to held-out prompts etc.? How important are these new empirical observations?
References:
[1] Dhruva Karkada, Daniel J. Korchinski, Andres Nava, Matthieu Wyart, and Yasaman Bahri. Symmetry in language statistics shapes the geometry of model representations. In International Conference on Machine Learning, 2026
[2] Gurnee et al., When Models Manipulate Manifolds: The Geometry of a Counting Task (2026)
[3] Sheridan Feucht, et al.. Arithmetic in the wild: Llama uses base-10 addition to reason about cyclic concepts. arXiv preprint arXiv:2605.01148, 2026
Questions for authors
- See Weakness 2
- Section 6 motivates candidate representation geometries through Sturm-Liouville boundary conditions, but its operational contribution remains unclear. The year experiment constructs coordinates by numerical kernel eigendecomposition and fits transitions from supplied year adjacencies. It does not demonstrate that selecting an operator or boundary condition improves representation prediction or behavioral control. Could authors clarify on the operational role of Sturm-Lioville framing in this work?
We thank the reviewer for the time taken with our manuscript and for the assessment. Taking these comments seriously, we have decided to withdraw the paper and appropriately rework our manuscript. On the comments: it is true that equations (1) and (2) follow from [1] under a uniform prior. We intended them as an illustration rather than a contribution. On the relation to [2]: the difference is that the operator in our work reads the current coordinate from a live activation, rather than being supplied it. We tested this on held-out prompts against a displacement-matched rate-permutation control. We agree this distinction should have been stated more explicitly.