- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper shows that models represent the order of concepts along a direction. The authors first show that they are able to find this direction using difference of mean activations. They then show that by combining this direction with the gradient of the answer logits, they can identify a causal direction that enables controlled manipulation of the model.
Critical strengths
The main strength of this paper is that they were able to "build" a steering vector without any training. They did it by adding to the difference of means vector the gradient of the answer logits.
Critical weaknesses
I don't find the results novel or exciting enough. I think there are already a few papers that indicates that models uses an ordinal direction in the residual stream [1,2,3]. In addition most of the analysis techniques, like difference of means [4] or DAS , are also not novel.
References: [1] https://arxiv.org/pdf/2607.04167 [2] https://arxiv.org/pdf/2409.05448 - see ordering IDs [3] https://aclanthology.org/2023.findings-acl.383.pdf [4] https://arxiv.org/abs/2406.11717
Questions for authors
I don't have question that will materially affect my assessment. I found the paper clear.
4 comments
Reviewer toQs reply (Part 2/2).
In summary, earlier work has described the geometry of continuous concepts such as space and time (Gurnee & Tegmark, 2024), cyclic concepts such as days and months (Engels et al., 2025), and categorical and hierarchical concepts (Park et al., 2025). Ordered categorical concepts were missing from this picture. We show across 26 ladders and four models that they are arranged along an axis fixed by their two endpoints, and that the model uses this axis when it compares them. We also explain why a trained direction is more effective, and build one without training by combining the endpoint axis with the answer-logit gradient, which the reviewer highlights as the main strength of the paper. Thus, we believe these results add a missing piece to the account of concept geometry rather than reapplying known methods.
- W2. "In addition most of the analysis techniques, like difference of means [4] or DAS, are also not novel."
[4] shows that a difference-in-means direction can causally mediate a behaviour, and we agree that our tools, difference-in-means, interventions and DAS, are standard. We do not claim them as a contribution. What we claim is what they reveal about ordered knowledge. The simplicity of the method is also what makes it valuable. The axis is built from the two endpoints alone, so the intermediate rungs play no part in it, and finding them in order along the axis is a held-out result.
All in all, we agree that these works are relevant and should have been discussed. We have added them to the related work of the revised version, [1] and [2] as studies of orders defined by the input (structural position and order of mention), [3] as evidence of magnitude structure in number embeddings, and [4], together with Belrose (2023), among the applications and foundations of difference-in-means. They do not, however, cover the core contributions of our paper: the endpoint axis for orders learned in pretraining, its causal role in comparisons, and the construction of an effective direction without training.
We hope that these clarifications and the additions to the related work address the reviewer's concerns, and that the reviewer will consider revising their assessment.
References:
[1] Bassi, S. and Tomar, S. Geometry of ordinal representations in language models. arXiv:2607.04167, 2026.
[2] Dai, Q., Heinzerling, B., and Inui, K. Representational analysis of binding in language models. arXiv:2409.05448, 2024.
[3] Shah, R. S., Marupudi, V., Koenen, R., Bhardwaj, K., and Varma, S. Numeric magnitude comparison effects in large language models. Findings of ACL, 2023.
[4] Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., and Nanda, N. Refusal in language models is mediated by a single direction. NeurIPS, 2024.
Belrose, N. Diff-in-means concept editing is worst-case optimal: Explaining a result by Sam Marks and Max Tegmark. EleutherAI Blog, 2023. https://blog.eleuther.ai/diff-in-means/
Engels, J., Michaud, E., Liao, I., Gurnee, W., and Tegmark, M. Not all language model features are one-dimensionally linear. ICLR, 2025.
Gurnee, W. and Tegmark, M. Language models represent space and time. ICLR, 2024.
Park, K., Choe, Y. J., Jiang, Y., and Veitch, V. The geometry of categorical and hierarchical concepts in large language models. ICLR, 2025.
Thank you for your response and for the further discussion. I will update my score.
Thank you for taking the time to consider our rebuttal. We are glad that our response addressed your concerns and appreciate your further assessment of our work.
Reviewer toQs reply (Part 1/2).
We thank the reviewer for their review and for pointing us to these relevant works. However, we respectfully disagree that [1–4] anticipate our results. They provide important precedents for studying representation geometry and causal interventions, but they address different questions. [1] and [2] mostly study structural positions and order of mention, defined by the input, [3] studies representations of numerical magnitude, and [4] identifies a direction that mediates refusal. To our knowledge, none of them looks at how LLMs represent ordered categorical knowledge acquired in pretraining, or at whether an axis built only from the endpoint categories recovers the order of the intermediate ones and is used by the model when it compares them. In addition, [1] and [3] make no causal claim about the model's behaviour. Changes related to this review are highlighted in purple.
These papers do find directions related to order in the residual stream, and we have now included them in our paper. However, the orders they study are of a different kind, as they are given by the prompt ([1], [2]) or by the number itself ([3]). We study orders between categories that the model learned in pretraining, and we show that an axis built from the two endpoint categories places the remaining ones and is used by the model when it compares them. We go through each paper below.
[1] (Bassi & Tomar, 2026) is the closest to our work in name. It studies four ordinal variables: bracket nesting depth, Python indentation level, position in a markdown table and numeric magnitude. In the first three, the value is determined by the input sequence (e.g. the nesting depth of a token is the number of brackets still open at that point in the prompt), and in the fourth the number itself carries its magnitude. Our ladders, such as noble titles, military ranks or physical scales, are not stated in the prompt, so the order must come from what the model has learned. The methods also differ. [1] fits probes and PCA on all values, while our axis is built from the two endpoints, with the intermediate rungs held out. Finally, [1] states that its patching "tests informational concentration in the subspace, not behavioral causality", while we test whether the model uses the axis when it decides which of two rungs ranks higher.
[2] (Dai et al., 2024) does make a causal claim, but of a different kind. It studies the "ordering ID" of entities in prompts such as "The coffee is in Box Z, the stone is in Box M, the map is in Box H... Box Z contains the ___". The ordering ID is the position at which an item is mentioned: "coffee" has index 0 because it comes first, and would have another index if the sentence were reordered. In contrast, a viscount outranks a baron, and Saturn is farther from the Sun than Mercury, whatever the order in which they appear in the prompt. [2] finds its direction with PCA over all positions and intervenes by adding steps of a chosen size along it, which moves the retrieved object down the list. We build the axis from the two endpoints alone, exchange the coordinates of the two compared concepts, so the size of the intervention is set by the data, and test whether the model's judgment is reversed, against norm-matched random directions.
[3] (Shah et al., 2023) asks whether the embeddings of the numbers 1–9 in pretrained models (BERT, GPT-2, T5, ...) show the distance, size and ratio effects known from human number comparison, by testing whether the cosine similarity between two number tokens varies with those quantities. Their variable is numerical magnitude, where the symbol itself denotes the quantity, and each number is embedded on its own, outside any prompt. Apart from our two number ladders, our ladders have no numerical metric, so nothing in the word reveals its rank. [3] does not construct a direction or intervene on the model. Its aim is the cognitive plausibility of number representations, while ours is the geometry of learned categorical orders and its use in comparisons.