- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how the functional role of class directions in language-model representations depends on the answer-token mapping. The authors reverse binary answer labels while keeping the numerical input and underlying rule fixed, and decompose the resulting class directions into symmetric (mapping-invariant) and antisymmetric (mapping-sensitive) components. In Qwen2.5, interventions show that the symmetric component has a substantially larger effect at an intermediate layer, whereas the antisymmetric component dominates at the final layer. The paper validates this pattern using natural projection removal, norm-matched perturbations, random controls, multiple calibration splits, and an alternative answer vocabulary. An exact analysis of the final readout explains the final-layer effect through alignment with the output head, while cross-vocabulary classification shows that task-label information can remain recoverable even when the model fails to produce the correct mapped answer.
Critical strengths
The paper introduces a clean and well-controlled way to separate answer-mapping-invariant and answer-mapping-sensitive components of class directions by reversing answer labels while holding the underlying input and rule fixed. Importantly, the analysis goes beyond decodability: the authors use projection removal, norm-matched perturbations, random-direction controls, multiple calibration splits, and an alternative answer vocabulary to establish that the two components have distinct functional effects across layers.
A second strength is the careful connection between representation geometry and model behavior. In particular, the exact final-readout analysis explains why the antisymmetric component becomes influential near the output, while the frozen cross-vocabulary classifier demonstrates that task-relevant label information can remain recoverable even when the model fails to express it through the required answer mapping. Together, these results provide a useful methodological warning against interpreting a class direction independently of its intervention site and output interface.
Critical weaknesses
The main weakness is that the proposed symmetric/antisymmetric decomposition does not uniquely isolate the semantic factors motivating the analysis. By construction, the two components characterize the mapping-invariant and mapping-sensitive parts of the fitted class directions, but neither necessarily corresponds to a semantic decision or answer-token encoding. Moreover, as acknowledged in the appendix, normalizing the class directions can mix task-label main effects with label–mapping interactions. Although the interventions establish distinct functional effects across layers, they do not fully identify the underlying representations responsible for those effects. This limits how strongly the observed change in layerwise intervention sensitivity can be interpreted as evidence of a mechanistic transition from decision-related to answer-related representations.
A second concern is the limited generality of the central intermediate-to-final shift. The transition from symmetric dominance at an intermediate layer to antisymmetric dominance at the final layer is demonstrated primarily in Qwen2.5 on parity and divisibility, with additional vocabulary validation remaining within the same model. In the second architecture, Gemma, the antisymmetric component already dominates at the selected intermediate layer, while the reversed mappings fail behaviorally. Consequently, the central pattern has not been independently reproduced in another architecture that reliably follows both answer mappings. Although the authors appropriately limit their conclusions to the capable Qwen conditions, further evidence is needed to determine whether the observed layerwise behavior reflects a broadly shared representational process or a model- and interface-specific phenomenon.
Questions for authors
-
Beyond cross-task transfer and frozen classification, what evidence distinguishes the symmetric component from shared prompt-structural or task-format effects? Could the authors provide a control that more directly isolates decision-related information?
-
Can the authors reproduce the intermediate-to-final shift in another model that reliably follows both answer mappings? Additionally, could they hold the final candidate order fixed within the AB/BA vocabulary to isolate label-assignment effects from candidate-order effects?
5 comments
Q1. What distinguishes the symmetric component from shared prompt-structural or task-format effects? Could you provide a control that more directly isolates decision-related information?
A decision donor differs from the recipient in more than its decision, so we tested the alternatives this leaves open.
It is a different number, so its state could carry a property of that number, such as its parity, that later layers evaluate under the recipient's question. Two donors hold the parity fixed: another number of the same parity asked "Is this number divisible by 3?" with the opposite answer, and the same number asked "Is this number odd?" (both under the other mapping, so their correct letter is the recipient's). Under AB and BA, "yes" is also always the first clause, so a decision donor changes which clause applies. Minimal pairs fix this: two instructions that differ only in the words yes and no ("If yes, answer A; if no, answer B" and "If no, answer A; if yes, answer B", and the same with B named first). Within a minimal pair the same-parity donor has the recipient's parity, letter and clause; of these, only the decision differs. Shares at the layer where the decision donor peaks (parity; a minimal pair's own peak layer is given where it differs):
| Model | Decision donor (layer) | Same parity, divisible by 3? | Same number, odd? | Same parity, divisible by 3?, within a minimal pair (A first / B first) |
|---|---|---|---|---|
| Qwen2.5-7B | 86% (21) | 98% | 73% | – / – |
| Qwen3-4B | 95% (24) | 24% | 42% | 15% / 44% |
| Qwen3-8B | 78% (24) | 64% | 20% | 41% / 48% |
| Qwen3-14B | 93% (28) | 78% | 53% | 45% (27) / 74% (27) |
| Llama-3.1-8B | 96% (17) | 59% | 57% | 57% (16) / 59% (16) |
| Gemma-2-9B | 99% (28) | 95% | 81% | 39% (26) / 82% |
| Gemma-3-12B | 59% (28) | 19% | 3% | 22% / 16% |
| Phi-4 | 92% (22) | 71% | 50% | 45% / 61% (24) |
| Mistral-Nemo-12B | 76% (19) | 60% | 50% | – / – |
| Qwen2.5-7B, chat template | 100% (21) | 99% | 97% | 66% / 99% |
(–: below our 80% accuracy criterion on that pair, which applies to both pairs of Mistral-Nemo and to Qwen2.5 with A first; for Qwen2.5 with B first, state replacement was not run with plain-text prompts. With its chat template, in the last row, Qwen2.5 follows all four instructions.)
The same-parity donor still changes more than half of the answers in seven of nine models (the exceptions, Qwen3-4B and Gemma-3-12B, reach 19–24%), with a letter-only control at most 20%, so in these models the parity of the number does not explain the transfer; in the two exceptions most of it is lost once the parity is held fixed. Within the minimal pairs this donor changes 15% to 99% of the answers, about half in most models, and often more into recipients whose instruction names yes first (Qwen3-8B with A first: 72% against 10%): the decision itself crosses over, and the rest of what crosses over in the standard pair depends on the wording and on the input. The decision donor of the minimal pair alone (other number, same letter and clause) changes 45% to 100%, less than in the standard pair in several models. The paper reports all of this in Section 3.3 and a new appendix table, and qualifies its reading of the first stage accordingly.
On normalization: without the scaling, U = (d_AB + d_BA)/4 is the effect of the decision and W = (d_AB − d_BA)/4 the effect of the correct letter. The scaling makes s proportional to U + ρW, with ρ set by the difference in length of the two class directions; in every model the cosine between s and U is above 0.999 at the chosen layer. On parity U and W are nearly orthogonal in most models, and removing W then costs about the same as removing a. In Qwen2.5 they overlap (cosine 0.44 on parity), so W contains part of the decision direction, which is why removing it costs more than removing a (0.52 against 0.09). The normalization mixes the two only by this small factor, so s stays essentially the decision effect U.
On edit size: where little of a is present, a small effect of removing it is partly built in. We therefore moved both components by the same amount at every layer from four before to four after the chosen layer, in every model. In a second version every state is pushed towards the wrong decision along s and towards the wrong letter along a, so that a noisy sign cannot weaken the push along a. At every such layer where the cosine between the two class directions exceeds 0.85 (73 cases), moving along s costs more than moving along a in both versions, with an interval excluding zero in 72; where it is below 0.7, moving along a costs more in 71 of 74 cases.
On task-format effects (Q1): a class direction is a difference between yes- and no-examples under the same template, so what the classes share, including most of the format, cancels, and the donors of part 2 hold the letter, the clause and the parity fixed. Changing the format also shows that the design does not find a decision-like stage everywhere: for the same 200 news articles, a decision donor changes 91–100% of the answers at its peak layer when they are asked a yes/no question ("Is the proposed topic correct?"), but moves at most 6% of them to its own topic when one of four topics must be chosen from lettered options, and at most 31% with an instruction.
Thank you for the careful reading and for two questions that asked for specific experiments. We ran them, together with further controls they led us to. The revised paper is uploaded. Numbers are on 100 held-out items unless stated otherwise, and brackets are 95% bootstrap intervals.
Q1 (part 2): three new donors fix the number's parity, the letter and the clause; the decision still crosses over, but in most models only part of what crosses over is the decision, and the paper now says so. Q2 (part 4): eight more models follow both mappings and show the two stages, clearly in seven of the nine, and with the final candidate list held fixed the ordering at layer 22 is unchanged. W1 (parts 1 and 3): the tests do not rely on the definition of the components.
W1. The decomposition does not uniquely isolate the semantic factors, and normalization can mix the main effect with the interaction.
We agree that the definition by itself identifies nothing, and the paper now says so. The revision tests what the components carry with experiments that do not rely on the definition.
The central one uses no fitted direction. We replace the entire last-token state of a recipient prompt at one layer by the state of a donor prompt and let the remaining blocks run. A decision donor has the opposite decision and the same correct letter (a number of the opposite class under the other mapping); a token donor has the same decision and the other letter (the same number under the other mapping). In all nine models there are intermediate layers at which a decision donor changes most answers and a token donor almost none, and later layers at which a token donor changes all of them. The table ties this state-level result back to the shared component s: at the layer where the decision donor peaks, transplanting only the coordinate along s carries about as much as the whole state in eight models (within four percentage points) and more in Mistral-Nemo (88% against 76%).
(Parity; shares over the recipients for which recipient and donor are answered correctly under both mappings.)
These are maxima over layers. At the layer chosen on calibration data, the decision donor changes at least 75% of the answers in six models and fewer in Qwen3-14B, Gemma-3 and Mistral-Nemo; Table 1 of the revision now gives both. Near the output the same holds for the flipping component a: transplanting only its coordinate from a token donor changes 99% of the parity answers at the last layer of Qwen2.5, and 99% at layers 26 and 27 (per-layer outputs in the supplementary material).