acceptodds
Under review as a conference paper at ICLR 2027

From Numerical Rules to Answer Tokens

Abstract

The effect of an intervention on a language model's hidden state can depend on the token used to report an answer. We test this dependence on parity and divisibility by three, exchanging the answer labels while keeping the number and rule fixed. Class directions fitted under the two mappings yield a symmetric component that keeps its sign under the exchange and an antisymmetric component that reverses it. In Qwen2.5-7B, natural projection removal of the symmetric component at layer 22 lowers the correct-answer logit margin by about three, while removing the antisymmetric component changes it by less than 0.1. At the final layer, their order of influence reverses. This change holds under norm-matched edits, across three calibration splits, and for divisibility under a second answer vocabulary. An exact analysis of the final readout explains the final-layer effect. The intermediate symmetric direction also supports classification across answer vocabularies: a classifier fixed on one vocabulary recovers all 100 held-out parity labels under another mapping where the model's own candidate answers are only 50% accurate.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic Interpretability, Representation Geometry, Causal Interventions, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Dfp3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how the functional role of class directions in language-model representations depends on the answer-token mapping. The authors reverse binary answer labels while keeping the numerical input and underlying rule fixed, and decompose the resulting class directions into symmetric (mapping-invariant) and antisymmetric (mapping-sensitive) components. In Qwen2.5, interventions show that the symmetric component has a substantially larger effect at an intermediate layer, whereas the antisymmetric component dominates at the final layer. The paper validates this pattern using natural projection removal, norm-matched perturbations, random controls, multiple calibration splits, and an alternative answer vocabulary. An exact analysis of the final readout explains the final-layer effect through alignment with the output head, while cross-vocabulary classification shows that task-label information can remain recoverable even when the model fails to produce the correct mapped answer.

Critical strengths

The paper introduces a clean and well-controlled way to separate answer-mapping-invariant and answer-mapping-sensitive components of class directions by reversing answer labels while holding the underlying input and rule fixed. Importantly, the analysis goes beyond decodability: the authors use projection removal, norm-matched perturbations, random-direction controls, multiple calibration splits, and an alternative answer vocabulary to establish that the two components have distinct functional effects across layers.

A second strength is the careful connection between representation geometry and model behavior. In particular, the exact final-readout analysis explains why the antisymmetric component becomes influential near the output, while the frozen cross-vocabulary classifier demonstrates that task-relevant label information can remain recoverable even when the model fails to express it through the required answer mapping. Together, these results provide a useful methodological warning against interpreting a class direction independently of its intervention site and output interface.

Critical weaknesses

The main weakness is that the proposed symmetric/antisymmetric decomposition does not uniquely isolate the semantic factors motivating the analysis. By construction, the two components characterize the mapping-invariant and mapping-sensitive parts of the fitted class directions, but neither necessarily corresponds to a semantic decision or answer-token encoding. Moreover, as acknowledged in the appendix, normalizing the class directions can mix task-label main effects with label–mapping interactions. Although the interventions establish distinct functional effects across layers, they do not fully identify the underlying representations responsible for those effects. This limits how strongly the observed change in layerwise intervention sensitivity can be interpreted as evidence of a mechanistic transition from decision-related to answer-related representations.

A second concern is the limited generality of the central intermediate-to-final shift. The transition from symmetric dominance at an intermediate layer to antisymmetric dominance at the final layer is demonstrated primarily in Qwen2.5 on parity and divisibility, with additional vocabulary validation remaining within the same model. In the second architecture, Gemma, the antisymmetric component already dominates at the selected intermediate layer, while the reversed mappings fail behaviorally. Consequently, the central pattern has not been independently reproduced in another architecture that reliably follows both answer mappings. Although the authors appropriately limit their conclusions to the capable Qwen conditions, further evidence is needed to determine whether the observed layerwise behavior reflects a broadly shared representational process or a model- and interface-specific phenomenon.

Questions for authors
  1. Beyond cross-task transfer and frozen classification, what evidence distinguishes the symmetric component from shared prompt-structural or task-format effects? Could the authors provide a control that more directly isolates decision-related information?

  2. Can the authors reproduce the intermediate-to-final shift in another model that reliably follows both answer mappings? Additionally, could they hold the final candidate order fixed within the AB/BA vocabulary to isolate label-assignment effects from candidate-order effects?

5 comments

Thank you for the careful reading and for two questions that asked for specific experiments. We ran them, together with further controls they led us to. The revised paper is uploaded. Numbers are on 100 held-out items unless stated otherwise, and brackets are 95% bootstrap intervals.

Q1 (part 2): three new donors fix the number's parity, the letter and the clause; the decision still crosses over, but in most models only part of what crosses over is the decision, and the paper now says so. Q2 (part 4): eight more models follow both mappings and show the two stages, clearly in seven of the nine, and with the final candidate list held fixed the ordering at layer 22 is unchanged. W1 (parts 1 and 3): the tests do not rely on the definition of the components.

W1. The decomposition does not uniquely isolate the semantic factors, and normalization can mix the main effect with the interaction.

We agree that the definition by itself identifies nothing, and the paper now says so. The revision tests what the components carry with experiments that do not rely on the definition.

The central one uses no fitted direction. We replace the entire last-token state of a recipient prompt at one layer by the state of a donor prompt and let the remaining blocks run. A decision donor has the opposite decision and the same correct letter (a number of the opposite class under the other mapping); a token donor has the same decision and the other letter (the same number under the other mapping). In all nine models there are intermediate layers at which a decision donor changes most answers and a token donor almost none, and later layers at which a token donor changes all of them. The table ties this state-level result back to the shared component s: at the layer where the decision donor peaks, transplanting only the coordinate along s carries about as much as the whole state in eight models (within four percentage points) and more in Mistral-Nemo (88% against 76%).

Model (layer of the peak)Whole stateOnly the coordinate along sEverything except sToken donor
Qwen2.5-7B (21)86%88%4%0%
Qwen3-4B (24)95%93%0%0%
Qwen3-8B (24)78%79%0%0%
Qwen3-14B (28)93%95%0%0%
Llama-3.1-8B (17)96%95%0%0%
Gemma-2-9B (28)99%100%0%0%
Gemma-3-12B (28)59%62%1%0%
Phi-4 (22)92%91%0%0%
Mistral-Nemo-12B (19)76%88%5%9%

(Parity; shares over the recipients for which recipient and donor are answered correctly under both mappings.)

These are maxima over layers. At the layer chosen on calibration data, the decision donor changes at least 75% of the answers in six models and fewer in Qwen3-14B, Gemma-3 and Mistral-Nemo; Table 1 of the revision now gives both. Near the output the same holds for the flipping component a: transplanting only its coordinate from a token donor changes 99% of the parity answers at the last layer of Qwen2.5, and 99% at layers 26 and 27 (per-layer outputs in the supplementary material).

Q1. What distinguishes the symmetric component from shared prompt-structural or task-format effects? Could you provide a control that more directly isolates decision-related information?

A decision donor differs from the recipient in more than its decision, so we tested the alternatives this leaves open.

It is a different number, so its state could carry a property of that number, such as its parity, that later layers evaluate under the recipient's question. Two donors hold the parity fixed: another number of the same parity asked "Is this number divisible by 3?" with the opposite answer, and the same number asked "Is this number odd?" (both under the other mapping, so their correct letter is the recipient's). Under AB and BA, "yes" is also always the first clause, so a decision donor changes which clause applies. Minimal pairs fix this: two instructions that differ only in the words yes and no ("If yes, answer A; if no, answer B" and "If no, answer A; if yes, answer B", and the same with B named first). Within a minimal pair the same-parity donor has the recipient's parity, letter and clause; of these, only the decision differs. Shares at the layer where the decision donor peaks (parity; a minimal pair's own peak layer is given where it differs):

ModelDecision donor (layer)Same parity, divisible by 3?Same number, odd?Same parity, divisible by 3?, within a minimal pair (A first / B first)
Qwen2.5-7B86% (21)98%73%– / –
Qwen3-4B95% (24)24%42%15% / 44%
Qwen3-8B78% (24)64%20%41% / 48%
Qwen3-14B93% (28)78%53%45% (27) / 74% (27)
Llama-3.1-8B96% (17)59%57%57% (16) / 59% (16)
Gemma-2-9B99% (28)95%81%39% (26) / 82%
Gemma-3-12B59% (28)19%3%22% / 16%
Phi-492% (22)71%50%45% / 61% (24)
Mistral-Nemo-12B76% (19)60%50%– / –
Qwen2.5-7B, chat template100% (21)99%97%66% / 99%

(–: below our 80% accuracy criterion on that pair, which applies to both pairs of Mistral-Nemo and to Qwen2.5 with A first; for Qwen2.5 with B first, state replacement was not run with plain-text prompts. With its chat template, in the last row, Qwen2.5 follows all four instructions.)

The same-parity donor still changes more than half of the answers in seven of nine models (the exceptions, Qwen3-4B and Gemma-3-12B, reach 19–24%), with a letter-only control at most 20%, so in these models the parity of the number does not explain the transfer; in the two exceptions most of it is lost once the parity is held fixed. Within the minimal pairs this donor changes 15% to 99% of the answers, about half in most models, and often more into recipients whose instruction names yes first (Qwen3-8B with A first: 72% against 10%): the decision itself crosses over, and the rest of what crosses over in the standard pair depends on the wording and on the input. The decision donor of the minimal pair alone (other number, same letter and clause) changes 45% to 100%, less than in the standard pair in several models. The paper reports all of this in Section 3.3 and a new appendix table, and qualifies its reading of the first stage accordingly.

On normalization: without the scaling, U = (d_AB + d_BA)/4 is the effect of the decision and W = (d_AB − d_BA)/4 the effect of the correct letter. The scaling makes s proportional to U + ρW, with ρ set by the difference in length of the two class directions; in every model the cosine between s and U is above 0.999 at the chosen layer. On parity U and W are nearly orthogonal in most models, and removing W then costs about the same as removing a. In Qwen2.5 they overlap (cosine 0.44 on parity), so W contains part of the decision direction, which is why removing it costs more than removing a (0.52 against 0.09). The normalization mixes the two only by this small factor, so s stays essentially the decision effect U.

On edit size: where little of a is present, a small effect of removing it is partly built in. We therefore moved both components by the same amount at every layer from four before to four after the chosen layer, in every model. In a second version every state is pushed towards the wrong decision along s and towards the wrong letter along a, so that a noisy sign cannot weaken the push along a. At every such layer where the cosine between the two class directions exceeds 0.85 (73 cases), moving along s costs more than moving along a in both versions, with an interval excluding zero in 72; where it is below 0.7, moving along a costs more in 71 of 74 cases.

On task-format effects (Q1): a class direction is a difference between yes- and no-examples under the same template, so what the classes share, including most of the format, cancels, and the donors of part 2 hold the letter, the clause and the parity fixed. Changing the format also shows that the design does not find a decision-like stage everywhere: for the same 200 news articles, a decision donor changes 91–100% of the answers at its peak layer when they are asked a yes/no question ("Is the proposed topic correct?"), but moves at most 6% of them to its own topic when one of four topics must be chosen from lettered options, and at most 31% with an instruction.

Reviewer_ZGXiOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper is mostly unreadable, and I tried to understand it. I cannot provide a summary.

I can certainly be wrong or unqualified. But I truly believe the problem is with the paper and not me.

Critical strengths

None.

The work done by the authors may be worthwhile, but this submission cannot be considered for publication.

Critical weaknesses

I cannot follow this paper. It lacks a reasonable introduction, it is impossible to follow, and I cannot even tell what problem they are investigating and why. The abstract itself is unreadable. The introduction starts with a statement that appears to be the main empirical conclusion.

  • (012) “Class directions fitted under the two mappings yield a symmetric component that keeps its sign under the exchange and an antisymmetric component that reverses it.” I cannot tell what that means. A large part of the paper reads like that sentence.

  • (030) “The labels remain readable under an output convention that the model fails to follow.” Why and when would you not be able to read a label?

  • (054) “Exchanging answer labels changes the required token while preserving the rule’s truth value” I can’t understand that either.

  • (135) “deterministic labels”. What kind of labels would be undeterministic? I guess the label definition could be ambiguous?

  • The notation in Figure 1 and Section 3.1 is hard to follow, and I believe just too imprecise (and wrong). For examples, “Layer l denotes the output of decoder block l”, and then using l to identify hidden and baseline states. I assume that l is the layer number, not the output of a decoder block. In 048, the paper reads “We retain both constituent states so that the contribution of baseline subtraction can be measured directly.” Well, how would you calculate the difference of two things unless you have access to (retain) them?

Questions for authors

None

3 comments

Thank you for quoting the exact sentences that stopped you. The abstract and the introduction used terms that we never defined, and the introduction opened with a result before it had stated the problem. We have rewritten the abstract, the introduction, the method section and Figure 1, and the revised paper is uploaded. Below we first say in plain words what the paper is about, and then answer each of your points with the sentence that now replaces it.

A standard way to find a concept inside a language model is to average its hidden states over the examples of each class and take the difference of the two averages. We call this difference a class direction. Suppose the model is asked: "Consider the number 12. Is this number even? If yes, answer A; if no, answer B." For every even number the right decision is yes and the right next token is A. A direction that separates even from odd numbers therefore also separates "about to output A" from "about to output B", and from this one prompt we cannot tell which of the two the direction represents.

We separate the two by asking the same question twice, once with "A means yes" and once with "B means yes". The decision stays the same and the letter that reports it is swapped. The part of the class direction that the two prompts have in common is built to follow the decision, and the part in which they differ is built to follow the letter; whether they do is what the experiments test. We then delete one part at one layer, or replace the model's internal state at the last prompt token, at one layer, by the state of another prompt, and watch which answer the model gives.

In Qwen2.5-7B, which has 28 layers, deleting the shared part at layer 22 lowers the model's preference for the correct letter by 3.2 (out of 3.7), and deleting the flipping part lowers it by 0.09. At the last layer it is the other way round. Two stages appear in all eight further models, less sharply in two of them. One result narrowed the scope of the paper: the first stage appears when a yes/no question is answered through an instructed letter mapping, also when news articles are asked whether a proposed topic is correct, and is weak or absent when the same articles must instead be assigned one of four topics.

Line 012: "Class directions fitted under the two mappings yield a symmetric component that keeps its sign under the exchange and an antisymmetric component that reverses it."

This sentence used four terms that had not been defined. The revised abstract first states the problem: "When the prompt tells the model which letter to give for each answer ('If yes, answer A; if no, answer B'), the yes- and no-examples also differ in the letter the model must output, so a class direction may carry the decision or the letter." It then reads: "We separate the two by asking the same question under two answer mappings, with the letters exchanged. Adding the two normalized class directions gives a shared component, which is what balancing the letters produces; subtracting them gives a flipping component, which reverses with the letters."

Line 030: "The labels remain readable under an output convention that the model fails to follow."

By "readable" we meant that the correct yes/no answer can be recovered from the hidden state by a linear classifier. With the letters C and D in reversed roles, Qwen2.5 outputs the same letter for every test number, so it is right for only half of them. A linear classifier that was fitted only on prompts with the letters A and B, and is then applied to the layer-22 states of these C/D prompts, still tells even from odd numbers for all 100. The revised text reads: "Qwen2.5 gives the same answer to every number, yet a classifier along ss at layer 22 recovers all 100 held-out parity labels". We describe this as an observation about what can be read out: because the model gives the same answer to every number under these letters, interventions cannot show there whether the direction is used.

Line 054: "Exchanging answer labels changes the required token while preserving the rule's truth value."

It meant that 12 is even in both prompts, so the correct decision is yes in both, while the letter that means yes changes from A to B. The revised introduction says: "A second prompt asks the same question about the same number but reads 'If yes, answer B; if no, answer A.' The decision is unchanged and the correct letter is swapped." Figure 1(a) shows the two prompts, one above the other.

Line 135: "deterministic labels."

We meant that the correct answer is fixed by an arithmetic rule, so there is no annotator disagreement and no label noise. The phrase is gone; the text now says: "The main tasks are two yes/no questions whose correct answer is fixed by a rule".

Notation: "Layer ℓ denotes the output of decoder block ℓ."

This was wrong as written, because ℓ is an index and not a state. The revision defines one object and one index: "For a model with LL decoder blocks, hℓm(n)h^{m}_{\ell}(n) is the vector at this position after block ℓ∈{1,…,L}\ell\in\lbrace 1,\dots,L\rbrace for number nn under mapping mm; each block adds its output to this vector, which is known as the residual stream." Section 2.1 defines each symbol where it first appears.

Line 048: "We retain both constituent states so that the contribution of baseline subtraction can be measured directly."

You are right: the sentence did not even say why a second prompt is involved. The revision explains it: "Even and odd numbers also differ in features that the model represents whatever it is asked, such as the last digit. To keep these out of the class direction, we subtract the state [...] of the same prompt with the question 'Is this number an integer?', whose answer is yes for every number". The result does not depend on this step: without it, deleting the shared part at layer 22 costs 3.23 and deleting the flipping part 0.09, against 3.24 and 0.09 with it.

Figure 1.

The old figure used symbols that its caption did not explain. The new Figure 1 has three panels: the two prompts, one above the other, with the decision and the correct letter marked, the two class directions with their sum and difference, and the measured effect of deleting each part at every layer.

Introduction.

The revised introduction states the problem (what a class direction confounds), the idea (exchange the letters), the question (which part matters where), the findings, their limits, and the contributions, in that order. The result that used to open the introduction now comes after the method has been explained.

If a sentence in the revised abstract or introduction is still unclear, please tell us which one, and we will fix it during the discussion period.

Thank you again for your review. Since the discussion closes on 18 November, we would be grateful to know whether the revised abstract, introduction and Figure 1 (pages 1 to 3) now state the problem and the method clearly. In the marked PDF, the sentences that replace the ones you quoted are labelled "ZGXi" in the right margin. If any sentence still stops you, please quote it and we will fix it.

Reviewer_RvemOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates the relationship between answer labels and LLM hidden state interventions (i.e., steering) that are designed to target something more fundamental than a specific answer token. Prior work on difference-in-means steering has to carefully balance classes to avoid this effect-- for example, in MCQA, by ensuring that 50% of the "true" class takes label A and 50% B, and same for the "false" class, so that the steering vector isn't simply promoting "A" or "B" tokens rather than the "truth" concept. This paper proposes a way to separate out the "valid" (symmetric) concept portion of a found direction from the specific output label it promotes ("asymmetric direction"), and show that when transferring to another setting with a different vocabulary, this "valid concept" part still works. The authors also track which components have a causal effect in which layers.

Critical strengths

Whether directions simply capture some basic statistics of a particular next-token prediction (instead of a "meaningful" representation of a concept) is an interesting research question with implications for steering and causal validation of interpretability localization experiments.

Critical weaknesses
  1. I can barely understand the paper. I didn't get a lot out of it.
  2. Uses a single, outdated model (Qwen2.5-7B).
  3. Limited to binary multiple-choice tasks.
Questions for authors

In 1-2 sentences, what is the main research question you answer? What are your main findings, and what are their significance?

Why or how is this method better or different from simply averaging over the two sets of data when constructing concept directions to remove noise from any particular output label?

5 comments

Thank you for the review. Your summary of the paper is accurate, and you are right that the writing made it far too hard to get there. We rewrote the abstract, the introduction, the method section and Figure 1, and we ran the experiments you asked about. One of them narrowed a claim, as described in part 3. The revised paper is uploaded. Numbers below are on 100 held-out items unless stated otherwise; shares of answers changed by state replacement are over the items answered correctly under both assignments, as recipient and as donor.

Q1. In 1–2 sentences, what is the main research question? What are the main findings and their significance?

Question: when a model reports a yes/no answer with an instructed letter, does a class direction act on the decision or on the letter, and at which layers? Finding: through the part shared by both letter assignments at a few intermediate layers and through the letter part afterwards, clearly in seven of nine models; at the last token this first stage is weak or absent for a four-way topic choice. Significance: a direction fitted under one letter assignment pushes answers under the other towards the wrong letter from one to three layers after that stage, and balancing removes this but leaves a direction that in seven models acts mainly within a few layers.

Q2. Why or how is this better or different from simply averaging over the two sets of data when constructing concept directions?

For obtaining the concept direction it is not different. The shared component is the averaged direction: the direction obtained by pooling both letter assignments has a cosine of at least 0.994 with it at the intermediate layer on parity, in all nine models. What the paper adds is what averaging alone does not tell you: where a direction fitted under a single assignment goes wrong, and where the averaged direction still acts.

First, we steered every prompt with the class direction fitted under the other assignment, with the same step size as for its own direction. The table gives the change of the yes-minus-no logit on parity.

Model (layer ℓ / last layer L)at ℓ: same assignment / other assignmentat L: same / otherother assignment pushes the wrong way from layer
Qwen2.5-7B (22/28)+4.77 / +4.39+7.30 / −7.1623
Qwen3-4B (24/36)+19.91 / +15.81+40.46 / −40.7725
Qwen3-8B (24/36)+17.47 / +15.39+36.05 / −37.3025
Qwen3-14B (29/40)+18.95 / +2.44+38.98 / −40.0330
Llama-3.1-8B (17/32)+9.35 / +7.50+14.05 / −14.1818
Gemma-2-9B (28/42)+9.22 / +8.51+18.37 / −18.5029
Gemma-3-12B (29/48)+8.80 / +3.36+27.61 / −26.9430
Phi-4 (25/40)+12.06 / +1.63+23.64 / −23.3726
Mistral-Nemo-12B (26/40)+4.12 / +0.77+4.19 / −4.1527

At the intermediate layer the direction from the other assignment moves the answer the right way, with 38–92% of the effect of the own direction in six models (less in Qwen3-14B, Phi-4 and Mistral-Nemo-12B, whose chosen layer already lies inside the transition). One layer later it already moves the answer the wrong way on parity, in all nine models (on divisibility up to three layers later), and at the last layer it does so as strongly as the own direction moves it the right way. Tan et al. (NeurIPS 2024) found that even with balanced letters the effect of a steering vector depends on which letter carries the target answer. Their vectors act at every position and ours at the last prompt token, so we do not claim to explain their result.

Second, the averaged direction acts in a narrow band of layers. Added at the last prompt token one layer at a time and averaged over the two assignments, it changes the yes-minus-no logit by more than half of its largest effect only at three to six consecutive layers, ending at most two layers after the chosen layer, in seven models. In Mistral-Nemo the band is wider, and in Gemma-3 a third to a half of the effect persists for twelve more layers on parity. At the last layer it changes the logit by at most 7% of its largest effect; under each assignment alone it shifts the margin towards one letter by up to 41% of that effect (Gemma-3, where no answer changes). Contrastive activation addition adds its vector at every position, which we did not test; the paper says so.

The part that averaging discards controls the output after the intermediate stage, as the output head leads one to expect: on parity, at the last layer, removing it removes at least 94% of the margin in every model, while removing the shared component changes the margin by at most 2% (removing the pooled direction itself, at most 4%).

W2. A single, outdated model (Qwen2.5-7B).

We added eight instruction-tuned models, each with its own chat template: Qwen3-4B, Qwen3-8B, Qwen3-14B, Llama-3.1-8B, Gemma-2-9B, Gemma-3-12B, Phi-4 and Mistral-Nemo-12B. Together with Qwen2.5 they cover five families. The main test needs no fitted direction. We replace the whole last-token state of a prompt by the state of a prompt with the opposite decision and the same correct letter (decision donor), or by the state of the same number under the other letter assignment (token donor). The table shows parity.

Model (layer ℓ / last layer L)Accuracy, both assignmentsRemove shared / flipping at ℓDecision / token donor at ℓDecision donor at its peak (layer) / token donor there
Qwen2.5-7B (22/28)100/94%3.24 / 0.0984% / 1%86% (21) / 0%
Qwen3-4B (24/36)94/94%10.51 / 0.6595% / 0%95% (24) / 0%
Qwen3-8B (24/36)95/95%5.20 / 0.1578% / 0%78% (24) / 0%
Qwen3-14B (29/40)98/98%8.57 / 5.0736% / 0%93% (28) / 0%
Llama-3.1-8B (17/32)93/96%5.70 / 0.5096% / 0%96% (17) / 0%
Gemma-2-9B (28/42)93/94%5.85 / 0.1599% / 0%99% (28) / 0%
Gemma-3-12B (29/48)98/99%2.96 / 0.6831% / 2%59% (28) / 0%
Phi-4 (25/40)98/97%4.65 / 1.8375% / 1%92% (22) / 0%
Mistral-Nemo-12B (26/40)91/85%1.12 / 0.6651% / 43%76% (19) / 9%

ℓ is chosen on calibration data only, as the last layer before the largest drop of the cosine between the two class directions; for Qwen2.5 it is layer 22, the layer of the submission. In every model a decision donor changes most answers at a few intermediate layers while a token donor changes almost none, and later a token donor changes all of them. The peak column is a maximum over layers; at ℓ the decision donor changes at least 75% of the answers in six models. Some cases are weaker, and the paper says so. In Qwen3-14B, Phi-4 and Mistral-Nemo the cosine falls in two steps, so ℓ already lies inside the transition. In Gemma-3 the decision donor peaks at 59%. In Mistral-Nemo the answers after replacement at ℓ largely stop depending on the recipient's assignment, so its shares near 50% are not a partial transfer.

Before running Qwen3-14B, Gemma-3 and Qwen2.5 with its chat template, we wrote down two layer rules and four pass criteria. Qwen2.5 with its chat template passes every criterion under both rules (decision donor 100%, token donor 0%). Qwen3-14B fails the equal-size and replacement criteria under the first rule, whose layer lies inside its transition, and passes all of them under the second (93% on parity). Gemma-3 fails the replacement criterion on parity under both rules (31%) and on divisibility under the first rule (67%), and passes it narrowly on divisibility under the second (76% against the required 75%; 66% counted over all held-out items). The paper reports every criterion for every run.

The dominance of the letter part at the last layer is largely expected from the output head, and the paper says so; the informative result is the intermediate stage.

W3. Limited to binary multiple-choice tasks.

We tested the result outside the binary numerical setting, and one test narrowed the claim. The table gives the largest share of answers that follow a decision donor at any layer. A token donor takes over later in every condition that was run.

Question and answer formatQwen2.5Qwen3-4BQwen3-8BQwen3-14BLlama-3.1Gemma-2Gemma-3Phi-4Mistral-Nemo
number, instruction869578939699599276
number, lettered options43395<80%57843<80%
film review, instruction10096959694999610096
news article, instruction91929697971009995<80%
news article, four topics, instruction<80%4325<80%31<80%417
news article, four topics, lettered options0414<80%6<80%<80%<80%

(Numbers: parity. <80%: the model answers fewer than 80% correctly under one assignment, so no share is reported. The news articles are the same 200 AG News articles in all three news rows. Four topics use four letter assignments that form a Latin square.)

The intermediate stage appears for yes/no questions about numbers, film reviews (SST-2) and news articles, which we asked whether a proposed topic is correct. When the same articles must be assigned one of four topics, it is weak or absent: a decision donor moves at most 6% of the answers to its own topic when the topics are listed as lettered options, and at most 4% with an instruction, except 17%, 25% and 31% in Mistral-Nemo-12B, Qwen3-14B and Gemma-2-9B. Listing the two answers of a yes/no question with their letters ("A. Yes B. No") weakens the stage on parity in every model we tested and removes it in some. This agrees with work on lettered multiple choice, where the content is matched to an option and then bound to its letter at other positions (Wiegreffe et al., ICLR 2025; Wong et al., 2026). We now state the scope plainly: the result concerns yes/no verdicts reported through an instructed letter mapping.

We ran three answer options in all nine models. None solves the remainder modulo 3; with three ranges given as an instruction, Gemma-3-12B and Phi-4 follow all six letter assignments. In Phi-4 state replacement shows the same two stages: at layers 22 to 24 a number from another range, given with the recipient's letter, makes 85–86% of the answers follow its range (66% at layer 25), and from layer 26 on the same number under another assignment sets the letter in at least 99%. In Gemma-3, whose first stage is weak also with two options, at most 39% follow the donor's range. Removal agrees in Phi-4: removing the plane of decision effects at layers 22 to 24 lowers the margin by 6.15–6.63 out of 10.59, against at most 0.49 for the plane of letter effects, and from layer 26 on the order is reversed.

One limit remains. The method needs answer labels that can be exchanged and a model that follows every assignment, so free-form generation is outside its scope.

W1. Clarity.

The introduction now starts from one example prompt and the confound in it, defines each term in plain words before using it, and ends with the contributions; the abstract states the same problem before the method. The two components are named by what they are, the part shared by the two letter assignments and the part that flips with them. Figure 1 shows the two prompts, the two components and the measured layer profile. If a passage of the revised abstract or introduction is still unclear, please tell us which one, and we will fix it during the discussion period.

Relation to prior findings. A stage in which the answer is represented independently of how it will be reported, followed by a stage that binds it to the output word, has been reported before: for in-context learning with remapped label words (Tao et al., Findings of EMNLP 2024) and, in concurrent work, for instructed conditional rules (Thavarasa and Sukumar, 2026) and for steering directions, which carry the label in earlier layers and the letter in later ones (Gao et al., 2026). The revision cites this work. What the paper adds is a causal test of both parts of the class direction itself, the object people fit for probing and steering, at every layer in nine models with input and question held fixed; minimal pairs showing that in several models a donor taken from the other letter assignment also transfers which clause applies, a caution for designs that swap answer words; and evidence about when the first stage is present at the answer position: for yes/no verdicts through an instructed mapping, but only weakly or not at all for a four-way topic choice.

Comments on the paper

1 comment

We thank the three reviewers. Two concerns were shared: the paper was hard to read, and its central result rested on one model. We have uploaded a revised paper that addresses both, with the experiments the reviewers asked for and controls their questions suggested. This note says what changed; the replies to each reviewer give the details.

The abstract, the introduction, the method section and Figure 1 are rewritten. The paper starts from one example prompt and the confound in it, defines each term before it is used, and names the two components by what they are: the part shared by the two letter assignments and the part that flips with them.

The evidence is broader. The revision tests every layer and repeats the experiment in eight further instruction-tuned models (Qwen3-4B, Qwen3-8B, Qwen3-14B, Llama-3.1-8B, Gemma-2-9B, Gemma-3-12B, Phi-4 and Mistral-Nemo-12B), five families in total, each with its chat template. The main test needs no fitted direction: we replace the whole last-token state of one prompt (the recipient) by that of another (the donor). In every model there are a few intermediate layers at which the model then reports the decision of the inserted state with the letters of its own prompt (59–99% of the parity answers where this peaks), and later layers at which it reports the inserted letter; the first stage is weak in Gemma-3 and overlaps with the second in Mistral-Nemo. Removing either part of the class direction agrees in every model, and so do edits of equal size at every tested layer where the class directions are still nearly parallel (73 cases; in Mistral-Nemo the cosine stays below 0.77 at all of them). For three further runs (Qwen3-14B, Gemma-3 and Qwen2.5 with its chat template) we fixed the layer choice and the pass criteria in advance: Qwen2.5 with its chat template passed every criterion, Qwen3-14B passed only with the second of two layer choices written down, and Gemma-3 failed on parity.

We also tested what crosses over. A donor of the same parity as the recipient, asked a different question with the opposite answer, still changes more than half of the answers in seven of nine models; inside instructions that differ only in the words yes and no, so that letter and clause are the recipient's, it changes 15% to 99%, about half in most runs. The decision itself crosses over, but not all of what crosses over is the decision, and the paper now says so.

The revision also tests when the intermediate stage exists. It appears for yes/no questions answered through an instructed letter mapping, about numbers, film reviews and news articles. On parity it is weaker when the two options are listed with their letters ("A. Yes B. No"), and it is weak or absent when the same 200 news articles must be assigned one of four topics (at most 6% of the answers follow a decision donor with lettered options, and at most 4% with an instruction, except 17%, 25% and 31% in Mistral-Nemo-12B, Qwen3-14B and Gemma-2-9B). We report this as a limit of scope: the result concerns yes/no verdicts reported through an instructed mapping, not every multiple-choice format.

On existing practice: the shared component is what balancing the letters produces (on parity, cosine of at least 0.994 with the pooled direction at the intermediate layer). What the revision adds is what the other part does and where each part matters. For edits at the last prompt token, a direction fitted under one letter assignment pushes the other assignment towards the wrong letter one to three layers after the intermediate stage, and the balanced direction acts mainly within a few layers around that stage in seven models and barely at the last layer in all nine. Stages of this kind have been described for in-context learning with remapped labels (Tao et al., 2024), for multiple-choice answers (Wong et al., 2026; Mueller et al., 2025) and, concurrently, for instructed conditional rules (Thavarasa and Sukumar, 2026) and steering directions (Gao et al., 2026); the revision cites this work and says what it adds.

What is unchanged: the question, the decomposition, and the Qwen2.5-7B experiment of the submission with its numbers (removing the shared part at layer 22 costs 3.24 of a margin of 3.67 and removing the flipping part 0.09; at the last layer 0.02 against 3.68), which a separate re-implementation reproduces, and the analysis of the output head. What is new answers specific requests: the eight models (Rvem W2, Dfp3 W2 and Q2), the donors that fix parity, letter and clause (Dfp3 Q1), the fixed candidate order (Dfp3 Q2), the other datasets and formats (Rvem W3), and the steering comparison (Rvem Q2). Code and per-example outputs are in the supplement. All edits act at the last prompt token, and the method needs exchangeable answer labels.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.