acceptodds
Under review as a conference paper at ICLR 2027

How Language Models Combine Multiple Binding Mechanisms

Abstract

Language models must bind entities together in context to later retrieve relevant information, for example when remembering which city a person lives in. Prior work has established three mechanisms (positional, lexical, and reflexive) that models use for this retrieval, but left open where the underlying binding signals are stored and how they are combined into an answer. In this work, we isolate the low-rank subspaces of the final token residual stream used by each mechanism and show that they can be individually manipulated. Further, we show that the subspaces contribute additively to the query key product of a single dominant attention head driving answer retrieval. Interventions on individual subspaces selectively redirect the head's attention and the model's final output. Finally, we examine the subspaces themselves: the positional subspace primarily encodes fact order, while the lexical and reflexive mechanisms share a semantic identity channel. Taken together, these findings demonstrate how known retrieval heuristics are represented and combined. More broadly, the work provides insight into how redundant mechanisms are implemented and reconciled within language models.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, variable binding, in-context learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_V7aqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how language models combine the three in-context retrieval mechanisms: positional, lexical, and reflexive. Using the 20-fact boxes task on Gemma 2-2B, the authors first identify L18H6 as the primary retrieval head via direct logit attribution and causal restoration, showing that its effect is almost entirely explained by where it attends among candidates. Then they estimate low-rank subspaces of the final-token residual stream at layer 18 for each mechanism using counterfactual pair differences, and show that these subspaces jointly reproduce the effect of a full counterfactual patch and can be edited individually to redirect the answer. The authors decompose L18H6's attention logits into contributions from a task-general component and each mechanism subspace, showing that the mechanisms cast additive "votes" resolved by the softmax. Finally, they characterize the subspaces, finding that the positional subspace encodes fact order while the lexical and reflexive subspaces share a semantically compressed entity channel.

Critical strengths
  • The central claims are supported by several complementary methods: DLA, whole-head versus attention-only causal restoration, sufficiency tests with a reverse control, targeted single-subspace edits, and a non-interventional attention decomposition on unpatched prompts.
  • Appendix E convincingly shows that the uncentered contrastive covariance estimator of Lee et al. (2026) absorbs the task-general query component into the order-ID subspace, which motivates separating out g in the residual stream.
  • The authors present replication on a second model family and released code.
Critical weaknesses

W1. Additivity holds by construction

  • Mechanisms come from Gur-Arieh et al. (2025); QK-logit decomposition follows Lee et al. (2026).
  • Since the query is linear in the residual stream, additive logit contributions are guaranteed by construction. The real contribution is that each component targets a mechanism-specific candidate and is causally separable, and the framing should say so.

W2. Narrow scope

  • Results rest on a single synthetic task (20 facts, two entity types) and two models.
  • Llama already shows a less concentrated circuit (Appendix A), so the "single primary head" picture may not generalize.

W3. Separability vs. shared content channel

  • Section 4.1 claims independent manipulability, but Section 6.2 reports strong lexical/reflexive overlap (cosine 0.83). I think this needs a bit of a explanation.

W4. Unexplained portions of the effect

  • The "Other / mixed" outcomes in Fig. 9 (~23–31%) are not analyzed.
  • Restoring L18H6 recovers only ~37–58% of the counterfactual margin.
  • The 99.7% variance explained may be dominated by the format component (g). Variance among same-type candidates should be reported.
Questions for authors
  1. In a setting where both lexical and reflexive mechanisms contribute substantially, does editing one content subspace also move the other mechanism's vote in L18H6?
  2. What do the "Other / mixed" outcomes in Fig. 9 look like? Does the model still output a candidate token, and which components dominate L18H6's attention logits in these cases?
  3. What fraction of attention-logit variance is explained among same-type candidates only?
  4. Can the word-order effect be explained by whether the answer token's key can encode the identity of its paired entity?
2 comments

We thank the reviewer for their detailed review and precise questions, and we’re glad they found our central claims supported by complementary methods.

W1:

Additivity holds by construction. We agree that additivity follows from construction and have reframed the contribution in the abstract and the introduction to be of the form: "We show that the same attention heads read all three subspaces, each directing attention to its mechanism's target." We also updated Section 5’s title to be “Retrieval Head Attention Decomposition” rather than “Additive Combination in QK Space.” Per-subspace decomposition of the attention logit is now just described as the method we use to show that the subspaces are directing attention, not the finding.

W2:

Single task with only two entity types

We have now added the music task (most of the other Gur Arieh et al. tasks are variations of this same format). The top heads are the same and the swaps work similarly. Because both the lexical and reflexive mechanisms are active, we were also able to examine that case. Added as Appendix K.

Circuit concentration

We’ve now extended our analysis to L19H1, L22H4 and L23H2 for Gemma and L24H24, L22H14, L27H7, and L25H7 for Llama and found that the pattern is consistent with L18H6. While this does imply a more distributed computation than a single primary retrieval head, we have softened that language in the manuscript, now stating that even if distributed beyond a single head, the core computation is the same: retrieval heads reading all three subspaces and softmax deciding over the contributions.

W3:

Explaining separability of lexical and reflexive mechanism despite the shared channel. We have now addressed this with the addition of the music task where both lexical and reflexive mechanisms are active. We show that they still can be separately manipulated even while having strong overlap.

These findings are compatible because our swap method subtracts one entity representation within the subspace and adds in another, it does not overwrite the entire subspace. The lexical mechanism uses the queried entities and the reflexive mechanism uses the answer entity, so the two mechanisms hold different entities in any one prompt. Swapping out one queried entity for another as a targeted intervention therefore does not disrupt the answer entity's representation even if it is occupying the same subspace. We have added this to Appendix K.

W4:

Unexplained portions:

  • “Other / mixed” (now added to Appendix B): Output is a non-candidate token in only 0.9% of cases (mostly “Box” token instead of the box letter). Most are near positional misses (69% are within two, 79% within 3 positions of the target). 26% are close content matches (e.g rose/flower, or clock/watch. This aligns with the diffuse positional and smentaically graded contributions in Figure 5a and Figure 5b.
  • Partial recovery of counterfactual margin: L18H6 is the largest causal contributor but still partial. Restoring the four heads from Table 1 recovers 63% (box → object) and 94% (object → box) of the margin and returns the base answer on 94% and 99.8% of prompts. We now show the decomposition of those additional heads in Appendix J.
  • Same type variance: see Q3.

Q1:

No, we are able to separately edit the two content mechanisms. In the newly added music task, at L18H6, swapping one content subspace moves its contribution from the original target to the swap target nearly entirely, for example: reflexive 1.90 → -0.09 at the original target and 0.06 → 2.06 at the swap target, while the other subspaces’ contributions change by at most 0.05. We showcase this in the newly added Appendix K and elaborate more in the reply to W3.

Q2:

For the "Other / mixed" outcomes in Fig. 9, does the model still output a candidate token? Yes, in 99.1%, see W4.

Which components dominate L18H6's attention logits in these cases?

In these cases, the token the model outputs generally has attention logit contributions from multiple mechanisms (positional 1.2-1.8, content 1.9-2.0). The positional signal within the head is diffuse and spread across nearby tokens (Figure 5a) and the content signal also leaks some attention onto semantic neighbors (Figure 5b).

Q3:

The fraction of attention-logit variance explained among same-type candidates only is 99.7%, the same as overall, now noted in Section 5.1. The task-general component g contributes 17-22%.

Q4:

Our results are consistent with this. Under causal attention: a token’s key can only carry the identity of entities that come before it in its fact. When the answer comes after the queried entity, the answer token’s key can carry the queried entity’s identity and the lexical subspace matches on it. When the answer comes before the queried entity in context, its key only carries its own identity and only the reflexive subspace can match on it. We have added this explanation to Appendix B.

Dear Authors,

Thank you for your detailed experiments. I do think these results improve the paper. Since I think the initial score is high enough for the paper, I will maintain my score.

Good Luck!

Reviewer_Hb5aOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work builds on Gur-Arieh et al. 2025, who originally found that language models use a mix of positional, lexical, and reflexive mechanisms to perform entity binding tasks. The authors attempt to understand whether they can identify the subspaces in which each mechanism lives, and how redundancy between mechanisms might shape a model's final answer. They do so by studying two models, Gemma 2 2B and Llama 3.1 8B, and a classic boxes containment task. For Gemma 2 2B, they find that one attention head in particular acts as the main answer retriever. At the final token position, they then find that they can extract a low rank subspace for each of the lexical, positional, and reflexive mechanisms, in addition to a generic formatting component. These subspaces can be validated individually using causal patching. Further, final outputs can be modelled as voting between mechanisms. The work therefore also demonstrates how multiple redundant mechanisms can contribute to the same function.

Critical strengths
  • The paper is clear and easy to follow. It was a pleasure to read!
  • Experiments appear to be fairly thorough and results are convincing, with nice causal tests
  • Nice to see results for two models
Critical weaknesses
  • Not sure I fully agree with the choice to study only a single head. Why not also study the other two heads in table 1, which have similarly large effects?
  • Didn't Gur-Arieh et al. study a bunch of different instantiations of the entity binding task? Why only study the boxes task here?
  • The work sits close to Gur-Arieh et al. and Lee et al. and at some level I think the novelty is fairly limited. Specifically what I mean by that is that the core insight is kind of peripheral to these existing papers.
  • Building on the above point, I guess it's not as clear to me why the result of this study might be framed as important, and I think the authors could polish that framing some more to make it clear why one should care about where and how the different mechanisms are represented
Questions for authors
  • Fig 1 is not super clear imo. I think the depictions of the position and content matching mechanisms should very obviously illustrate what is meant by position and content matching. But as is I don't think I can figure out the difference between the two from the figure alone.
  • I would really love to see how the mechanism decomposition changes across pretraining. Not essential for this paper, but rerunning the experiment on different pythia checkpoints might be very interesting
2 comments

We thank the reviewer for their constructive review and helpful suggestions, especially on the framing. We’re glad they found the results convincing!

W1:

Studying the other two heads in Table 1. We’ve now extended our analysis to L19H1, L22H4 and L23H2 for Gemma and L24H24, L22H14, L27H7, and L25H7 for Llama and found that the pattern is consistent with L18H6. In L18H6, each mechanism’s contribution to the attention logit landed on its counterfactual target and the combination of task-general attention and mechanism subspaces account for 99.7% of the attention logit variance across tokens. The other three Gemma retrieval heads (L19H1, L22H4, L23H2) show the same pattern: each subspace’s contribution lands on its own target, with 99.1–99.4% of variance explained. We added some of these results to Section 5.1 and the rest as Appendix J.

W2:

On adding more entity binding tasks like Gur-Arieh et al. We have now added the music task (most of the other Gur Arieh et al. tasks are variations of this same format). The top heads are the same and the swaps work similarly. Because both the lexical and reflexive mechanisms are active, we were also able to examine that case. Added as Appendix K.

W3:

On distinction from prior work by Gur-Arieh et al. and Lee et al. Gur Arieh et al. established the three mechanisms and model how often each one wins as a function of position. Lee et al. introduce a query-key decomposition method and apply it to order and lexical signals. Neither characterizes the signals themselves or how they are combined into a final answer.

We show that the three signals are in low rank subspaces of the residual stream which can be causally swapped one at a time, that the same retrieval head reads all three at once, with each subspace directing attention to its mechanism’s target while softmax collapses their votes into a final retrieval. We also show that the lexical and reflexive mechanisms share an identity representation channel.

None of these findings are obvious a priori given the prior work. Each mechanism could have had its own set of heads, with competition decided additively in the residual stream. The winning mechanism could have been determined earlier than at the retrieval step itself, with retrieval heads only seeing the winning reference. The lexical and reflexive mechanisms could have used completely unrelated code systems. We also show that Lee et al.'s estimator absorbs the task-general query component into its order subspace, and suggest an adaptation (Appendix E).

W4:

On polishing the framing to make clear why one should care about how and where the different mechanisms are represented. Prior work could predict which mechanism would win but could not act on one mechanism at a time. This subspace-level understanding, especially the identity channel, also predicts a category of errors (semantic neighbors) previously unmentioned in relation to variable binding which focused more on pointer/dereference and ID motif.

More broadly, this is a worked example of how a model handles redundant mechanisms that disagree. The model carries the conflicting information all the way through to the final token position where relatively late stage retrieval collapses the votes into a token selection via softmax. Whether other redundant computations are resolved the same way is open, but this gives a concrete pattern to test for. We appreciate the note that clarity could be increased here and have added references to these impacts in paragraphs 2, 3, and 4 of the introduction.

Q1:

Improving Figure 1 readability, specifically the difference between position and content matching. We have reworked Figure 1 with icons as labels to better differentiate the mechanisms. Tokens targeted by the positional mechanism are now labeled with icons for 1, 2, 3, 4 where 2 is the lookup at the final token residual stream. The content subspace (in this case, lexical) holds an entity, pie, it matches pie to pie and has a weaker match to jam because it is a similar food item. It does not match hat and vest, which are clothing items. We added this to the figure caption to guide the viewer through the figure. We hope this makes it clearer!

Q2:

Mechanism decomposition changes across pretraining. We agree this would be very interesting, especially to track emergence of the different mechanisms across time! We leave it to future work.

I thank the authors for their detailed response and I appreciate their engagement with my feedback. I maintain my rating as a weak accept. I continue to think that the work adds only marginal insight on top of the original mixing mechanisms paper. At the same time, it does not explain why `each [mechanism] wins as a function of position' in that original paper, which is perhaps what I would want a paper that builds on it to explain. I realize that I did not make that suggestion in my initial review, and I of course also do not view it as a critical weakness. However, these thoughts do reflect the limits of my enthusiasm for the premise of this work. I wish the authors all the best with their current and future research endeavors!

Reviewer_2BfZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates how language models represent and combine three previously identified binding mechanisms—positional, lexical, and reflexive—during in-context entity retrieval. Using a synthetic boxes task, the authors first identify a primary retrieval head (L18H6 in Gemma 2-2B) through direct logit attribution and causal restoration. They then isolate low-rank subspaces in the final-token residual stream using counterfactual prompt pairs and demonstrate that interventions on these subspaces can selectively redirect the model's predictions.

The authors further decompose the query-key computation of the retrieval head, showing that the three subspaces contribute additively to attention logits. Finally, they find that positional representations encode fact order, while lexical and reflexive representations share a semantically structured identity space. The main findings are also replicated in Llama 3.1-8B, although retrieval appears more distributed across attention heads.

Critical strengths
  1. The paper goes beyond studying individual binding strategies by examining how positional, lexical, and reflexive signals contribute to a common retrieval process. This provides useful insights into how language models may integrate multiple sources of information during entity retrieval.

  2. The paper provides an empirical analysis of how positional, lexical, and reflexive binding signals are represented and contribute to retrieval within language models. These experimental observations offer useful insights for the interpretability research community.

Critical weaknesses

W1. Limited Evidence for the Underlying Computational Mechanisms

The three binding mechanisms are distinguished primarily through counterfactual patching, where each mechanism is associated with a different predicted answer (Section 2). After identifying L18H6 as the primary retrieval head (Section 3, Table 1), the subsequent analyses mainly examine how the corresponding residual subspaces contribute to its attention computation (Sections 4–5). Although the interventions demonstrate causal effects on retrieval, they do not necessarily establish that these representations correspond to distinct computational mechanisms rather than different signals associated with counterfactual retrieval outcomes. In particular, the individual interventions are evaluated under conditions where each mechanism is dominant (Section 4.1). Evaluating whether the same pathways operate under positional-only or lexical-dominant conditions, together with selective upstream interventions, would provide stronger evidence for the proposed mechanism-level interpretation.

W2. Limited Analysis of Final-Token Representations

The main experiments focus on the final-token residual stream entering Layer 18 (Sections 4–5), where information relevant to the final answer may already have been integrated. This is particularly important for the reflexive mechanism, which relies on an answer-entity representation resolved by an earlier layer. Consequently, the observed reflexive signal may reflect a downstream retrieval or readout process rather than an independently established binding mechanism. Although Appendix Figures 15–17 visualize representations across layers, they do not causally explain how these signals emerge or propagate.

W3. Limited Novelty of Additive Combination

The additive decomposition of attention logits in Section 5.1 (Equations 2–3) follows naturally from the linearity of the query-key computation. While identifying mechanism-related subspaces and their contributions is useful, the additive structure itself does not establish a new principle of mechanism combination. The analysis primarily demonstrates that multiple signals contribute to a selected retrieval head, rather than explaining how their relative importance is determined.

W4. Insufficient Analysis and Presentation of Distributed Computation

Although L18H6 is identified as the primary retrieval head in Gemma 2-2B, Table 1 and Appendix D reveal additional retrieval heads, including L19H1, L22H4, and L23H2. Furthermore, Appendix A shows that retrieval in Llama 3.1-8B is more distributed, with some later heads exhibiting reflexive-specific behavior. These findings suggest that different binding signals may be processed or combined across multiple heads rather than through a single shared retrieval pathway. However, the detailed attention decomposition focuses primarily on L18H6 in Gemma and L17H24 in Llama. The current presentation also does not sufficiently convey how these distributed representations and computations contribute to the paper's central claim about combining multiple binding mechanisms. A more systematic comparison and clearer visualization of mechanism-specific contributions across retrieval heads and layers would provide a more comprehensive and intuitive understanding of their distributed implementation.

W5. The functional independence of lexical and reflexive mechanisms remains insufficiently established.

Section 6.2 reports substantial overlap between the lexical and reflexive subspaces, with a mean principal cosine similarity of 0.83 for their top ten directions (Figure 6). Importantly, the authors acknowledge that the individual interventions in Section 4.1 are conducted in settings where one content-based mechanism dominates and the other contributes relatively little (Section 6.2, p. 9). Therefore, it remains unclear whether the two mechanisms can be independently manipulated when both contribute substantially to retrieval.

W6. Limited generalizability beyond synthetic entity retrieval.

The experiments are conducted primarily on a synthetic boxes task involving 20 facts and two entity types (Section 2). While this controlled setting facilitates mechanistic analysis, the models already achieve near-saturated accuracy on the original task, and the evaluation does not establish whether the same binding mechanisms and retrieval heads generalize to more complex settings. In particular, natural-language entity tracking, multiple relation types, and compositional reasoning may involve different representations or additional retrieval pathways. Evaluating whether the identified heads and subspaces retain their causal roles in more naturalistic contexts would substantially strengthen the broader implications of the work.

Questions for authors

Q1. Generalizability of Binding Mechanisms. To what extent do the experimental findings support the claim that positional, lexical, and reflexive binding mechanisms are implemented through a sparse set of attention heads in general natural-language tasks? Could the authors clarify how broadly and precisely the identified mechanisms explain binding computations beyond the controlled boxes task?

Q2. Adequacy of the Boxes Task. Is the boxes task sufficiently expressive to distinguish the three binding mechanisms, particularly the reflexive mechanism? Since reflexive retrieval relies on an answer representation already resolved by earlier layers, it remains unclear whether the current experimental setting adequately captures its underlying computational process rather than its final retrieval behavior.

Q3. Linearity and Nonlinear Interactions Between Mechanisms. The paper distinguishes its approach from prior work by tracing binding-related representations from the residual stream into the QK computation. However, additive representations and linear read/write operations are already common assumptions in mechanistic interpretability. Could the authors clarify whether the proposed framework captures nonlinear interactions or conflict resolution when the three mechanisms provide inconsistent evidence? For example, consider the statements "Alice's box does not contain a book" and "Alice's box does contain a book." While the lexical entities remain the same, the conflicting propositions may require positional or contextual information to determine the answer. Could such cases exhibit nonlinear competition or uncertainty over mechanisms, potentially motivating a distributional analysis, such as a Dirichlet-based formulation, beyond additive attention-logit decomposition?

2 comments

We thank the reviewer for their thorough review and are glad that they think our work offers useful insights for the interpretability research community.

W1:

Establishing distinct computational mechanisms vs different signals associated with counterfactual retrieval outcomes. We build on the three retrieval mechanisms found by Gur-Arieh et al. and study how their signals combine at retrieval. The subspaces themselves are not defined by counterfactual outcomes, they are estimated from prompt pairs that differ in one variable at a time (fact position, queried entity, or answer entity).

Interventions are evaluated under conditions where each mechanism is dominant. While the two box task formats have one dominant content mechanism each, the positional mechanism is live and tested against the dominant content mechanism in both formats and targeted swaps work on both in both. But we agree that evaluating multiple content mechanisms at the same time is a useful experiment and have added the evaluation of the music task from Gur Arieh et al (Appendix K) where the lexical and reflexive mechanisms are both active in the same prompts.

W2:

Why do we examine the final-token residual stream entering layer 18? Gur-Arieh et al. (https://arxiv.org/pdf/2510.06182, Fig. 2) showed that the final-token state in Gemma carries binding information used for retrieval from layer 16, so we start our search for retrieval heads there. The most prominent retrieval heads start in layer 18 with L18H6. So we analyze its input, the residual stream entering layer 18. We added clarification for this in Section 3.

The observed reflexive signal may reflect a downstream retrieval or readout process. Gur-Arieh et al. directly test for this - whether the final token reflexive signal already carries the final answer (https://arxiv.org/pdf/2510.06182, Section 3.4, Figure 4). There they conclude that the model relies on a reflexive mechanism, distinct from the positional and lexical ones, where a direct pointer to the answer entity is used to retrieve it. How earlier layers construct these signals remains open. We instead study how the signals are used at time of retrieval.

W3:

Additive decomposition of attention logits follows naturally. We agree that additivity follows from construction and have reframed the contribution in the abstract and the introduction to be of the form: "We show that the same attention heads read all three subspaces, each directing attention to its mechanism's target." We also updated Section 5’s title to be “Retrieval Head Attention Decomposition” rather than “Additive Combination in QK Space.” Per-subspace decomposition of the attention logit is now just described as the method we use to show that the subspaces are directing attention, not the finding.

How the relative importance of signals is determined. We demonstrate that within the retrieval head, relative importance is determined by summing each subspace’s contribution per candidate token, with softmax selecting among the totals (Section 5.1). The positional contribution is spread over neighboring facts, the content contribution is concentrated on the matching entity (Section 5.3), and word order determines which content mechanism contributes (Appendix B). Why each signal arrives with the strength it has is upstream of what we study.

W4:

Additional retrieval heads. We have now successfully replicated our findings on L18H6 across the other retrieval heads. We ran the attention logit decomposition on L19H1, L22H4 and L23H2 for Gemma and L24H24, L22H14, L27H7, and L25H7 for Llama and found that the pattern is consistent with L18H6. In L18H6, each mechanism’s contribution to the attention logit landed on its counterfactual target and the combination of task-general attention and mechanism subspaces account for 99.7% of the attention logit variance across tokens. The other three Gemma retrieval heads (L19H1, L22H4, L23H2) show the same pattern: each subspace’s contribution lands on its own target, with 99.1–99.4% of variance explained. The Llama heads (L24H24, L22H14, L27H7, L25H7) show the same pattern, with 98.5–98.8% of variance explained. We added results to Appendix J and referred to them in Section 5.1.

Reflexive-specific behavior. We have removed the remark in Appendix A on later reflexive-specific heads. The per-head decomposition of the DLA-identified heads mentioned above shows that while they carry a larger reflexive contribution than L17H24 in the object → box format, none is reflexive-only: all four also read the positional and lexical subspaces.

The current presentation also does not sufficiently convey how these distributed representations and computations contribute to the paper's central claim about combining multiple binding mechanisms Extending our decomposition results across multiple heads implies a more distributed computation than a single primary retrieval head and we have softened that language in the manuscript. Our central claim is that even if distributed beyond a single head, the core computation is the same: retrieval heads reading all three subspaces at once and softmax deciding over the contributions, which we now state more clearly at the end of Section 5.1.

W5:

Independence of lexical and reflexive mechanisms. We have now addressed this. In the newly added music task extension, both lexical and reflexive mechanisms are active and we find that they can still be separately manipulated. In the boxes task, swapping one subspace moves 98-105% of the logit shift from the original target to the swap target while the other mechanisms’ targets change by at most 11% as much. On the music task, swapping one content subspace moves 79-120% of the logit shift from the original target to the swap target while the other content mechanism's target changes by only 3-17% as much.

This finding is compatible with the mentioned overlap between the lexical and reflexive subspaces because our swap method subtracts one entity representation within the subspace and adds in another, it does not overwrite the entire subspace. The lexical mechanism uses the queried entities and the reflexive mechanism uses the answer entity, so the two mechanisms hold different entities in any one prompt. Swapping out one queried entity for another as a targeted intervention therefore does not disrupt the answer entity's representation even if it is occupying the same subspace. We have added this to Appendix K.

W6:

Limited generalizability beyond synthetic entity retrieval We agree that our work does not cover natural-language entity tracking and have made this limitation explicit in the Discussion. As mentioned we have also now extended beyond the boxes task by adding the music task, which features three entities and queries based on two of them.

Models already achieve near-saturated accuracy on the original task. We see this as a requirement for studying how the model retrieves the (correct) answer. The counterfactual design then adds the necessary conflict between the mechanisms.

Q1:

Our evidence concerns the retrieval step of in-context binding tasks, and we do not extend the claim to general natural-language tasks. Within that scope, a small, consistent set of heads carries retrieval in Gemma 2-2B on both tasks we tested and in both Gemma and Llama the retrieval heads read all three mechanism subspaces.

Q2:

Yes, the boxes task is sufficient to distinguish between the three binding mechanisms at retrieval time because it is possible to build a counterfactual design where the three relevant variables for the three mechanisms (the position of the queried entity, the answer entity and the queried entity) predict different answers. Regarding the reflexive mechanism, we are not studying the upstream computational process, we are studying final retrieval behavior.

Q3:

The counterfactual is designed so the three mechanisms are in conflict. So our framework demonstrates that their contributions add to each candidate’s attention logit, and this is then resolved by softmax, a nonlinear computation. It is possible that negation could also make it to the final retrieval but we don’t study that here.

Comments on the paper

1 comment

We thank the reviewers for their thoughtful feedback. We have incorporated your suggestions into the revised manuscript and uploaded an updated version with newly added text highlighted in blue color.

We are encouraged that the reviewers found our results useful for understanding how language models integrate multiple sources of information and for providing useful insights to the broader interpretability research community (R1), the paper clear, easy to follow and a pleasure to read (R2) and the experiments thorough and convincing, with nice causal tests and several complementary methods (R2, R3). We are glad that reviewers valued the replication on the second model family (R2, R3), the released code (R3) and the convincing motivation to factor out the task-general component from the subspaces (R3).

We summarize the main changes here and respond to each reviewer below individually.

  1. We replicated our analysis on a second task in Appendix K (R1, R2, R3). We added the three-entity music task of Gur-Arieh et al. in Gemma 2-2B in order to both extend beyond a single task to include a more complex one, as well as cover a task where both the lexical and reflexive mechanisms are active on the same prompts. We found the same top retrieval heads (L18H6, L22H4, L19H1) to handle retrieval on this task and also showed that the subspaces can be edited individually, including the content mechanisms.

  2. We extended our attention logit decomposition to the other retrieval attention heads (R1, R2, R3). On the other prominent retrieval heads in Gemma and in Llama, we found that the pattern is consistent with the results we reported on the single primary head (Appendix J). All read the same three subspaces, with 99.1–99.4% (Gemma) and 98.5–98.8% (Llama) of attention-logit variance explained.

  3. We reframed our contribution and its novelty (R1, R2, R3). We agree that additivity of the attention logits follows from the linearity of the query, so the abstract, introduction and Section 5 now present the per-subspace decomposition as a method, with the subspaces directing the retrieval heads’ attention to their mechanisms’ targets as the finding. We further emphasize novelty over prior work by more explicitly calling out what was left open in prior work (manipulating one mechanism at a time) and our novel contributions in the introduction, such as predicting a new type of retrieval error (semantic neighbor).

  4. We clarified the separability of the lexical and reflexive subspaces (R1, R3). In the newly added music task extension where both lexical and reflexive mechanisms are active, we find that they can still be separately manipulated. This finding is compatible with the overlap between the lexical and reflexive subspaces due to our swap method. Our swaps target the entity representations themselves within the shared identity channel. Because the lexical and reflexive mechanisms have separate representations (queried entity vs answer entity), their representations can be edited separately even while occupying a shared channel. More detail in Appendix K.1.

We state what remains unaddressed as limitations in the revised Discussion section. We study the mechanism at retrieval, but don’t answer the question of how the signal was created and moved to the final token (R1). Our tasks synthetic and the results in more naturalistic settings remain untested (R1, R3). We thank the reviewers again for the feedback which has substantially improved the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.