acceptodds
Under review as a conference paper at ICLR 2027

A Sparse Circuit for Context To Query Information Transfer

Abstract

A number of head families have been identified for in-context learning (ICL), ranging from heads that promote token and semantic-level copying to heads that perform retrieval and task-level functions. While several of these head families encode dedicated functions that generalize across ICL tasks, less is known about which heads causally mediate information flowbetween the context and the query. Using a causal intervention that patches only the attention connections between query tokens and context tokens, we first identify attention heads that mediate this flow directly. We then identify a shared set of 20 such heads, dubbed ICL20, that supports a diverse range of in-context learning functions. Ablating ICL20 degrades performance across eight diverse evaluation tasks. Moreover, individual ICL20 heads adopt different functional roles by task. Our approach recovers compact, general-purpose circuitry underlying several previously studied ICL functions and reveals two new roles assumed by attention heads during ICL: verbalizing relations between context segments and the query, and using chat-template markers as retrieval anchors.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, safety, model behavior

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SBfiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies which attention heads mediate information transfer from a context to a fixed query. It adapts desiderata-based masking to replace context keys and values at query positions, learns task-specific sparse masks, and aggregates their rankings into a shared set of 20 heads, ICL20. Experiments primarily use Llama-3.1-8B-Instruct, with an additional Llama-3.2-3B study. The authors report cross-task ablation effects, investigate evidence transfer and label mapping using larger masks, and propose task-dependent roles involving source selection, relational representations, and chat-template anchors.

Critical strengths
  1. The effects extend beyond the tasks used to identify the heads. ICL20 is selected using persona, affect, and reading-comprehension tasks, but ablating it also hurts translation, copying, synthetic sequence tasks, and free-form QA. For the 8B model, retrieval accuracy drops from 0.89 to 0.08 and rule-inversion accuracy from 0.85 to 0.13 in Section 5.2.1. Figure 4 shows a similar broad pattern for a separately identified set in the 3B model. This is an interesting result because the heads appear to matter for more than the particular tasks used to find them.

  2. The label-remapping experiment helps separate using evidence from choosing an output label. In Appendix B.1.1, the evidence mask can change the model’s answer while leaving it in the original Yes/No vocabulary. Adding 49 Ba/Ga-specific heads in layers 16–31 increases the rate of correct new-label answers from at most 0.1% to 89% on the Ba/Ga joint-intervention pairs, and the effect also transfers to Portuguese and French labels. This gives a concrete indication that carrying contextual evidence and mapping the answer into the requested vocabulary involve partly different sets of heads.

Critical weaknesses
  1. I am not yet convinced that the experiments isolate context transfer in the fixed ICL20 set. The discovery procedure replaces context K/V at query positions, but the main evaluation in Section 5 zeroes whole head outputs. This can also disrupt other computations those heads perform. Meanwhile, the high-IIA results and the label-remapping and multilingual experiments use task-specific masks, often with roughly 120 heads, rather than the exact 20-head aggregate. I could not find held-out IIA results for that fixed set, or a control that separates its context-to-query contribution from its other connections. The ablation results show that these heads matter across tasks, but they do not yet show that the damage comes specifically from removing contextual information. The same issue applies to the chat-anchor interpretation: attending to markers and causing errors when ablated does not establish what information the heads read from those markers.

  2. I would have expected a direct comparison with function-vector heads. Todd et al. [1] already identify a small shared set of heads by averaging causal effects across ICL tasks. Yin and Steinhardt [2] compare FV and induction heads across 12 models and find FV heads more important for few-shot ICL, especially in larger models. The paper cites both, but Sections 4–5 only compare ICL20 with token copiers, concept copiers, and retrieval heads. Restricting the intervention to context K/V is a meaningful methodological change, but a different way of finding heads does not necessarily identify a different mechanism. Measuring overlap and comparing equal-sized head sets under the same interventions would help establish what ICL20 adds to the existing picture.

[1] Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, and David Bau. Function Vectors in Large Language Models. ICLR, 2024.

[2] Kayo Yin and Jacob Steinhardt. Which Attention Heads Matter for In-Context Learning? ICML, 2025.

Questions for authors
  1. What held-out IIA does the exact, fixed ICL20 set achieve under the Section 2 intervention? Do the large QA effects persist when intervention is restricted to its context-to-query contributions, rather than zeroing whole head outputs? This would clarify whether the main result is transfer-specific or reflects broader head importance.

  2. How much does ICL20 overlap with the top 20 FV heads in each model, and how do their intervention effects compare on the same evaluation examples? A distinct effect, including after accounting for shared heads, would materially strengthen the claimed contribution.

Reviewer_EJ7bOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates attention heads involved in context-to-query information transfer during in-context learning. It adapts desiderata-based component masking to perform counterfactual interventions on context-to-query attention and learns sparse task-specific head masks evaluated using interchange intervention accuracy (IIA). Aggregating pruning-order rankings across persona, affect, and reading-comprehension tasks yields a 20-head set (ICL20), which is evaluated primarily through whole-head zero ablation across 18 tasks, with comparisons against token-copying, concept-copying, and retrieval heads. Additional experiments investigate label-space and cross-lingual transfer, individual-head behavior, and replication on Llama-3.2-3B-Instruct.

Critical strengths

1.Targeted causal interventions. The context-to-query K/V patching procedure is well aligned with the research question. The task-specific IIA results (10–65 heads achieving at least 80% IIA; Section 3.2), matched-sparsity random-mask controls (Section 2), cross-task transfer (Appendix A.3), and cross-lingual transfer (AUC 0.92–0.95; Appendix B.1.2) provide meaningful evidence of non-arbitrary head selection.

2.Concrete mechanistic observations. The single-head translation experiments (Figure 3) and the analysis of template-attending heads (Section 5.2.1) reveal interesting, testable behaviors. Replication on a second Llama model (Appendix A.1) further supports the reproducibility of the reported phenomena within this model family.

Critical weaknesses

1. The central mechanistic claim is not sufficiently established for the final ICL20 set.

(a) Mismatch between discovery and validation. ICL20 is selected by aggregating pruning-order rankings obtained from task-specific context-to-query patching masks (Section 4). However, its principal cross-task validation relies on whole-head zero ablation at all positions (Section 5.1). Such ablation establishes computational importance without isolating the claimed information-transfer function. The edge-restricted IIA of the final ICL20 set is not reported for any task. Consequently, the principal cross-task evaluation does not directly test the defining mechanistic property attributed to the final circuit.

(b) Tension between the proposed mechanism and the head-level evidence. The authors report that six question-side template-attending heads account for nearly all the performance loss caused by ablating nine such heads on Retrieval and Rule Inversion, while also stating that these heads cannot be patched to steer the output (Section 5.2.1). These are the two tasks on which ICL20 exhibits its largest ablation advantage over the compared head families (Figure 1b: 0.81 versus at most 0.05; 0.72 versus at most 0.03). Although this does not exclude an information-transfer role, it leaves unresolved whether the observed degradation reflects the proposed context-to-query transfer mechanism or other computations involving contextual anchors.

(c) Insufficient controls for alternative explanations. The whole-head ablation experiments lack layer-matched random head sets, mean/resample ablation, and control tasks that do not require in-context information. The matched-sparsity random-mask controls used during circuit discovery do not directly address these alternative explanations for the ablation results. Consequently, the evidence does not sufficiently distinguish a specialized transfer circuit from a collection of heads broadly important for contextual computation.

2. The functional distinctiveness, generality, and empirical interpretation of ICL20 require stronger support.

(a) Limited differentiation from established ICL mechanisms. The methodology builds on existing component-masking and causal-intervention techniques, making the identification of a functionally distinct circuit central to the paper's contribution. The limited overlap between ICL20 and token-copying, concept-copying, and retrieval heads (Section 4.2) does not, by itself, establish functional distinctiveness. In particular, a relevant comparison with function-vector heads, discussed in the cited literature (Todd et al., 2024; Yin & Steinhardt, 2025), is missing. The relationship between ICL20 and previously identified ICL mechanisms therefore remains insufficiently characterized.

(b) Generality and statistical evidence require qualification. Six of the 18 evaluation tasks are also used in circuit discovery. Among the remaining tasks, at least one copier-head set produces a larger performance drop than ICL20 on each of the five translation tasks and on Copy (Figure 1b). This heterogeneity does not invalidate the reported average advantage, but warrants a more qualified interpretation of generality. Furthermore, retrieval-head ablation results are displayed for only two tasks in Figure 1b, whereas Section 4.2 reports an average comparison involving all three head families ((p<0.001), (n=18)) without specifying the statistical test or comparison set. Both evaluated models also belong to the Llama family, leaving cross-family generalization untested.

(c) Quantitative claims and interpretation need greater precision. Section 5.2.1 states that ablating head 13.18 reduces accuracy by more than 75% across all translation tasks, whereas Figure 3 shows smaller relative reductions for Es-En (0.40 to 0.14; 65%) and Pt-En (0.39 to 0.11; approximately 72%). The same figure shows improved accuracy when head 13.18 is redirected on four of five language pairs, as well as substantial degradation from ablating the single concept-copier head 11.6; these observations warrant further discussion. The statement that ablation causes the model to produce the contrastive response (Section 5.2) is also stronger than aggregate accuracy reductions alone establish. Moreover, the characterization of context and label information as being transported through "distinct circuits" (Section 3.2) would benefit from qualification, given the shared heads and their role in the joint interventions reported in Appendix B.1.1. The reported layer range of the Ba/Ga heads also differs between the main text (8–31) and appendix (16–31). Finally, the manuscript contains unresolved presentation issues, including broken cross-references in Appendix B.1.1 and Figure 9, inconsistent terminology, and an incorrectly titled Appendix B.

Questions for authors

1.Mechanistic validation. What is the edge-restricted patching IIA of the final ICL20 set on the discovery tasks and suitable held-out tasks? How do query-position-restricted ablation and mean ablation compare with whole-head zero ablation?

2.Alternative explanations and baselines. How does ICL20 compare with layer-matched random 20-head sets (across multiple seeds) and function-vector heads under identical ablation protocols? Does ablating ICL20 also impair tasks that do not require in-context information?

3.Statistical reporting. Which statistical test supports the reported (p<0.001), and which tasks and head families are included? Please also clarify why Figures 1b and 3 report different performance drops for apparently corresponding ablations (e.g., Pt-En, ICL20: 0.41 versus 0.30).

4.Head-level interpretation. How do the authors reconcile the template-attending heads' substantial ablation effects with their reported inability to steer outputs through patching? Why does redirecting head 13.18 improve translation accuracy on four language pairs?

Reviewer_Z7CoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how large language models transfer information from context to query during in-context learning (ICL). The authors propose a causal intervention that patches only the attention connections from context tokens to query tokens, and use it to identify attention heads that directly mediate this information flow. On several natural-language reasoning tasks, they train desirability-based masks and then aggregate pruning orders to derive a compact set of 20 heads, termed ICL20. They report that ablating ICL20 substantially degrades performance across classification, reading comprehension, translation, copying, retrieval, and some free-form reasoning tasks. They further analyze the roles of individual heads and claim two additional functions: some heads verbalize relations between context segments and the query, and some attend to chat-template markers as retrieval anchors. Overall, the paper presents cross-task, cross-lingual, and cross-format validation in support of a sparse, shared circuit for context-to-query information transfer.

Critical strengths

The paper addresses an important and timely question in mechanistic interpretability: not just which heads are correlated with ICL, but which heads causally mediate information transfer from context to query. The proposed intervention is more targeted than standard attention analysis or probing, and therefore has the potential to yield more meaningful mechanistic conclusions. This substantially strengthens the paper’s relevance and its potential impact on the field. Besides, the authors validate their claims across multiple task families, multiple languages, and multiple prompt formats, and they also compare ICL20 against previously studied head families such as token copiers, concept copiers, and retrieval heads that makes the result more convincing

Critical weaknesses
  1. The approach of injecting patch context K/V into the query can show that ‘some read-in pathway of these heads is important,’ but whether this is באמת equivalent to ‘the minimal causal circuit for context → query information transfer’ still lacks a rigorous argument. 2.The definition and selection of ICL20 appear somewhat heuristic and post hoc, and the paper lacks a systematic sensitivity analysis of key choices, including seed selection and the top-cutoff threshold
Questions for authors
  1. How stable is ICL20 across random seeds, task subsets, and cutoff choices? For example, do the identified heads remain largely the same if the cutoff is changed from 20 to 10/30/50, or if only a subset of the training tasks is used? 2. Since the patching intervention replaces context K/V and can indirectly affect later query-query attention and residual-stream states, did you run more localized control experiments such as path patching, mean/noise ablation, or layer-restricted interventions to verify that the results are not driven by broader dynamical perturbations? As a future-work suggestion, I would also encourage more direct causal tests for the “verbalization” and “chat-template anchor” interpretations, e.g., by perturbing template tokens or label spaces in a controlled way.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.