- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper takes a step back from the usual circuit analysis paradigm and asks a relatively simple but underexplored question: when we identify circuits for the same task, do they actually look alike across inputs? And do they actually differ from circuits identified for other tasks? The authors formalize these as consistency (do circuits for the same task overlap?) and specificity (are circuits for one task distinguishable from those for another?), and then measure both across five models (Gemma-2, Llama-3.2, Qwen3, ranging 2B–4B) and six tasks spanning arithmetic, logic, and reasoning. They use two attribution methods (EAP-IG and RelP) and look at three granularity levels — attention heads / MLP blocks (component level), individual MLP neurons, and SAE features. The main takeaway, in my reading, is that component-level circuits turn out to be consistent but not specific: across different inputs to the same task, the identified heads and blocks overlap substantially, but the same heads and blocks get recruited for quite different tasks as well, suggesting that at this granularity circuits capture general-purpose computation rather than task-specific mechanisms. Neuron-level circuits show the opposite pattern — they are specific but not consistent — which creates a real tension for the field. The attribution method does matter (EAP-IG and RelP produce circuits with different compositions and layer preferences), but the high-level patterns hold for both, so the conclusions are not an artifact of a single method. SAE features sit somewhere in the middle, and an analysis of pretraining checkpoints suggests that consistency emerges early and may later be refined or partially overwritten. Overall, the paper makes a solid case that the field's reliance on component-level circuits may be overstated, and that the choice of attribution method and granularity level are far more consequential than previous work has acknowledged.
Critical strengths
-
Expriments are comprehensive and systematic. Covers 5 models of different scales and architectures. Encompasses 6 diverse task. Uses two independent attribution methods (EAP-IG and RelP) for cross-validation. Analyzes at multiple granularity levels (component-level, neuron-level, feature-level). Appendix contains rich supplementary experiments.
-
The core finding that component-level circuits are consistent but not specific, challenges a common assumption in the mechanistic interpretability field that component-level circuits can provide task-specific explanations. This finding has important implications for the field.
Critical weaknesses
-
Lack of deep mechanistic analysis of "Why Component-Level circuits are not specific?". The paper observes that component-level circuits are not specific, but the analysis of the root cause is insufficient. The authors mention several possibilities but do not validate or rule them out with additional experiments.
-
The paper primarily focuses on foundational findings, but does not deeply discuss the practical implications. In particular, if component-level circuits are not specific, then how effective are model editing methods based on component-level circuits? This question deserves further discussion.
Questions for authors
The paper uses EAP-IG and RelP but does not employ a third independent method (such as Direct Patching) for validation. Although both methods show consistent trends, given that the paper itself finds that "circuit structure depends on the attribution method," could there be some attribution bias that leads to the observed phenomena?
4 comments
Response to Reviewer QuEp (2/2)
Weakness 2 (Practical implications)
We discuss this in the last paragraph of Section 6.2, although not in detail. While writing the paper, we could not find many model editing methods that use component-level circuits, so we felt it would be hard to ground a longer discussion in existing work. That being said, we think the implications for model editing come down to how accurately a circuit localizes the behavior we want to change.
One way to think about this is in terms of precision (whether the edit changes only the intended behavior) and recall (how broadly the edit generalizes). The overlap we find across tasks suggests that edits to component-level circuits may have low specificity, a problem that prior work has also observed in model editing (Hoelscher-Obermaier et al., 2023). Neuron-level edits might be more precise, but since neuron-level circuits are less consistent across inputs, such an edit may not generalize across the task.
More broadly, we think designing reliable edits requires a better understanding of how the model produces the targeted behavior. For example, Rizwan et al. (2024) show that editing methods that decide when to apply an edit based on similarity in representation space are vulnerable to lexical bias, and often apply edits to unrelated prompts that share words with the edited prompt. We believe that the properties we propose, consistency and specificity, can help build that understanding.
Question 1 (lack of third independent method)
This is a fair question. We originally did not run activation patching ourselves because the papers introducing EAP-IG (Hanna et al., 2024) and RelP (Jafari et al., 2025) both compare their method with it. However, for completeness, we have now run activation patching at the component-level for all five models (Appendix F). To summarize, we find that both EAP-IG and RelP scores correlate positively with activation patching scores, EAP-IG more so than RelP (Spearman correlations of 0.92-0.97 and 0.66-0.75). This suggests that both methods, and EAP-IG in particular, approximate activation patching well.
We hope this addresses your concerns. If anything is still unclear or you have further questions, please let us know.
References
Wang et al. (2023): https://arxiv.org/abs/2211.00593
Lieberum et al. (2023): https://arxiv.org/abs/2307.09458
Hoelscher-Obermaier et al. (2023) https://aclanthology.org/2023.findings-acl.733/
Rizwan et al. (2024) https://arxiv.org/abs/2408.10411
Hanna et al. (2024) https://arxiv.org/abs/2403.17806
Jafari et al. (2025) https://arxiv.org/abs/2508.21258
ok
Thank you again for your review. We wanted to check whether our response has addressed your concerns. We think the expanded mechanistic case study you suggested has greatly improved the paper, and we'd like to hear whether you find it convincing and whether it's what you had in mind. If your concerns are addressed, we'd be grateful if you would consider reflecting that in your score.
Response to Reviewer QuEp (1/2)
Thank you for your review! We appreciate that you found our findings important. We address your concerns and questions below.
Weakness 1 (lack of in-depth mechanistic analysis)
We agree that this is a relevant concern. In Section 6.1 we proposed several explanations for why component-level circuits are not specific, but we did not test them directly. We felt that a full mechanistic analysis of all six tasks was out of scope (each of these analyses would be a full paper in and of itself), but we think a narrower analysis is still useful. In the paper, we already include an initial case study of the attention heads shared across tasks in Llama-3.2-3B (Section 5.1). We have now expanded this case study to all five models (Appendix J).
The first explanation is that the shared components perform the same function in every task, meaning that the tasks share part of their computation. To test this, we look at the attention heads that are shared by all six task circuits. We first classify every head into roles identified in prior work, such as name mover heads in IOI (Wang et al., 2023) and correct-letter heads in multiple-choice tasks (Lieberum et al., 2023), using simple correlational metrics like attention weights and direct logit attribution. Since these roles are only based on correlational metrics, we then test them causally. For each shared head with a task-relevant role, we test whether the head actually performs that role, by steering its attention to a different token and checking whether the model’s answer changes accordingly.
If the shared heads performed the same function in every task, we would expect them to have a task-relevant role in multiple tasks. In two of the three model families (Llama and Gemma), we do find one shared head for which this holds. We refer to this head as an “answer mover”, since it acts like a name mover head in IOI and a correct-letter head in the multiple-choice tasks. When we steer the attention of this head (and the rest of its key-value group) to a different token, the model outputs that token as its answer a majority of the time (and much more often than when we steer random heads). This behavior also generalizes to a variety of new tasks we construct where the answer is a word in the prompt.
However, heads like this are rare, so we think this explanation only accounts for a small part of the overlap. Only 26% of the shared heads have a task-relevant role in at least one task, while 50% have no role from prior work, and the rest either amplify the model’s chosen answer or have a generic attention pattern, like previous-token heads. This is similar to the proportion among heads that are not shared (24%). Overall, our steering experiments identify only 7 of the 98 shared heads as having a task-relevant role in more than one task. Therefore, most of the shared heads do not seem to perform the same task-relevant function across tasks, and understanding what they are doing is an interesting direction for future work.
The second explanation was about the granularity of the components. A single attention head or MLP block can be involved in many different computations, so two tasks that rely on different features in the same component will share that component. Our neuron-level results support this explanation. In particular, MLP blocks make up most of the overlap between component-level circuits, but when we zoom into the neuron-level circuits of different tasks, the overlap is close to zero (Figure 4). This suggests that different tasks use different neurons within the same MLP blocks, and that these blocks are likely not performing the same computation for every task.