acceptodds
Under review as a conference paper at ICLR 2027

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

Abstract

The circuits framework in mechanistic interpretability aims to identify sparse subgraphs of model components that are causally responsible for a behavior, typically evaluated by measuring _necessity_ and _sufficiency_. But these criteria say little about whether a circuit consistently captures how a model performs a task, or if it is specific to that task. We study these two properties, _consistency_ and _specificity_, across six tasks and five models, extracting circuits at the component level (attention heads and MLP blocks) and at the level of individual MLP neurons. We find that component-level circuits are highly consistent and causally important on most tasks, but they are not specific: ablating one task's circuit damages another task's performance about as much as that task's own circuit does. Neuron-level circuits, on the other hand, exhibit higher task-specificity but are far less consistent within tasks. This is explained by circuit overlap: component-level circuits share most of their components across all task pairs, related or not, while neuron-level circuits overlap only between closely related tasks. In a case study of the components shared by the task circuits of `Llama-3.2-3B` we show that they consist mostly of MLP blocks, we show that they consist mostly of MLP blocks, while the few attention heads within turn out to have general-purpose roles. Overall, our findings raise questions about the degree to which circuits can support targeted understanding of, and intervention on, model behavior.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, circuit discovery, circuit evaluation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_QuEpOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper takes a step back from the usual circuit analysis paradigm and asks a relatively simple but underexplored question: when we identify circuits for the same task, do they actually look alike across inputs? And do they actually differ from circuits identified for other tasks? The authors formalize these as consistency (do circuits for the same task overlap?) and specificity (are circuits for one task distinguishable from those for another?), and then measure both across five models (Gemma-2, Llama-3.2, Qwen3, ranging 2B–4B) and six tasks spanning arithmetic, logic, and reasoning. They use two attribution methods (EAP-IG and RelP) and look at three granularity levels — attention heads / MLP blocks (component level), individual MLP neurons, and SAE features. The main takeaway, in my reading, is that component-level circuits turn out to be consistent but not specific: across different inputs to the same task, the identified heads and blocks overlap substantially, but the same heads and blocks get recruited for quite different tasks as well, suggesting that at this granularity circuits capture general-purpose computation rather than task-specific mechanisms. Neuron-level circuits show the opposite pattern — they are specific but not consistent — which creates a real tension for the field. The attribution method does matter (EAP-IG and RelP produce circuits with different compositions and layer preferences), but the high-level patterns hold for both, so the conclusions are not an artifact of a single method. SAE features sit somewhere in the middle, and an analysis of pretraining checkpoints suggests that consistency emerges early and may later be refined or partially overwritten. Overall, the paper makes a solid case that the field's reliance on component-level circuits may be overstated, and that the choice of attribution method and granularity level are far more consequential than previous work has acknowledged.

Critical strengths
  • Expriments are comprehensive and systematic. Covers 5 models of different scales and architectures. Encompasses 6 diverse task. Uses two independent attribution methods (EAP-IG and RelP) for cross-validation. Analyzes at multiple granularity levels (component-level, neuron-level, feature-level). Appendix contains rich supplementary experiments.

  • The core finding that component-level circuits are consistent but not specific, challenges a common assumption in the mechanistic interpretability field that component-level circuits can provide task-specific explanations. This finding has important implications for the field.

Critical weaknesses
  • Lack of deep mechanistic analysis of "Why Component-Level circuits are not specific?". The paper observes that component-level circuits are not specific, but the analysis of the root cause is insufficient. The authors mention several possibilities but do not validate or rule them out with additional experiments.

  • The paper primarily focuses on foundational findings, but does not deeply discuss the practical implications. In particular, if component-level circuits are not specific, then how effective are model editing methods based on component-level circuits? This question deserves further discussion.

Questions for authors

The paper uses EAP-IG and RelP but does not employ a third independent method (such as Direct Patching) for validation. Although both methods show consistent trends, given that the paper itself finds that "circuit structure depends on the attribution method," could there be some attribution bias that leads to the observed phenomena?

4 comments
AuthorsOpenReview ↗

Response to Reviewer QuEp (1/2)

Thank you for your review! We appreciate that you found our findings important. We address your concerns and questions below.

Weakness 1 (lack of in-depth mechanistic analysis)

We agree that this is a relevant concern. In Section 6.1 we proposed several explanations for why component-level circuits are not specific, but we did not test them directly. We felt that a full mechanistic analysis of all six tasks was out of scope (each of these analyses would be a full paper in and of itself), but we think a narrower analysis is still useful. In the paper, we already include an initial case study of the attention heads shared across tasks in Llama-3.2-3B (Section 5.1). We have now expanded this case study to all five models (Appendix J).

The first explanation is that the shared components perform the same function in every task, meaning that the tasks share part of their computation. To test this, we look at the attention heads that are shared by all six task circuits. We first classify every head into roles identified in prior work, such as name mover heads in IOI (Wang et al., 2023) and correct-letter heads in multiple-choice tasks (Lieberum et al., 2023), using simple correlational metrics like attention weights and direct logit attribution. Since these roles are only based on correlational metrics, we then test them causally. For each shared head with a task-relevant role, we test whether the head actually performs that role, by steering its attention to a different token and checking whether the model’s answer changes accordingly.

If the shared heads performed the same function in every task, we would expect them to have a task-relevant role in multiple tasks. In two of the three model families (Llama and Gemma), we do find one shared head for which this holds. We refer to this head as an “answer mover”, since it acts like a name mover head in IOI and a correct-letter head in the multiple-choice tasks. When we steer the attention of this head (and the rest of its key-value group) to a different token, the model outputs that token as its answer a majority of the time (and much more often than when we steer random heads). This behavior also generalizes to a variety of new tasks we construct where the answer is a word in the prompt.

However, heads like this are rare, so we think this explanation only accounts for a small part of the overlap. Only 26% of the shared heads have a task-relevant role in at least one task, while 50% have no role from prior work, and the rest either amplify the model’s chosen answer or have a generic attention pattern, like previous-token heads. This is similar to the proportion among heads that are not shared (24%). Overall, our steering experiments identify only 7 of the 98 shared heads as having a task-relevant role in more than one task. Therefore, most of the shared heads do not seem to perform the same task-relevant function across tasks, and understanding what they are doing is an interesting direction for future work.

The second explanation was about the granularity of the components. A single attention head or MLP block can be involved in many different computations, so two tasks that rely on different features in the same component will share that component. Our neuron-level results support this explanation. In particular, MLP blocks make up most of the overlap between component-level circuits, but when we zoom into the neuron-level circuits of different tasks, the overlap is close to zero (Figure 4). This suggests that different tasks use different neurons within the same MLP blocks, and that these blocks are likely not performing the same computation for every task.

AuthorsOpenReview ↗

Response to Reviewer QuEp (2/2)

Weakness 2 (Practical implications)

We discuss this in the last paragraph of Section 6.2, although not in detail. While writing the paper, we could not find many model editing methods that use component-level circuits, so we felt it would be hard to ground a longer discussion in existing work. That being said, we think the implications for model editing come down to how accurately a circuit localizes the behavior we want to change.

One way to think about this is in terms of precision (whether the edit changes only the intended behavior) and recall (how broadly the edit generalizes). The overlap we find across tasks suggests that edits to component-level circuits may have low specificity, a problem that prior work has also observed in model editing (Hoelscher-Obermaier et al., 2023). Neuron-level edits might be more precise, but since neuron-level circuits are less consistent across inputs, such an edit may not generalize across the task.

More broadly, we think designing reliable edits requires a better understanding of how the model produces the targeted behavior. For example, Rizwan et al. (2024) show that editing methods that decide when to apply an edit based on similarity in representation space are vulnerable to lexical bias, and often apply edits to unrelated prompts that share words with the edited prompt. We believe that the properties we propose, consistency and specificity, can help build that understanding.

Question 1 (lack of third independent method)

This is a fair question. We originally did not run activation patching ourselves because the papers introducing EAP-IG (Hanna et al., 2024) and RelP (Jafari et al., 2025) both compare their method with it. However, for completeness, we have now run activation patching at the component-level for all five models (Appendix F). To summarize, we find that both EAP-IG and RelP scores correlate positively with activation patching scores, EAP-IG more so than RelP (Spearman correlations of 0.92-0.97 and 0.66-0.75). This suggests that both methods, and EAP-IG in particular, approximate activation patching well.

We hope this addresses your concerns. If anything is still unclear or you have further questions, please let us know.

References

Wang et al. (2023): https://arxiv.org/abs/2211.00593

Lieberum et al. (2023): https://arxiv.org/abs/2307.09458

Hoelscher-Obermaier et al. (2023) https://aclanthology.org/2023.findings-acl.733/

Rizwan et al. (2024) https://arxiv.org/abs/2408.10411

Hanna et al. (2024) https://arxiv.org/abs/2403.17806

Jafari et al. (2025) https://arxiv.org/abs/2508.21258

Thank you again for your review. We wanted to check whether our response has addressed your concerns. We think the expanded mechanistic case study you suggested has greatly improved the paper, and we'd like to hear whether you find it convincing and whether it's what you had in mind. If your concerns are addressed, we'd be grateful if you would consider reflecting that in your score.

Reviewer_RS1rOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that circuit quality should be measured beyond necessity and sufficiency. The authors propose that circuits should also be consistent (within the same task, the same components should be reused) and specific (different tasks should have different circuits). They trace per-prompt circuits using two counterfactual-based attribution methods (EAP-IG and RelP) at two levels of granularity (component and neuron levels), across six tasks and five models. They use overlap metrics and ablation experiments and show that component-level circuits are consistent but not specific, while neuron-level circuits are specific but not consistent. The paper discusses important questions about the granularity of circuit analysis, component reuse across tasks, and how circuits can depend on the attribution method.

Critical strengths
  1. The paper asks a good question about how circuits should be evaluated beyond necessity and sufficiency, and defines clear metrics to quantify their proposed desirable properties (consistency and specificity). The experimental setup is extensive, covering multiple tasks, models, attribution methods, and ablation strategies, making the empirical analysis valuable.
  2. The paper investigates how the level of granularity affects circuit analysis, which I believe is an important question for mechanistic interpretability. Even considering the concerns about the proposed metrics, the differences between component-level and neuron-level circuits are interesting and worth exploring further. Another interesting finding is that different attribution methods produce very different circuits, even at the same level of granularity. This shows how strongly circuit analysis can depend on methodological choices.
Critical weaknesses
  1. The paper argues that useful circuits should have high consistency across inputs of the same task. However, this requirement does not account for variation that may reflect input-dependent mechanisms (Franco et al., 2026; Rai et al., 2026; Wu and Barez, 2026). The authors' suggestion that circuits could be defined as "fuzzy" or input-dependent restates observations that have already been extensively made in the literature (Franco et al., 2026; Rai et al., 2026). More fundamentally, if circuits are not consistent across the same task, perhaps the task is not in fact coherent, or else a notion of task itself needs to be redefined. For instance, Franco et al. (2026) show that in IOI, ABBA prompts have different circuits than BABA prompts, and Rai et al. (2026) show that different mechanisms can be used within the same task. So, by simply defining IOI ABBA as a different task from IOI BABA, we could have more consistent circuits. Therefore, the paper needs a clearer justification for consistency as a desirable property of circuits, a more robust investigation of the influence of the task definition in the paper’s result, and how the paper’s conclusions and recommendations are different from the related work.
  2. Per-prompt circuit tracing using counterfactual-based attribution methods may introduce spurious components, and related instability has been documented by Méloux et al. (2026) and Wu and Barez (2026). Méloux et al. (2026) show that even exact single-input causal mediation analysis exhibits high variance, which is amplified by gradient-based approximations such as the ones used by EAP-IG. Wu and Barez (2026) show that applying EAP-IG to find per-prompt circuits produces less faithful circuits because of high gradient noise and not considering the interactions between edges, and come up with a best-of-N sampling method to mitigate this problem. The most common methodology to find circuits averages the effect of ablating a certain component over a dataset as a way to have a better estimation of the effect of that component on model performance, which is not possible with a simple application of this method in a per-prompt fashion. Therefore, part of the results can be an artifact of the noisy circuit extraction rather than mechanistic differences.

References

Questions for authors
  1. Are the necessity results averaged over multiple independently sampled random control circuits? If so, how much variability is observed across these controls? For this metric, having a range of necessity values makes more sense, especially for component-level circuits, since it may vary depending on the random circuit choice.
  2. Do neuron-level circuits in a task cluster together? For example, can the authors check whether different IOI templates (e.g., ABBA vs. BABA) produce different clusters, and whether consistency increases within those clusters?
4 comments
AuthorsOpenReview ↗

Response to Reviewer RS1r (1/2)

Thank you for your detailed and constructive review! We appreciate you pointing us to closely related work, which we have added to the paper. We address your concerns and questions below, including an updated framing that we think strengthens the paper.

Weakness 1 (Consistency and task definition)

We agree that the mechanism a language model uses may not be the same across prompt-specific variations of a task. However, we don’t think this makes consistency any less useful as a property to measure. For example, if our IOI dataset mixed the ABBA and BABA formats, we would expect to measure lower consistency, which would tell us that our circuit was missing something about how the model solves the task. This would then motivate splitting the task into groups of similar prompts and finding a separate circuit for each group, which is what Rai et al. (2026) do.

In our case though, the IOI dataset we use from MIB only contains BABA prompts, so the ABBA/BABA split doesn’t materially affect our results. Our other tasks also follow fairly narrow, fixed templates. That said, one of our main findings is that neuron-level circuits have low consistency, even on these narrowly defined tasks. Following the same reasoning as above, this suggests that no single set of neurons explains how the model solves every prompt of a task. One possible reason is that the model uses different mechanisms for different prompts. For example, Nikankin et al. (2025) find that language models solve arithmetic using a “bag of heuristics”, where individual MLP neurons respond to specific patterns in the operands, which means that different neurons would be important for different subsets of our Addition prompts. Another possible reason is redundancy among neurons that implement the same mechanism. Figuring out which of these is the case would be an interesting direction for future work.

We also think that our work differs from Franco et al. (2026) and Rai et al. (2026) in its overall goal. Both papers propose new methods for finding circuits when the mechanism varies within a task. In particular, Franco et al. trace prompt-specific circuits, while Rai et al. group examples before finding circuits. On the other hand, our goal is to propose properties for evaluating circuits, regardless of how they were found. The first property, consistency, interplays with both works: low consistency is a strong signal that a single, task-level circuit is not enough, and is thus a strong motivation for using methods like theirs. The second property, specificity, measures whether a task’s circuit is different from the circuits of other tasks, which isn’t the main focus of either paper. In our paper, we find that these two properties don’t always go together. For example, our component-level circuits are highly consistent, but they are also largely shared with other tasks.

We have updated the paper to include this discussion. In particular, we added Franco et al. and Rai et al. to the related work (Section 7), and in the introduction we now note that low consistency may mean that the model uses different mechanisms for different prompt-specific versions of a task. Finally, in the conclusion, we now cite both papers in our discussion of “fuzzy” circuits.

Weakness 2 (Noise in per-example circuits)

It’s true that attribution scores for single examples can be noisy, and this is an issue for nearly all circuit discovery methods, including exact activation patching (Méloux et al., 2026). To limit its effect, our task circuits only keep components that appear in at least P% of the circuits of 1,000 examples, and all ablations are measured on a separate set of 200 examples. Furthermore, our main trends also hold under both EAP-IG and RelP, even though the two methods only partly agree with each other, and under both zero and mean ablation (Appendix E). If our results were mostly an artifact of noise, we would not expect them to hold across all of these settings. Additionally, since multiple reviewers asked for it, we have run activation patching on every component of all five models (Appendix F). We find that EAP-IG ranks components almost identically to activation patching (Spearman correlations of 0.92-0.97 across models), so our EAP-IG results closely match what exact patching would give. RelP also correlates with activation patching, though less strongly (0.66-0.75).

Question 1 (control circuits)

In our experiments, we use a single randomly sampled control set for each model and task (and each top-K and P threshold). We agree that sampling multiple control sets and reporting the range would give a better estimate of necessity. Due to compute and time constraints, it is not straightforward for us to rerun all of our experiments this way, but we are planning to add results with multiple control sets by the end of author response for at least one model.

AuthorsOpenReview ↗

Response to Reviewer RS1r (2/2)

Question 2 (neuron-level circuit clustering)

This is an interesting idea, fairly similar to the approach of Rai et al. (2026). We think it is complementary to our work. In particular, if clustering identified groups of prompts with higher consistency, this would support using consistency to detect when a task should be split. As we mention above, our IOI dataset only contains BABA prompts, so clustering our neuron-level circuits for IOI would not tell us anything about the ABBA/BABA split.

Therefore, we are planning to rerun our IOI experiments on a dataset that contains both ABBA and BABA prompts, and check whether the neuron-level circuits cluster by format and whether consistency is higher within each cluster. We are happy to add this analysis to the appendix, and it should be done by the end of the author response period.

Based on Franco et al., we would expect ABBA and BABA prompts to produce different neuron-level circuits, but we would not expect consistency within each cluster to be high, since our current IOI results already use only the BABA template and show low consistency.

We hope this addresses your concerns. If anything is still unclear or you have further questions, please let us know.

References

Nikankin et al. (2025): https://arxiv.org/abs/2410.21272

Franco et al. (2026): https://arxiv.org/abs/2602.13483

Rai et al. (2026): https://arxiv.org/abs/2605.09129

Méloux et al. (2026): https://arxiv.org/abs/2510.00845

Reviewer_RS1rOpenReview ↗

Response to Rebuttal

I thank the authors for their responses. All my concerns have been addressed.

AuthorsOpenReview ↗

Follow-up to Reviewer RS1r

Thank you for responding, we are glad that all of your concerns have been addressed! We would be grateful if you would consider updating your score to reflect this. Additionally, as promised, we have run your two requested experiments, and report the results below.

Question 1 (control circuits)

We resample the control set 20 times for every task circuit of Qwen3 4B, using the EAP-IG circuits at P=50% and at top-K=5% and top-K=10%. We show the results in this figure (https://imgur.com/a/Zt2rgDI), with top-K=5% in the top panel and top-K=10% in the bottom panel.

Each dot is one control set, and the y axis is the accuracy drop when we ablate it. The star marks the control set that we use in the paper, and the red line marks the accuracy drop when we ablate the task circuit.

We observe that the neuron-level circuits (yellow) are necessary, i.e. the control sets cause a lower accuracy drop than the task circuit. At the component level (blue), we also observe that the control sets cause a lower accuracy drop than the task circuit, although the drop varies more between control sets in Addition and Boolean Logic at top-K=10%. We also see a smaller gap in ARC (Challenge) at top-K=5%, where every control set lowers accuracy by 42 to 49 points, compared with 60 points for the circuit.

Overall, we find that, on average over the control sets, the necessity of the circuits from our original paper still holds, although, in the Boolean Logic task, there is one (outlier) control set that causes a larger accuracy drop than the component-level circuit.

Question 2 (neuron-level circuit clustering)

To test whether neuron-level circuits cluster by prompt format, we use the ABBA versions of our original BABA IOI prompts from MIB, in which the first two names are swapped (e.g. "When Joe and Roy were about to leave the music store, Joe gave a needle to" becomes "When Roy and Joe were about to leave the music store, Joe gave a needle to"). We extract neuron-level EAP-IG circuits for these ABBA prompts across all five models, and compare them with our original BABA circuits.

We find that the per-prompt circuits strongly cluster by format, in line with Franco et al. (2026). This figure (https://imgur.com/a/mtmhIMJ) shows a heatmap of the Jaccard overlap between the circuits of every pair of prompts. BABA-BABA and ABBA-ABBA pairs overlap much more than BABA-ABBA and ABBA-BABA pairs. Interestingly, this contrast is weakest in Qwen3 4B, the most capable model we study, which might be related to stronger models being less sensitive to the prompt format. Note that the thin diagonal lines in the top right and bottom left are ABBA prompts paired with their original BABA counterparts, such as the two prompts in our example above. These matched pairs overlap more than other cross-format pairs.

As we predicted, splitting the prompts by format increases consistency (Reuse@50 goes from 0.6-1.4% for the mixed prompts to 2.6-9.5% within a single format), but it remains low overall.

Looking at the heatmap again, we also observe smaller clusters within each format, which turn out to be the nine sentence templates used in MIB for the IOI prompts, e.g.

While {name_A} and {name_B} relaxed, {name_C} gave a {object} to

The {place} {name_A} and {name_B} went to had a {object}. {name_C} gave it to

Motivated by the idea that the model may organize its computation around such prompt-specific details, we split each format further by template, which increases consistency further (Reuse@50 of 9-20% within a single template).

Overall, these results suggest that consistency can help identify meaningful subdivisions of a task, such as the prompt format or the sentence template used in the IOI task.

Reviewer_UYwROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Typical circuit discovery methods use metrics, such as sufficiency and necessity, to demonstrate that the final circuit is causally responsible for a certain model behavior. However, there are other desired properties that circuits should satisfy, namely consistency and specificity. Consistency is where much of the identified circuit should be active across different samples of the same task, and specificity is where the identified circuit is specific to the current task and does not affect the performance of other tasks. Thus, the authors examined two levels of granularity when performing circuit discovery: component-level, where the nodes are attention heads and MLP layers, and neuron-level, where the MLP layers are split into individual neurons. The authors evaluated these two granularities across various model families and tasks to determine if the resulting circuits are consistent and specific. The results revealed that the component-level circuits are consistent but not specific, and the neuron-level circuits are the opposite.

Critical strengths
  1. Extensive results: the authors explored the two properties (consistency and specificity) across multiple tasks and model families to show how the conclusions are general.
  2. Novel problem formulation: The problem the authors aim to analyze, are circuit consistent and specific, is a valid concern in MI. The final circuits should be consistent across samples and specific to the task it is responsible for. Viewing an entire MLP layer as a single node is oversimplification since a layer can have thousands of neurons and, as a result, have many different functionalities.
Critical weaknesses
  1. Limited baselines: the authors mentioned that there was little overlap between the circuits found with EAP-IG and RelP, even though the two methods are two different ways to approximate activation patching and should be aligned. This could mean that the dataset was too diverse to begin with, possibly the samples varied too much from each other, etc. It would be better to directly perform activation patching, at least at the component level, to determine if the results from the other two baselines are valid or due to the dataset.
  2. Cross task overlap results: the data points for boolean are consistently near-zero, which indicates that ablating the circuit does not affect the current task's performance. But, addition is consistently causing a near perfect accuracy drop, so it means that the discovered circuit for boolean is not necessary?
Questions for authors
  1. What is the reason behind using zero ablation? What if the experiments used a different ablation method (counterfactual or mean)? Zero-ablation can significantly disrupt the intermediate computations in unintended ways.
  2. In section 6.1, the authors proposed potential reasons for why circuits overlap. Then, should we really require that circuits should be specific?
3 comments
AuthorsOpenReview ↗

Response to Reviewer UYwR

Thank you for your review! We appreciate that you found our work interesting. We address your concerns and questions below.

Weakness 1 (Limited baselines)

We don’t think dataset diversity fully explains the disagreement between EAP-IG and RelP. First, our datasets are fairly controlled. IOI, CopyColors MCQA, and both ARC tasks come from MIB (Mueller et al., 2025), and Addition and Boolean Logic follow fixed templates (Appendix C). Second, we compute the Spearman correlation between the two methods’ scores on the same example (Section 4), so diversity among examples doesn’t explain the low correlation. However, as you suggested, we have run activation patching at the component-level for all five models (Appendix F). We find that both EAP-IG and RelP scores correlate with activation patching scores, EAP-IG more so than RelP (Spearman correlations of 0.92-0.97 and 0.66-0.75). This suggests that much of the disagreement comes from RelP, so our results are indeed valid and not due to the dataset.

Weakness 2 (Does the boolean circuit have low necessity)

The Boolean points appear near zero because the accuracy drop from ablation is bounded by the gap between the model’s baseline accuracy and chance. Boolean logic is a binary task, so chance is 50%, and some of the models we study do not do much better than that (Table 3), leaving little room to drop. On Addition, however, chance is essentially 0%, so ablation can remove nearly all of the accuracy.

This also explains why Boolean has low necessity in Figure 2. Since models are not far above chance, it is easy for even the random control set to bring accuracy down to chance, so ablating the circuit and ablating the control cause similar drops. This is likely why Qwen3 4B, which has the highest Boolean accuracy (88%), shows higher necessity than Gemma 2 2B Instruct, which has the lowest (57%). Therefore, necessity magnitudes are somewhat confounded by baseline and chance accuracy, so we’d caution against directly comparing necessity between tasks.

Question 1 (Ablation method)

We discuss this concern in footnote 4 of the main text (page 4) and in Appendix E. To address it, we re-ran our main experiments with mean ablation, replacing each component’s activations with its average value over the counterfactual/corrupted inputs of the task. Our conclusions about necessity and specificity hold under both ablation methods. In particular, accuracy drops under zero and mean ablation are highly correlated, with r = 0.97 (Pearson correlation) for component-level circuits and 0.80-0.86 for neuron-level circuits (Figure 6, Table 4). We do notice that at the neuron-level, zero ablation results in larger accuracy drops than mean ablation. In the revised paper, we have moved this result from the footnote into the main text to make it more clear that we ran this robustness check.

Question 2 (Should we require specificity?)

This is a fair point, and we agree that specificity should not be a hard requirement. Indeed, we observe that language model circuits reuse components across tasks, and as we discuss in Section 6.1, this reuse could plausibly be a driver of generalization. Therefore, we think it is more natural to view specificity as a property to be measured, not a hard requirement. As such, we have reworded the introduction, which previously said that circuits "should be" specific, to reflect this framing.

We hope this addresses your concerns. If anything is still unclear or you have further questions, please let us know.

References

Mueller et al. (2025) https://arxiv.org/abs/2504.13151

Thank you for the detailed response! My concerns have been addressed.

Thank you for engaging with our response, and we're glad the additional experiments addressed your concerns. If you feel the revisions have strengthened the paper, we'd be grateful if you would consider updating your score. Either way, we appreciate your time and thoughtful feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Reviewers

We’d like to thank all reviewers for the time they have spent on our paper. Based on the reviews, we have uploaded a revised version of the paper, with changes highlighted in magenta. For convenience, we outline the relevant changes here, in a general comment to all reviewers.

  • We have added a new section, Appendix F, where we run activation patching at the component-level for all five models and compare the results to EAP-IG and RelP.

  • We have expanded our case study of the attention heads shared across task circuits from Llama-3.2-3B to all five models (Section 5.1 and Appendix J).

  • We have reworded the introduction (Section 1) to present consistency and specificity as properties to be measured, not hard requirements. We also note that low consistency may mean that the model uses different mechanisms for different prompt-specific versions of a task.

  • We have updated the related work (Section 7) and our discussion of why circuits overlap (Section 6.1, "Reuse as a feature, not a bug") to discuss how a task should be defined for a language model. The tasks that we define may not correspond to how the model organizes its computation. For example, if a model solves tasks by combining more basic skills, we expect two tasks that draw on the same skill to have overlapping circuits.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.