acceptodds
Under review as a conference paper at ICLR 2027

Data-driven Circuit Discovery for Interpretability of Language Models

Abstract

Circuit study aims to explain how language models (LMs) implement a task by localizing and interpreting a circuit: a computational subgraph responsible for a specific model's behavior. Existing methods for circuit localization are largely hypothesis-driven: they define a task through a dataset, assume that all examples in the task are implemented by the same mechanism, and apply circuit discovery to obtain a single circuit. This workflow does not account for mechanistic diversity within a task, where different examples may rely on different mechanisms. We first show that when a dataset mixes tasks processed through distinct mechanisms, current circuit discovery can combine these mechanisms into a single circuit while retaining faithfulness comparable to circuits discovered on each task separately. Across four tasks and three models, we further provide evidence for such mechanistic diversity within individual tasks. To account for this diversity, we propose Data-driven Circuit Discovery (DCD), which modifies the circuit-discovery workflow by adding a grouping step before the discovery. The grouping step aims to partition the dataset into subsets of examples that rely on similar mechanisms. We instantiate this step with clustering over per-example attribution patterns (DCD-Cluster). Across four tasks and three models, DCD-Cluster decomposes the dataset into multiple subsets whose circuits exhibit distinct faithfulness patterns, providing useful hypotheses about mechanistic differences among examples.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, circuit study, interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7oRYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper shows that a faithful circuit can combine different computations across examples. It proposes Data-driven Circuit Discovery (DCD), which groups examples before circuit discovery, with DCD-Cluster using per-example edge attributions. Experiments on four tasks across three models examine circuit transfer and compare clustering with random grouping. Cross-task separation works well, while within-task results are less consistent.

Critical strengths
  1. Conceptual Insight: The paper highlights a critical blind spot in current circuit discovery—showing convincingly that high faithfulness does not guarantee a single, coherent underlying mechanism. This serves as a valuable sanity check for the community.

  2. Dataset Sensitivity: The within-task analysis offers a helpful empirical demonstration of how sensitive recovered circuits are to variations in dataset composition.

Critical weaknesses
  1. Limited methodological novelty. The paper offers a useful empirical observation, but the proposed method mainly adds clustering of per-example attributions to an existing circuit-discovery pipeline. I find the methodological advance modest. Comparison with random grouping does not establish whether attribution-based grouping provides more insight than simpler grouping based on input or hidden-state representations.

  2. Dependence on a known variant count. The within-task experiments set the number of clusters to the known number of input variants because automatic selection tends to over-split. This leaves it unclear how reliably DCD can identify useful groups when the dataset’s structure is not known in advance.

Questions for authors
  1. Requested comparison: How does attribution-based clustering compare with clustering input or hidden-state representations, using the same number of groups and the same circuit-discovery settings? This would help establish what the attribution information adds.

  2. Clarification: How do the within-task conclusions change when the number of clusters is selected automatically? Does the reported over-splitting mainly create redundant groups, or does it change which circuit distinctions can be recovered?

  3. Clarification: For one within-task example, could you explain what different computations the recovered circuits suggest, and which intervention results support that interpretation? This would help me understand what is learned beyond recovering the known input variants.

  4. Contribution and novelty: Could you explain more concretely how the contribution differs from the closest prior work? I currently see the main contribution as the empirical evidence for within-task heterogeneity. What does the proposed grouping procedure add beyond this observation and the use of existing attribution and clustering methods?

5 comments
AuthorsOpenReview ↗

New Experiments: Causal Validation of Distinct Mechanisms Within the Same Task; Automatic Cluster Selection and Its Impact on Circuit Recovery; Comparison of Attribution-Based, Hidden-State, and Sentence-Embedding Grouping

We thank the reviewer for recognizing the importance of DCD's central observation and its within-task analysis. We address the concerns below; new results and clarifications will be included in the revision.

W1 – Novelty: We believe DCD's contribution goes beyond adding clustering to an existing circuit-discovery method; DCD-Cluster is only one instantiation of the broader framework. Our central contribution is identifying and addressing a limitation in the standard workflow, which typically discovers a single circuit for a task dataset without accounting for mechanistic diversity within it. This limitation cannot be resolved simply by improving edge-attribution methods such as EAP and EAP-IG, since it concerns how examples are grouped before circuit discovery.

We agree that random grouping alone is insufficient to establish the value of attribution-based grouping. In Appendix D.3, we compare against a representative-circuit baseline adapted from Franco et al. (2026), showing that constructing circuits from aggregated group evidence achieves higher faithfulness. We also compare hidden-state and sentence-embedding clustering under matched settings (Q1).

W2/Q2 – Automatic K: We evaluated gap statistic, elbow, and silhouette across seven within-task datasets, which selected K = 9–12 compared with 2–8 known variants. On GPT-2 IOI names-vs-letters, automatic K selects 12 instead of the fixed K = 2. We observe that (1) known variants remain separated, forming seven names-only and five letters-only clusters, also observed on Llama and Qwen; (2) additional groups may be redundant, as swapping sibling circuits produces little faithfulness loss (median 0.015); and (3) some groups become very small, with four containing fewer than 10 examples. Thus, automatic K can preserve variant distinctions without knowing their count, but over-splitting may create redundant groups and hinder recovery of the original circuit distinctions.

Q1 – Grouping baselines: DCD is a general framework for grouping examples before circuit discovery. Attribution-based DCD-Cluster, hidden-state, and sentence-embedding are all valid instantiations. To evaluate these instantiations, we compared them on Llama-3.1-8B, keeping K, EAP-IG circuit discovery, circuit size (5% of edges), and evaluation fixed. Across seven within-task datasets, best-of-K faithfulness is comparable: 1.074 (attribution), 1.061 (hidden states), and 1.064 (sentence embeddings). These results demonstrate that DCD can achieve comparable circuit-set coverage using different grouping strategies. However, comparable faithfulness does not necessarily imply that the resulting groups capture the same mechanisms; establishing such distinctions requires additional transfer and causal analysis.

Q3 – What the circuits compute: To answer this, we investigated the two circuits discovered by DCD-Cluster from a mixed dataset of IOI names and letters on GPT-2. Building on the IOI circuit characterized by Wang et al. (2023), we find that both the names and letters circuits share duplicate-token, induction, and S-inhibition heads, but differ in their downstream answer-retrieval mechanisms:

  • Names: S-inhibition heads steer name movers (9.9, 9.6, 10.0), which copy the indirect object's name.
  • Letters: Two name movers (9.6, 10.0) fail to copy letters. Instead, through path patching, we identify a new head, 6.1, which receives inputs from induction heads and influences backup name mover 10.10, whose OV circuit copies 88% of letters.

We counterfactually patched components in the full model on 200 held-out prompts per variant. Values are fractions of recoverable logit difference lost:

InterventionNamesLetters
Induction heads0.550.53
S-inhibition heads0.890.82
Name movers0.240.05
Head 6.10.020.30
Names-exclusive edges0.340.05
Letters-exclusive edges0.000.40

Variant-exclusive edge interventions show a double dissociation: disrupting edges specific to the names circuit primarily affects names, while disrupting edges specific to the letters circuit primarily affects letters. Together, these findings demonstrate how DCD can generate useful hypotheses about mechanistic differences within the same task, which can then be investigated and validated through causal interventions.

Q4 – Contribution: Beyond demonstrating within-task heterogeneity, DCD introduces a change to the standard circuit-discovery workflow to account for it: grouping examples before constructing circuits. This limitation cannot be addressed simply by improving attribution methods such as EAP or EAP-IG, since it concerns how examples are grouped for circuit discovery. Importantly, attribution-based clustering (DCD-Cluster) is only one instantiation of this broader framework, rather than the primary contribution itself.

Thank you again for the constructive review. As the discussion period draws to a close, we wanted to follow up to see whether our rebuttal adequately addresses the points you raised. We would gladly engage further on any aspect that remains unclear. Thanks again!

Thank you again for the constructive review. As the discussion period ends tomorrow, we wanted to follow up to see whether our rebuttal adequately addresses the points you raised. We would gladly engage further on any aspect that remains unclear. Thanks again!

Reviewer_26GrOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether averaging attribution scores across a task can hide differences in how a model handles individual examples. A circuit may be highly faithful while combining several mechanisms. The authors propose DCD: group examples first, then discover a circuit for each group. DCD-Cluster uses per-example edge attributions for this grouping, treating the clusters as hypotheses about mechanistic differences.

Critical strengths
  1. The paper makes a useful point: high faithfulness alone does not mean a circuit captures one coherent mechanism.

  2. The experiments cover several models and a good range of tasks.

Critical weaknesses
  1. It is still unclear how coherent each cluster is internally. Would splitting a cluster further reveal meaningful differences? Some within-cluster tests would help.

  2. Within-task experiments use the known number of variants as K. The authors acknowledge this, but it leaves automatic discovery with an unknown number of groups untested.

  3. Most results concern transfer, edge overlap, and faithfulness. I would like to see a closer look at what the circuits actually compute and how that differs across clusters.

  4. Both grouping and circuit discovery rely on attribution scores. Faithfulness does test intervened model behavior, but more targeted interventions would help check whether the clusters reflect different mechanisms rather than quirks of the attribution method.

Questions for authors
  1. How stable are the clusters and circuits across random seeds and resampled discovery sets?

  2. The clustering setup is confusing. Appendix D.2.2 says k-means is run after PCA/SVD, but also says all clustering uses Jaccard distance on the original binary vectors. Which representation and distance are used at each step?

  3. The appendix explains why arithmetic is tested only on Llama. Could the authors score full multi-token answers to check the findings on another model?

  4. Could the authors provide an inital test on longer contexts, more complex reasoning, or multi-token outputs? It is hard to judge how far the results extend beyond these synthetic next-token tasks.

5 comments
AuthorsOpenReview ↗

New Experiments: Causal Validation of Distinct Mechanisms Within the Same Task Variants; Within-Cluster Analysis Revealing Finer Mechanistic Differences; Cluster and Circuit Stability Across Seeds

We thank the reviewer for recognizing our central finding and the breadth of our evaluation. We address each concern below; new analyses and corrections will appear in the revision.

W1 – Within-cluster coherence: To investigate whether the discovered clusters contain further mechanistic differences, we progressively subdivided the two clusters identified by DCD-Cluster on Llama-3.1-8B IOI names/letters, examining K = 2 through K = 6 using Ward hierarchical clustering. We characterized the resulting groups by their prompt differences using GPT-6 Astra and evaluated their circuit transfer gaps and edge Jaccard against 10 matched random splits.

KDistinguishing propertySizesGapJaccard
2Named people vs. “Person + letter”99/990.1970.377
3Names: “had a long argument … said to” vs. other templates10/890.0130.440
4Letters: “had a long argument … said to” vs. other templates13/860.0690.480
5Letters: repeated-person order (ABBA vs. BABA)38/480.0200.477
6Names: “When A and B got a [object] at the [place]” vs. other templates 16/730.0150.474

Only K = 2 and K = 4 exceed both the 0.05 transfer-gap threshold and the 95th percentile of matched random splits. Thus, some finer splits reveal distinct circuit dependencies, but not every subdivision is mechanistically meaningful.

W2 – Auto K: We evaluated gap statistic across seven within-task datasets, which selected K = 9–12 compared with 2–8 known variants. On GPT-2 IOI names-vs-letters, auto K selects 12 instead of the fixed K = 2. We observe that (1) known variants remain separated, forming 7 names-only and 5 letters-only clusters, also observed on Llama and Qwen; (2) additional groups may be redundant, as swapping sibling circuits produces little faithfulness loss (median 0.015); and (3) some groups become very small, with 4 containing fewer than 10 examples. Thus, auto K can preserve variant distinctions without knowing their count, but over-splitting may create redundant groups and hinder recovery of the original circuit distinctions.

W3 – What the circuits compute: We investigated the two circuits discovered by DCD-Cluster from a mixed dataset of IOI names and letters on GPT-2. Building on the IOI circuit characterized by Wang et al. (2023), we find that both circuits share duplicate-token, induction, and S-inhibition heads, but differ in their downstream answer-retrieval mechanisms:

  • Names: S-inhibition heads steer name movers (9.9, 9.6, 10.0), which copy the indirect object's name.
  • Letters: Two name movers (9.6, 10.0) fail to copy letters (0% vs. 94–96% for names). Through path patching, we identify a new head, 6.1, linking induction heads to backup name mover 10.10, whose OV circuit copies 88% of letters.

We tested these differences through counterfactual patching on 200 held-out prompts per variant. Values indicate the fraction of recoverable logit difference lost:

InterventionNamesLetters
Induction heads0.550.53
S-inhibition heads0.890.82
Name movers0.240.05
Head 6.10.020.30
Names-exclusive edges0.340.05
Letters-exclusive edges0.000.40

The variant-exclusive edge interventions show a double dissociation: disrupting each circuit's exclusive edges primarily affects its corresponding variant, further showing the unique mechanisms between 2 circuits. Together with path patching, these findings demonstrate how DCD can generate hypotheses about distinct mechanisms within the same task, which can then be causally validated.

W4 – Causal validation: Beyond attribution similarity and faithfulness, we provide new experiments with targeted causal interventions in W3 as evidence that DCD can generate hypotheses about distinct mechanisms within the same task, which can then be causally validated.

Q1 – Stability: We evaluated cluster and circuit stability across 10 random seeds and 10 resampled discovery sets. Cluster assignments are generally stable, with adjusted Rand index (ARI, measuring agreement between cluster assignments) of 0.88–0.99 across cross-task settings and 0.81–1.00 for six of seven within-task settings. Discovered circuits are also largely consistent across reruns, with substantially higher edge overlap for matched circuits (Jaccard 0.88–0.98) than for circuits from different clusters (0.43–0.50).

Q2 – Clustering pipeline: We will correct Appendix D.2.2: binarize attributions (γ = 0.99 cross-task; 0.9 within-task), apply PCA to 20 dimensions, then Euclidean k-means. Jaccard on original binary vectors is used only for K-REPRESENTATIVE medoid selection.

Q3/Q4 – Scope: We agree that multi-token outputs, longer contexts, and complex reasoning deserve further study, but reliable circuit discovery and evaluation remain challenging in these settings. We restrict arithmetic experiments to Llama, following prior works like Nikankin et al., 2025b. We will clarify these limitations in the revision.

Thank you again for the constructive review. As the discussion period draws to a close, we wanted to follow up to see whether our rebuttal adequately addresses the points you raised. We would gladly engage further on any aspect that remains unclear. Thanks again!

Thank you again for the constructive review. As the discussion period ends tomorrow, we wanted to follow up to see whether our rebuttal adequately addresses the points you raised. We would gladly engage further on any aspect that remains unclear. Thanks again!

Reviewer_cBDeOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

One human-defined task can actually consist of multiple sub-tasks, where each subtask can be implemented by a separate circuit. Classical automated circuit discovery methods output by definition one (merged) circuit given any human-defined task, no matter how many different sub-tasks or mechanisms are involved. The paper demonstrated this phenomenon empirically. Furthermore, they suggest automatically group prompts(-pairs) of a task into mechanistically distinguishable sub-tasks. They suggest a workflow: first dividing a task into sub-tasks, then discovering a separate circuit for each of these sub-tasks. They call this workflow Data-driven Circuit Discovery (DCD). They propose an example algorithm for grouping by clustering the edge-attribution scores. Empirical results of this algorithm look promising. There is probably a lot of room for improvement to come up with better grouping algorithms.

Critical strengths

The high-level idea makes a lot of sense: There is no reason to believe that human-defined tasks are always implemented by the same circuit across all examples of the task. It seems very plausible that many human-defined tasks consist of multiple sub-tasks that are implemented via different circuits, but current automated circuit discovery methods would not automatically detect this. The paper clearly demonstrates this in the case of deliberately mixed tasks and provides soft evidence that this also occurs for tasks, where it was a-priori not clear if they consist of mechanistically different sub-tasks.

Critical weaknesses
  1. Clarify novelty relative to prior work. Franco et al. (2026) cluster per-prompt circuits into families, while Marks et al. (2025) cluster contexts and discover one circuit per cluster. Discuss what DCD adds beyond these approaches, especially since grouping is presented as its key contribution.

  2. Strengthen the baseline comparison. K-REPRESENTATIVE uses DCD’s clusters; the paper does not evaluate groupings derived from Franco et al.’s ACC++ circuits.

  3. Qualify robustness claims. The main text should acknowledge the two EAP exceptions, including the reversed Arithmetic–EB result, and clarify that GPT-2 and E-Act checks each cover only one task pair.

  4. Moderate claims about transfer and patterns. “Neither circuit explains” the other variant is too strong when transferred faithfulness remains ≥0.91 in three of seven pairs. Section 5.2 should also acknowledge the two pairs without a clear pattern.

Questions for authors
  1. Evaluation: Report faithfulness clipped at 1 per prompt-pair, so large gains on some prompts cannot compensate for failures on others.

  2. Cluster analysis: Explore shared properties within clusters, potentially using a frontier LLM. Explain over-splitting and report performance and circuit Jaccard similarities for automatically selected KK.

  3. Figure 5: Explain the inconsistent sorting, identify whether EAP or EAP-IG is used, and reconcile the transfer interpretation with Table 27. Group-level averages may communicate the results more clearly.

  4. Clarity and corrections: State consistently whether all experiments use prompt-pairs, define K-RANDOM as random grouping, and use “validation set” consistently. Correct the Figure 3b numbers in lines 267–269 and fix minor legend, cross-reference, and formatting issues.

6 comments
AuthorsOpenReview ↗

New Experiments: Improved Baseline Comparison Using ACC Grouping Signal, Causal Evidence for Distinct Answer-Retrieval Mechanisms Within the Same Task Variants, and Within-Cluster Analysis of Finer Mechanistic Differences

We thank the reviewer for recognizing the motivation behind our work and the significance of mechanistic diversity in circuit discovery. Below, we address each concern, including additional experiments and clarifications that we will incorporate into the revised paper.

W1 - Novelty: Our motivation for grouping differs from Marks et al. (2025): while they cluster contexts to discover previously unidentified behaviors, DCD treats groups as hypotheses about mechanistic diversity. Franco et al. (2026) cluster per-example circuits and select representatives; DCD instead groups examples before circuit discovery and constructs new circuits from aggregated group attributions. This difference matters empirically: group-level construction achieves higher faithfulness than representative-example circuits (Appendix D.3).

W2 - Baseline comparison: Following the suggestion, we compared ACC (Franco & Crovella, 2025) and EAP-IG as grouping signals. Since Franco et al. (2026) is still a preprint and its ACC++ implementation is not publicly available, we used ACC as an alternative. After grouping, we compared representative-example circuits against DCD's group-level circuits. We evaluated GPT-2 Small on cross-task mixtures and within-task IOI names-vs-letters examples.

SettingGroupingRepresentative (Franco)Group-level (DCD)
cross-taskEAP-IG0.360.67
cross-taskACC0.420.69
IOI names/lettersEAP-IG0.540.75
IOI names/lettersACC0.680.75

Across both settings and grouping methods, constructing circuits from aggregated group attributions consistently outperforms selecting a representative-example circuit, as done by Franco et al. (2026). This suggests that how circuits are constructed from clusters matters more than how the grouping signals.

W3 - robustness: We will include these exceptions to claims in Sec. 4.1. Mixing occurs in 10/10 tested pairs with EAP-IG and 7/9 with EAP; GPT-2 and E-Act each cover only IOI–SC. We will state that faithful circuits can mix mechanisms, not that mixing is universal.

W4 - Transfer: “Fails to transfer” means a shortfall relative to the target variant’s own circuit exceeding bootstrap noise, not low absolute faithfulness (Sec. 4.2, lines 249–250). We will report gaps of 5.9–53.8 percentage points and clarify that Sec. 5.2 shows four clear patterns, one weaker, and two without clear patterns.

Q1 - Evaluation: Following the suggestion, we conducted additional experiments with clipped faithfulness. Per-prompt normalized clipping is unstable when the full-model and empty-circuit logit differences nearly coincide. Instead, we cap each prompt's circuit logit difference at the full-model value before computing dataset-level faithfulness. On Llama-3.1-8B cross-task mixtures, mixed-circuit faithfulness remains 0.57–0.91. It matches or exceeds single-task circuits for arithmetic, sequence completion, and entity binding; IOI is lower (0.81–0.89 vs. 0.95), as it was without clipping. This alternative clipping procedure therefore preserves our qualitative conclusion.

Q2 - Cluster analysis: To investigate whether the discovered clusters contain further mechanistic differences, we progressively subdivided the two clusters identified by DCD-Cluster on Llama-3.1-8B IOI names/letters, examining K = 2 through K = 6 using Ward hierarchical clustering. We characterized the resulting groups by their prompt differences using GPT-6 Astra and evaluated their circuit transfer gaps and edge Jaccard against 10 matched random splits.

KDistinguishing propertySizesGapJaccard
2Named people vs. “Person + letter”99/990.1970.377
3Names: “had a long argument … said to” vs. other templates10/890.0130.440
4Letters: “had a long argument … said to” vs. other templates13/860.0690.480
5Letters: repeated-person order (ABBA vs. BABA)38/480.0200.477
6Names: “When A and B got a [object] at the [place]” vs. other templates 16/730.0150.474

Only K = 2 and K = 4 exceed both the 0.05 transfer-gap threshold and the 95th percentile of matched random splits. Thus, some finer splits reveal distinct circuit dependencies, but not every subdivision is mechanistically meaningful.

We additionally evaluated auto K using gap statistic across 7 within-task datasets, which selected K = 9–12 compared with 2–8 known variants. On GPT-2 IOI names-vs-letters, auto K selects 12 instead of the fixed K = 2. We observe that (1) known variants remain separated, forming 7 names-only and 5 letters-only clusters, also observed on Llama and Qwen; (2) additional groups may be redundant, as swapping sibling circuits produces little faithfulness loss (median 0.015); and (3) some groups become very small, with 4 containing fewer than 10 examples. Thus, auto K can preserve variant distinctions without knowing their count, but over-splitting may create redundant groups and hinder recovery of the original circuit distinctions.

*Q3 – Figure 5: All panels use EAP-IG, and both halves share the same prompt ordering, sorted by each variant circuit's faithfulness. We will reconcile interpretation with Table 27, and add group-average faithfulness.

Q4 - Clarity and corrections: All experiments use clean/corrupted prompt pairs (App. A.4). K-RANDOM matches DCD-Cluster's K and cluster sizes. We will standardize “validation set,” fix the legend and cross-references, and correct lines 267–269: on P8, P1's circuit scores 0.27 versus P8's own 0.93 (gap 0.66); P1's circuit scores 0.80 on P1.

Thank you again for the constructive review. As the discussion period draws to a close, we wanted to follow up to see whether our rebuttal adequately addresses the points you raised. We would gladly engage further on any aspect that remains unclear. Thanks again!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.