acceptodds
Under review as a conference paper at ICLR 2027

Slaying the Hydra: Interaction-Aware Circuit Discovery in Language Models

Abstract

Localizing behavior to individual components of a language model is a central goal of mechanistic interpretability. However, scoring components one at a time misses context-dependent effects: a _primary_ component can inhibit the activation of a _backup_, leading to issues with ranking components. _Actual causality_ studies the structure of such interactions via _witnesses_: variables that provide contextual information to resolve interaction terms. However, estimation with witnesses typically requires combinatorial enumeration and is infeasible in practice. We introduce the _witness-integrated set effect (WISE)_, a family of causal estimands that build on the witness mechanism while taking expectations over _sets_ of causes and witnesses to remain computationally feasible. Building on this approach, we introduce _JuntaLearner_, a gradient-based circuit discovery method that learns to rank components by their causal impact across varying-sized sets of components and witnesses. Alongside faithfulness metrics, we introduce measures of necessity and task specificity, and the _circuit recognition score (CRS)_ to summarize each metric across circuit sizes while emphasizing effects achieved by small circuits. Across tasks and models of increasing size, _JuntaLearner_ achieves higher mean CRS compared to attribution baselines on all metrics. Since its cost does not grow with the number of candidate components, _JuntaLearner_ scales to large models while accounting for set-level interactions and avoiding first-order approximations.

Primary area:
interpretability and explainable AI
Keywords:
Localization, Circuit Discovery, Mechanistic Intepretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ayjzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses an important problem: how to accurately assess the causal impact of a model component on a given task, given the presence of interactions between components that occur when performing the task. Naive approaches to this problem are prohibitively expensive.

The paper thus works in the same setting as classical Shapley-style analysis (see below). Where the method goes beyond Shapley-style analysis in its introduction of the role of witnesses and also in defining the learning problem that allows it to do subset selection rather than individual-component attribution.

The paper introduces the use of witnesses (components with frozen outputs in the causal reach of the mediators) as a way to separate interacting sets of components. It then defines a learning problem to identify sets of components that appear to have strong effect together (either in necessity or sufficiency or both). The learning problem addresses subset selection for a problem evaluating model effect size on counterfactual inputs.

Critical strengths

The problem that the paper attacks is an important one, one that has bedeviled the circuit-discovering community for some time. The paper adds good conceptual ideas to the problem (witnesses, mainly, and the idea of witness-integrated effects) and it shows that one can define a learning problem that can be addressed via gradient descent to attack the subset-selection problem that is implied by the witness-integrated effects concept.

Critical weaknesses

I think the general problem the authors attack is important, and the paper represents a solid effort to do so. However it falls short for me for three reasons.

  1. The evaluation is problematic. The paper focuses its conclusions on the "tuned" values of the evaluation metrics using the CRS version of each score. The paper's conclusion that the method outperforms attribution baselines on average is mainly based on the "tuned" results. There are a number of problems:
  • First, the tuned configurations were chosen using the same scores the paper reports. Tuning is performed over five categories of configuration settings using a process that optimizes for the evaluation metric. When not tuning configuration for each given tesk/model setting, ie, using the "default" configuration, Table 1 shows that the results are essentially the same as Per-head NIE. Thus the value of the subset-selection aspect of the work seems be called in question.

  • Second, the settings used for "tuned" in each row of Figure 4 are different, ie, the setting used to choose Net, Faithfulness, and Kill are all differerent for the same model and task. The paper should pick one configuration per cell for all three goals.

  • Third, introducing the CRS version of each score is favorable to the paper's methods. CRS favors small circuits, which the paper's methods also favor.

  • Fourth, some baselines are tunable (eg EAP-IG) but they were not tuned as far as I can tell, unlike the proposed method.

  1. The main conceptual contribution of the paper is witness-pinning. However, pinning only helps on one out of nine model-metric combinations, and the default method does not seem to use witness pinning.

  2. The portion of the paper that estimates equation (5) is addressing the problem of how to assign credit to a set of contributors when there are interactions between contributors. This general problem has been studied using Shapley value as its tool.
    I believe that the sampling estimator in Appendix A produces the Shapley ranking. In fact it appears that the method used by Algorithm 2 is the stratified Shapley representation used by SVARM (Kolpaczki et al., AAAI 2024).

There is literature following this direction already. It should be cited and compared against:

  • Neuron Shapley (Ghorbani & Zou, 2020): seems to me to have almost the same goal, finding a few high-impact components cheaply by sampling.
  • Shapley Head Pruning (Held & Yang, EACL 2023)
  • Asymmetric Shapley values (Frye et al., 2020) - this addresses to some extent the same problem that addition of witnesses in this paper does
  • Interaction indices: Grabisch & Roubens 1999; Shapley-Taylor (Sundararajan et al., 2020); Faith-Shap (Tsai et al., 2023). The paper needs to compare its methods against a sampled Shapley ranking.
Questions for authors

See above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer ayjz (1/2)

We thank the reviewer for their time and feedback on our manuscript. We address the reviewer's concerns below and in the revised PDF. The PDF on this page is now the revised version; the submitted version remains available under Revisions. In the revised PDF, we annotated all changes with specific numbers, which we refer to as (Point #); points raised only by this reviewer are marked with "A" in magenta, and concerns shared with other reviewers are marked in blue.

Tuning is performed over five categories of configuration settings using a process that optimizes for the evaluation metric.

This is correct, thank you for pointing it out! Selection used the same held-out pairs we report scores on. We have therefore rescored the selected configurations and the baselines on fresh prompt pairs that no run had drawn. Mean scores drop by at most 0.03, and per-head NIE, which involves no selection, drops as much as our configurations, so selecting on the held-out pairs did not inflate the reported scores. The net margin over per-head NIE holds (0.08 on fresh pairs, against 0.07 on the held-out pairs). Cf. Appendix C. (Point #1).

When not tuning configuration for each given tesk/model setting, ie, using the "default" configuration, Table 1 shows that the results are essentially the same as Per-head NIE. [...] The main conceptual contribution of the paper is witness-pinning. However, pinning only helps on one out of nine model-metric combinations, and the default method does not seem to use witness pinning.

It's true that in aggregate over models and datasets, we tie per-head NIE; however, sets and witnesses help in various places (Point #2 and Point #3, respectively):

  • Training on sets raises kill over single heads (by 0.06 to 0.11 on Qwen3 and Llama; new App. H), while the large net gain on Qwen3 comes from pinning witnesses: our default, which trains on sets without pinning witnesses, has 0.35 there, against 0.34 for per-head NIE. Sets alone also do not explain the comparison with MAttr, which also learns over sets: our default pins no witnesses and beats it on kill and on net in 14 of 15 cells each.
  • While JuntaLearner often ties per-head NIE, per-head NIE is more likely to fail when more set-level interactions are present.
    • For example, on Qwen3, the net-tuned JuntaLearner rankings beat per-head NIE by 0.19 to 0.23 net on average in three tasks.
    • On Llama, our default trains only on sets (its smallest set has seven heads, so no head is ever patched alone), yet it beats per-head NIE on kill and net in three of the five datasets (by up to 0.05) and ties it within 0.003 in the other two.
    • Finally, per-head NIE pins no witnesses; pinning them raises mean net over the five Qwen3 tasks from 0.34 to 0.48. See App. D (held per-head NIE) and the new App. E.

We've introduced more experiments and discussion on witnesses in Appendix D (cf. Figure 19).

(Continued in the next comment.)

AuthorsOpenReview ↗

Response to Reviewer ayjz (2/2)

(Continued from the previous comment, part 1/2.)

Third, introducing the CRS version of each score is favorable to the paper's methods. CRS favors small circuits, which the paper's methods also favor.

Yes, this is true (Point #8). This is by design: a circuit is useful when a few components carry the behavior, and removing more components risks damaging other behaviors. Under CPR and 1−CMD, which weight all circuit sizes equally, our net lead is larger (0.69 against 0.56 for per-head NIE; under CRS, 0.54 against 0.48). New: Table 7 (App. F).

Fourth, some baselines are tunable (eg EAP-IG), but they were not tuned as far as I can tell, unlike the proposed method.

Thanks for pointing this out (Point #5). In our revised experiments, we run a grid search over the EAP-IG hyperparameter, and additionally consider signed and unsigned variants. The best score out of these becomes the "task-optimized EAP-IG". We run MAttr (which we note is concurrent work to ours) at its default values only due to compute and time constraints; however, our main comparison to MAttr (in Figure 1) uses JuntaLearner's defaults, so we believe this comparison is fair. We also further discuss this in Appendix C.

This general problem has been studied using Shapley value as its tool.

Thank you for the pointer to this literature! We now include sampled Shapley as a baseline (Point #4). In particular, with all set sizes weighted equally, our Algorithm 2 gives the Shapley ranking and is closely related to Stratified SVARM (App. A). Notably, JuntaLearner is the method primarily used in our experiments, and does not estimate (5) directly.

Sampled Shapley is now a baseline on the ten GPT-2 and Qwen3 cells. On GPT-2, the six rankings of Table 8 are within 0.04 of each other on every mean. On Qwen3, Shapley has the highest mean kill and faithfulness (0.78 and 0.57, against 0.77 and 0.54 for Task-optimized) but a much lower net (0.19, against 0.49 for Task-optimized and 0.35 for our default), because its circuits also remove the control tasks. Sampled Shapley is >100x more expensive than per-head NIE, and would become infeasible on Llama. Cf. Appendix G and Table 8.

We hope the answers and the revised paper resolve your concerns.

Reviewer_ayjzOpenReview ↗

Response

I've read and considered the authors' responses. However the results suggest that key parts of the method are equivalent to Shapley, and so the responses don't demonstrate in my view that there is a significant enough advance over prior work to merit raising my score.

AuthorsOpenReview ↗

Factual correction: JuntaLearner is not equivalent to Shapley

We thank the reviewer for reading our responses. We respectfully disagree that key parts of the method are equivalent to Shapley, and we apologize if our presentation gave that impression.

  1. The equivalence holds only for the Monte Carlo estimator, in one special case, not for what JuntaLearner learns.

  2. The two methods differ in results and in cost. If they were equivalent, they would score alike. The figure shows both on Qwen3-1.7B (App. G, Fig. 32 shows both models; see this more clearly in this figure:

https://imgur.com/a/KH72hV7

) Qwen3-1.7B. Left: net CRS per task, sampled Shapley against Task-optimized JuntaLearner. Right: minutes per task on one GPU

  • Results. On GPT-2 small all rankings are close. On the larger Qwen3-1.7B, sampled Shapley has lower net than Default JuntaLearner in every task (by 0.07 to 0.27; mean 0.19 against 0.35, and 0.49 for Task-optimized). It ranks first heads that every computation needs, many in layer 0, so its circuits also remove the control tasks.
  • Cost. At our budgets, sampled Shapley takes 124 to 252 times as long as per-head NIE on both models, and 25 to 190 times as long as Default JuntaLearner.
  1. The revised paper states the differences in App. A (end of the SVARM paragraph) and in App. G (paragraph "Differences from JuntaLearner"). App. A ends with: "JuntaLearner trains on a ladder of a few sizes and, in its held regimes, with witnesses, so its target is not the Shapley value." We summarize them here.

    a. The estimator, not the learning objective. Without witnesses, and with all set sizes weighted equally, the expected score of the sampler in Algorithm 2 is an increasing affine function of the Shapley value (App. A). JuntaLearner instead learns a distribution over sets by gradient descent on the expected set score JkJ_k, and that objective is not a Shapley value.

    b. Sets are drawn from learned logits, not uniformly. Shapley averages a head's marginal contribution over all coalitions. The link to WISE holds at initialization, when the logits are uniform (Prop. 1), and even there it covers only the first set size, one term of the Shapley average.

    c. Only a few set sizes. Training runs on a short ladder (e.g. 72-32-16-4-1). With only some sizes weighted, the score is neither Shapley nor Banzhaf.

    d. A different readout. The ranking comes from recorded gradients and, on ladders ending at k=1k=1, from single-head scores (Eq. 9), not from estimated Shapley values.

    e. Witnesses change the game. In the held regimes we pin downstream heads at their clean values, so v(S)v(S) is no longer the NIE game, and no Shapley identity applies to any estimator.

Since these points are in the appendices, they are easy to miss. We will move a short version into the main text (Sec. 4, in the paragraph on sampled Shapley values) and make App. A and App. G more explicit.

Reviewer_vnViOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper is motivated by the observation that the importance estimated by intervening on a single component may be obscured by backup components and other interactions, and that individual effects cannot simply be combined into set-level causal effects. Existing component-wise patching and gradient attribution methods can miss interactions, exact patching becomes expensive as the number of candidates grows, and exhaustive enumeration of sets and contexts incurs exponential cost. The authors therefore propose WISE (Witness-Integrated Set Effects), a family of set-level causal estimands that take expectations over mediator and witness sets. Building on WISE, they introduce JUNTALEARNER, a circuit discovery method that uses Gumbel-top-k sampling, straight-through gradient estimation, and a set-size schedule to optimize WISE and ultimately produce a component ranking.

Critical strengths
  1. The authors’ motivation is reasonable, and studying the causal roles of components at the set level is a worthwhile direction.
Critical weaknesses
  1. The rationale for the set-size schedule is insufficiently established. The main text primarily shows that a descending schedule outperforms training exclusively on singleton sets. However, the configuration labeled as the net winner for Llama in Figure 14 is reverted + half held, which runs counter to the paper’s narrative of starting with large sets and progressively shrinking them. A direct comparison of ascending, descending, and fixed-size schedules under the same budget, objective, and witness settings is needed to establish the advantages and applicability of the descending schedule.
  2. Although the witness mechanism has a causal motivation, when it is effective remains insufficiently explained. The default configurations for all three models use the free regime, and the benefits of witness pinning are mainly reflected in Qwen3’s net metric. The current results therefore do not sufficiently establish how much additional benefit witness integration provides over set-level search alone, or why its benefits vary across models and metrics.
  3. Hyperparameter tuning is asymmetric in the main experiments: the authors search over many configurations for JUNTALEARNER, while MAttr uses its default settings. Appendix E acknowledges this issue and partially addresses it through transfer experiments on MIB, but a comparison under comparable tuning budgets is still missing.
  4. The main experiments search only over attention heads. Although the supplementary MIB experiments include MLPs and input embeddings, kill and net are still evaluated only over attention heads. The method’s effectiveness across mixed component types and at other granularities, such as edges, neurons, or SAE features, remains insufficiently validated.
Questions for authors
  1. When backup mechanisms involve multiple tiers or chains of redundancy, can the current set-sampling and witness mechanisms reliably identify the roles of primary components? For example, after only some backups are pinned, could unpinned components still compensate through other pathways? Synthetic tasks with two-tier or multi-tier redundancy could help evaluate identification performance and failure conditions under different witness fractions.
  2. How are the data used for hyperparameter search and Default/Tuned configuration selection determined in the main experiments? How do these data relate to the held-out prompt pairs? Are the CRS values driving the GP/CatBoost search and configuration selection computed on a validation set that is independent of the final evaluation set?
2 comments
AuthorsOpenReview ↗

Response to Reviewer vnVi (1/2)

We thank the reviewer for their time and feedback on our manuscript. We address the reviewer's concerns below and in the revised PDF. The PDF on this page is now the revised version; the submitted version remains available under Revisions. In the revised PDF, we annotated all changes with specific numbers, which we refer to as (Point #); points raised only by this reviewer are marked with "V" in green, and concerns shared with other reviewers are marked in blue.

The rationale for the set-size schedule is insufficiently established.

We conducted a more thorough experiment on set-size schedules to better explore this (Point #7). This includes the requested controlled comparisons for descending, ascending, and flat k=1 schedules, with the same budget, objective, witness share, and seed. It is contained in Appendix H.

Both multi-size schedules beat k=1 on kill, so most of the gain comes from training on sets; order matters for net on Qwen3, where descending wins 29 of 36 cases. Training at one fixed size, the median rung of the default (k=6 on GPT-2, k=33 on Qwen3) lowers net on all five Qwen3 tasks by 0.15 to 0.25 and stays close to the default on GPT-2. The Llama "reverted + half held" winner was chosen only among narrow ladders, whose widest sets hold 40 of Llama's 1,024 heads, and it leads only on IOI. On a wide ladder (sets of 512, 228, 114, then 80 heads, at the same 90 steps), descending matches or beats ascending on all five Llama tasks.

Although the witness mechanism has a causal motivation, when it is effective remains insufficiently explained. [...] The current results therefore do not sufficiently establish how much additional benefit witness integration provides over set-level search alone, or why its benefits vary across models and metrics.

The net-best pinning regime differs by model and task (Point #3). We investigated this further and discuss it in the revised paper. In short, where heads shared across tasks act only through later heads (Qwen3), pinning every witness ranks those heads low and raises mean net from 0.34 to 0.48. As zETw notes, this is the first paper to use witnesses in circuit discovery; we do not claim a general theoretical account of when pinning helps, but we think the evidence makes it worthy of further study. We discuss this more in Appendix D.

(Continued in the next comment.)

AuthorsOpenReview ↗

Response to Reviewer vnVi (2/2)

(Continued from the previous comment, part 1/2.)

Hyperparameter tuning is asymmetric in the main experiments: the authors search over many configurations for JUNTALEARNER, while MAttr uses its default settings.

Thanks for pointing this out (Point #5). In our revised experiments, we run a grid search over the EAP-IG hyperparameter, and additionally consider signed and unsigned variants. The best score out of these becomes the "task-optimized EAP-IG". We run MAttr (which we note is concurrent work to ours) at its default values only due to compute and time constraints; however, our main comparison to MAttr (in Figure 1) uses JuntaLearner's defaults, so we believe this comparison is fair. We also further discuss this in Appendix C.

The main experiments search only over attention heads.

We focused on testing with heads only, as most documented backups are attention heads (Wang et al., 2023; Rushing & Nanda, 2024), and two of our tasks, IOI and induction, were first explained through heads (Olsson et al., 2022). While the method is extensible to MLPs, we defer investigation with edges, neurons, and SAE features to future work; we now state this as a limitation and drop the scaling claim from the abstract. On MIB, we train on heads and MLPs; one untuned run beats MAttr on CMD, and on kill and net scored over its heads (patching the first-layer MLP of GPT-2 alone removes every task). (Point #9).

When backup mechanisms involve multiple tiers or chains of redundancy, can the current set-sampling and witness mechanisms reliably identify the roles of primary components?

Yes (Point #10)! We test this in a new synthetic model with up to three tiers of backups: pinning every downstream head ranks the primary first at every depth; pinning a share m of them still does so from m = 0.5 with one tier and from m = 0.75 with two or three. It fails when a compensator outside the candidate pool writes directly to the output; we now list this as a limitation. This new experiment can be found in Appendix I.

How are the data used for hyperparameter search and Default/Tuned configuration selection determined in the main experiments? How do these data relate to the held-out prompt pairs? Are the CRS values driving the GP/CatBoost search and configuration selection computed on a validation set that is independent of the final evaluation set?

They weren't; thank you for pointing this out! Rankings train on 70% of each cell's prompt pairs, and the GP/CatBoost search and both configuration selections used scores on the other 30%, which are also the scores we report. We have therefore rescored the selected configurations and the baselines on fresh prompt pairs that no run had drawn. Mean scores drop by at most 0.03, and per-head NIE, which involves no selection, drops as much as our configurations, so selecting on the held-out pairs did not inflate the reported scores. The net margin over per-head NIE holds (0.08 on fresh pairs, against 0.07 on the held-out pairs). Cf. Appendix C. (Point #1).

We hope the answers and the revised paper resolve your concerns; if they do, we would be grateful if you would reconsider your score.

Reviewer_zETwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces WISE and JuntaLearner which are the witness set effect score and the circuit discovery method maximizing WISE, respectively. The main motivation is that prior circuit discovery works usually compute per-component scores which can distort indirect effect (NIE, PIE) and mislead that circuit-level effect is compositional, while interactions between components could not be expressed as a sum of two individual effects. To address this problem, authors adopt "witnesses" from actual causality. "Witness" is a concept (and also a component) that enables to control the downstream effect of components. But the full search is combinatorial and infeasible, JuntaLearner does sampling with gumbel noise and update to optimize the witness-integrated score. Results show that JuntaLearner outperforms baselines with many different metrics including task-sensitivity

Critical strengths
  1. Clear motivation and corresponding design. The authors identify a blind spot in circuit discovery (specifically, joint effects), and the adoption of the witness approach appears appropriate.
  2. Authors provide substantial analyses so that it is easier to examine the method in different ways.
  3. Also authors provide limitations honestly.
Critical weaknesses

My concerns are mostly about the advantages of considering "witnesses". I articulate my concerns as follows:

  1. It is unknown how much JuntaLearner does faithfully reflect WISE since the convergence to maximize WISE is not guaranteed and also metrics reported do not directly reflect WISE estimation, while authors mention in Appendix B. I do not mean that the method should guarantee some kinds of convergences but it is not clear whether the method benefits from faithfully maximizing WISE or it gathers some group effect information during training considering that authors state a regime for witnesses is free as a default in Appendix C and trade-off for witness pinning reported in Section 4.2..
  2. While authors point out that the existing methods suffer from not considering the joint effect but per-head NIE baseline which is a per-component baseline and also direct computation (compared to gradient-based approximation baselines) is already comparable to JuntaLearner. The plausible hypothesis is that the main reason of gap in reported scores is gradient-based approximation is not faithful to approximate the per-component effect, rather the compositionality is not true in those tasks.

I suggest that the authors explicitly discuss how the limitations connect to WISE, and how WISE relates to JuntaLearner optimization and performance.

Besides, it is not very clear whether the method is scalable considering that experiments are mainly conducted analyzing attention heads not the full nodes including MLP layers, while stated in Discussion section.

Questions for authors
  1. To my best knowledge, this is the first paper that integrates "witnesses" in circuit discovery. However, it would be helpful if the authors provided more detail on which methods have been proposed to mitigate the Hydra effect (or the other problems mentioned) and why those methods are infeasible or less faithful. Currently, these prior works are introduced only in the last sentence of page 1, and the baselines in the experiments do not directly aim to resolve these problems.
  2. Please add a label for x-axis in Figure 2.
3 comments
AuthorsOpenReview ↗

Response to Reviewer zETw (1/2)

We thank the reviewer for their time and feedback on our manuscript. We address the reviewer's concerns below and in the revised PDF. The PDF on this page is now the revised version; the submitted version remains available under Revisions. In the revised PDF, we annotated all changes with specific numbers, which we refer to as (Point #); points raised only by this reviewer are marked with "Z" in orange, and concerns shared with other reviewers are marked in blue.

It is unknown how much JuntaLearner does faithfully reflect WISE since the convergence to maximize WISE is not guaranteed and also metrics reported do not directly reflect WISE estimation, while authors mention in Appendix B. [...] I suggest that the authors explicitly discuss how the limitations connect to WISE, and how WISE relates to JuntaLearner optimization and performance.

Indeed, we agree this is an important question, which we've added to Appendix B (Point #6). We have added two new propositions that connect the training objective to WISE. At the start of training, the objective's exact gradient is proportional to each head's WISE contrast at the first set size, and under exact gradient flow, a head that dominates another keeps a logit at least as large (Props. 1–2). Training follows this gradient only approximately, with a straight-through estimate and Adam, and the ranking comes from recorded gradients and, in the GPT-2 and Qwen3 defaults, from single-head scores on a final k=1 rung, so the propositions do not guarantee the ranking. We've also added discussion of this point to the limitations paragraph in the discussion section.

While authors point out that the existing methods suffer from not considering the joint effect but per-head NIE baseline which is a per-component baseline, and also direct computation (compared to gradient-based approximation baselines) is already comparable to JuntaLearner.

It's true that in aggregate over models and datasets, we tie per-head NIE; however, sets and witnesses help in various places (Point #2 and Point #3, respectively):

  • Training on sets raises kill over single heads (by 0.06 to 0.11 on Qwen3 and Llama; new App. H), while the large net gain on Qwen3 comes from pinning witnesses: our default, which trains on sets without pinning witnesses, has 0.35 there, against 0.34 for per-head NIE. Sets alone also do not explain the comparison with MAttr, which also learns over sets: our default pins no witnesses and beats it on kill and on net in 14 of 15 cells each.
  • While JuntaLearner often ties per-head NIE, per-head NIE is more likely to fail when more set-level interactions are present.
    • For example, on Qwen3 the net-tuned JuntaLearner rankings beat per-head NIE by 0.19 to 0.23 net on average in three tasks.
    • On Llama, our default trains only on sets (its smallest set has seven heads, so no head is ever patched alone), yet it beats per-head NIE on kill and net in three of the five datasets (by up to 0.05) and ties it within 0.003 in the other two.
    • Finally, per-head NIE pins no witnesses; pinning them raises mean net over the five Qwen3 tasks from 0.34 to 0.48. See App. D (held per-head NIE) and the new App. E.

(Continued in the next comment.)

AuthorsOpenReview ↗

Response to Reviewer zETw (2/2)

(Continued from the previous comment, part 1/2.)

Besides, it is not very clear whether the method is scalable, considering that experiments are mainly conducted analyzing attention heads not the full nodes, including MLP layers, as stated in the Discussion section.

On scaling: exact per-head NIE needs a forward pass per component, so its cost grows with the number of components, and exhaustive set-level scoring grows exponentially. Each JuntaLearner step instead scores a fixed number of sampled sets on a fixed number of prompt pairs, whatever the number of components. At the sizes we tested (up to 1,024 heads), this does not make training cheaper than batched per-head NIE, which is why we removed the scaling claim from the abstract. What the cost buys is set-level interactions: sampled Shapley values, the other set-level method we ran, cost 124 to 252 times as much as per-head NIE (App. G).

With regard to MLP layers, we focused on testing with heads only, as most documented backups are attention heads (Wang et al., 2023; Rushing & Nanda, 2024), and two of our tasks, IOI and induction, were first explained through heads (Olsson et al., 2022). While the method is extensible to MLPs, we defer investigation with edges, neurons, and SAE features to future work; we now state this as a limitation and drop the scaling claim from the abstract.

On MIB, we train on heads and MLPs; one untuned run beats MAttr on CMD, and on kill and net scored over its heads (patching the first-layer MLP of GPT-2 alone removes every task). (Point #9).

However, it would be helpful if the authors provided more detail on which methods have been proposed to mitigate the Hydra effect (or the other problems mentioned) and why those methods are infeasible or less faithful.

Thanks for pointing this out (Point #11). We now further discuss this in Appendix J.

In particular, we break down existing methods into: methods that require the circuit a priori, methods that score one component at a time (and thus, cannot compensate for set-level interactions), and qualitative studies on compensation that do not directly address circuit discovery.

We adapted the scores from the second group as baselines, per-head PIE and per-head NIE+PIE, which now appear in Table 1. Their mean net is 0.43 and 0.46, against 0.48 for our default, and they fall furthest behind on GPT-2 IOI, where the backup heads are known (0.23 and 0.44 against 0.65). New: App. J.

Please add a label for the x-axis in Figure 2.

Done, thank you!

We hope the answers and the revised paper resolve your concerns; if they do, we would be grateful if you would reconsider your score.

I appreciate the authors' efforts to clarify my main concerns. Since the revised manuscript addresses these points, and appears to address the other reviewers' concerns as well, I raise my score to 3 and recommend that the AC accept the paper to the conference.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers for their careful reviews. On soundness, zETw has "no substantive concerns about the paper's soundness", and vnVi and ayjz have "minor or fixable concerns". We have addressed each of these concerns, as summarized below and detailed in our replies to each reviewer.

In response to reviewer feedback, we have run a new set of experiments and revised the paper. In the revised paper, each change is colored and tagged with the number of the answer that explains it (1 to 12) and the initials of the reviewers who raised the point: Z = zETw (orange), V = vnVi (green), A = ayjz (magenta); points raised by several reviewers are blue.

We've addressed reviewers individually and replaced the manuscript with a revised version. The PDF on this page is now the revised version; the submitted version remains available under Revisions. This entails new experiments, including:

  • rescoring the selected configurations and the baselines on fresh prompt pairs (App. C)
  • adding five seeds for the Qwen3 net picks
  • comparing many different set-size schedules (App. H)
  • computing per-head NIE with downstream heads held, our estimand at set size one
  • a new toy example illustrating the benefits of our method with multi-tier backup heads

We've also included substantial further discussion and experimental results, including five new appendices (E, G, H, I, J).

We hope that our responses help address reviewer concerns, and we look forward to further discussion!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.