acceptodds
Under review as a conference paper at ICLR 2027

Causal Localization of Evidence Effects in Transformers

Abstract

Can controlled symbolic inputs reveal attention heads that influence a language model's decisions from natural language evidence? We introduce controlled evidence aggregation (CEA), which varies the counts and order of two symbols in otherwise fixed prompts and measures changes in a frozen Transformer's preference between two answer tokens. The preference is measured by their logit difference; the prompts prescribe no correct answer. Balanced utility causal selection (BUC) identifies compact head sets by jointly evaluating how much ablating their outputs removes the count effect and how closely patching mean outputs between count conditions recovers the effect in both directions. We derive conditional bounds linking head attribution to intervention effects and specifying when the attribution shortlist can be recovered from finite samples. In Pythia-410M, ablating sets of 1–12 heads selected separately for four ordering regimes removes 65.5–91.6% of the count effect. The sets retain their influence on count comparisons not used for selection. To test relevance beyond symbolic inputs, we evaluate a natural language task asking which state is supported by more reports. Ablating eight heads selected only on symbolic yes/no prompts lowers mean accuracy by 8.2 percentage points, versus 0.8 for random sets matched by size and layer. Our results demonstrate that controlled symbolic probes can identify compact attention head sets whose causal influence extends to natural language decisions.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, causal localization, activation patching, attention heads, controlled evidence aggregation, transformers

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EWCuOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies Transformers on symbolic inputs to understand whether findings on attention head behavior transfer to a specific natural language task the authors propose.

Critical strengths

The understanding of causal mechanisms of behaviours in large language models and in their components, transformers, are a very important problem. Studying them in synthetic experiments is a good proposition because, due to the controlled nature, many boundary conditions can be prescribed and transparently followed.

The paper proposes an experimental design which tries to remove certain biasing aspects, for example, with their label names being non-informative. The paper has many experimental ablations (although their exact meaning is hard to discern for myself).

Critical weaknesses

My biggest concern with the paper is that the writing is extremely hard to understand. They are using many terms, and do not always explain them. Even in cases where their meaning might be inferred (BF, FP, "write" as a noun), by putting at least a reference or a short definition, would significantly improve ease of understanding for the reader.

I find it also hard that many quantities are defined, but not explained: what does it mean that a contrast is "clean"? the utility of what does J represent?

Regarding the related work section, I see that the authors do not refer to the literature in studying formal grammars or symbolic tasks to understand model behaviour, including their extrapolation performance or whether they can compose different steps ahead. I would recommend the authors investigate this literature to contextualise the work, e.g., Meszaros et al. Also regarding the causal analysis, many of the formally grounded methods are missing, e.g., works by Geiger et al., many of which relate to causal abstarctions. Also, from checking the mentions of BUC in multiple sections, I was not entirely sure whether that is a method proposed by the authors or is coming from previous works (I could not find any statement that has the spirit of "we propose BUC" - although my understanding is that this is also a contribution of the paper).

The theoretical statements are hard to contextualize, as there is no explanation of what the results mean.

On the experimental design side and the metrics, the descriptions in the paper can't really be followed because many of the metric symbols are perhaps only named but not explained. The metric ranges are not given (at least not in the tables), and some metrics are not defined in the main text (e.g., slope reduction - I found the definition in the appendix, but I could only find it in teh appendix by searchign the pdf, as there were no clickable references, same for PairAcc/MinPairAcc). The same holds for task descriptions

For this reason, the paper is basically impossible to judge for me, thus I recommend rejection.

Questions for authors

In 5.6, some scope checks mention different models, what exactly does that mean? What were the experiments, and how were they run?

4 comments

Response to Reviewer EWCu (1/2)

Thank you for the thoughtful review. We appreciate your recognition that understanding causal mechanisms in Transformers is important, that controlled synthetic settings let boundary conditions be set and followed transparently, and that our design tries to remove biasing factors such as informative label names. We also agree with your main concern. The submitted version made the paper much harder to judge than it should have been, including the meaning of the many ablations. Below we address each Weakness (W1–W6) and your question.

Abbreviations: BUC = balanced utility causal selection; BF16 = bfloat16 precision; FP32 = 32-bit floating-point precision; FR/BA/IN/RA = Front/Back/Interleave/Random orderings; pp = percentage points.


W1. Undefined terms (BF, FP, "write")

All three are now defined at first use. A head's write is its contribution to the residual stream, the model's running hidden representation (Section 1). BF16 and FP32 are spelled out in Section 3.3, where they first appear. We also checked the rest of the main text for the same problem, so that abbreviations and named quantities are defined before they are used.

W2. What "clean" means, and what the utility JJ represents

Clean means computed with no intervention. The clean contrast is the ordinary high-count minus low-count difference in the readout (Section 3.1), and every intervention is compared against it (Section 3.2).

JJ (Section 4, Equation 4.1) is the signed raw utility. It is a weighted average of how much ablating a head set reduces the count contrast and how much patching it moves the readout in the intended direction, both in logit units. The difference from BUC's score QintQ_{\mathrm{int}} is normalization. QintQ_{\mathrm{int}} divides by the clean contrast and rewards landing close to it, whereas JJ keeps growing when a patch overshoots. The greedy guarantee in T3 is stated for JJ, and Section 4 now says in one sentence how the two differ.

W3. Related work on formal languages and causal abstraction

Thank you for these pointers. Section 2 now discusses Mészáros et al. (2024) on rule extrapolation in formal languages, together with studies of compositional failures and task-trained symbolic models. We explain the difference: those works ask whether a completion follows a prescribed rule, while our symbolic prompts prescribe no correct answer and we measure a frozen model's preference. Section 2 also covers the causal-abstraction line (Geiger et al., 2021; 2024). Interchange interventions test an alignment between neural components and a high-level causal model. BUC tests removal and recovery of a controlled contrast without assigning the heads a high-level variable. Appendix F extends this comparison in a table.

W4. Is BUC our proposal?

Yes, and we now say so explicitly: "We propose balanced utility causal selection (BUC)…" (Section 1, third paragraph). Our contribution is the joint selection criterion, which scores removal and recovery in both directions for the same head set, together with its controlled evaluation.

Response to Reviewer EWCu (2/2)

W5. What the theoretical results mean

Section 4 now opens by stating the question each result answers, and each theorem is followed by a plain-language reading:

  • T1. When can cheap attribution (projecting each head's output onto the answer direction) predict BUC's intervention score for a fixed head set? The bound says the prediction error is at most the measured intervention residual divided by the clean contrast.
  • T2. With finitely many prompts, when does the estimated top-mm attribution list equal the true one? It does when the true scores have a clear gap at rank mm relative to the sampling error.
  • T3. When is greedy selection close to the best set? It is when the raw utility behaves like a (weakly) submodular function, in which case greedy achieves at least a 1−e−γ1-e^{-\gamma} fraction of the optimum.

We also measure the quantities each result depends on (Tables 25, 26, 5, and 50) and state where a guarantee does not apply. For example, T3 concerns JJ, not the normalized BUC score.

W6. Metric definitions, ranges, task descriptions, and links

  • Definitions in the main text. Shrink and match are defined in Section 3.2 with a worked example. Slope reduction is defined where it is used, in Section 5.3: one minus the ratio of the ablated to the clean regression slope of the readout on count. Excess damage and the majority-minus-read-one interaction are defined in Section 5.4. PairAcc and MinPairAcc are no longer used in the main text. They appear only in an appendix table (Table 28), whose caption now links to their definition in Appendix B.3.
  • Ranges. Shrink and match are at most 1, equal 1 for complete removal or recovery, equal 0 when the intervention changes nothing, and can be negative. This is stated in Section 3.2 and now also in the captions of Tables 1 and 2. Accuracies are fractions, and damage is in logits or percentage points as labeled.
  • Task descriptions. The symbolic task is described with explicit example inputs in Section 1. The language tasks (majority, read-one, KEEP/DROP) are described in Section 5.4, with templates and scene design in Appendix B.3 (Tables 7–8).
  • Links. All cross-references and citations are now colored, clickable links.
  • Ablations. We consolidated the appendix so the many ablations are easier to navigate. Appendix C collects the supporting results for Section 5, Appendix D holds the extended experiments, and overlapping tables were merged.

Q. Section 5.6 (now Section 5.5): what are the scope checks with different models?

This check asks whether the symbolic localization result is specific to Pythia-410M. For each of six frozen models, we reran the same BUC selection from scratch on the same symbolic prompts and readout, separately for each ordering, and evaluated the selected heads against layer-matched random sets. The six models are Pythia-70M, -160M, -410M, and -1B, GPT-2, and Qwen3-4B-Base. Before running, we checked that the answer words "various" and "successful" are single tokens in every model's tokenizer (Table 43). Averaged over the four orderings, the selected sets give shrink 0.650–0.895 and match 0.700–0.925 across models, versus at most 0.048 and 0.059 for random sets (Table 24; per-ordering results in Tables 39–40, Appendix D.7). The heads are reselected in each model, so this shows that the procedure finds effective small sets in other models; it does not transfer one head set between architectures.

Two other model checks appear in the revision. The fixed-budget method comparisons in Section 5.2 use both Pythia-410M and GPT-2. The language tests in Section 5.4 repeat the majority versus read-one comparison in Pythia-1B and GPT-2, each with its own symbolically selected heads.


We hope the rewritten text, the clarified theory, and these explanations make the paper possible to evaluate. We would sincerely appreciate it if you could take another look, and we are happy to clarify anything that is still unclear.

I thank the authors for the detailed response. The clarifications help understanding to some extent, though as I checked the uploaded pdf again, parts such as sections 6&7 still do not read in a self-contained manner. I'd consider teh authors to rework the presentation of their results and resubmit.

Response to Reviewer EWCu (follow-up)

Thank you for reading the revised PDF again and for telling us exactly where it still falls short. You are right about Sections 6 and 7. Both were written as compressed summaries that assumed the reader had just finished Section 5, so they relied on terms such as the matched task pair, the screened pool, and within-seed intervals without saying what they meant. We have now revised the main text so that each part can be read on its own, and the updated PDF is uploaded:

  • Sections 6 and 7. Section 6 (Discussion and limitations) now has four labeled paragraphs that follow the order of Section 5: symbolic localization, selection objective, language transfer, and theory and model scope. Each paragraph restates the terms it uses, says in plain terms what the experiment establishes, and then gives its limits. For example, it explains that ablation and mean patching show that the count effect (the shift in the model's answer preference when evidence counts change) reaches the answer through the selected heads' final-position outputs, but not that these heads compute the count alone. It also explains why larger damage on majority than on read-one points to aggregation, together with the confounds that remain, and says what the method comparison, each of the three theorems, the confidence intervals, and the cross-model tests do and do not establish. Section 7 (Conclusion) now tells the complete story with the key numbers.
  • Section 5. It opens with five questions, one per subsection, and restates what shrink and match measure. Each paragraph now states its finding first and then gives the evidence in short sentences. For example, Section 5.4 states the exact majority and read-one questions and defines a scene where it first uses the term.
  • Definitions throughout. Every term is now defined where it first appears, including direct attribution, the attribution shortlist, the screened pool, sensitivity ranking, donor inputs, and the three language tasks. When a symbol or term reappears far from its definition, the text restates what it is, for example "GREEDY-R, the recovery-only baseline of Section 3" or "the signed raw utility JλJ_\lambda of Section 4", so readers do not need to search back.

If any other part still does not read on its own, we would be grateful if you could point us to it, and we will revise it right away. We hope the revised presentation makes the contribution easier to evaluate, and we would appreciate it if you could take it into account in your final assessment.

Reviewer_GPXoOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

See the critical weakness section.

Critical strengths

None

Critical weaknesses

The writing prevents me from properly reviewing the work. The second paragraph of the introduction, for instance, could do a much better job of introducing the setup. After reading it many times without seeing the rest of the paper, I can maybe squint at what the authors may be doing, but the “Front ordering” is fully unclear. Using terminology that has not been introduced is also an issue in the next paragraph where “compact head sets” are talked about. In the second-to-last paragraph of the introduction, it is not clear what “mutually exclusive states” is referring to. Or e.g., in lines 110--113 where many terms without clear semantics are casually used. On line 138 the paragraph starts “Ablation sets the selected writes to zero.” and then it just jumps to something else. Or on line 166, it casually drops “the fp32 rule” that is not defined until much later. The point is that the paper can not be reasonably read in one go by a human without major detective work; in many ways the writing appears like a collection of technical claims rather than trying to communicate the reasoning and understanding that should hopefully follow. For a paper, it is simply not enough to do good experiments or have enough author understanding of valuable findings or insights. There may be good work here but it needs to be communicated clearly and I could simply not follow it along.

Questions for authors

None

1 comment

Response to Reviewer GPXo

Thank you for reading the paper so carefully and for being direct about the main problem. You are right: the submitted version read like a list of technical claims and did not walk the reader through the reasoning. Your pointers to specific paragraphs and lines were extremely helpful, and we used each of them as a test case while rewriting Sections 1 and 3. Below we go through them one by one.


1. Introduction, second paragraph, and "Front ordering"

We agree this paragraph did not set up the task. It now starts from one concrete input pair and only then names anything. Under the Front ordering (all B's before all C's), the two prompts are

50 B | 150 C | 50 A    <->    150 B | 50 C | 50 A

where each number counts repetitions of one space-separated symbol. Both prompts begin with "Tokens:" and end with "Choose one word: various or successful" followed by "Answer:". We measure the model's next-token logit for "various" minus that for "successful". The difference between the two prompts is what we call the count effect. The other three orderings (Back, Interleave, Random) are defined right after, in Section 3.1.

2. "Compact head sets"

We removed this term. The text now says "small head sets" and gives their sizes (1–12 heads). A head's write, its contribution to the residual stream, is defined in the sentence before it is first used (Section 1, third paragraph).

3. "Mutually exclusive states"

Section 1 (fifth paragraph) now explains this with an example. Each language prompt states that exactly one of two states holds, for example valid or invalid, and individual reports may disagree. Six reports asserting "valid" and six ruling out "invalid" support valid, whereas two asserting "invalid" and two ruling out "valid" support invalid. Each state word appears eight times, yet the support is 12:4. So the model cannot answer by counting words.

4. Lines 110–113 (terms without clear semantics)

Section 3.1 now introduces each symbol before it is used, in this order: the readout (a logit difference, which equals the log-odds between the two answers), the evidence and tail lengths, the four orderings, the input distribution for each ordering and count, and the high-minus-low contrast. Clean is defined there: the contrast computed from forward passes without any intervention.

5. Line 138 ("Ablation sets the selected writes to zero" and then a jump)

Section 3.2 now describes the procedure as a sequence. We first run a clean forward pass and record the readout. In a separate forward pass, we modify the selected heads at the final position, let the rest of the model run as usual, and read the same two logits. Only then are ablation (set the selected outputs to zero) and mean patching (replace them with their average under the opposite count) introduced. The scores follow, with a numerical example: a clean contrast of 2 logits and a patch gain of 3 logits give match 0.5.

6. Line 166 ("the fp32 rule")

BF16 (bfloat16) and FP32 (32-bit floating point) are now spelled out where they first appear (Section 3.3). The stopping rule is stated in the same paragraph: return the shortest prefix with shrink ≥ 0.85 and mean match ≥ 0.70, otherwise the full 12-head budget with a failure flag. The paragraph also says that fixed-budget comparisons use exactly kk heads instead.

Overall flow

Beyond these spots, the introduction now follows question → concrete example → what we measure → our method → what the theory asks → language tests → findings. In Section 5 we state the interpretation right after each result, for example what a non-significant interval does and does not imply. We also consolidated the appendix so that each table answers one question.


We hope the revised Sections 1 and 3 can now be read in one pass. If you have time to take another look, we would be very grateful for any remaining places where the text still gets in the way.

Reviewer_8rCcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Controlled Evidence Aggregation (CEA) to identify attention heads responsible for count-based evidence aggregation in Transformers and Balanced Utility Causal Selection (BUC) to select compact head sets using ablation and activation patching. Empirical studies on Pythia-410M and other language models demonstrate that selected heads influence symbolic count comparisons and transfer partially to natural-language decision tasks.

Critical strengths

The combination of controlled symbolic tasks, causal interventions, and natural-language transfer experiments provides a useful framework for studying attention-head functionality.

Critical weaknesses

W1. The theoretical guarantees rely on strong assumptions about attribution errors, monotonicity, and weak submodularity, while the greedy approximation result concerns a different objective from the normalized BUC selection criterion.

W2. The natural-language evaluation mainly considers binary evidence-comparison tasks and final-token interventions, making it unclear whether the identified heads represent general evidence aggregation mechanisms rather than task-specific decision features.

W3. Improvements over sensitivity-based selection are small or inconsistent across ordering regimes, and the FP32 fixed-budget comparisons lack convincing statistical evidence of consistent superiority.

Questions for authors

Central Query - Theoretical Guarantees

The theoretical results rely on restrictive assumptions that are not empirically validated, and the greedy approximation guarantee applies to a surrogate objective rather than the actual BUC selection criterion, limiting their relevance to the proposed algorithm.

Q1. Can the authors establish whether the assumptions underlying the theoretical guarantees hold empirically and explain how these guarantees apply to the actual BUC objective?

In light of this, two immediate questions arise:

The experimental validation primarily focuses on controlled symbolic counting and relatively simple binary natural-language decisions, leaving it unclear whether the identified attention heads capture general evidence-aggregation mechanisms or task-specific response patterns.

Q2. Do the selected heads generalize to more complex reasoning tasks involving multi-step evidence aggregation, multiple answer choices, or longer contexts?

The improvements over existing head-selection approaches are not consistently substantial across evaluation settings, while the limited statistical uncertainty analysis makes it difficult to establish whether the observed gains are robust and attributable to the proposed selection strategy.

Q3. Can the authors provide statistically supported comparisons against stronger head-selection baselines across multiple models and intervention budgets?

4 comments

Response to Reviewer 8rCc (1/3)

We thank you for the careful and constructive review, and for recognizing that combining controlled symbolic tasks, causal interventions, and natural-language transfer gives a useful framework for studying what attention heads do. Your three questions shaped most of the new analyses in the revision. Below we answer each Weakness together with the matching Question (W1/Q1, W2/Q2, W3/Q3) and point to the corresponding changes.

Abbreviations: BUC = balanced utility causal selection; QintQ_{\mathrm{int}} = BUC's normalized score (0.5 shrink + 0.25 HL match + 0.25 LH match); JJ = signed raw utility in logit units; GREEDY-R = greedy search that maximizes recovery only; FR/BA/IN/RA = Front/Back/Interleave/Random orderings; CI = 95% paired-bootstrap interval; pp = percentage points.


W1 / Q1. Do the theoretical assumptions hold empirically, and how do the guarantees apply to the actual BUC objective?

You're right that the three results have different targets, and that the greedy bound concerns a different set function from BUC's normalized score. The revision now says this up front (Section 4, first paragraph) and reports a measured diagnostic for each assumption. In short: T1 covers BUC's actual score, T2 covers the attribution shortlist that BUC actually uses, and T3 covers the raw utility only.

T1 (fixed-set attribution error) applies to QintQ_{\mathrm{int}} directly. Equation 4.5 bounds the gap between BUC's normalized score and its attribution-based prediction. Its only assumption is a bound on intervention residuals. The identities themselves always hold, so the empirical question is how large those residuals are, and we measure them (Table 25, eight model–ordering conditions, final FP32 run). Mean prediction errors are 0.30–1.16 score units in Pythia-410M and 0.41–4.50 in GPT-2, and every mean error lies within its measured envelope. So we do not claim the residual is small. On the contrary, the diagnostic shows that direct attribution is often a poor stand-in for an intervention, especially in GPT-2. That is exactly why BUC uses attribution only to build the shortlist (in GPT-2 the shortlist contains all 144 heads), and makes every screening and greedy decision from measured interventions.

T2 (shortlist recovery) applies to the shortlist BUC actually uses. Its assumption is a population tail scale, which no finite sample can certify. As an empirical check we plug in the sample standard deviation and repeat the diagnostic for Pythia-410M RA at seeds 1729–1740 (Table 26). Against the mean plug-in threshold 2η^=0.0001012\widehat\eta=0.000101, the rank-192/193 boundary gap passes in 11 of 12 seeds, and the margin of the eight selected heads passes in all 12. We report these as margins, not as a certified recovery probability.

T3 (greedy bound) covers the raw utility JJ, not QintQ_{\mathrm{int}}. We now say this explicitly, and we checked both functions on finite pools. For JJ, we enumerate all 256 subsets and 1,024 single-head additions in two eight-head pools per ordering (Tables 5 and 50). All eight pools have zero negative marginals on evaluation data, the minimum empirical submodularity ratio is 0.820, and exact greedy reaches at least 0.944 of the optimum (maximum regret 0.0248). For the normalized score itself, the premises do fail: 17–31% of marginals are negative (Table 27). So the T3 guarantee does not transfer to QintQ_{\mathrm{int}}. What we can say for QintQ_{\mathrm{int}} is empirical: in an exhaustive audit of QintQ_{\mathrm{int}} itself on eight-head pools, greedy selection on QintQ_{\mathrm{int}} stays within 0.021–0.043 of the best subset of at most the same size (Table 27, last column).

Response to Reviewer 8rCc (2/3)

W2 / Q2. Do the selected heads generalize beyond binary decisions, and are they more than task-specific decision features?

This was the most useful question for us, and the revision adds a dedicated set of language tests (Section 5.4, Tables 3–4). The design throughout is the same. We select eight heads on symbolic yes/no prompts only, freeze them, and ablate them in language prompts that never entered selection. We then compare the damage with eight size- and layer-matched random sets. The key comparison is majority versus a read-one control that uses the identical reports, answer tokens, and query directions but asks about a single designated report. Shared answer processing therefore affects both tasks, and the majority-minus-read-one interaction II measures damage beyond it.

On 128 held-out test scenes with 16 reports, excess damage (selected minus random) is 0.519 logit for majority but only 0.092 for read-one. The interaction is 0.428 logit [0.417, 0.438], or 23.78 pp [20.86, 26.58] in accuracy. Your three directions:

  • Longer contexts. With 32 and 64 reports (same 3:1 support ratio and the same frozen heads), II is 0.461 [0.452, 0.472] and 0.497 [0.486, 0.508], positive in all 8 ordering–query cells at each length.
  • Multiple answer choices. With three states (8:4:4 support) and answers A/B/C, II is 0.440 [0.428, 0.451] (24.56 pp; clean accuracy 0.910 for majority, chance 1/3). Notably, these answer tokens differ from the yes/no readout used to select the heads, which argues against a pure yes/no decision feature.
  • Multi-step aggregation. KEEP/DROP requires filtering marked reports before counting. The overall support is balanced 8:8, while the KEEP subset is 6:2. Clean test accuracy is 0.902 [0.871, 0.934], and selected-head excess damage is 0.479 logit [0.462, 0.495].

The interaction also holds with model-specific selections in Pythia-1B (0.462 [0.450, 0.472]) and GPT-2 (0.396 [0.386, 0.407]).

On final-token interventions: we intervene at the final position because that is where the measured effect enters the readout. Ablating the same heads at the symbolic sequence positions instead gives shrink of only 0.059, 0.022, and 0.001 in FR, BA, and IN, versus 0.906, 0.655, and 0.916 at the final position (Table 22). Earlier positions still feed the selected heads through attention, and the component checks show that output–value patching recovers more than query–key patching in FR, IN, and RA (Table 23).

To be upfront about scope, since you ask directly: the tasks use restricted answer choices over structured reports. Read-one also differs from majority in which report it queries and in clean accuracy (0.979 vs. 0.933). So shared answer processing and task difficulty can still contribute, and Section 6 says so. Free-form, multi-token decisions over heterogeneous documents remain open.

Response to Reviewer 8rCc (3/3)

W3 / Q3. Are BUC's gains over other selection rules statistically supported across models and budgets?

Your point is fair for the original BF16 comparison. There, at each ordering's own head count, BUC traded a little removal for better recovery, and its QintQ_{\mathrm{int}} advantage over sensitivity ranking was only 0 to 0.039 (Table 18 reports the summed score 2Qint2Q_{\mathrm{int}}). The revision replaces that with fixed-budget FP32 comparisons in Pythia-410M and GPT-2 at k=4,8,12k=4, 8, 12 (Section 5.2, Appendix C.1). All methods share the candidate pool, selection data, evaluation inputs, and donors, and the uncertainty is a paired bootstrap over 96 matched RA input pairs.

  • Sensitivity ranking. BUC is higher in 23 of 24 model–ordering–budget cells, by +0.0383 to +1.0672. The remaining cell is a tie because both rules pick the same heads (Table 13). All six RA intervals exclude zero, and under the paired-mean estimand five of six do (Table 14).
  • Three single-score rules (single-HL patch, single bidirectional patch, write norm; Pythia-410M, k∈1,2,4,8,12k \in \\{1,2,4,8,12\\}, all orderings). Of 80 differences, 65 favor BUC and 15 are ties with identical sets, and 13 of the 20 RA intervals exclude zero (Table 15).
  • Head-level attribution patching. In GPT-2 RA, BUC is higher by +0.5126 [+0.4785, +0.5406]. In Pythia-410M RA the two rules share six of eight heads and score nearly the same (+0.0012 [−0.0003, +0.0065]). BUC is higher in all six deterministic-ordering cells (Table 16).
  • Random controls. In Pythia-410M, BUC exceeds the 97.5th percentile of three 30-draw random-control families at k∈4,8k \in \\{4, 8\\} in every ordering (Appendix C.1).

Against GREEDY-R, which shares BUC's pool and greedy search and changes only the objective, the advantage is real but modest. All six pooled differences are positive (+0.0012 to +0.0827), and two intervals exclude zero (Table 2). Under the paired-mean estimand, all three Pythia-410M intervals exclude zero, while GPT-2 at 8 and 12 heads is slightly negative with intervals that include zero (Table 14). Across seeds 1729–1731, which regenerate the inputs and rerun both selections, the eight-head RA difference is positive every time in both models, and three of the six seed-specific intervals exclude zero (Table 17). We state the conclusion at that strength in the paper. BUC's joint objective clearly beats rules that rank heads one at a time. Against a recovery-only greedy search it helps consistently in sign but only sometimes significantly.


We hope these clarifications and new results address your concerns about the scope of the theory, the generality of the transfer, and the statistical support for BUC. We would be glad to follow up on any point.

Thank you to the authors for the detailed rebuttal and additional experiments, which address several of my earlier concerns. However, my main concern remains that the theoretical guarantee applies to the surrogate objective �0��0�, rather than the actual BUC objective �1��1�, for which the required monotonicity assumption does not hold. The eight-head experiments are useful but do not establish whether these findings generalize to larger candidate pools. Additionally, the stronger intervention effects at the final answer position leave some uncertainty about whether the identified heads perform evidence aggregation or primarily influence the final readout. Therefore, despite the improved empirical evidence, I would like to maintain my original score of 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.