acceptodds
Withdrawn from ICLR 2027

CONTROLS ARE INTERVENTIONS: WHAT COMPLEMENT ABLATIONS ERASE, AND HOW TO AUDIT THEM

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Circuit analyses often replace everything outside a circuit with mean, zero, or resampled activations, then knock out parts of it. This complement ablation is treated as a control but is an intervention, erasing each input’s own activations and the replaced components’ reactions to knockouts. We audit it against the full model on knockout effects and the head choices they induce. In three published GPT-2 circuits, for indirect object identification (IOI), acronyms, and greater-than, mean-ablating the non-circuit heads keeps clean accuracy within five points. Yet on every fresh input set, its per-input choice of the most damaging head pair misses more full-model effect than our tolerance allows. Freezing these heads at each input’s clean activations keeps clean outputs exact and blocks only their reactions; it removes most of this error in IOI, about half elsewhere. Unfreezing the 48 non-circuit heads that attribution patching ranks highest passes in IOI and greater-than, not acronyms. In the original completeness test of the published IOI circuit, knockout sets found by greedy search have large gaps under mean ablation but small ones under freezing; a frozen-complement search finds the reverse. In IOI, freezing attention to the first token, the attention sink, removes over a third of the self-repair after zeroing the name-mover heads. Auditing the decisions a complement ablation supports, not its clean behavior, makes its faithfulness testable.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, activation patching, ablation, circuit analysis, attention sink, self-repair, causal interventions, evaluation methodology

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_agCFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

When you do a circuit analysis, one common type of ablation is a mean ablation on the complement of the circuit. The complement are things that are thought to not really matter in the circuit. This paper shows that they do and that you can miss important behaviors when you do it. For example, if you knock out a circuit component like a head and self-repair happens in some of the outside components, then you would have missed that because you did amine ablation on the outside components.

Critical strengths

I think this paper identifies an interesting problem with a protocol sometimes used in circuit analyses (the complement mean ablation) by connecting to prior work in the area (self-repair). Put another way, they point out and demonstrate the issue of doing mean ablations on the complement, because when you then try to get importance for the in-circuit components (through e.g., ablating them), you now fail to measure effects like self-repair, which can make up for that ablation.

Critical weaknesses

The exact contribution is a little specific and niche within the field. I think this paper has limited impact even if the observation is interesting and useful in limited applications

The attention sink rresult is interesting but I feel like it doesn't contribute much to the main story

I would be more willing to give a positive score if the paper contributed a concrete scientific finding that researchers would change after these experiments and establish cleaner results (e.g., the acronym result fails the arbitrarily chosen threshold, while the other two pass)

Questions for authors

N/A

Reviewer_xX2iOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper describes a series of experiments that vary the typical approach to circuit discovery and analysis. The paper does not clearly describe what readers should conclude based on these experiments nor what general methodological lessons to take away from them.

Critical strengths

The paper appears to identify some serious problems with the current methodologies for circuit discovery and analysis.

Critical weaknesses

As currently written, the paper appears to almost entirely avoid the most useful classes of narrative and description that is present in most scientific papers. The paper describes a set of experiments, but does not explain either what readers should conclude based on those experiments or the general class of methodological improvements embodied in those experiments.

Specifically, the paper does a good job of describing a set of experimental results, but it rarely provides an analysis of what those experimental results mean for the larger problem of circuit discovery and the project of mechanistic interpretability. What should readers conclude based on the experimental results? For example, the authors might say that: “Current methods for curcuit analysis provide biased estimates about the utility of different potential elements of the circuit.” This analysis should be the primary backbone of the paper. The authors should lead with the analysis, and then provide experimental results that provide the evidence to support that analysis. In the absence of such analysis, the experimental results won’t mean anything to most readers.

The paper also does a good job of describing the specific classes of experiments (e.g., “An audit of complement ablations…”), but it does not describe the methodological innovations that make these experiments possible. Instead, the authors should foreground clear descriptions of the methodological improvements that allow the audits that they perform. Other researchers may want to use these methodologies, so they are a major contribution.

Finally, the “Implications and Limitations” section explains what analysts should do, rather than what readers should conclude based on the results in the paper. This section should explain what readers should conclude about current circuit discovery and testing methods based on the experimental results (along with stating possible limitations).

Questions for authors

What conclusions do you most hope readers will take away from the paper? How would you state those conclusions in clear and concise language?

What are the methodological innovations that you have made that enabled your experiments?

Reviewer_H1GSOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates how complement ablation affects causal conclusions in mechanistic interpretability. The authors argue that replacing non-circuit activations with fixed values not only removes input-specific information but also prevents these components from responding to subsequent interventions. They introduce an audit based on knockout-effect errors and decision regret, which measures whether component choices derived from an ablated model agree with their actual effects in the full model.

Experiments on three published GPT-2 circuits show that mean ablation can preserve clean accuracy while changing the identification of important attention heads. The authors further examine clean-activation freezing and selective unfreezing as ways to reduce these discrepancies. They revisit the IOI circuit completeness evaluation and investigate the contribution of first-token attention to downstream self-repair.

While the observations are potentially useful, the paper's conceptual contribution is limited, and the extensive experimental analysis does not establish sufficient generality or practical value.

Critical strengths
  1. The paper raises a relevant concern regarding the interpretation of component importance under complement ablation.

  2. The authors revisit previously published circuit analyses and provide several controlled comparisons between mean ablation, clean-activation freezing, and selective unfreezing.

Critical weaknesses
  1. The central observation appears largely incremental. Replacing intermediate activations with constants necessarily prevents those components from responding to upstream interventions. The distinction between direct and total causal effects is already well established, and previous work has investigated the sensitivity of circuit faithfulness and component importance to ablation choices [1–3]. The proposed audit offers a different evaluation perspective, but the paper does not sufficiently demonstrate a substantial conceptual advance beyond these established observations.

  2. The main experimental evidence is restricted to GPT-2 Small and three relatively simple, previously studied circuits. Despite the broad methodological framing, the paper does not validate its central audit and selective-unfreezing procedure on modern language models or more complex mechanistic circuits. The additional Qwen experiments in the appendix do not adequately address this limitation.

  3. The proposed remedy lacks generality. Although unfreezing 48 AtP-ranked heads achieves low regret for the selected IOI candidate pairs, Appendix N.5 reports substantially larger errors when the same configuration is evaluated on other circuit heads and head classes. Consequently, the method does not provide a reliable circuit-level guarantee and requires intervention-specific validation. This substantially weakens its usefulness as a general evaluation methodology.

  4. The practical contribution is questionable. The audit fundamentally relies on full-model knockout effects as its reference, while the successful selective-unfreezing configuration restores computation in a substantial fraction of the non-circuit heads. It remains unclear what practical advantage this procedure offers over directly analyzing the original model. The paper provides neither a compelling efficiency benefit nor a broadly transferable alternative for circuit evaluation.

  5. The presentation quality is unsatisfactory, particularly the clarity and readability of the tables. Many tables are overly dense, with small fonts, crowded columns, and excessive numerical details, making it difficult to identify the key findings or compare experimental settings. The manuscript also contains numerous auxiliary tables that repeat similar observations without providing sufficiently clear summaries or additional insights. The authors should substantially improve the table formatting, consolidate redundant results, and present the main findings in a more concise and accessible manner.

Ref:

[1] Transformer Circuit Faithfulness Metrics Are Not Robust. COLM.

[2] Hypothesis Testing the Circuit Hypothesis in LLMs. NeurIPS.

[3] Optimal Ablation for Interpretability. NeurIPS.

Questions for authors
  1. What is the fundamental conceptual novelty of the proposed audit beyond established causal mediation analysis and previous studies showing that circuit faithfulness depends on the chosen ablation method [1–3]?

  2. Given the poor transfer of the 48-head selective-unfreezing configuration to other knockout sets in Appendix N.5, what practical guarantees can the proposed methodology provide beyond the specific interventions used during evaluation?

  3. Why should practitioners adopt this auditing and selective-unfreezing procedure instead of evaluating knockout effects directly in the full model? Can the authors demonstrate a concrete practical advantage on a modern model or a substantially more complex circuit?

Reviewer_YtGwOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper makes the observation that mean/zero ablating components of an LLM judged not to be in a circuit means that when the heads of the circuit for a given behavior are knocked out, the zero or mean ablated components cannot compensate for the knocked out components.

They propose unfreezing those components that rank highly by AtP to allow them to compensate for knockouts.

Critical strengths

The paper provides a thoughtful analysis of how mean ablations are not actually a control but instead an intervention that changes the way self-repair happens when a circuit has its components knocked out.

They demonstrate that mean ablation removes self-repair in IOI tasks.

They provide a method for better controlling for the effects of knockouts (unfreezing components ranked as important from AtP).

Critical weaknesses

The main weakness is the presentation of results. A lot of results are presented inline and it feels like the listing the outputs of agent driven experiments. I think the results are sound, I just would prefer to have them packaged into a story that is easy to follow. Also, I would prefer to have a background section that sets up the notation and nomenclature of the experiments rather than just diving straight into your methods after the introduction. In general the structure of the paper (e.g. the section headings) feels LLM generated and I would prefer to walked through the experiments by more natural text. I would also prefer to allude to the implications and to reference them throughout rather than putting them at the end. The paper would also benefit from a diagram explaining their audit.

Questions for authors

How did you decide the tolerance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.