acceptodds
Under review as a conference paper at ICLR 2027

Every Ablation Is a Dose: Counterweights and the Semblance of Self-Repair

27 views

Abstract

Ablate a component of the language model, and it's often observed that other components adjust and appear to compensate. This phenomenon, termed self-repair, has been observed repeatedly, but its mechanism remains unclear. The most systematic study to date concluded that self-repair is noisy and unlikely to have a single explanation. We argue that it has one, a gain already present before any ablation. Any intervention on a causally important component can be viewed as a point on a coordinate axis , the signed strength of a counterfactual contrast. Hence, any conventional ablation methods are uncalibrated points on this axis. We show that the causal repair response for a fine-grained unit is governed by an affine law . The slope is a fixed coefficient that consistently influences the model, with or without ablation, and its sign determines whether the unit counteracts or reinforces the removed signal. On a factual-verdict task across four models from distinct families (Gemma, Qwen, LLaMA, and Mistral), we identify components (MLP neurons, OV neurons, and singular directions) that follow this affine law, 68 of 81 downstream directions in all. Moreover, we can anticipate the magnitude of from the fixed weights. On the IOI circuit of GPT-2 Small, seven of the ten heads the intervention can reach follow the law, and all seven are counterweights. From this perspective, what may appear as self-repair is merely a counterweight doing its usual operation when the contrastive signal emerges at the core.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, self-repair, ablation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_FU3UOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This work studies the phenomenon of self-repair in LLMs: the behavior of a model appears to compensate for ablated internal components. They do so by defining a systematic study whose conclusion is somehow in contrast with our current understanding: instead of considering self-repair as a noisy, difficult to explain phenomenon, the authors find that the causal repair response of an LLM abides to a simple law, with one important "slope" parameter. The slope parameter influences how the model reacts to a removed "signal". Their work helps in identifying the role of LLM internal blocks, and which and how closely they follow the simple law. They also demonstrate that, in some cases, the slope parameter can be inferred by model weights alone.

In conclusion, this paper argues that the so called self-repair might be merely a usual operation of counter-weighting reacting to a contrastive signal emerging from the model.

Critical strengths
  • This work is very interesting, and sets off to study an important phenomenon, self-repair, that hinders ablation stuides on LLMs. The authors posit that self-repair is not an unreachable mechanism to understand: on the contrary, it abides to a simple affine law, with a slope coefficient that is not only interpretable, but also predictable. This slope coefficient can be readily used to understand the effects of individual components of an LLM, in terms of counteracting or reinforcing a removed signal.
  • The "do-calculus" implied by this work is used to study a variety of models, which caters to the generality of the conclusions discussed by the authors. The task used to study self-repair is simple and clear, which is a virtue in this kind of empirical analyses of causation.
Critical weaknesses
  • The text is very hard to read, sentences are terse, mixing terms that I find colloquial (that technically prepared devs would use to argue in a discussion) with mathematical notation that is not always explicit, and in places confusing.
  • As discussed by the authors themselves, the single task used in this work, despite its virtues discussed in the strengths, might not be sufficient to state that the methodology and the findings (the affine law) apply to more complex tasks.
Questions for authors
  1. Figure 1: why the dose axis is inverted w.r.t. the usual convention of having negative values on the left of 0, and positive values to the right?
  2. In section 1 you present equation 1, but ownr\textrm{own}_r is not defined (yet). In the introduction, you define "directions" as internal units, such as a neuron, and indicate that you perturb an upstream neuron and check the opposition a downstream neuron excerts. Are you modifying the neuron activation output? Overall, for a reader that parses sequentially your paper, the introduction appears vague. Would it make sense to be a bit more precise on the "mechanics" rather than using a narrative that is reminiscent of biological interventions?
  3. In section 2: concerning weight-space bases, by calling Neuron directions and SVD basis oco_c in both cases, I think the notation might be confusing. Similarly, the "single interface" is difficult to grasp. What is vv (it is defined much later, and turns out to be a clamp value)? Overall, a pass on this section to clarify and explicit the notation would be useful to improve the readabiliy of the paper.
  4. Section 3, which is the most important, is somehow elusive to me, technically. I have an overall understanding of the various quantities described here, and ultimately the estimand Erb(x)E_r^b(x) is clear. My main issue is with the notation: I think the authors made an effort to "clean it up" such that it is not too heavy, but I'm frequently lost in double checking past definitions to make sure everything "adds up".
  5. Section 4: the intro to this section really reads like Claude before Opus 5.5 release. I suggest a pass to remove "Claudisms": it's this, not that. here we do this, it certifies, the verdict, ...
  6. Section 4: What do you mean by "pole"? It's defined in sec 2, and used in sec 4. A pole is the product of a direction ωr\omega_r by the unembedding matrix WUW_U. You say a pole sets its direct effect. Its who? The pole?
  7. Section 4: What is a bench? "A direction enters the bench"...
  8. In figure 3, authors say that in the shaded band, "each crosses zero on both classes at a sufficient dose". This seems not to be true for Qwen and Mistral. Why? (see also Figure 4).
2 comments
AuthorsOpenReview ↗

Response to Reviewer FU3U

We greatly appreciate Reviewer FU3U’s thoughtful engagement. The reviewer found the paper hard to follow due to flaws in the presentation. Nevertheless, they found it very interesting, reconstructed its arguments, examined sections, and provided valuable feedback.

The central idea of the paper is that conventional ablations (zero, mean, resample) can cause a semblance of self-repair due to pre-existing coupling interactions between the upstream core(s) (e.g., a neuron, Table 1) and downstream counterweights (e.g., neuron, singular direction, Tables 6-9). Any ablation is an uncalibrated point that this coupling interaction surfaces. This was measured by using the continuous dose axis and a nested counterfactual.

Conventional ablations are single-point deletions and are ill-equipped to capture this coupling interaction. The approximate affine law is one such concrete interaction that quantifies this receptiveness and, surprisingly, generalizes across four different models and to the IOI circuit identified by [Wang et al. 2023]. This dose-response could show up as a different curve, but the main thesis will still stand; it's not contingent on a single curve form. The affine law is a parsimonious approximation that consistently showed up, so we thought it was worth reporting.

A1: The dose axis λ\lambda is ordered by corruption fraction of upstream core(s), moving from clean (left) to counterfactual (right). Any intervention ν\nu on a core(s) neuron can be written as ν=m−λ2Δ\nu=m-\frac{\lambda}{2}\Delta, where mm and Δ\Delta are the midpoint and difference between counterfactual and clean scalar activation of the neuron.

A2: Yes, we modify the neuron's scalar activation in the upstream core; more precisely, the scalar that multiplies the fixed output column. We have defined ownr\mathrm{own}_r in the introduction now.

A3: We chose this notation to make clear that for any component c, its output could be expressed in either of the two bases (neuron directions and SVD basis), i.e., oc=∑jzcjwcj=∑kαckvcko_c= \sum_{j} z_{cj}w_{cj} = \sum_{k} \alpha_{ck}v_{ck}. Yes, ν\nu is any scalar intervention clamp on a neuron or singular value, it should be ν\nu and not vv. It was a mistake.

A4: We have added an algorithm box (Box 1, pg 5) containing a concrete example and procedure. Figure 2 can also help in encapsulating the whole procedure - there are two kinds of interventions working here:

  • (a) Primary on upstream core(s) component(s) - neuron
  • (b) Secondary on downstream component - neurons and SVD.

The dose (λ\lambda) intervention is applied to the core, and the downstream component is swapped with the activation value it will assume in the same mirror-dosed model (See Box 1, pg 5).

A5: Taken. We have revised it.

A6: The directions (ωr\omega_r) ‘pole’ is a set of tokens that are promoted or suppressed at extremes of unembedding projection ωr⊤WU\omega_r^\top W_U. Examples in Tables 6-9.

A7: Bench is the set of directions in weight space. See Tables 6-9 for all such chosen directions; we chose them based on the interpretability of the poles for the relevant task.

A8: The reviewer is right, and the caption was unclear. By sufficient dose, we meant any point on the whole tested range λ∈{+1,+0.5,0,−0.5,−1,−3,−5,−7}\lambda \in \lbrace +1, +0.5, 0, -0.5, -1, -3, -5, -7 \rbrace.

Thank you again for your detailed engagement, and your questions made the revision clear. Please let us know if we can clarify anything further.

Reference. Wang, Variengien, Conmy, Shlegeris, and Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. ICLR 2023.

Reviewer_FU3UOpenReview ↗

Thank you for your rebuttal

Dear Authors, thank you for your rebuttal. Some of my questions have been sufficiently addressed, but the revised paper still has the same weaknesses presented in my first review. Section 3 is still challenging to parse, despite the additional example and algorithm.

I will maintain my score.

Reviewer_3P42OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors of this paper study self repair -- i.e. cases where causal ablations to a language model can produce misleading results because downstream model components can adapt to the ablation. From what I can discern, the core argument is that self-repair doesn't imply that some downstream component is actually adapting to the ablation, but instead that the ablation is merely surfacing 'a gain already present before any ablation'. They try to illustrate this using some notion of a 'dose' axis, although I find the definition of the dose axis to be inscrutable.

Critical strengths
  • The figures are nicely designed! I especially enjoyed the aesthetic of figure 6
  • It's an interesting idea that I think could be pretty compelling if it was explained better, with more intuitive examples and clearer explanations.
Critical weaknesses
  • The paper engages with causal mediation but not causal abstraction. I think it would definitely be worth exploring how the author's ideas translate to concerns about causal abstraction. I think that's basically necessary for this paper.
  • The writing is impossible to follow. One example: it repeatedly refers to a 'dose' and a 'core' but I really have no idea what either of those two things means. Below I've also included a couple examples of other lines that I couldn't really understand, but there are many many more in the actual paper.
  • The authors should test out their idea on a larger set of tasks, and a larger set of models
Questions for authors
  • some phrasing eg. line 077 'every prompt has a counterfactual partner of the opposite class' is just really unclear -- what is a counterfactual partner of the opposite class? Also line 054 'compensation is a standing coefficient' this is not an insightful statement or proposal. It's just confusing.
  • More examples of the above. line 094 'Compensation is standing. The directions that self-repair already oppose the core in the clean model, so deletion releases a gain rather than recruiting one.' What do you mean by this?
  • Also like the above. line 436 'the opposition is a standing coupling in the clean model' What is the opposition?
3 comments

We are sincerely grateful to reviewer 3P42. Despite the prose being difficult to follow, the reviewer engaged with the work's essence, called the idea “interesting,” suggested it “could be pretty compelling if it was explained better,” and provided valuable feedback on points we can improve.

The core thesis is that conventional ablations (zero, mean, and resample) contribute to apparent self-repair through pre-existing coupling between an upstream core (e.g., a neuron) and a downstream counterweight (e.g., a neuron or a singular direction). This interaction could be observed using a continuous dose axis. We demonstrate that any ablation is an uncalibrated point on this dose axis that can show up as self-repair.

The core It's the upstream component that is causally significant and is coupled to downstream components. In our case of factual tasks, core(s) are neurons. See (pg. 5, Table 1).

The dose axis Consider a pair of prompts (xx and x′x'):

  • xx: “..New York is in America. This statement is:”
  • x′x': “..New York is in Asia. This statement is:”

Let the core neuron c take values aca_c and ac′a_c' on the above prompts, respectively. Define m=ac+ac′2m=\frac{a_c+a_c'}{2} and Δ=ac′−ac\Delta=a_c'-a_c. Now, any clamp value ν\nu we use to intervene in the core can be written as:

ν(λ)=m−λ2Δ\nu(\lambda)=m-\frac{\lambda}{2}\Delta

Or, in other words, any ablation clamp ν\nu on a neuron carries a dose:

λ(ν)=2(m−ν)Δ\lambda(\nu)=\frac{2(m-\nu)}{\Delta}

Revision adds a detailed example (pg. 5, Box 1).

W1 (Causal Abstraction) We interpret this as a request to frame our design in a causal abstraction framework (Geiger et al., 2021; Geiger et al., 2025). We have added a correspondence table that maps causal abstraction vocabulary to quantities we study (See App. Q, Table 36). In short, the secondary intervention we perform is: lower-level models (Models as SCM) map to a higher-level model (Eq. 6), where λ\lambda is a high-level abstraction that has predictive power in interchange ablations. This is the initial correspondence we propose, and the reviewer is right that this direction deserves further exploration.

W2 (Writing) Noted, and we apologize for the difficulty. We have made revisions to the paper; see the list of changes in the general comment. The core and dose are defined in the paragraphs above.

W3 (more tasks and models) We tested our method on two tasks with different structures:

  • Factual verification across four model families, in both instruction-tuned and base checkpoints (App. H). In base checkpoints, this compensation pattern was also present (but we didn't quantify the predictive laws).
  • Indirect Object Identification (IOI) on GPT-2 Small (App. L), a syntactic task with a bench (set of upstream core and downstream counterweights) fixed by prior work [Wang et al. 2023] (e.g., Backup Name Mover heads). The IOI core was a set of heads.

The coupling effect holds in both settings. There was an interesting exception: head 11.10 for the IOI task, which was dissolved once the correct core set was identified. Its response was a non-monotonic curve (initially decreasing, then increasing). We hypothesized that it was coupled to Head 10.7 (Negative Name Mover Head), and the response became monotonic after the inclusion of Head 10.7 (See App. L.3).

We agree that an extension to include further tasks is necessary to establish generality. However, certain things are to be noted:

  • Coupling could exist between distinct feature types (e.g., neurons and singular directions), with no constraint on what kind of feature it must be. In our case, it turned out to be in standard bases, which were semantically readable. We used a heuristic based on legible unembedding logit labels to identify components, noting that it's a lower bound.
  • The coupling could emerge in non-standard features and be dispersed throughout the model. To find such coupling across any given task, we need a discovery method to identify such components, which we leave to future work.
  • We need a task where the conceptual attributes can be separated for causal analysis. Other tasks may involve an entanglement of multiple concepts, which can make them harder to analyze.

Questions

  1. x′x' is the counterfactual partner of xx. They are statements with identical structure but different truth values. We agree that the statement is vague and doesn't convey information transparently; we have revised it. A standing coefficient is the slope γr\gamma_r, which quantifies the coupling.
  2. Noted and revised. The upstream core neuron and downstream counterweight already interact by later contributing in opposite directions to the former (in our case, through an affine equation). When ablation (deletion) occurs, we see compensation.
  3. The opposition of the upstream core (neuron) by downstream counterweights (neurons and singular directions) due to coupling that is present in the clean model.

We look forward to further discussion.

I thank the authors for their response. I maintain my score. I continue to find statements such as 'It's the upstream component that is causally significant and is coupled to downstream components' to be quite vague. I understand what each term in that statement means, but somehow the statement as a whole remains unclear. I recommend that the authors do another complete pass over their writing, and without the use of an AI. I do believe that that would likely help make the paper accessible to future human reviewers. I wish the authors all the best.

We appreciate the feedback and the opportunity to clarify the work further. In our case, the core (the upstream causally significant component) is a set of neuron(s), which, under swap intervention from the opposite class (e.g True or False), removes approx 50% of the logit margin ( pg. 5, Table 1). The core is coupled to some downstream components that scale with the core's intervention strength (i.e., increased dose).

We have also given a detailed illustration in Box 1 (pg. 5)

Reviewer_Zd85OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates the dynamics of self-repair mechanisms in language models. Instead of fully ablating the component being studied, the authors use a continuous intervention that interpolates between the clean pass and a counterfactual value. This allows them to fit a dose-response curve to intervention strength, where linear fits have some predictive power (although they are the worst of all models fitted). The significance of this to understanding self-repair is unclear.

The paper is extremely hard to read due to being LLM-written, and this is a sufficiently strong issue that I recommend rejection - this is already policy at TMLR (https://medium.com/@TmlrOrg/increasing-emphasis-on-clear-writing-in-acceptance-criteria-bfd75349f5e9) and I believe it should be adopted more broadly.

Critical strengths

Self-repair is an interesting phenomenon and some of the mechanistic analyses are of minor interest

Critical weaknesses
  1. The paper is essentially unreadable.

  2. The core claim of the paper - that a linear dose-response curve applies to self-repair mechanisms - is disproven by the paper's own results (see appendix K.2). The authors are presumably aware of this, but the text of the paper foregrounds the linear fit, which I think is misleading.

  3. The significance of the existence of a dose-response curve (whether linear or not) to self-repair and the mechanistic origins of that curve are unclear from this paper.

Questions for authors

None

1 comment
AuthorsOpenReview ↗

Response to Reviewer Zd85

We thank the reviewer for their time. The central idea of the paper is that conventional ablations (zero, mean, resample) can appear to induce self-repair due to pre-existing coupling interactions between the upstream core(s) (e.g., a neuron; Table 1) and downstream counterweights (Tables 6-9). The nature of the coupling interaction is invisible under usual ablations and becomes apparent when using a dose axis. Any ablation clamp is an uncalibrated point on this dose axis, and this uncalibrated point creates the semblance of self-repair via pre-existing coupling.

W1 The ideas, hypotheses, findings, experimental design, figure design, and conclusions were all originated entirely by the authors. AI assistance was used to rephrase some of the authors' text and compress the narrative. This was disclosed as required by the AI policy.

W2: The statement that “The core claim of the paper - that a linear dose-response curve applies to self-repair mechanisms - is disproven by the paper's own results (see Appendix K.2).” is factually and logically incorrect.

  • App. K.2 does not contain any such comparison of the curve fits. It tests whether the estimated coupling is statistically distinguishable from zero.

  • App. K.5 is the comparison of the functional form of dose response. It's the stress test we ran to check our own fit, detect curvature, and report it (as referenced in the main paper of the initial draft, lines 363-365, Sec 5). It detects it (Table 29), but the curvature is small in Table 24 and App. K.4, adding a quadratic coefficient only removes 3-9% of residual misfit, and the coupling slope γr\gamma_r just shifts by the median of 0.6%-2.5% (below its standard error).

  • The draft makes it clear that it is an approximate affine law (Introduction - Contribution 3, Conclusion) and reports R2R^2=0.65-0.91 in the main text. We claim that this affine relationship is a leading-order, reasonably robust description of the dose-response relationship. An acknowledged approximation cannot be discarded by its reported approximation error unless it affects the conclusions drawn from the approximation.

  • Most importantly, the paper's central thesis is independent of the precise functional form of the dose-response curve. The existence of dose-dependent coupling does not strictly require an exact affine form. The affine form is a parsimonious quantification of this coupling interaction within a given window (convex hull of clean and counterfactual), and it remains stable when higher-order curvature is introduced. The function already takes a visible non-linear form when the dose value ∣λ∣>1|\lambda| > 1, the receptiveness of the dose axis remains intact (Figure 3).

W3: The significance of self repair The dose-response curve establishes that the downstream component's compensation effect is a predictive function of upstream intervention. It has a pattern that varies systematically with the dose of the intervention applied to the upstream component. It lays the foundation for the idea that this interaction could be analyzed causally and quantified rather than reported as unpredictable noise.

Once the dose axis is constructed, one can see that any conventional ablation is an uncalibrated point on it (Fig 6), and the standard coupling interaction results in self-repair. The standard ablations are one point on this axis, which is insufficient to discern if compensation/self-repair is a consistent pattern or noise.

The mechanistic origin of self-repair: We define mechanism as a function that maps cause variables to effect variables in a causal model, identified among intermediate representations by causal methods (Saphra & Wiegreffe, 2024). In those terms, the paper established the following:

  • The dose-response curve reveals a coupling between upstream and downstream components (e.g., neurons), with the latter causally opposing the former. This coupling is present in clean instruction models, in their base checkpoints (App H).
  • We observe that the contribution of downstream counterweights (e.g., a neuron, a single direction) is a continuous function of the dose applied to the core. That is, dose response Er=f(λ)E_r=f(\lambda). The approximate affine law is one instance of ff that has predictive power over four model families on factual tasks. The same holds for the IOI circuit on GPT-2 (App L).
  • We identify the weight-space components/directions that carry this coupling: MLP neurons, OV channels, and singular directions of both, and catalog them (App D). We demonstrate that the magnitude of the coupling coefficient γr\gamma_r is ranked by alignment of the upstream write and downstream read vectors (Sect 6).

What remains is to express the identified function in closed form in terms of model weights, which could be explored in future.

References Saphra, Wiegreffe. Mechanistic? BlackboxNLP Workshop 2024

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions and locations of changes

We appreciate the reviewers' effort and actionable feedback. All three found the paper difficult to follow; the first revision is mainly focused on clarity and transparent communication of the idea. The compression of the narrative introduced layers of opacity in the text; we are genuinely sorry for this and will make it easier for readers in any further revisions.

Locations (revised PDF; changes in blue)

SectionPageLines
Abstract115–19
§1 Introduction1–253–54, 75–89
§1 Contributions2–395–101, 108–109
§2 Setup3123–125
§3 Dose axis, Eq. 34178–199, 215
Box 1 (new)5216–238
§4 Census5–6257–267, 288–310
Fig. 3 caption6281–285
§5 Coupling law6–7315–323, 367–377
Fig. 6 caption8395–397
§7 Deletion on the dose axis8422
§8 Related work9446–447
App. Q, Table 36 (new)653456–3488

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.