acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Circuit Discovery with Learnable Corruptions

Abstract

Patching-based circuit discovery has a hidden degree of freedom: the corruption that defines the behavioral contrast. Existing methods fix this corruption and optimize only the sparse circuit, even though a patching score is not a property of a circuit alone, but of a circuit under a chosen behavioral contrast. We make this dependence explicit by recasting circuit discovery as a joint optimization over sparse circuits and reference-anchored admissible corruptions. In our reformulated problem, the benchmark reference is retained as an anchor for constructing semantically admissible and behaviorally disruptive counterfactual families, with fixed-corruption discovery recovered as a boundary case. We instantiate this formulation with SEED (**S**tructure **E**dit and **E**dge **D**iscovery), a tractable alternating procedure that updates the corruption coordinate under the current circuit and updates a shared sparse circuit under the resulting contrasts. Across eight circuit-discovery tasks, under both denoising (sufficiency) and noising (necessity) discovery objectives and with both Faithfulness and Completeness measured on held-out data, learning corruptions improves over optimizing the circuit alone, although the gains are task-dependent. Yet the corruptions selected by SEED do not transfer uniformly to non-SEED methods, indicating that they behave as circuit-coupled best responses rather than universally better corruptions. These results suggest that circuit discovery should be understood as a coupled problem over circuits and behavioral contrasts.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, language model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_dny5OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper addresses a fundamental yet often overlooked assumption in patching-based circuit discovery: that the baseline corruption is fixed a priori. To tackle this, this paper introduce SEED, an alternating optimization method that iteratively picks counterfactual tokens and updates circuit edges via a straight-through surrogate. Experiments on 8 tasks show that SEED yields improvements in held-out faithfulness and completeness compared to fixed-corruption controls.

Critical strengths
  1. Questioning the fixed-corruption convention in activation patching is timely and thought-provoking.
  2. The paper covers 8 standard benchmarks across a wide range of budgets, evaluates both sufficiency (denoising) and necessity (noising), and sensibly tests everything on held-out benchmark corruptions to avoid trivial metric cheating.
Critical weaknesses
  1. While the paper claims to rethink fixed corruptions, the contribuytion is still extremely constrained. The algorithm still requires human-designed reference templates, manually pre-selected editable slots (κ\kappa), and token-length matching. The most difficult parts of creating corrupted text—deciding how to intervene, and fixed token counts—are still solved by humans. In reality, SEED doesn't escape the fixed-corruption paradigm; it merely tunes a single token inside an already human-curated counterfactual slot.

  2. In tasks like IOI, different corrupted text designs serve distinct causal purposes (e.g., change which name is direct and indirect object without changing the names or positions, like "John and Mary ... John" to "John and Mary ... Mary", is for the value and position of the indirect object). These fixed contrasts are intentionally crafted to isolate specific sub-mechanisms. Simply picking tokens that maximize a generic behavioral drop (Δ\Delta) risks collapsing these fine-grained causal inquiries.

  3. In mechanistic interpretability, we generally view a "circuit" as an intrinsic property of the model's weights—the actual computational subgraph implementing a capability. If a circuit is real, it shouldn't be fragile to the specific optimization path. However, RQ3 shows that using SEED’s learned corruptions on other algorithms (like EAP / EAP-IG) often leads to negative transfer. The authors frame this as corruptions being "circuit-coupled best responses", but to me, this raises a red flag: does SEED actually uncover the model's true underlying mechanism, or is it just finding an ad-hoc subnetwork overfitted to a co-adapting prompt contrast?

  4. The manuscript has a clear tendency to using AI tools in math and method derivation:

Theorem 3.2 is essentially a restatement of subset maximization. Theorem 4.1: Framing a simple scoring and ranking over ≤50\le 50 discrete candidate tokens as a variational optimization feels like massive overkill. In practice, it's just a softmax score. Eq. 16 vs. Eq. 27: Introducing an abstract "recovery-aligned shaping utility" Φed\Phi_{ed} in the main text, only to reveal in the appendix that it is simply a smooth function.

Stripping away this artificial complexity would make the paper much more transparent and enjoyable to read.

Questions for authors
  1. Following RQ3, if a discovered circuit cannot be reliably replicated under alternative optimizers using the same contrasts, why should we consider it the model's "true" mechanism rather than an artifact of SEED’s joint trajectory?

  2. How would SEED scale if the editable coordinate κ\kappa is not a single token, but spans multiple tokens or disjoint sites across a long-context prompt or larger size model?

  3. In Table 18, SEED incurs a ~1.46x wall-time overhead compared to SEED-EDGE (327s vs. 223s), why SEED-EDGE is faster than SEED?

Reviewer_w4eFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SEED, which alternates between selecting constrained corruptions and optimizing a sparse circuit in a frozen language model. Experiments cover eight tasks on GPT-2 Small, with additional results on Qwen3-0.6B. Evaluation uses fixed reference corruptions. SEED improves average faithfulness over a matched fixed-corruption baseline, although gains vary across tasks.

Critical strengths

The SEED-EDGE control and shared test corruptions make the main comparison useful. In particular, SEED cannot improve its reported test score simply by choosing a more favorable evaluation corruption.

Critical weaknesses
  1. Limited novelty. The contribution appears more incremental than the framing suggests. SEED combines corruption selection with sparse circuit optimization using standard optimization tools. Theorem 3.2 follows from enlarging the feasible set and provides no guarantee for the proposed algorithm under fixed test corruptions. The theoretical result therefore adds limited support to the methodological contribution.

  2. The value of adaptive selection remains unclear. The comparison with fixed corruptions does not separate circuit-dependent selection from simply using more varied training corruptions. The candidate-pool ablations do not resolve this distinction. Without a matched non-adaptive baseline, the evidence for the proposed coupling is incomplete.

Questions for authors
  1. Section 2 explains how SEED differs from Optimal Ablation in what it optimizes. What is the main practical or interpretive benefit of choosing input corruptions rather than replacement activations, and which results support that benefit?

  2. How does SEED compare with sampling corruptions from the same admissible pool according to the proposal prior, without circuit-dependent reweighting? Keeping the disruptiveness constraint and circuit optimizer unchanged would help distinguish adaptive selection from simply using more varied training corruptions.

  3. RQ3 shows that the selected corruptions do not transfer consistently to EAP and EAP-IG. Within SEED itself, does continued corruption updating outperform fixing each example’s corruption at an intermediate checkpoint and continuing circuit optimization? This would more directly test the need for continued coupling.

  4. In RQ1, Negation is described as having more diverse corruption contrasts, while RQ2 groups it with tasks having more constrained corruption families. Could you clarify this distinction and the evidence linking corruption diversity to the gains from SEED?

Reviewer_P1hpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents a new circuit discovery method SEED for language models. The paper argues that circuits implicitly depend on the contrasting example. The core idea is to jointly optimize the circuit and the contrasting example, rather than fixing it in advance. In addition, the authors also propose a new optimizer for circuit discovery.

Critical strengths

The paper controls for the effect of the optimizer. SEED-Edge isolates the effects of the optimizer, without optimizing the contrasting samples. SEED adds only the optimized samples. This is consequential because it allows to separate the effects of the proposed optimizer and the effects of the corruption optimization on that optimizer.

Evaluation on held-out data under the fixed benchmark corruptions. The paper evaluates the method on standard benchmarks, in standard settings, and reports both sufficiency and necessity. This matters because a method that picks its own corruptions could otherwise trivially inflate the scores. Evaluation on standard benchmarks and metrics prevents this.

Critical weaknesses

The method doesn't address the problem the paper motivates, and the evaluation cannot establish it. The paper argues that circuits shouldn't depend on a "hidden degree of freedom: the corruption". But the the circuit patching literature, the corruption explicitly defines which variable is explained, and carefully selected. SEED doesn't remove this dependency. It merely replaces a researcher-specified one with an optimizer-selected one that makes the current circuit look good. Furthermore, all results are only reported wrt. the original, predetermined benchmark corruption. This means the method and evaluation only establishes a much narrower claim: selecting alternative corruptions for circuit discovery slightly improves numbers on the standard fixed-corruption objective. That's a very different claim from "rethinking circuit discovery", and the paper's main conceptual selling point remains unsupported.

The experimental gains are narrow, and the headline results overstate them. Under the denoising objective, SEED improves on only 3 out of 8 tasks in Faithfulness over SEED-Edge, and while average task gains are positive, medians are near zero: Faithfullness (D -0.008, N+0.001) and Completeness (D +0.001, N +0.0005). The positive averages are mostly driven by a few tasks. Which tasks benefit changes between evaluation protocols, and the paper's explanation that richer admissible families mean larger gains ist contradicted by Table 7. Against the baselines SEED (D) is the most faithful method only on 2 out of 8 tasks. This means the abstracts claim of improvement “under both objectives and both metrics” holds only on task average. The evidence for a general effect is weak.

The gains cannot be attributed to the proposed mechanism. The paper introduces both a new circuit optimization mechanism, and the contrast sample optimization. While the optimization is ablated without the contrast optimization, the inverse is not the case. It is unclear if the measured improvement is optimizer-specific, or if the paper's claim of generality holds and improvements can also be measured for other baseline circuit discovery methods. Furthermore, it is unclear why the modified circuit optimizer is introduced in the first place, as it is neither part of the paper's main claims, nor does it consistently improve circuit discovery.

Questions for authors

See weaknesses. Points that would materially affect my assessment:

  • Null tests for key results and claims. The reported gains are inconsistent, mostly small, and confounded. A clearly separable measurement with a null hypothesis and significance tests would materially strengthen the conclusions.
  • Structure Edit with another method. Combine the corruption-picking step with the other baseline circuit discovery methods.
  • Run SEED-Edge with corruptions sampled from the MLM prior without circuit conditioning, to test whether the gains come from circuit-dependent selection or simply from training on more varied corruptions.
  • Evaluate the circuit over the entire admissible family. The paper's motivation is that circuits shouldn't depend on arbitrary corruptions. Yet the current evaluation is limited to that exact corruption. It should evaluate the average, best, and worst-case over all permissible corruptions.
  • An experiment should establish that the variance RQ1 is actually due to the choice of corruption. Circuit discovery is known to be sensitive to general noise and parameter choices. A null could e.g. be date resampling, template variation, or changing unrelated parts of the prompt.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.