acceptodds
Under review as a conference paper at ICLR 2027

Finding Subspace Circuits with Subspace Attribution Patching

Abstract

Edge attribution patching (EAP) has been widely adopted as a tool for finding circuits — small, task-specific computational subgraphs — within language models. A growing body of evidence suggests that the model components within a circuit communicate via low-rank subspaces of the residual stream. However, while circuit localization is well-studied, the study of the subspaces used by a circuit has received little attention. This work proposes subspace attribution patching (SAP), a finer-grained formulation of EAP, which finds circuit edges jointly with the causally-relevant subspace for each edge. We rigorously evaluate a range of methods for constructing subspace bases for SAP, relying on spectral decompositions of model weights and activations. Furthermore, we demonstrate that several plausible SAP approaches can perform well while not being faithful to the model's computations, falling prey to the Interpretability Illusion of Makelov et al. (2024). We conclude by discussing a method to diagnose this illusion. Our findings enable researchers 1) to jointly localize circuit edges and low-rank information subspaces for a given task, and 2) to detect illusory results, paving the way for more reliable research in this area.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, subspace patching, attribution, circuits, patching, faithfulness, language model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CLEuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Subspace Attribution Patching (SAP), which refines component-level EAP by projecting messages onto custom orthonormal basis axes in the residual stream. The authors evaluate various basis construction methods, reveal that top-performing bases can suffer from the "Interpretability Illusion", and propose a conditional variance metric (NormCondVar) alongside qualitative analyses to diagnose these illusions.

Critical strengths
  1. Exposing the Interpretability Illusion: The paper does a great job demonstrating that finer-grained/subspace attribution methods can easily hallucinate causal effects through cancelling directions, cautioning the community against blind faith in standard faithfulness metrics.

  2. Extensive Empirical Evaluation: The authors evaluate their methods across a diverse set of tasks and benchmarks (e.g., IOI, Greater-Than, Gender-Bias), accompanied by thorough statistical testing.

Critical weaknesses
  1. The constructed "basis axes" are not intrinsic, persistent properties of the model. Because they shift depending on the specific heuristic, task, or data split, claiming that these axes carry standalone "semantics" feels quite forced—there is no stable mapping between a mathematical axis and a real semantic concept. Furthermore, these axes neither correspond to specific physical weights/neurons nor offer monosemantic, sparse representations like SAE features. In that sense, the paper’s primary take-home message seems to be showing how artificial subspace splitting leads to interpretability illusions, rather than establishing basis axes as valid, fundamental computational units.

  2. Compared to sparse feature circuit discovery (e.g., SAE- or Transcoder-based circuits), SAP is less fine-grained and offers much less justifiable interpretability. It is difficult to see SAP standing on its own as an appealing new circuit discovery paradigm. The authors should reconsider the core positioning of the paper—perhaps framing it primarily as a diagnostic study of subspace illusions—and include a head-to-head comparison and deeper discussion against genuinely fine-grained sparse circuit methods.

Questions for authors

none

3 comments
AuthorsOpenReview ↗

Response to Reviewer CLEu

We thank the reviewer for their time in reviewing our paper and for their critique. We hope to have addressed their concerns below.

On basis axes as properties of the model

We agree with the reviewer that several bases which we evaluate are not intrinsic properties of the model. These are the data-derived bases Eig, SVD, RGrad, WMsg, NDM and NDMAx. We note that an SAE or a Transcoder basis is also not an intrinsic property of the model, and is similarly data-derived. However, we argue that the RWght and WWght bases are intrinsic, being constructed from an SVD of model weights. In this way (as Reviewer ApgJ points out), our work connects to prior work on SVD-based circuit analysis, which finds that singular vectors of model weights are highly interpretable with respect to the function of a given model component. We refer the reviewer to our discussion with Reviewer ApgJ for more details.

It is also true that the axes found shift between scoring methods, tasks and data splits; this is simply because the circuit which we are trying to find is task-specific. In our view, all subspace circuit localization methods would share this property. Finally, as we discuss in the paper, it is optimistic to expect the subspace axes for a particular edge to be aligned to the weight or neuron basis (please see: Arora et al. 2018, Elhage et al. 2022, Mueller et al. 2026, as cited in our paper). SAEs are better in this respect, but please see below.

On comparing SAP with SAE-based approaches

We thank the reviewer for raising this point, which was also raised by Reviewer ksrg. Our formulation of SAP has a number of advantages with respect to SAE-based circuit discovery methods.

  1. A basis of SAE features does not (in general) form an admissible basis for an attention head writer at a given layer. Thus, an SAE basis is not guaranteed to produce patches which lie in the writer's column space, and is likely not to, as it spans the output space of all the attention heads in a given layer together. This could be overcome by training an SAE on the outputs of individual attention heads separately; however, this is expensive and not commonly done. Our bases are constrained to be a complete basis of the column space of each writer.

  2. An SAE basis is not a complete orthonormal basis of the residual stream (it is overcomplete). This means that the scores of the basis axes separately will not add up to the score of the edge as a whole, due to the non-orthogonality of the axes. SAP ensures this property by construction, capturing the notion that (estimates of) their separate causal effects add up to the (estimated) causal effect of the whole.

To get around these problems, Marks et al. (2025) and other approaches dispense with the notion of edges between specific model components (attention heads and MLPs) and trace circuits between SAE features as the fundamental unit of analysis. This approach, however, ignores the fact that individual attention heads as such play important functional roles within many circuits.

  1. Many subspaces used within a circuit are multidimensional, a fact which cannot be captured by tracing circuits with respect to an SAE basis. As discussed in Engels et al. (2025), SAE directions necessarily split multidimensional feature spaces (e.g., a 2D circle subspace for the days of the week) into a sum of one-dimensional features coding for the most common clusters within the space (e.g., each day separately). This is a property intrinsic to the SAE training objective. The bases we define have no such constraint, which allows us to find such multidimensional subspaces. We illustrate one such edge in Figure 1.

  2. SAE-based circuit localization approaches frequently make use of "error terms" that store residual activation which could not be decomposed using the SAE basis. This means that a potentially large part of the edge score is carried by uninterpretable "dark matter" (in the parlance of Engels et al., 2024) rather than being decomposed into interpretable axes. Marks et al. find that such error terms are crucial parts of the circuit, as removing them from the circuit drastically reduces circuit faithfulness on both Pythia-70M and Gemma2-2B. An error term is an integral part of an SAE decomposition; our approach decomposes the entire edge score using a complete basis by construction, and has no error term.

We have added a discussion of these points to Appendix C of the amended manuscript. We hope that this alleviates the reviewer's concerns regarding the advantages of SAP over SAE-based approaches.

Area_Chair_PDYhOpenReview ↗

Please respond to the authors

Hello!

Just a friendly bump that you should engage with the authors response

Dear Reviewer CLEu,

Thank you again for reviewing our paper and for your thoughtful and constructive feedback. We have posted our responses to your comments along with a revised manuscript, and would greatly appreciate any further thoughts you may have before the discussion period ends.

If our responses have addressed your main concerns, we would be grateful if you would consider revising your rating upward. If any concerns remain, please let us know and we would be happy to provide further clarification.

Best regards,
The Authors

Reviewer_ApgJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a subspace-based framework for mech interp that aims to characterize information flow between components of transformer language models at a finer granularity than conventional edge-level attribution. The main contribution is the explicit integration of low-rank subspace analysis with causal attribution and activation patching, providing a potentially useful way to investigate the relationship between representational geometry and functional computation/role, complementing existing approaches based on edge-level attribution.

Critical strengths
  • The paper modifies/changes the object of analysis from edges/nodes to subspaces, making the subspace an explicit object of analysis and providing a way to study the dimensional structure of information flow in GPT2. The paper introduces subspace activation patching (SActP) and subspace attribution patching (SAP) to investigate which portions of a component's residual-stream message are relevant to downstream computation, opening up new questions of interp over subspaces.

  • The paper connects the hypothesis that Transformer components communicate through low-dimensional subspaces with intervention-based methods for studying computational circuits, and if validated empirically, this connection could improve the state of interp and provide a more compact characterization of causal interactions.

Critical weaknesses
  • The distinction between finding the causally relevant or only behavioral influence is not completely established, and more investigation would be required on that end; Makelov et al. explicitly demonstrate this problem for subspace activation patching, where a successful intervention can activate a dormant parallel pathway rather than reveal the mechanism naturally responsible for the behavior. Overall, the paper leaves room for future investigations, as highlighted in the paper, but the current version seems a little unfinished and does not provide a clear finding to rely on.

  • The paper assumes, or tries to solve, the problem of finding causally relevant information transmitted along a computational graph edge that can be captured by a low-rank subspace, but the criteria for identifying these subspaces and selecting their ranks are insufficiently justified. In particular, the relationship between the proposed subspace activation patching method and existing approaches based on SVD or other low-rank decompositions remains unclear, missing significant improvements in the field in the last 2 years. It would be good to consider findings from recent work that emphasize interpretable structures obtained via singular value decomposition (Merullo et al., 2024; Singh et al., 2024; Ahmad et al., 2025; Franco et al., 2026; Xue & Andrzejak, 2026). The information flow between nodes is well studied by Merullo et al. (2024), with Ahmad et al. (2025) identifying circuits on an SVD basis. It would be good to either clarify the distinction or present an empirical comparison to support the claim of improved circuit localization. Without stronger empirical validation, it is difficult to determine whether the identified subspaces reflect genuine causal mechanisms or primarily capture correlations and artifacts of the chosen decomposition.

  • The claimed approximation of SAP may require independent validation and may be good to report in the paper. In general, it would be good to consider a held-out sample and to show that those scores reliably approximate the actual SActP intervention effects. The current version of the paper directly follows works on EAP, and risks establishing the properties of an attribution approximation rather than those of the causal method it is intended to approximate. This additional set of experiments would add additional validation.

[1] Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching. Aleksandar Makelov, Georg Lange, Atticus Geiger, Neel Nanda. (ICLR 2024)

[2] Talking heads: Understanding inter-layer communication in transformer language models, J Merullo, C Eickhoff, E Pavlick, NeurIPS, 2024.

[3] What needs to go right for an induction head? a mechanistic study of in-context learning circuits and their formation AK Singh, T Moskovitz, F Hill, SCY Chan, AM Saxena.

[4] Beyond components: Singular vector-based interpretability of transformer circuits, A Ahmad, A Joshi, A Modi, NeurIPS 2025

[5] Singular Vectors of Attention Heads Align with Features G Franco, C Loughridge, M Crovella, 2026.

[6] SVD as a Fast Interpretability Method for Transformers M Xue, A Andrzejak, ICML 2026

Questions for authors
  • Since the paper uses a popular MIB benchmark, I wonder why not consider replicating the entire set of experiments, with multiple models of different scales, that would add great value to the paper if the proposed approach of SAP works better at circuit finding.

Minor comments:

  • The Huang & Hahn (2026) citation is not coming through correctly; the hyperlink is missing
  • The link for Arora et al., 2018 is coming out to be wrong in the following lines in section 3.2: “In general, we cannot expect this basis to be aligned with the neuron basis (Arora et al., 2018; Elhage et al., 2022; Geiger et al., 2024).”
  • Not sure what the word parsimony means: “Furthermore, we would like Suv to be of minimal rank, to maximize circuit parsimony and interpretability.” Is it faithfulness combined with localization/sparsity, or only sparsity?
6 comments
AuthorsOpenReview ↗

Response to Reviewer ApgJ (part 1)

We sincerely thank Reviewer ApgJ for their insightful critique and references to prior work. We hope to have addressed their concerns and questions below.

On causal relevance vs. merely behavioural influence

We agree that empirically illustrating the Interpretability Illusion with respect to SAP (e.g., on the axes found by the Eig basis) in the style of the Makelov et al. would be valuable. However, as we point out in Footnote 2 (line 213), our basis construction method already rules out the straightforward case of Makelov et al.'s Interpretability Illusion. Specifically, constraining all bases' axes to the writer's output space, we remove the ability to patch a provably "causally disconnected" direction, i.e., one which is in the nullspace of the writer. The patch also cannot be "disconnected" by being in the nullspace of all downstream readers, given that we are patching in the residual stream.

Therefore, compared to Makelov et al., our version of the Interpretability Illusion is "soft": it arises when distribution of writer messages is far enough away from the distribution of reader gradients, such that a projection onto a basis axis (e.g., one which lies halfway between the two distributions, as in Eig) is too far from the natural axes of variance in either distribution. Such an axis is then an unfaithful description of the natural operation of the model, and we call such a basis axis "illusory". Because of the soft nature of the Illusion in this case, we cannot illustrate it as straightforwardly as Makelov et al. Instead, we attempt to diagnose it in Section 4.3 by exploiting the fact that the variance of scores across the axes must sum to the total variance of the edge score, which is constant across all possible bases. Therefore, any method which generates such an illusory axis will also generate a complementary axis that will cancel the variance along the first. As we explain in the paper, the Eig basis (which manufactures illusory axes by construction) scores lowest on the diagnostic metric. We hope that this result can serve as proxy to an explicit illustration of the Interpretability Illusion in SAP.

On demonstrating that SAP reliably approximates SActP

We agree with the reviewer that it would be good to demonstrate that the ranking of bases by SAP matches the ranking by SActP. We did not do this in the paper because running SActP on all the methods listed would be prohibitively computationally expensive. Additionally, as the reviewer has pointed out, the validity of SAP approximating SActP follows directly from the fact that EAP approximates EActP, which is well-established in the literature and which we recap in our derivation in Appendix B.

On Questions For Authors

We agree with the reviewer that it would be good to validate the proposed approach on more models and datasets, and to compare it with the existing MIB results. To this end, we have added results on Qwen2.5-0.5B on the following tasks: IOI, MCQA, ARC-E, ARC-C. We also evaluate Qwen on SVA, which is not in MIB, but allows for further comparison with our GPT-2 results. The conclusions which we make on GPT2 also hold on Qwen and on the new datasets. However, we still do not replicate the full suite of MIB experiments on Llama-3.1 8B and Gemma-2 2B due to compute constraints. We hope that this alleviates the reviewer's concern.

AuthorsOpenReview ↗

Response to Reviewer ApgJ (part 2)

On comparisons with existing SVD circuit localization methods

We thank the reviewer for pointing us to prior work. We have not conducted comparisons with the provided works because none of them are directly comparable in their capacity as circuit localization methods. However, several of these works are related and reinforce our approach, and can inspire valuable future extensions of SAP. We address each comparison in detail below.

With respect to Talking Heads (Merullo et al., 2024), there a number of core differences in methodology. Merullo et al. identify potential circuit edges u→vu \to v as the dot product between the singular value vectors of uu and vv. Here, u,vu, v are QK and OV matrices of attention heads, rather than individual WQ,WK,WV,WMLP,inW_Q,W_K,W_V,W_{MLP,in} as readers and WO,WMLP,outW_O,W_{MLP,out} as writers as in our framework. This formulation does not allow the authors to find circuits involving MLPs, as MLPs are nonlinear and cannot be factorized as the SVD of WMLP,inWMLP,out⊤W_{MLP,in} W_{MLP,out}^\top. Additionally, having found a potential subspace mediating edge u→vu \to v using composition scores, they do not validate its causal importance using activation patching; instead, they do a form of node-level zero-ablation by zeroing out singular values of uu and observing behavioural changes in the vv of interest. This means that most of the subspaces which they identify for a given edge are not empirically validated and cannot claim to reflect "genuine causal mechanisms"; in contrast, the circuits we find via SAP are validated using SActP to measure faithfulness values, and have a stronger claim in this respect.

Beyond Components (Ahmad et al. 2025) do not consider edge-based circuits at all. Instead, they find subspaces within the output space of a given head or MLP by taking an SVD of its weight matrix, and learning a sparse binary mask which masks out causally-irrelevant singular values. In doing so, they find that different singular vectors of a given model component mediate different functions within the model. However, they do not find the subspaces which carry the causally-important information for a given edge. In contrast, we do so with SAP. For example, in Figure 1 we illustrate that the edge between S-Inhibition Head 7.9 and the query of Name Mover 9.9 encodes both the name and the position of the subject in separate subspaces.

Franco et al. (2026) and Xue and Andrzejak (2026) are valuable concurrent work which we were unaware of. Similarly to the above works, Franco et al. also analyze the singular vectors of attention head weights and find that they tend to align to features of interest. In so doing, they do not consider edges, or even indeed circuits at all. However, their results may shed light on the reason that SAP is able to find such compact circuits, often with only a few axes per attention-head edge. In a similar vein, Xue and Andrzejak (2026) conduct an in-depth investigation of the subspaces used by the MLP sublayers of Transformers. To do this, they rely on SVD of the MLP weights. They also find interpretable low-rank structure within the weights, and trace the effect of every low-rank component on a given model prediction. In doing so, they also do not consider edge-based circuits or the subspaces which span them. Furthermore, they note that their analysis "is strictly confined to MLP layers," and leave the extension of their framework to attention heads to future work. Given the large body of evidence underscoring the importance of attention heads for circuits, Xue and Andrzejak (2026) is largely parallel to our work.

Finally, Singh et al. (2024) appears to the authors to be an investigation into the training dynamics of induction head formation in Transformers. While it does not appear to be directly relevant to the present work, it would be an interesting future direction to explore using SAP. We would be glad to hear the reviewer's thoughts on the connection between Singh et al. and the current work.

To sum up, the prior work suggested by the reviewer does not directly compare with the current work, in that they do not concern circuits which are composed of edges between model components. However, they do all deal with low-rank informational structure of language models. In this way, these approaches may inform the design and analysis of further SAP bases. In fact, we tried an approach inspired by Talking Heads in early testing (using an SVD of the product of the writer and reader matrices), but this approach was deemed out of scope as it does not produce a basis that is orthonormal in the residual stream. In our work, we did not aim to present an exhaustive list of all possible bases, but rather to present the SAP methodology and offer a starting point for future work.

We thank the reviewer again for their time and valuable critique. We have uploaded a revised version containing the changes above as an OpenReview revision. Please see our Official Comment for more details.

Area_Chair_PDYhOpenReview ↗

Please respond to the authors

Hello!

Just a friendly bump that you should engage with the authors response

Reviewer_ksrgOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies subspaces in circuit localization. Existing work either localizes edges (EAP and its variants) or learns subspaces for known causal variables (DAS and related methods); the two have not been combined. The authors propose subspace attribution patching (SAP), which projects each edge's message onto a low-dimensional subspace before computing the attribution score, thereby recovering the edge and the subspace it uses at the same time. They first formalize the notion of an admissible basis, namely that the patch must lie in the column space of the writer's output weights, and then show that any complete orthonormal basis splits the edge score into a sum of per-axis scores. All axes can therefore be scored in parallel, and the unit of a circuit moves from an edge to an edge plus a rank-one axis. They compare reader-side, writer-side, weight-derived, data-derived, CrossEig, CrossSVD and NDM bases on six tasks with GPT-2 Small, using the CPR and CMD metrics from MIB. The central finding is that the most natural basis, Eig, performs best on the unadjusted metrics, but the authors argue this is an interpretability illusion and introduce NormCondVar, a conditional-variance diagnostic. After weighting by this quantity the ranking reverses: NDM and RGrad are best and Eig is worst. The paper closes with a qualitative analysis of 131 edges on IOI, showing that on a single edge the top two axes encode the subject's position and the remaining axes encode its name.

Critical strengths
  • The paper turns subspace-level circuit localization into a well-defined and computationally feasible problem. The admissibility constraint together with the additivity of axis scores means that all axes can be scored in one parallel pass without combinatorial search, and the circuit unit refines naturally from an edge to an edge plus an axis. The framework is clean and can be attached directly to existing EAP pipelines.
  • The analysis of the Eigen illusion and NormCondVar are valuable contributions. Symmetrization blends the message and gradient directions, so the eigenvectors necessarily bisect them, and a zero-effect edge is split into a positive and a negative axis; keeping only one of them in the top-k selection manufactures a causal effect out of nothing. This two-dimensional counterexample is elegant and correct. NormCondVar uses a Schur complement to measure the residual variance of the kept axes given the dropped ones; it is well defined, bounded, cheap, and reusable as a diagnostic.
Critical weaknesses
  • The adjusted ranking is circular, and this directly affects the paper's main positive conclusion. NDM is trained to minimize total correlation between subspaces, and NormCondVar measures precisely the correlation between kept and dropped axes. In Table 4, NDM scores 0.65 to 0.90 on NormCondVar, while Eig scores 0.10 to 0.22 under signed (CPR-style) selection and 0.28 to 0.61 under absolute-value selection, so once this quantity is used as a weight, NDM naturally ranks first. The paper's own tables contain an internal control: NDMAx uses the same NDM rotation but retains individual axes instead of blocks, and its NormCondVar immediately drops to 0.11 to 0.30 (signed) and 0.20 to 0.38 (absolute), with the adjusted ranking dropping accordingly. If the metric really measured whether an axis is used by the model, changing only the unit of retention should not produce such a large gap. The conclusion that NDM and RGrad are better therefore cannot currently be separated from a preference built into the diagnostic itself.
  • Several claims exceed the evidence. First, the text states that Eig ranks first or second on both CPR and CMD across all datasets, but in Table 3, on IOI Eig's CPR (1.58) is numerically fifth and its CMD (0.501) sixth among the SAP bases, and on GB its CMD is fourth; unless these gaps are statistical ties, the statement does not hold. Second, on STok SAP scores 5.71, random subspaces score 3.22, and no intervention scores 5.39, so patching is worse than doing nothing. This conflicts with the claim of finding the causally relevant subspace for each edge, and the abstract does not mention it. Third, the abstract says several plausible bases fall prey to the illusion, but only Eig is analyzed in detail. Fourth, the closest prior work, Sparse Feature Circuits, already performs feature-level circuit localization over SAE features, and it is not used as a baseline, so the novelty claim of jointly finding edges and subspaces lacks a direct comparison.
Questions for authors
  1. Can you add Sparse Feature Circuits, or any SAE feature circuit, as a baseline? Please also state what per-edge admissible bases buy over a global SAE dictionary in substance, rather than only as an implementation difference.
  2. Can you add a control showing that NormCondVar is not simply rewarding NDM's objective? For example, use random rotations with the same block structure, the NDM rotation before training, or a basis that was not trained with a correlation objective. How do you explain the gap between NDM (0.65 to 0.90) and NDMAx (0.11 to 0.38) in Table 4?
  3. The text says Eig ranks first or second across all datasets, but on IOI it is numerically fifth (CPR) and sixth (CMD) in Table 3. Are these ties under your significance test? If not, please correct the statement and explain why Eig is not ahead even before adjustment on the largest dataset.

Minor (does not affect my assessment): the main text says only complete orthonormal bases of the residual stream are considered, whereas Definition 1 requires every axis to lie in the writer's column space; please reconcile the two. The notation is dense, and the appendix contains a broken reference ("??" in the caption of Table 5).

5 comments
AuthorsOpenReview ↗

Response to Reviewer ksrg (part 1)

We sincerely thank Reviewer ksrg for their detailed reading and constructive criticism. We hope to have addressed their concerns and questions below.

On the circularity of the adjusted ranking.

Note that NormCondVar measures the extent to which, for a given edge with picked subspace KK and unpicked subspace UU, the variance in UU is predictive of that in KK. Thus it measures the redundancy between the set of picked and unpicked axes in a given edge, not "whether an axis is used by the model" as suggested. Furthermore, NDMAx differs from NDM in that the former does not respect the subspace blocks which NDM forms by merging irreducibly-correlated axes into one subspace. Therefore, it is expected that NDMAx will score much lower on NormCondVar, because many sets of such irreducibly-correlated axes will end up split between KK and UU.

We do however agree that NormCondVar can privilege NDM in some sense. As pointed out, NDM rotations are trained to minimize total correlation between subspaces, and NormCondVar is driven by the covariance between the picked and unpicked subspaces of an edge. Thus, NDM naturally scores highly on NormCondVar. Having said that, we don't consider this a problem, because the justification for NDM in Huang and Hahn (2026) largely agrees with our justification for NormCondVar in that subspaces which encode separate features should be decorrelated. Indeed, one axis of variance being split into two illusory axes in our formulation can be considered as two extremely statistically-dependent features in the Huang and Hahn framework, which NDM will merge into one block. In this way, NDM implements a proxy for NormCondVar and rightly scores highly. RGrad (and the other bases), however, do not optimize for NormCondVar in any way, which makes it interesting that RGrad comes in second to NDM.

On claims exceeding evidence.

We address the reviewer's comments in the order that they are presented.

  1. We thank the reviewer for pointing out the imprecision in our wording. Although the Eig basis is first or second on almost all datasets, there is substantial variance in the rankings of the methods across datasets and Eig is neither first nor second on several of them. This effect is statistically significant. However, in the CMD-scoring regime, it is expected that Eig may not rank first. As we point out in Section 4.2, in the presence of illusory axes, selecting axes by top-k |score| may select both of the mutually cancelling high-scoring axes, because each has a high-magnitude score. If the Eig basis forms a large number of illusory axes on a given dataset, it therefore "wastes" a lot of the circuit budget on selecting subspaces which have no causal effect. Thus, in the CMD regime, this yields a circuit with a lower faithfulness score.

    We do not have a definitive explanation for why Eig is not the highest-scoring basis on IOI for the CPR setting. One thing to note is that SAP is a linear approximation to SActP, which is what drives the CPR and CMD scores. Eig is guaranteed to capture the most SAP score in the fewest axes; however, there is no guarantee that they will also capture the most realized SActP effect. In a certain sense, the illusory axes manufactured by Eig are especially pathological with respect to the causal mechanisms in the model itself, in that they do not correspond to the natural directions of variance in writer messages, reader gradients, or the singular directions of weights (which frequently correspond to important data features; see Ahmad et al. 2025, Franco et al. 2026). Therefore it is possible that patching such axes will in some cases cause OOD activations, which may reduce model performance under nonlinear downstream interactions between components. We have added a discussion of this effect to Appendix D.1 and referenced it in the corresponding Results section.

AuthorsOpenReview ↗

Response to Reviewer ksrg, part 2

  1. We thank the reviewer in pointing out the confusion with respect to the interchange intervention scores on StokS_{tok}. To clarify, the MSE when interchanging StokS_{tok} is higher when patching SAP axes than under No Intervention due to the fact that the SAP axes are not trained to isolate StokS_{tok} vs SposS_{pos}. Thus, the subspaces found by SAP encode both variables, sometimes even on one edge (as we illustrate in Figure 1). Therefore, when performing an interchange intervention on SAP axes, we transfer both position and name signal from the counterfactual prompt, no matter what the prompt is. However, the high-level causal model (as well as DAS and DBM) can interchange StokS_{tok} while keeping SposS_{pos} fixed at its value in the clean prompt, and vice versa. Therefore it expects low MSE on interchanging StokS_{tok} on the IO$\leftrightarrow$S1 dataset, which flips position only. In this cases, No Intervention does the correct thing by not changing the logit difference. However, SAP does not isolate StokS_{tok} vs SposS_{pos}. Therefore, performing this interchange on SAP axes causes high MSE.

In the paper, we followed MIB's convention of averaging the MSE across the three counterfactual datasets. This was the cause of the confusion. The following table illustrates this effect:

methoddimsS_PosS_PosS_PosS_PosS_TokS_TokS_TokS_Tok
IO↔S1IO↔S2IO↔bothmeanIO↔S1IO↔S2IO↔bothmean
DAS1282.202.162.212.191.751.723.272.25
DBM+PCA2.452.102.272.272.101.893.212.40
Full Vector3.131.982.222.442.521.764.212.83
No intervention023.1222.622.8316.192.796.496.885.39
SAP473.752.372.042.728.394.404.355.71
Random4714.9011.312.339.512.262.225.193.22
SAP, top two85.014.522.494.016.222.096.404.90
Random820.7319.762.7314.412.495.206.674.78
SAP, remaining3917.6012.492.1910.762.292.504.653.15
Random3918.2314.292.3511.632.213.035.313.52

Note that when interchanging SposS_{pos} on IO$\leftrightarrow$S1 and StokS_{tok} on IO$\leftrightarrow$both (i.e., the datasets which flip just the interchanged variables), SAP outperforms both Random and No Intervention by wide margins. The apparent discrepancy on StokS_{tok} is driven by the high MSE on IO$\leftrightarrow$S1, where SAP also transfers the position signal and the causal model expects it to stay at the baseline value.

Note that interchanging StokS_{tok} on IO$\leftrightarrow$S1 and IO$\leftrightarrow$S2 on Random subspaces also has the same effect of also inadvertently transferring the position signal. However, a random subspace does a poorer job than SAP of localizing the effect of the variable to a few axes. Therefore, it transfers less of both signals than SAP. This is beneficial in this case. When interchanging StokS_{tok} on IO$\leftrightarrow$S1, the target logit difference given by the MIB causal model is the same as baseline. Therefore, transferring less signal gets Random closer to the target. When interchanging StokS_{tok} on IO$\leftrightarrow$S2, transferring some name signal (correctly) and some position signal (incorrectly) may land Random nearer to the target logit difference than SAP, which likely "overshoots" the target by transferring most of both signals.

We have added the per-dataset breakdown table to the amended manuscript, subject to the RGrad correction (please see the Official Comment). We have also added further discussion of this effect in the main text and Appendix I. The table above is for the RWght circuit analyzed in the submission version; we present both for completeness' sake.

  1. We thank the reviewer for pointing this out. Although no basis achieves perfect NormCondVar score, only the Eig basis directly optimizes for the Interpretability Illusion. Therefore it is the one we analyze in detail. We will correct the manuscript to say this more precisely.
AuthorsOpenReview ↗

Response to Reviewer ksrg (part 3)

  1. We do not compare against SAE-based circuit discovery methods as it is would not be an apples-to-apples comparison. This is largely because a basis of SAE features does not (in general) form an admissible basis for an attention head writer at a given layer. Specifically, an SAE basis is not guaranteed to produce patches which lie in the writer's column space, and is likely not to, as it spans the output space of all the attention heads together. Furthermore, an SAE basis is not a complete orthonormal basis of the residual stream (it is overcomplete). This setting complicates the math, as:
  1. for a rank-64 attention head writer, an orthogonal projector Πuv\mathbf{\Pi}_{uv} onto 64 or more SAE feature directions will trivially patch the entire message, dropping nothing in Equation 2, and

  2. one cannot score directions using a projection through Πuv=Suv⊤Suv\mathbf{\Pi} _ {uv}=\mathbf{S} _ {uv}^\top\mathbf{S} _ {uv}, because the axes si\mathbf{s}_i are not orthogonal and therefore the scores of the axes will not add up to the score of the edge.

Finally, SAE-based circuit localization approaches frequently make use of "error terms" that store residual activation which could not be decomposed using the SAE basis. This means that a potentially large part of the edge score is allocated to the "dark matter" of the SAE error term (in the parlance of Engels et al., 2024) rather than being decomposed into interpretable axes. In fact, Marks et al. find that such error terms are crucial parts of the circuit, as removing them from the circuit drastically reduces circuit faithfulness on both Pythia-70M and Gemma2-2B. An error term is an integral part of an SAE decomposition; however, it is not clear how to integrate this approach with ours, which decomposes the entire edge score using a complete basis by construction.

Therefore the manuscript leaves the exploration of non-orthonormal bases for future work.

In this way, the advantage of per-edge admissible bases over global SAE dictionaries is three-fold:

  1. the restriction of the basis to the column space of the edge's writer ensures that the result of patching any subset of the basis axes is a message which the writer could have emitted in a normal forward pass. This prevents a basic form of Makelov et al.'s Interpretability Illusion, in which a patch which lies outside the writer's column space activates a dormant pathway and manufactures illusory causal effects;

  2. admissible bases ensure that the scores of the axes add up to the score of the edge, correctly capturing the notion that (estimates of) their separate causal effects add up to the (estimated) causal effect of the whole;

  3. As discussed in Engels et al. (2025), SAE directions frequently split multidimensional feature spaces (e.g., a 2D circle subspace for the days of the week) into a sum of one-dimensional features coding for the most common clusters within the space (e.g., each day separately). This is a property intrinsic to the SAE training objective. The bases we define have no such constraint.

We have added a section discussing the connection to SAE-based circuit localization approaches to the Appendix.

We hope we have answered the reviewer's Questions for Authors in the discussion above. We have also addressed the reviewer's minor comment as: "In this work, we only consider Suv∗\mathbf{S}^*_{uv} which are admissible and orthonormal in the residual stream, and leave exploration of other bases to future work."

We thank the reviewer again for their time in reading and reviewing our manuscript. Their insightful comments have allowed us to improve the paper, making our results clearer and more precise. We have uploaded a revised version containing the changes above as an OpenReview revision. Please see our Official Comment for more details.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of changes in the revised manuscript

We sincerely thank all three reviewers for their feedback. We uploaded a revised manuscript: new text is marked in blue, corrections are red, content-preserving clarifications are green. Corrections specific to Point 5 are in violet. All additions and corrections continue to support the main findings and conclusions.

1. New model and tasks [Reviewer ApgJ]

To ease comparison with existing MIB results, we have added results for Qwen2.5-0.5B on the following MIB circuit tasks: IOI, MCQA, ARC-Easy and ARC-Challenge. We also run it on SVA as an added comparison with GPT-2 Small.

2. Comparison with SAE-based circuit discovery methods [Reviewers ksrg, CLEu]

We have added an Appendix section clarifying the distinction between SAP and SAE-based circuit discovery approaches. We have added a discussion on the main points preventing direct comparison, as well as the advantages of SAP over SAE-based approaches.

3. More precise claims [Reviewers ksrg, ApgJ]

  • Section 4.2 now states our findings more precisely
  • We have added an Appendix section explaining why the Eig basis does not rank first on all datasets.

4. Causal variable localization [Reviewers ksrg]

We have clarified our approach to the causal variable localization track in Section 4.1. New results in Appendix I break the scores down by counterfactual dataset. They show that SAP's high average error on interchanging STokS_{\text{Tok}} comes from the two datasets that also flip the subject's position, because SAP's axes carry both signals.

5. Correction to the Section 4.1 analysis

While preparing the revision, we found an error in Section 4.1 of the submitted version. The circuit analyzed there came from the RWght basis, not from RGrad as the text stated. This affects Figure 1, Tables 5 and 7, and the heatmaps in Appendix J. We reran all of these analyses on the RGrad circuit at the same 0.1% budget. This circuit has 4,020 axes on 1,446 edges, and we analyze 180 of its edges instead of 131. On the edge from head 7.9 to the query of head 9.9, the top two axes separate the subject's position and the other six separate its name. In Table 7, SAP now uses 55 dimensions. On SPosS_{\text{Pos}}, it scores an MSE of 3.68, against 8.46 for random subspaces, 16.19 for no intervention and 2.19 for DAS. On STokS_{\text{Tok}}, it scores 4.36, against 2.98 for random subspaces. All qualitative findings and conclusions continue to hold.

6. Recomputed GPT-2 results

We have fixed a minor implementation detail to bring the RGrad basis more in line with the other bases. When computing gradients, RGrad now takes gradients only from token positions at and after the first position where the clean and counterfactual prompts differ. Earlier positions have a zero message difference, so they do not change edge score. This brings this basis in line with the other bases, which are also restricted to the same set of token positions that affect the score. This detail slightly changes the metric numbers for RGrad. All qualitative findings still hold and most method rankings are unaffected. The full list of rerankings is:

  • On CPRw\text{CPR}_w, GT dataset: RGrad is now tied for first, SVD is bumped to second, WMsg third.
  • On CMDw\text{CMD}_w, GB dataset: WMsg is no longer tied for first and is bumped to second. Eig is bumped to third, NDM to 4th.
  • On CMDw\text{CMD}_w, IOI dataset: RGrad is no longer tied with RWght and drops to third. Eig is bumped to 4th.
  • On CPR\text{CPR} , GT dataset: RGrad rises to third, WWght bumped to 4th.
  • On signed NormCondVar\text{NormCondVar}, IOI dataset: RGrad rises to second, WMsg third, SVD bumped to 4th.
  • On absolute-value NormCondVar\text{NormCondVar}, GT dataset: RGrad rises to second, SVD bumped to third.
  • On absolute-value NormCondVar\text{NormCondVar}, IOI dataset: RGrad rises to second, WMsg remains third, SVD bumped to fourth.

Finally, minor edits for clarity have been applied throughout the paper.

We sincerely thank all revieweres for their time, and we are glad to answer further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.