acceptodds
Withdrawn from ICLR 2027

Calibrating Causal-Localization Scores: Feature-space comparisons depend on intervention budget and write-back semantics

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Causal-localization benchmarks compare representations by intervening on selected features and measuring the resulting counterfactual behaviour, and the scores are often read as properties of the representations themselves. We argue that this reading is incomplete: the measured effect is jointly determined by the representation and by the intervention used to interrogate it. We therefore treat causal localization as an intervention-relative measurement. Instead of assigning each feature space one intrinsic score, we characterise an intervention-response profile: causal response as a function of the residual-space intervention actually realized, together with its selection, write-back and evaluation conditions. Across RAVEL, MIB's IOI setting and a synthetic model with known causal structure, this distinction is consequential. Method orderings change across effective residual-space budgets, and rank-matched random subspaces acquire substantial causal effect. Nominally comparable learned interventions can occupy residual subspaces whose ranks differ by up to in our replay of MIB's learned-mask procedure, and re-optimising at a common residual-support rank changes the ordering in both successfully optimised large-mismatch tests. Write-back semantics add a separate asymmetry: changing only the operator substantially changes an SAE's causal score while leaving orthonormal bases invariant, and swapping an entire dictionary recovers 7% to 99% of the corresponding full-state causal effect across the settings we measure. These results motivate a calibration principle: compare interventions in the residual space where they act, report the intervention alongside the score, and distinguish localization from the capacity and transmission of the measuring intervention.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, causal localization, sparse autoencoders, activation interventions

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4Ra9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work is about that causal-localization scores from interchange-intervention benchmarks (RAVEL, MIB, SAEBench) measure a representation–intervention pair, and separates the nominal intervention (swap kk features) from the realized edit in residual space. Instead of a single score, it proposes reporting an intervention-response profile: held-out cause measured across a sweep of effective residual rank r/dr/d, with the selector, write-back operator, and coverage condition stated explicitly. For dictionaries, it adds a full-swap recovery diagnostic, Rswap=Cswap/CstateR_{\text{swap}}=C_{\text{swap}}/C_{\text{state}}, the fraction of the full-state patch's causal effect that survives swapping every dictionary coordinate. On RAVEL/gemma-2-2b, IOI/GPT-2-small, and a synthetic transformer with a known causal site, it shows that (i) the leading family flips with rr (DAS at low rank, SAE at high rank); (ii) rank-matched random subspaces have a causal floor that depends on r/dr/d and answer cardinality; (iii) changing only the selector, or only the write-back operator, moves SAE cause substantially while orthonormal bases are invariant; and (iv) a replay of MIB's learned masks yields SAE and neuron supports differing in residual rank by up to 9.9×9.9\times.

Critical strengths
  1. Cross-family comparisons in RAVEL/MIB/SAEBench do equate a kk-dimensional orthonormal subspace with kk overcomplete dictionary latents, and no prior benchmark pairs a method's score with what a rank-matched random subspace achieves. The nominal/realized distinction, the rank-matched random control (an explicit analogue of probing control tasks), and the full-swap diagnostic are each cheap, well-defined, and immediately adoptable.

  2. The paper pre-registered effect-size criteria, reports which predictions failed, discards two invalid rank-matching procedures before adopting a third, and explicitly withdraws its own initial hypothesis that write-back alone explains MIB's SAE-vs-neuron ordering.

Critical weaknesses
  1. The projected donor hb+WW⊤(hs−hb)h_b + WW^\top(h_s-h_b) bypasses the encoder: it orthonormalises kk selected decoder columns and replaces the entire component of hbh_b in that span, so its realized rank is always kk. The featurizer swap moves hh by DS ΔfSD_S\,\Delta f_S, whose realized per-example rank is ∣{i∈S:Δfi≠0}∣|\{i\in S:\Delta f_i\neq 0\}|. For sparse codes far below kk on most graded pairs. The paper notes this but still presents 4.2 as a separate write-back semantics finding. By its own principle, the 8×8\times median operator gap (Table 7) and the 0.924→0.0000.924\to0.000 IOI result compare interventions of very different realized rank. Without that number, write-back semantics matter cannot be separated from the featurizer swap barely edits anything at this kk. The same operator choice means Table 2's headline (an SAE dictionary leads at rank 128) is a result about a label-selected 128-dimensional subspace of span⁡(D)\operatorname{span}(D) versus a 128-dimensional fitted rotation.

  2. The two results behind the cross-family claim (the rank-dependent ordering (Table 2, Fig. 2) and the common-rank reversal (Table 4)) are each explained as well by estimator variance or optimization failure. DAS at rank kk fits ∼d⋅k\sim d\cdot k continuous parameters (≈295k at k=128k=128) from 1024 entities, whereas the sae selector picks 128 fixed columns. The paper's own numbers show DAS is heavily overfit already at rank 8 and that its standing depends on nfitn_{\text{fit}}. A high-variance fitted subspace losing to a low-variance selected one as kk grows is the bias–variance prediction; reporting the fit-vs-held-out gap along the rank sweep, or scaling nfitn_{\text{fit}} with kk, would separate the two. The paper already has cheap lower bounds on what an axis-aligned subspace of that size can achieve but does not report them. The abstract's "changes the ordering in both successfully optimised large-mismatch tests" presents n=2n=2, after a post-hoc exclusion, as a robust finding.

Questions for authors
  1. For every featurizer-swap cell in Tables 3 and 7, please report the distribution over graded pairs of the realized per-example rank ∣{i∈S:Δfi≠0}∣|\{i\in S:\Delta f_i\neq 0\}| and the edit norm ∥h′−hb∥\|h'-h_b\|, alongside the same quantities for the projected donor. Does the operator gap survive when cause is plotted against realized per-example rank?

  2. Table 2 reports cause only. Please add isolation for each leader/runner-up pair. At rank 128 the SAE's isolation under the projected donor is 0.410 (§4.2). Does the rank-128 SAE lead survive under any isolation-constrained comparison, or does DAS lead at every rank once isolation is held fixed? (You criticise RAVEL's mean aggregate for rewarding non-intervention.)

  3. Along the rank sweep in Fig. 2 / Table 2, please report DAS's fitting-entity cause beside held-out cause, and/or re-run DAS with nfitn_{\text{fit}} scaled with kk (or with a regularized objective). Does the DAS$\to$SAE crossover move with nfitn_{\text{fit}}?

  4. For the two reversing cells in Table 4, what cause does the labelled neuron selector achieve at k=mk=m (445 and 402), and what does supervised achieve at its effective rank 127?

  5. Why was rank-matching done only downward (neuron →\to SAE's mm)?

Reviewer_hCYVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that a causal-localization score is not a property of a representation but of the (representation, intervention) pair, and that benchmarks such as RAVEL, MIB and SAEBench fold their own intervention choices into what they present as representation quality. Three distinctions carry the argument: capacity, construction and transmission. The authors report an intervention response profile instead of a scalar value. Causal response as a function of realized residual budget, paired with a rank-matched random control. Evidence comes from RAVEL city attributes in gemma-2-2b, MIB's IOI setting in GPT-2-small and a 25M synthetic transformer whose bottleneck makes localization known by construction.

Critical strengths
  • The reframing is correct and the paper keeps it bounded. It does not claim a universally preferred family, operator or rank, and it says explicitly which comparisons are limited to matching the dimension of the subspace rather than its orientation, norm or contents.
  • The orthonormal-basis invariance is set up as a correctness control rather than as a result, and it is exact: neuron agrees under both operators to the last reported digit in 110 of 110 comparisons, as does an independent random rotation. That is what makes the SAE asymmetry credible rather than a simple artifact.
  • Decodability is identical (0.776) for a random basis and for the unsupervised selector while their causal effects are 0.003 and 0.825, and the two preregistered delta-fidelity metrics reconstruct the state difference better at layer 18 than at layer 14 while layer 18 transmits roughly seven times less.
Critical weaknesses
  • The paper does not apply its own operator argument to the result that carries it. Sec. 4.2 shows that changing only the write-back takes an SAE's cause from 0.481 to 0.060 on RAVEL and from 0.924 to 0.000 on IOI token. The rank ladder behind the central claim (Fig. 2, Table 2, Sec. 3.1) never says which operator produced it, and Sec. 4.5 uses the projected donor throughout. If the operator can move a single family's score by an order of magnitude, the crossover location is as operator-dependent as it is rank-dependent. Either report the ladder under both operators, or say why the projected donor is the benchmark's operator.
  • Scope is narrow where the claims are general. Two pretrained models, one synthetic model, two benchmark families, single-layer interventions and effectively one site per benchmark. The 47.0% against 96.5% cross-setting comparison also changes model, benchmark and SAE together, as the paper notes. So the abstract's range describes the sites measured and is not established as a property of dictionaries.
  • The four families carrying the cross-family conclusions are sae, neuron, supervised (LDA) and DAS. The other rows are controls built for this paper. That is thin coverage for a general claim about family comparisons, and it makes the rank-matched random baseline the main external check. The paper handles LDA's rank cap honestly (127 rather than a requested 512), which is precisely the trap it describes.
Questions for authors
  • Can you re-run MIB's causal-variable comparison with rank-matched budgets under your protocol and report whether the ranking changes?
  • For the common-rank claim, how sensitive is the result to the chosen m? One cell left the ordering intact at an 8% reduction, two changed it at about 70%, and one failed outright at 90%. What happens at 30% or 50%, and would more seeds change any of the three outcomes?
  • Is there a hypothesis for the site effect that survives the density confound, given that layers 14 and 18 have code densities within 1% and differ 6.5x? For example, does write-back error align with the readout direction at layer 18?
Reviewer_kaKPOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper was very difficult to comprehend. I believe the bulk of it, if not all, was written using AI and never proofread even one. The key message that I was able to comprehend from the paper is the following: when using representational interventions to probe causal effects, considering only the number of features probed (in effect, the subspace rank) is insufficient.

Critical strengths

I did not find any specific strengths. The only positive thing I learned from the paper is that the rank of the subspace is not all that matters in causal abstraction studies/benchmarks, especially in overcomplete spaces. But I don't know if that's a new lesson, as I will explain in the "weaknesses" section.

Critical weaknesses
  1. The paper was very hard to understand. I believe it was fully written by AI due to some telltale signs, such as being highly defensive about the claims (lines 114, 156, 380) and inventing technical terms on the fly without defining them. Some of these technical terms are inconsistent. For example, "causal-localization," used in the abstract and the title(!!!), v.s. "interchange-intervention," used in the very first line of the introduction. While using AI to write the paper itself is not a problem, the authors should have at least proofread the manuscript. In its current writing state, the paper cannot be accepted.

  2. The primary research question/claim of the manuscript rests on the idea that it is wrong to compare causal abstraction on just the rank of the intervened representation space. This idea is captured in lines 052-053, albeit very confusingly. But this is not a real concern, to my knowledge

Questions for authors
  1. Can the authors produce any evidence in prior works that at least some people compare causal abstraction on the basis of the rank?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.