acceptodds
Under review as a conference paper at ICLR 2027

Path-Space: Estimating Transformer Component Effects and Their Interactions from Shared Backward Passes

Abstract

Which components of a transformer matter for a prediction, and which matter only in combination? Ablation answers this at one forward pass per component and one per pair, and the cheap readings used instead, the logit lens and direct-logit attribution, ignore everything downstream of the component. We present Path-Space, a method that estimates component effects and their pairwise interactions from shared backward passes. It reads each block’s write through the exact per-context transport to the output, as attribution patching does, at every block and position from one backward pass; it obtains a removal’s exact effect as the line integral of the gradient along the removal, evaluated by Gauss–Legendre quadrature; and it introduces second-order attribution patching, which predicts a component’s interaction with every other component from one backward pass on the run with that component removed, with an exact error identity and a symmetrised form. On five models from 0.36B to 70B parameters, scored against measured removals on the model’s own next-token log-probability, the per-context reading predicts removal effects at Spearman 0.69–0.82, against 0.20–0.29 for the identity reading behind the logit lens; six quadrature nodes reproduce the effects to a median error below 2 × 10−4 at every block but the first; and the symmetrised interaction estimate reaches 0.65–0.76 up to 8B and 0.56 at 70B in bf16, recovering strong interactions between individually weak components that rankings by single effects miss. Its setup costs about 4L forward passes against L 2/2 for all pairs: at 80 blocks it beats measuring pairs in the order of their single effects at equal cost from a fifth of all pairs on, a comparison we did not pre-register, while at 24–32 blocks it does not yet pay for itself

Primary area:
interpretability and explainable AI
Keywords:
LLM, mechanistic interpretability, attribution patching, logit lens, transformer language models, residual stream, ablation, causal mediation, circuit analysis, feature attribution, model editing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GGzbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates the effects of Transformer components (blocks and attention heads) on model outputs, with a particular focus on second-order interactions between components. At the block-write level, the authors use per-context gradients (one backward pass) to measure the contribution of each write to the output after propagation through subsequent layers (equivalent to attribution patching). They use 6-node Gauss–Legendre quadrature to recover the magnitude of deletion effects (equivalent to integrated gradients along the ablation path) and propose second-order attribution patching: a single backward pass on the run with component kk deleted estimates its interactions with all other components. They also provide an error identity and a symmetrized formulation. Experiments across five models (0.36B–70B parameters) use actual ablations as ground truth. The per-context readout achieves a Spearman correlation of 0.69–0.82 with deletion effects (compared with 0.20–0.29 for logit lens/DLA). The second-order estimator achieves 0.65–0.76 for models up to 8B parameters and 0.56 for the 70B model in bf16. Additional experiments investigate sign, mediation, head-level IOI, comparisons with AtP*, and nine attempts to "turn attribution into capabilities."

Critical strengths
  1. Explicitly models second-order interactions between components. It can identify component pairs whose individual effects are weak but whose joint effects are strong due to redundancy or self-repair—interactions that methods ranking components solely by their individual effects cannot discover.

  2. Relatively low computational cost. Estimating all pairwise interactions requires approximately 4L4L forward-pass equivalents, compared with L2/2L^2/2 forward passes for pairwise measurements. The advantage becomes increasingly significant as the number of components grows, such as when analyzing hundreds of attention heads.

Critical weaknesses
  1. The baselines are too weak. The primary baselines for the first-order method are logit lens, DLA, and average transport (Jacobian lens and tuned lens), none of which were designed to predict deletion effects. The authors themselves acknowledge that this "does not test what lenses were built for," making it an unfair comparison to evaluate these methods against actual deletion effects. More relevant baselines (AtP*, EAP-IG, and Hessian–vector-product corrections) are only partially evaluated: AtP* is compared on Pythia-410M using just 10 prompt pairs, yielding nearly identical rankings (0.950 vs. 0.945). No quantitative comparisons with EAP-IG or HVP corrections are provided. For the second-order method, the comparisons are limited to an additive assumption and rankings based on individual component effects, with no comparisons against other budget-constrained search methods or second-order estimators.

  2. The methodological novelty is relatively limited overall. The authors acknowledge in Appendix O that the first-order readout is simply attribution patching, path integration is equivalent to integrated gradients (EAP-IG), and response decomposition is standard causal mediation analysis. The primary novel contribution is therefore the second-order estimator, which is essentially a finite-difference approximation based on changes in gradients after ablation.

Questions for authors
  1. Compared with prior methods, is the contribution of this paper limited primarily to modeling second-order effects? What are the precise differences between the first-order readout and AtP, and between the curvature correction and IG/EAP-IG?

  2. Could the authors provide quantitative comparisons on the same metric and dataset against AtP* (including QK correction and GradDrop), EAP-IG (with varying numbers of integration steps), and the second-order HVP correction proposed by Zhang & Wang? These comparisons should report both attribution accuracy and computational cost.

  3. Could the authors report the computational overhead across models of different sizes and at the attention-head level, where there may be hundreds of components, including setup and initialization costs?

1 comment

Thank you for pressing on novelty and baselines; we answer your three questions in order.

Novelty and precise differences (W2, Q1). Methodologically, yes: the contribution is the second-order estimator. Gradient-times-write is AtP with a removal (zero) baseline, its removal-path integral is integrated gradients, and the mediation decomposition is standard; we claim none of these. What is new is that one backward pass on the component-removed run estimates that component's interaction with every other component, with an exact marginal-error identity and a symmetrised form.

The difference from EAP-IG is the path, not the principle. EAP-IG integrates along one interpolation of inputs or patched activations shared by all edges; our integral scales a single component's write from 1 to 0 with everything downstream recomputed, so it equals that component's removal effect. On that path the paper already reports a step sweep (Sec. 4.4, Fig. 9a, Qwen3-0.6B): pooled over blocks 1-6, median relative error 0.69 with one node (AtP), 0.47 with a local second-order correction (one extra gradient at s=0.1) and 0.38 with the trapezoid rule; six Gauss-Legendre nodes reach below 1e-4 over blocks 1-27, and Llama-3-8B repeats the ladder (first order 0.40, trapezoid 0.12, six nodes at most 0.003).

Relative to a second-order Taylor estimate, the finite step carries most of the gain. Against a clean-point gate-Hessian (HVP) baseline at full removal (Sec. 4.5, Table S.1, Fig. S.1), the symmetrised estimate reduces paired MAE by 0.056 [0.043, 0.069], 0.077 [0.049, 0.113] and 0.028 [0.015, 0.041] nats on pythia-410m, Qwen3-0.6B and SmolLM2-360M (40 documents each) and raises signed Spearman by 0.280, 0.279 and 0.263 (from 0.46-0.54 to 0.74-0.81); forward-only conditioning alone accounts for 0.21-0.25 of that gain. At doses up to 0.1 both are accurate, with the HVP's MAE 1.7-2.0 times ours, as a third-order expansion predicts; the ratio falls to 1.2-1.4 at full removal. This baseline is our implementation in removal-gate coordinates, not a native replication of Zhang and Wang's activation-patching HVP or multi-step HVP.

Baselines (W1, Q2). The new comparisons add a second-order estimator (the HVP) and further budget-constrained search strategies (AtP-product ordering, conditional row scans) to the additive and single-effect baselines you mention. AtP*: the existing pythia-410m comparison includes both the QK correction and GradDrop but covers only ten prompt pairs; verification counts for the true top-5/10/20 are 50/58/89 for AtP against 20/26/54 for AtP* (Table 20). We treat it as a limited comparison, not a replication. HVP and search: in a fixed pool of 128 pythia-410m heads (8,128 pairs, 40 IOI prompt families) with measured costs (Tables S.2-S.3, Fig. S.2), at 10% of the exhaustive time the symmetrised ranking recovers 0.617 of the exact top-10% interactions against 0.396 for single-effect ordering (+0.221 [0.203, 0.240]; 40/40 families), 0.380 for AtP-product ordering and 0.368/0.352 for single-/AtP-ordered row scans, with both rankings at 512 verified pairs (ceiling 0.630), i.e. at equal verification counts. The clean-point HVP ranks well (rank correlation 0.923 versus 0.972), but its longer setup delays its first batch to 10.3-10.6% of the exhaustive time, so it scores 0 at 10% and 0.825 at 20% against 0.981. EAP-IG: the step ladder above. Native expanded AtP*, input-level EAP-IG step sweeps, Zhang-Wang's activation-patching HVP/MS-HVP and Fisher-energy CoAx were not run; the paper lists them as limitations rather than presenting our adaptations as those benchmarks. At block level, measuring all pairs exactly costs only 2.6-3.2 estimator runs (Table S.1), so we claim accuracy per evaluation, not a speed-up.

Overhead (Q3). Per document, one estimator run costs 3.61, 6.48 and 5.63 s on pythia-410m, Qwen3-0.6B and SmolLM2-360M (HVP 3.85/6.75/5.88 s; exact all pairs 9.54/18.94/18.23 s). At head level the median setup per family is 1.62 s for the symmetrised estimate and 3.05 s for the HVP, 3.3% and 6.3% of the median exhaustive time of 48.5 s. Loading model and tokenizer takes 1.2-1.4 s per head worker and 1.2-2.2 s for the block models; the 83.5-84.3 s benchmark warm-up includes one full exact reference over all 8,128 pairs (48.7-49.2 s) plus ten policy warm-ups; cached singles cost 0.77 s per family (Appendix S.4). These are FP64 timings of our batched implementation on one GPU type, not generic cold-start latency.

All new measurements are FP64 runs on the same saved weights, with no new 8B or 70B runs; the head result is conditional on its restricted pool and the IOI template and name generator.

Reviewer_qksKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper contributes a method for estimating transformer component effects and their pairwise interactions from shared backward passes, and attempts to convert the findings into capability gains; those attempts close negative, but are reported informatively.

The paper (i) benchmarks three readings of a block's residual-stream write against measured ablation on the model's own next-token log-probability, across 5 models ; (ii) notes the removal effect is exactly a line integral of the gradient ; (iii) introduces second-order attribution patching; and (iv) reports nine pre-registered attempts to convert the object into capability, all negative or trades.

Critical strengths

The negative capability results are informative: redundancy among weak components is many-body not pairwise, sensitivity ≠ editability because the most sensitive cells are the most curved, and the per-token route lives in the backward pass.

The methodological hygiene is great in the context of a mechanistic-interpretability submission: pre-registered pass/fail criteria with a committed design document, shuffle/permutation nulls throughout, document-clustered intervals, a ledger of every experiment (Appendix R), an explicit table of artefacts the nulls caught, and criteria reported as failed when they failed.

Critical weaknesses

The headline comparison invites a stronger conclusion than it supports. The logit lens and DLA were not built to predict finite ablations, and Section 4.1 says so in its last sentence. A 0.69–0.82 vs 0.20–0.29 gap presented in the abstract therefore risks being read as refuting the lenses rather than as measuring one specific thing they do not do. This should be stated earlier in the paper.

The cost claim doesn't land at relevant depth. The pre-registered criterion fails at every depth, setup alone is 10.2% of pairs at 80 blocks, and the one favourable comparison (from 20% of pairs) is explicitly flagged as not pre-registered, yet it is the comparison the abstract leans on.

Questions for authors

how does symmetrised Î compare empirically against conditional co-ablation at matched cost on the same pairs?

at head granularity the cost share falls to 1–2% of pairs; can you run the fixed-budget comparison there, where you expect it to pay off, rather than only at block level?

1 comment

Thank you for the precise reading of the headline and the cost claim; both points changed the paper.

Lens framing (W1). We agree, and the abstract and introduction now say so up front: the lens/DLA comparison measures prediction of finite removals, not the representational or distributional objectives those methods were designed for. The numbers remain, without the implication that they refute lenses. First-order attribution and path integration are credited to AtP and integrated gradients, and our component-removal path is distinguished from EAP-IG's intervention path.

Cost claim (W2, Q2). The abstract no longer relies on the 80-block comparison: it states that the pre-registered efficiency criterion fails at every tested depth and reports the measured head-level result instead, while Sec. 4.6 keeps the 80-block comparison as exploratory (400 sampled pairs per position, costs normalised to all 3,160 pairs), not as a speed-up. Two further points on cost: the 10.2% setup share at 80 blocks charges the single effects twice, and the corrected nominal share is 7.7% (Sec. 4.6); and at 24-32 blocks measuring all pairs exactly costs only 2.6-3.2 estimator runs (Table S.1), so we claim no block-level saving.

At head granularity, where we expected the method to pay off, we ran the fixed-budget comparison you suggested: pythia-410m, an effect-independent pool of 128 of its 384 attention-head outputs at the final position, all 8,128 pairs measured exactly, and 40 IOI prompt families. Each policy is charged its measured setup and credited only for completed 256-pair batches (Table S.2, Fig. S.2); our setup is 3.3% of the measured exhaustive time T_full for 128 heads (the paper's 1-2% was a nominal figure for several hundred heads). At the pre-specified 10% budget the symmetrised ranking recovers 0.617 of the exact top-10% interactions against 0.396 for single-effect ordering (paired difference 0.221 [0.203, 0.240]; positive in 40/40 families) and 0.380 for AtP-product ordering. Both have then verified 512 pairs (recall ceiling 0.630), so the contrast is at equal verification counts, and per verified pair ours leads in 40/40 families at every 256-pair step up to 4,096. At the other pre-specified budgets (5, 20, 30 and 50% of T_full) the symmetrised ranking recalls 0, 0.981, 0.991 and 0.997 against 0.245, 0.622, 0.766 and 0.880: below 10% single-effect ordering, with its shorter setup, completes its first batch earlier, and from 10% on the symmetrised estimate is never behind. The clean-point HVP's 0 at 10% is first-batch latency, not a ranking failure (first batch at 10.3-10.6% of T_full; rank correlation 0.923 versus 0.972, Table S.3); it reaches 0.825 at 20% against 0.981.

Conditional co-ablation (Q1). For the same scalar target, co-ablating a pair gives its exact signed interaction, and with one conditioning component the conditional effect change is the pair interaction itself. A matched-cost comparison on the same pairs is therefore a comparison of acquisition strategies: measuring whole conditional rows versus individually ranked pairs. Our scalar row scans condition on heads in order of measured single effect or AtP score and measure every pair in each row; at 10% of T_full they recover 0.368 and 0.352 of the exact top-10% interactions, against 0.617 for the symmetrised ranking (Table S.2). CoAx's native score instead measures growth in Fisher-weighted logit-displacement energy, and its revised versions also compare two-state attribution contrasts; we therefore present row scans as adapted comparators, not as a native CoAx benchmark, and do not treat differently defined scores as interchangeable.

Boundaries. All new measurements are FP64 runs on the same saved weights, with no new 8B or 70B runs. Native Fisher-energy CoAx at matched cost, input-level EAP-IG step sweeps, expanded AtP* and Zhang-Wang's HVP/MS-HVP were not run and are listed as limitations. The head result is conditional on its restricted pool and the IOI template and name generator.

Reviewer_TxouOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper evaluates how well gradient-based component attribution and linear lenses predict the effects of zero-ablating transformer components. It compares per-context Attribution Patching with Direct Logit Attribution (DLA) across five models ranging from 0.36B to 70B parameters, with learned and averaged lens comparisons on the small models. Its main methodological contribution is an estimator of pairwise ablation interactions, I^(k,j)\hat I(k,j), that uses O(L)O(L) backward passes to estimate all (L2)\binom{L}{2} interactions. The paper also evaluates numerical integration along ablation paths and explores downstream applications, most of which fail their pre-registered criteria or offer limited advantages over simpler baselines.

I lean marginally below the acceptance threshold. The interaction estimates and careful empirical controls are strengths, but my main concerns are the incomplete demonstration of a practical efficiency advantage and the absence of a clean-point second-order comparison.

Critical strengths

Careful experimental controls and transparent reporting. It reports pre-registered criteria, shuffle nulls, and numerical precision checks, including an investigation of how bf16 affects measured interactions. Its candid reporting of negative results helps establish the limits of the methods.

Useful pairwise interaction estimates. Adding I^(k,j)\hat I(k,j) to measured single-component ablation effects reduces the median error in predicting joint two-component ablations by approximately 51–60% on the models up to 8B. Estimated interactions correlate with measured interactions at Spearman 0.65–0.76 on blocks and 0.70–0.91 on heads. These results support the estimator as a tool for identifying pairwise interactions.

Critical weaknesses
  1. The practical computational advantage is incompletely demonstrated. The estimator fails its pre-registered compute-efficiency criterion: at 24–32 blocks, its setup costs approximately 26–36% of measuring all pairs directly, and cost-matched search trails ranking by measured single-component effects below a budget of half the pairs. An exploratory comparison at 70B shows an advantage from a 20% budget, but no cost-matched evaluation establishes the benefit at attention-head granularity, where the method should be most useful. This materially limits the evidence for the paper’s central promise of efficient interaction estimation.

  2. A clean-point second-order baseline is missing. The estimator compares first-order attributions before and after actually removing a component. Comparing it with Hessian-vector products with respect to component-ablation gates at the unablated model would clarify the accuracy and computational benefit of this finite intervention over a second-order Taylor approximation. Without that comparison, the results establish improvement over additive predictions but leave an important alternative unexplored, weakening the evidence for the proposed estimator’s methodological advantage.

Questions for authors
  1. How does the interaction estimator compare with a clean-point Hessian-vector-product baseline in accuracy and computational cost?
  2. Can a cost-matched interaction-search evaluation at attention-head granularity establish an advantage over ordering pairs by measured single-component effects?
1 comment

Thank you for asking for the two comparisons that decide the estimator's practical value. We ran both, and both favour the estimator: at full removal it is more accurate than a clean-point HVP in all three small models (signed Spearman 0.74-0.81 versus 0.46-0.54), and at head level, with measured costs, it recovers 0.617 versus 0.396 of the exact top-10% interactions (+0.221 [0.203, 0.240]; positive in all 40 IOI prompt families). Both are reported in the new Appendix S (Tables S.1-S.3, Figs. S.1-S.2).

Clean-point HVP (W2, Q1). We implemented the requested baseline: the mixed Hessian with respect to component-removal gates at the unablated model, computed with Hessian-vector products. The exact interaction integrates this mixed derivative over the unit gate square; the HVP evaluates it at the intact corner, our finite estimate integrates it along the edge from the intact model to the k-removed model, and the symmetrised form averages the two edges at that corner. On pythia-410m, Qwen3-0.6B and SmolLM2-360M (40 documents each, all block pairs, five doses), the symmetrised estimate at full removal lowers MAE by 0.056 [0.043, 0.069], 0.077 [0.049, 0.113] and 0.028 [0.015, 0.041] nats and raises signed Spearman by 0.280 [0.256, 0.303], 0.279 [0.255, 0.303] and 0.263 [0.246, 0.281]. Most of this comes from the finite step: forward-only conditioning already adds 0.21-0.25 (in all 40 documents of each model), and symmetrisation adds the rest and secures the MAE gain in SmolLM2, where the forward estimate alone is level with the HVP (0.1538 versus 0.1530). At doses up to 0.1 both are accurate (signed Spearman at least 0.98), with the HVP's MAE 1.7-2.0 times ours, as a third-order expansion predicts; the absolute gap grows with the removed fraction while the ratio falls to 1.2-1.4 at full removal (Fig. S.1). Per dose the two cost about the same (HVP/finite 3.85/3.61, 6.75/6.48 and 5.88/5.63 s per document); the HVP is cheaper only when one Hessian is reused across many doses. Numerical checks in Appendix S.3 confirm the block comparisons and the head verifier against recomputed truth.

Head-level matched budget (W1, Q2). Yes, for budgets from 10% of the measured exhaustive time upward, in the one setting we tested. On pythia-410m we fixed an effect-independent pool of 128 of the 384 attention-head outputs at the final position and measured all 8,128 pair interactions exactly in 40 IOI prompt families. Every policy pays its measured setup and earns credit only for completed 256-pair verification batches within a fraction of the measured exhaustive time T_full. At the pre-specified 10% budget the symmetrised ranking recovers 0.617 of the exact top-10% interactions against 0.396 for single-effect ordering (paired difference 0.221 [0.203, 0.240]; positive in 40/40 families), 0.380 for AtP-product ordering and 0.368/0.352 for single-/AtP-ordered row scans (Table S.2, Fig. S.2). Both rankings have then verified 512 pairs (recall ceiling 0.630), so the gap is ranking quality at equal verification counts; per verified pair the symmetrised ordering leads in 40/40 families at every 256-pair step up to 4,096 pairs. Its setup is 3.3% of T_full against 1.7% for single effects, so below 10% single-effect ordering completes its first batch earlier and is ahead or level at all but one 1% step; from 10% on, the symmetrised estimate is never behind. The clean-point HVP scores 0 at 10% because its first batch completes at 10.3-10.6% of T_full, not because it ranks poorly (rank correlation 0.923 versus 0.972, Table S.3); at 20% it reaches 0.825 against 0.981, and the symmetrised estimate is never behind it at any budget. Of the top-10% interactions whose two single effects are both below the median, the symmetrised ranking finds 41% at 10% and single-effect ordering none. With cached singles and no free gradients the paired difference is 0.123 [0.103, 0.143].

Scope. At block level the pre-registered efficiency criterion still fails at every tested depth. The 26-36% setup share you cite counts the single effects twice, but the corrected nominal share of 20-27% (Sec. 4.6) is still too large to pay off at low budgets, and at 24-32 blocks measuring all pairs exactly costs only 2.6-3.2 estimator runs (Table S.1), so we claim accuracy per evaluation there, not a saving; the 80-block comparison remains exploratory (400 sampled pairs per position, nominal costs). The gate-Hessian baseline is our implementation in removal-gate coordinates, not a native replication of Zhang and Wang's activation-patching HVP or multi-step HVP, and the row scans are scalar adaptations, not native Fisher-energy CoAx; those native protocols and input-level EAP-IG step sweeps were not run and are listed as limitations. All new runs are FP64 on the same saved weights, with no new 8B or 70B runs; the block result covers three small models, and the head result is conditional on its restricted pool and the IOI template and name generator.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.