acceptodds
Under review as a conference paper at ICLR 2027

Full-Path Gradients for Mechanistic Data Attribution in Language Models

Abstract

Mechanistic data attribution (MDA) links interpretable units in language models to influential training documents, enabling data-based interventions on a chosen mechanism. Existing methods often restrict attribution to selected parameters. The derivative paths required to predict the resulting update remain underexplored. Later attention heads and multilayer perceptrons transmit changes in a target head's output even when their weights are fixed. We show how these downstream derivatives affect predictions of head-only updates. Under local smoothness and a fixed update direction, full-path prediction has error quadratic in the step size, whereas omitting downstream derivatives can introduce a first-order error. A selective-head backward pass matching released MDA code preserves forward values while blocking derivatives through most downstream attention heads. To compare these paths under a common update construction, we introduce Full-Path Target-Response Matching (FP-TRM), which weights document losses so that one descent step is predicted to increase the target head's contribution. We define this contribution as the observed next-token logit drop under target-head ablation, with a fixed upstream head ablated in both evaluations. On Pythia-1B, full-path predictions correlate with measured contribution changes at 0.99 versus 0.35 for selective predictions at one target, where the two paths also select nearly disjoint document sets. Across two targets, full-path gradients yield larger mean contribution changes on held-out text in every evaluated condition. FP-TRM also produces positive mean contribution changes at the evaluated targets in Pythia, OLMo-2 and Qwen3.5 models spanning 31M to 4B parameters. These results show that selecting data to modify a mechanism benefits from retaining the downstream computations that transmit its effects.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
mechanistic interpretability, training data attribution, data selection, gradient matching, attention heads, large language models, causal intervention

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4xnkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper makes a distinction between restricting which parameters get updated and restricting gradient propagation through downstream layers. It proposes FP-TRM, which selects and weights documents so as to increase an attention head’s ablation-defined contribution using a small head-only update. Experiments compares full and selective differentiation on Pythia-1B and evaluates the full method on Pythia, OLMo-2, and Qwen3.5.

Critical strengths

The matched path experiment isolate a meaningful implementation issue. For identical deployed updates, full-path predictions correlate with the measured responses at 0.9904 and 0.9999, compared to 0.3495 and 0.7334 for selective predictions at the two heads tested.

The intervention comparisons controls update magnitude and uses held-out text. Full-path actions beat selective-path actions in all ten target–corpus–route conditions, with positive multiplicity-adjusted lower bounds.

The paper is transparent about its limitations, including losses against OMP and GradSim, comparison counts that are conditional rather then independent, and cases where increasing head contribution actually makes output prediction worse.

Critical weaknesses

The methodological novelty is limited compared to how broad the presentation is. Proposition 1 follows directly from the standard Taylor remainder bound, and FP-TRM combines full differentiation with existing gradient matching. The strongest isolated finding is about one particular selective-backward implementation at two heads in a single Pythia-1B checkpoint; the other architectures dont repeat that matched path comparison.

The behavioral significance of the objective is still unclear. Contribution is measured while an upstream carrier head is forcibly ablated, so increasing it does not neccessarily improve the intact model’s computation. In the Qwen replay experiment, FP-TRM drops from third to eighteenth out of 19 actions at the higher dose, and none of the six reported output-gain lower bounds are positive (Tables 8–9). This limits the shown usefulness beyond optimizing the chosen internal score.

The broad baseline comparisons change several factors at once. MDA uses an unablated query and equal positive document weights, while FP-TRM uses carrier-conditioned fitting, optimized weights, and possibly negative coefficients (Section 5.1; Appendix G.4). These comparisons can’t isolate the benefit of full-path differentiation. In the more closely matched OMP comparison, FP-TRM gets three wins, three ties, and four losses in mean response on Pythia-1B (Table 3).

Questions for authors

Does the full-path advantage hold up across more independently selected heads and architectures under the same matched backward-path experiment?

Does optimizing carrier-conditioned contribution improve an independently defined mechanism behavior when the carrier is left intact?

How much of the advantage remains when MDA and FP-TRM share the same query context, coefficient constraints, and support-search procedure?

1 comment

Thank you for your detailed comments. To test response prediction and behavioral transfer, we evaluated six additional heads under MDA-style controls and measured copying with the carrier intact. Full-path derivatives predict the same updates more accurately at every head, while the copying results establish transfer of many contribution-based actions to intact-model behavior.

Q1: Does the matched result extend to more heads and architectures?

Full-path prediction is more accurate at all six added Pythia-14M and 31M heads. The matched derivative evidence spans these Pythia models, while OLMo-2 and Qwen test the complete intervention across architectures. We retained each model's original MDA head coordinate and chose two further heads by prefix-matching scores on 64 independent sequences, before measuring path differences or outcomes. Both derivatives predict the same 16 fixed head-only updates at relative norm 3×10−43\times10^{-4}.

Full-path correlations exceed 0.999 at all six heads, compared with 0.858–0.965 for selective differentiation. Full-path RMSE is also 99.1%–99.6% lower. The added heads include H0, H1, H3, and H6 in layers 3 and 4, broadening the controlled finding beyond two H4 targets.

Q3: What remains when query, coefficients, and support search are matched?

Full-path differentiation still reduces response-prediction error and changes document selection when the MDA query, coefficient constraints, and search rule are matched. Both paths use the released MDA probe, shared QKVO EK-FAC geometry, 256 candidate documents, and identical query contexts, positive equal weights, and top-ten selection rules. Top-ten overlap is 30%–90% across heads, alongside the prediction improvements above. Associated actions always deploy ordinary full-path document-loss gradients, so the attribution path changes document selection while preserving the update definition.

Full-path FP-TRM also yields larger contribution increases in all ten matched Pythia-1B conditions. Both paths share the contribution query, candidate pool, coefficient-family rule, support-search procedure, and update norm. All ten target–corpus–route contrasts have positive multiplicity-adjusted bounds. These matched comparisons isolate differentiation. The broader adapted-MDA comparisons instead assess the complete action constructions, including their different queries and weighting rules.

OMP remains competitive as a support solver under the corrected gradient. With matched target and geometry, Pythia-1B gives three higher means, three identical actions, and four lower means for FP-TRM. Thus the derivative finding supports using the correct response gradient with gradient matching, including OMP as a support solver.

Q2: Does contribution optimization improve behavior with the carrier intact?

Carrier-conditioned actions improve intact-copy log probability in 11 of 12 tests, with simultaneous 95% confidence intervals above zero. We evaluated six heads with both attribution paths, using equal positive top-ten actions at the common dose above. The carrier remained intact during all behavior measurements. Evaluation used 256 held-out copy sequences and 128 natural-text examples. Eleven of twelve no-carrier fitting actions also improve copying, and all 24 natural-text mean losses decrease slightly.

Contribution prediction measures the targeted internal intervention, while intact-copy gains measure its behavioral consequence. Across the twelve full-versus-selective copying comparisons, full is higher in two, selective is higher in eight, and two are unresolved under the simultaneous intervals. For example, at 14M L3H3, full-path selection produces a larger internal-contribution increase, while selective selection produces a larger intact-copy gain. Thus the new experiment establishes behavioral transfer for many actions and directly measures which objective each comparison supports.

The Qwen replay similarly shows that contribution gain is an objective-specific effect. Its higher-dose contribution increase accompanies a negative output change and rank 18/19, while the primary-dose output rank is 3/19. None of its six output-gain lower bounds is positive. Accordingly, output improvement is established by independent output measurements, and the contribution score answers the specific conditional mechanism question.

What is the methodological contribution beyond Taylor expansion?

Our contribution is to identify and measure the response error caused by selective differentiation under a fixed intervention. Proposition 1 expresses that error as an omitted first-order projection plus the standard second-order remainder. FP-TRM then uses established gradient matching to turn the complete response gradient into document-based actions. The matched experiments demonstrate the practical consequence of this distinction while holding the intervention and fitting choices fixed.

Reviewer_sfqhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how downstream gradient truncation affects document-based updates to a selected attention head. It identifies a selective-backward implementation in released MDA code and introduces FP-TRM, which matches weighted document gradients to a head-contribution objective. Controlled experiments show that full-path gradients improve response prediction and the resulting contribution changes.

Critical strengths
  • The implementation issue in MDA is real and concrete.
  • The main comparison is carefully controlled, and the full-path updates outperform selective controls in all tested conditions.
Critical weaknesses
  • The conceptual and theoretical novelty is limited.
    • Parameter freezing does not require gradient truncation;
    • Proposition 1 follows directly from a standard Taylor expansion.
    • FP-TRM largely applies existing gradient matching to a different target.
  • The closest baseline, OMP using the same target and gradient geometry, is competitive and sometimes even better. The proposed support search does not show a consistent benefit in the experiments.
  • The contribution objective does not have a clear, useful mechanistic interpretation.
Questions for authors
  • Why do we need a carrier head? What if we don't search for and close a carrier head?
2 comments

Thank you for your constructive review. We tested path correction with fixed updates and assessed the carrier's role by fitting actions with and without carrier ablation. These comparisons show that full-path derivatives improve response prediction, while intact-copy gains in both fitting contexts establish that the carrier is an optional choice of mechanism context.

What is new beyond parameter freezing, Taylor expansion, and gradient matching?

Our contribution is to isolate the first-order response error caused by truncating derivatives despite a complete forward computation, and measure its effect on attribution. Frozen later weights still transform a changed target-head output, so predicting a target-only update requires differentiation through those transformations. Selective backpropagation preserves forward values while omitting some derivative paths, which changes predicted responses and selected documents.

Specifically, Proposition 1 identifies the first-order error for the same deployed action. The full derivative predicts the response up to a standard second-order Taylor remainder. The selective derivative adds the projection of the omitted gradient onto that action, which is first order in update size. The value of this expression is that it separates path error from finite-step error and leads to a direct controlled test. On the two Pythia-1B targets, full-path predictions correlate with measured responses at 0.990 and above 0.999, compared with 0.350 and 0.733 for selective predictions.

Full-path predictions are also more accurate at all six added Pythia-14M and 31M heads. We retained each original MDA head coordinate and selected two more per model on independent prefix-matching sequences. With the released MDA probe and identical updates, full-path correlations exceed 0.999 at all heads versus 0.858–0.965 for selective differentiation. Full-path RMSE is also lower throughout. Thus the contribution is a reproducible attribution and intervention-prediction effect, with established gradient matching providing the action-construction tool.

What does the competitive OMP baseline imply?

OMP remains a competitive support solver, while the full-path advantage persists when the support-search procedure is held fixed. Under the same Pythia-1B target and geometry, FP-TRM has three higher means, three identical actions, and four lower means across ten comparisons. By contrast, the matched full-versus-selective FP-TRM experiment holds the search procedure fixed and favors full differentiation in all ten contribution-response conditions, with positive adjusted bounds. Therefore the derivative finding is compatible with using OMP to solve the subsequent support-selection problem.

Why use a carrier, and what happens without searching for or ablating one?

A carrier is optional: ablating it asks how much the target contributes when that upstream input is removed. Without carrier ablation, the objective measures the target's contribution in the intact upstream context. With carrier cc and target uu, the objective is the correct-token logit with cc ablated minus that with both cc and uu ablated. Because the same context appears in both terms, the difference isolates the target-head intervention within that context.

Without a carrier, we compare the intact model with target-head ablation, eliminating carrier search. To test this alternative, we fitted actions both with and without carrier ablation on six heads. Both conditions shared the candidate pool, QKVO EK-FAC geometry, equal positive top-ten rule, and relative update norm 3×10−43\times10^{-4}. Independent evaluation always kept the carrier intact.

No-carrier fitting produces positive copying gains in 11 of 12 actions, as does carrier-conditioned fitting. Each count includes both attribution paths, and positive gains have simultaneous 95% intervals above zero on 256 held-out copy sequences. The no-carrier mean gain is larger in eight of twelve matched context comparisons. Hence carrier ablation is useful for specifying the conditional intervention question, rather than a prerequisite for attribution or intact-copy improvement.

A contribution increase measures the specified head intervention, while an intact-copy gain measures its behavioral consequence. Across the twelve path comparisons spanning both fitting contexts, full-path copying gains are higher in two, selective gains are higher in eight, and two are unresolved by the simultaneous intervals. Accurate prediction of a contribution change therefore answers the selected mechanistic question, while intact-copy measurements establish its behavioral effect. This distinction gives the objective a precise interpretation and connects the analysis to independently measured behavior.

Thank you for the rebuttal. However, it still doesn't change my understanding that the main contribution of the paper is identifying an implementation issue in MDA and there is limited novelty.

Reviewer_Py6jOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FP-TRM (Full-Path Target-Response Matching) is a method for choosing training documents that strengthen one attention head, built on the idea that the selection gradient must pass through every later layer that reads the head's output. It argues that frozen later layers still carry the head's effect forward, so the released code of mechanistic data attribution (MDA) wrongly predicts the effect even though the forward pass is unchanged. The paper measures a head's contribution as how much the correct next token's logit drops when the head is switched off, and aims to raise it with one small update to the head's own weights. The paper develops a test procedure that takes a weighted combination of ten document gradients to the full-path gradient of this contribution and applies it as that single update. The same procedure applies to FP-TRM and selective gradients.

On two heads in Pythia-1B, the paper reports that full-path predictions match measured changes much more closely (correlation 0.99 vs. 0.35 at one head) and that the two gradients pick almost entirely different documents, with full-path picks giving larger increases. It also reports gains over adapted MDA and AIR, and on models from 31M to 4B parameters.

Critical strengths
  1. The core simple and useful point is that freezing later layers does not freeze their computation, so predicting what a head-only update does requires gradients through those layers. Fixing this makes a big difference in computational correctness and attribution outcomes. It's also good to see that the approach doesn't cause additional computational overhead.
  2. The study is extensive and yields several interesting insights. It also compares against a wide range of selection methods under a common update. The interesting insights include The gap between the two gradients is larger for the earlier head (layer 2) than for the later one (layer 7). This is consistent with more of an early head's effect being carried by later layers.
Critical weaknesses
  1. MDA's own paper defines its probe as the head's direct logit contribution and claims only a direct-effect approximation. The finding is therefore a correction to an implementation detail part of MDA. The paper does not show that MDA's actual conclusions would change.
  2. The experimental breath is limited, especially the comparison of the two gradients is narrow. Full and truncated gradients are compared only on Pythia-1B, for two heads that both have index 4. It is unclear if the same conclusion holds on other heads.
Questions for authors
  1. Will the same conclusion hold on heads other than the two picked in the main experiments? How's the difference and variance between heads?
  2. In MDA's original setting (induction heads in small Pythia models like 14M, 31M), how much do the top-ranked documents from MDA's code and from a full-path version overlap? Would MDA's relevant conclusions change?
1 comment

Thank you for your helpful feedback. We repeated the matched-path test at six heads in Pythia-14M and 31M, including MDA's original head coordinates. Full-path derivatives predict the same deployed updates more accurately at every head, while head-dependent document overlap identifies which local attribution conclusions change.

Q1: Does the result extend beyond the two H4 targets?

Yes, full-path prediction is more accurate at all six heads, although the size of the selective-path error varies across heads. We retained MDA's 14M L3H3 and 31M L4H3 coordinates and selected two additional heads per model using prefix-matching scores on 64 independent repeated sequences. The additional heads were fixed before measuring gradient differences or update outcomes. Each path then predicted the same 16 fixed head-only updates at relative norm 3×10−43\times10^{-4}. The table reports response correlation and the fraction of shared documents in the top-ten sets from a common 256-document pool.

Model/headFull correlationSelective correlationTop-ten overlap
14M L3H3>0.9990.85830%
14M L3H1>0.9990.92650%
14M L4H0>0.9990.96590%
31M L4H3>0.9990.95370%
31M L4H1>0.9990.96360%
31M L4H6>0.9990.95030%

As the table shows, accurate full-path prediction holds across different head indices, while the selective-path discrepancy depends on the head. Full-path RMSE is also 99.1%–99.6% lower, establishing a magnitude improvement as well as a correlation improvement. The 30%–90% overlap range shows that derivative choice can materially alter document ranking, with heterogeneous effects across heads. It complements the larger discrepancy at the earlier Pythia-1B head without assuming a universal depth ordering.

Q2: What is the document overlap in MDA's small-model setting?

Top-ten overlap is 30% at 14M L3H3 and 70% at 31M L4H3 under the released MDA probe and official QKVO EK-FAC geometry. Both paths share the 256 candidate documents, 64 independent curvature documents, query context, and positive equal-weight top-ten rule. The checkpoints are public Pythia-14M step 2,000 and Pythia-31M step 1,000. This identifies the tested model instances precisely, since MDA's original study used its own training checkpoints.

In terms of selected documents, these overlaps mean that changing the derivative path replaces seven of ten documents at the original 14M head and three of ten at the original 31M head. The additional heads show that overlap can be either larger or smaller, so the result supports measuring head-specific sensitivity instead of extrapolating one overlap rate to all induction heads.

Does this change MDA's direct-effect interpretation or its original conclusions?

The derivative choice changes local response predictions and document rankings. It does not by itself overturn MDA's document-removal and retraining conclusions, which concern a different intervention. MDA's stated direct-effect approximation and the response of a deployed head-only parameter update answer different questions: after an actual update, later layers still process the changed head output. Our fixed-update comparison evaluates that complete computation while changing only the derivative used to predict it. The formal parameter-subspace restriction in MDA is compatible with full downstream differentiation, so the finding specifically concerns the selective backward implementation.

Consequently, the relevant conclusion is that attribution derivatives should match the intervention being predicted. Our fixed-update comparison makes that distinction observable through response accuracy and changed document rankings, while preserving MDA's stated direct-effect approximation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.