acceptodds
Under review as a conference paper at ICLR 2027

DeltaTrace: Efficient Signed Attribution for Reasoning Language Models

Abstract

We study how words in the context support or oppose a language model's response. We present DeltaTrace, a novel attribution framework that explains the effect of input tokens on a fixed model response. DeltaTrace measures how the response's log-probability changes when the context tokens being explained are masked. To distribute this difference among source tokens, it constructs operator-specific local rules for finite changes and composes them backward through the computational graph using effective secant slopes. Each token is assigned a signed attribution as the inner product of its embedding displacement and the corresponding backward-propagated slope. Local conservation makes these contributions sum to the measured score difference and preserves their meaning when aggregated over sentences or passages. DeltaTrace supports both attention and linear-attention models, with auxiliary memory growing linearly with sequence length at fixed model dimensions and chunk size. On the evaluated attention model, DeltaTrace achieves the lowest deletion-faithfulness scores (RISE and MAS) on most retrieval and reasoning tasks and the highest needle-in-a-haystack Recall, exceeding FlashTrace by 5.67-24.33 percentage points. On the evaluated hybrid attention and linear-attention model, across 13 tasks and 1,243 examples, DeltaTrace lowers RISE on 9 tasks and MAS on 12 relative to FlashTrace.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language model, Intepretability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_R8vROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DeltaTrace decomposes the change in a fixed response's log-probability, caused by replacing selected source tokens with EOS, into signed token contributions that sum exactly to that change. It builds finite-difference rules for attention, softmax, RMSNorm, and Gated DeltaNet, and uses tiled/chunked computation to keep auxiliary memory linear in sequence length. It is evaluated on 13 tasks (1,243 examples) released with FlashTrace, using Qwen3-8B and Qwen3.5-9B.

Critical strengths
  1. Contributions are signed and sum exactly to the measured score change.

  2. One procedure covers both attention and Gated DeltaNet, so it works on hybrid models.

  3. Strong results with linear auxiliary memory. On Qwen3-8B, DT has the best RISE on 10/13 tasks, the best MAS on 11/13, and the best NIAH/VT recall among seven methods. On Qwen3.5-9B, it has lower RISE and MAS than FlashTrace on 9/13 and 12/13 tasks.

Critical weaknesses
  1. Only EOS is used as the reference, and no alternative reference is tested. The deletion evaluation also masks with EOS, and its fully masked endpoint is the same input as DT's reference.

  2. Only two models, both from the Qwen family, are used.

  3. Settings and hyperparameters for the baselines other than FlashTrace are not reported, and it is not stated whether their numbers were rerun.

  4. For scalar nonlinearities, the secant rule has the same form as DeepLIFT's Rescale rule, and the symmetric product split coincides with Shapley/IG for a single product. Both DeepLIFT and IG are cited, but this overlap is not discussed, and neither appears in Tables 2-4.

Questions for authors
  1. Which EOS token is used? Contexts filled with EOS may be out of distribution. How do results change with other references (e.g., PAD, mean embedding, deletion) or masking schemes?

  2. Do the results hold on other model families or scales?

  3. What are the settings of the baselines other than FlashTrace, and were their numbers rerun?

  4. How does DT compare with DeepLIFT and IG under the same setup?

  5. The local maps are not unique. How do results change with other choices, e.g., an asymmetric split for attention or a per-product symmetric split for the GDN recurrence?

  6. What are the generation settings and length distribution of the stored responses? For Qwen3.5-9B, were the responses generated by the same model? How does attribution quality vary with length?

  7. How well does a sentence's score sum match the effect of masking only that sentence?

1 comment
AuthorsOpenReview ↗

We report reference and deletion tests, four more models, the baseline settings, DeepLIFT and IG, two other allocation rules, response length, and the sentence-level correlation.

Thank you for your insightful review. Addressing your comments has improved the paper. Where no task is named, tables report averages over all thirteen tasks.

W1, Q1. EOS is <|im_end|>, the eos_token of the tokenizer. PAD is <|endoftext|>. Deletion shifts positions, so we trained one embedding vector so that replacing a span matches deleting it. The masked endpoint of the test is indeed the DT reference, so we also ran the test with PAD masking and with deletion. Test cells give RISE / MAS. KL is the divergence between the responses after replacing and after deleting a span, times 1000.

ReferenceEOS testPAD testDeletion testNIAH recallKL
EOS (paper)0.128 / 0.1940.145 / 0.2180.112 / 0.18148.012.2
PAD0.121 / 0.1790.131 / 0.1990.104 / 0.16549.634.7
Mean embedding0.165 / 0.2620.197 / 0.3100.146 / 0.24742.839.6
Learned vector0.130 / 0.1980.144 / 0.2210.113 / 0.18648.76.3
FlashTrace0.144 / 0.2200.143 / 0.2280.122 / 0.20836.0

EOS, PAD and the learned vector give close results, and the mean embedding is worse. EOS is closer to deletion than the other tokens we tried, and the learned vector is closer still. With these three references DT is below FlashTrace on both metrics under every test, except RISE under the PAD test, where EOS and the learned vector match FlashTrace. The deletion test has no reference as its endpoint, and Recall does not use the test. We will add these tests.

W2, Q2. We ran DT and FlashTrace on four more models. Kimi-Linear-48B is a second linear attention model. FlashTrace is set as in the paper. Cells give DT / FlashTrace.

ModelRISEMASNIAH recallVT recall
Qwen3-14B0.143 / 0.1470.205 / 0.22445.8 / 34.981.4 / 78.8
Qwen3-32B0.163 / 0.1720.225 / 0.25545.4 / 32.284.3 / 76.0
Llama-3.1-8B0.173 / 0.1950.235 / 0.26652.8 / 38.787.6 / 77.4
Kimi-Linear-48B0.166 / 0.2140.229 / 0.29854.8 / 35.493.1 / 83.4

DT has higher recalls in every cell. DT has lower RISE and MAS on Qwen3-14B and Qwen3-32B, Llama-3.1-8B and Kimi-Linear-48B. FlashTrace on Kimi is our port.

W3, Q3. The five baselines use the code and settings released with the FlashTrace benchmark. Perturbation and CLP replace ablated units with EOS, and REAGENT samples replacements from longformer-base-4096. Units are sentences on MQ-Q2 and MoreHopQA, and segments with source_k 20 elsewhere. IFR uses chunk_tokens 128 and sink_chunk_tokens 32, and AttnLRP uses neg_handling drop and norm_mode norm. Tables 2 and 3 and the NIAH rows of Table 4 report the released results. We ran the VT and HotpotQA rows ourselves with the same settings. We will add this to App. C.

W4, Q4. Both points are correct, and we will state them in Sec. 2.2. The table compares DT with DeepLIFT with the Rescale rule on the activation functions (Shrikumar et al., 2017) and with IG with 100 steps, under the same reference, target and tests.

MethodRISEMASNIAH recallVT recall
DT0.1280.19448.087.8
DeepLIFT0.3520.52622.313.8
IG0.1610.24136.853.7

IG is better than DT on RISE for three of thirteen tasks, on MAS for five, and on Recall for one of six NIAH tasks, and DT is better on the rest.

Q5. We ran both. The asymmetric split is ΔY=ΔPV0+P1ΔV\Delta Y=\Delta P V_0+P_1 \Delta V. The other GDN rule applies Eq. 18 to each product of Eq. 15. The table gives the change from the paper rule.

RuleRISEMASNIAH recallVT recall
Asymmetric attention, Qwen3-8B-0.009-0.025-0.5-3.6
Eq. 18 per GDN product, Qwen3.5-9B0.0000.0000.20.3

The GDN rule changes almost nothing. The asymmetric split lowers RISE and MAS, which is better, and both recalls, which is worse. We will report both and state that the split is a choice.

Q6. The responses come with the benchmark. They were generated by qwen3-235b-a22b-2507, not by Qwen3-8B or Qwen3.5-9B, and the release gives no decoding parameters. The table gives their length in tokens and DT results for the shorter and longer half of each task.

TasksMedianMinMaxRISE shortRISE longRecall shortRecall long
NIAH380.51889520.1440.14953.042.7
VT3902276110.1160.11187.787.6
HotpotQA49.5132110.0280.027
MATH2048936640.2300.232
MoreHopQA1796810960.0740.066

RISE shows no trend with length. Recall is lower for longer responses on NIAH and unchanged on VT.

Q7. We masked each sentence alone and measured the drop in response log probability. The table gives its Pearson correlation with the DT sentence sum within each example.

MQ-Q2MQ-Q4MQ-Q8MV-V2MV-V4MV-V8VT-H2-C3VT-H4-C1VT-H6-C1VT-H10-C1HotpotQAMATHMoreHopQA
Pearson0.790.540.160.810.730.560.570.380.480.530.650.210.74

The two agree on most tasks, least on MQ-Q8 and MATH. The sum is a share under joint masking, so the two need not be equal.

Reviewer_AW62OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces DeltaTrace, a reference-based attribution method for assigning signed contributions from prompt tokens to the log-probability change of a fixed multi-token response. The method replaces local derivatives with finite-change secant rules, propagates these rules through attention and Gated DeltaNet operations, and guarantees that token attributions sum to the measured score difference. Experiments on Qwen3-8B and Qwen3.5-9B report strong deletion faithfulness and evidence recovery, with an implementation designed to use auxiliary memory linear in sequence length at fixed model dimensions and chunk size.

Critical strengths
  1. The paper clearly specifies the counterfactual being explained and produces signed attributions that sum to the full-response score difference. This makes token, sentence, and passage aggregation easy to interpret.

  2. The finite-change rules cover both standard attention and recurrent linear attention, while tiled and chunked implementations address the otherwise prohibitive memory cost of long-context attribution.

  3. The evaluation covers retrieval, variable tracking, multi-hop QA, and mathematics. DeltaTrace substantially improves evidence recall and is best on most RISE and MAS comparisons, with especially large gains on MATH and several multi-hop settings.

Critical weaknesses
  1. The explanation is highly dependent on a single, potentially unnatural reference. Replacing every selected source token with EOS can create an out-of-distribution prompt. Because all selected tokens are masked jointly, the resulting attribution also depends on the chosen interaction-allocation rule. The paper does not show sensitivity to alternative references, masking scopes, or allocation choices.

  2. The experiments use two closely related Qwen models, and the linear-attention evidence comes from one hybrid architecture. It remains unclear whether the gains transfer to other model families, sizes, tokenizers, and recurrent architectures.

  3. Some empirical claims need uncertainty estimates and more careful qualification. DeltaTrace loses to FlashTrace on several multi-query tasks, is not the best method on every evidence-recovery comparison, and some datasets are small (for example, HotpotQA has 48 examples). The tables provide point estimates without confidence intervals or significance tests.

Questions for authors
  1. How stable are the signed attributions, deletion scores, and evidence-recovery results under alternative references such as padding, learned baselines, span deletion, or multiple reference prompts?

  2. Can the authors report uncertainty across examples and extend the evaluation to at least one non-Qwen attention model and one additional linear-attention or state-space family?

4 comments
AuthorsOpenReview ↗

We report sensitivity to the reference, the masking scope and the allocation rule, DT and FlashTrace on four more models, and per-example intervals for the comparison.

Thank you for your insightful review. Addressing your comments has improved the paper. Where no task is named, tables report averages over all thirteen tasks.

W1, Q1. We ran the alternatives you list. PAD is <|endoftext|>. One row averages scores under five special tokens. Deletion shifts positions, so we trained one embedding vector so that replacing a span matches deleting it. Same sign is the share of source tokens with the same sign as under EOS. KL is the divergence between the responses after replacing and after deleting a span, times 1000.

ReferenceRISEMASNIAH recallVT recallSame sign (%)KL
EOS (paper)0.1280.19448.087.812.2
PAD0.1210.17949.690.584.734.7
Mean of five special tokens0.1240.19448.289.093.2
Mean embedding0.1650.26242.878.487.739.6
Learned vector0.1300.19848.786.292.66.3

Token references and the learned vector give close results, and most tokens keep their sign. The mean embedding is worse. EOS is closer to deletion than the other tokens we tried, and the learned vector is closer still.

On masking scope, we masked each sentence alone, and for a sample of sentences reran DT with only that sentence masked. Pearson compares the DT sentence sum under joint masking with the drop in response log probability from masking only that sentence. Same sign is the share of its tokens with the same sign under both scopes.

MQ-Q2MQ-Q4MQ-Q8MV-V2MV-V4MV-V8VT-H2-C3VT-H4-C1VT-H6-C1VT-H10-C1HotpotQAMATHMoreHopQA
Pearson0.790.540.160.810.730.560.570.380.480.530.650.210.74
Same sign (%)63.059.778.760.461.271.083.975.174.674.459.776.564.7

Sentences agree across scopes on most tasks, least on MQ-Q8 and MATH. Token scores inside a sentence change with the scope, and we will state in Sec. 2.1 that scores are specific to the scope.

On allocation, we ran two other rules. The asymmetric split is ΔY=ΔPV0+P1ΔV\Delta Y=\Delta P V_0+P_1 \Delta V. The other GDN rule applies Eq. 18 to each product of Eq. 15. The table gives the change from the paper rule.

RuleRISEMASNIAH recallVT recall
Asymmetric attention, Qwen3-8B-0.009-0.025-0.5-3.6
Eq. 18 per GDN product, Qwen3.5-9B0.0000.0000.20.3

The GDN rule changes almost nothing. The asymmetric split lowers RISE and MAS, which is better, and both recalls, which is worse. We will report both and state that the split is a choice.

W2, Q2 second part. We ran DT and FlashTrace on two larger Qwen3 models, Llama-3.1-8B and Kimi-Linear-48B, a second linear attention model. The last two have their own tokenizers. FlashTrace is set as in the paper. Cells give DT / FlashTrace.

ModelRISEMASNIAH recallVT recall
Qwen3-14B0.143 / 0.1470.205 / 0.22445.8 / 34.981.4 / 78.8
Qwen3-32B0.163 / 0.1720.225 / 0.25545.4 / 32.284.3 / 76.0
Llama-3.1-8B0.173 / 0.1950.235 / 0.26652.8 / 38.787.6 / 77.4
Kimi-Linear-48B0.166 / 0.2140.229 / 0.29854.8 / 35.493.1 / 83.4

DT has higher recalls in every cell. DT has lower RISE and MAS on Qwen3-14B and Qwen3-32B, Llama-3.1-8B and Kimi-Linear-48B. FlashTrace on Kimi is our port.

W3, Q2 first part. We agree. Both methods explain the same responses, so we compare per example. The table gives the mean of DT minus FlashTrace with its 95% bootstrap interval. Negative favors DT for RISE and MAS, positive favors DT for Recall.

TaskRISEMASRecall
MQ-Q20.004 [0.000, 0.008]-0.045 [-0.050, -0.040]21.0 [20.0, 22.0]
MQ-Q40.031 [0.026, 0.037]0.008 [-0.003, 0.019]7.6 [6.8, 8.4]
MQ-Q80.077 [0.070, 0.083]0.178 [0.165, 0.191]5.9 [5.6, 6.2]
MV-V2-0.009 [-0.014, -0.004]-0.052 [-0.056, -0.048]24.3 [22.8, 25.8]
MV-V4-0.012 [-0.016, -0.008]-0.055 [-0.058, -0.051]7.4 [6.4, 8.4]
MV-V8-0.067 [-0.081, -0.053]-0.076 [-0.085, -0.067]5.7 [5.1, 6.2]
VT-H2-C3-0.031 [-0.034, -0.028]-0.038 [-0.041, -0.036]21.2 [20.1, 22.3]
VT-H4-C1-0.009 [-0.011, -0.007]-0.025 [-0.028, -0.022]2.9 [2.4, 3.4]
VT-H6-C1-0.012 [-0.014, -0.009]-0.025 [-0.028, -0.022]13.0 [12.5, 13.5]
VT-H10-C1-0.002 [-0.005, 0.000]-0.014 [-0.017, -0.011]7.1 [6.8, 7.5]
HotpotQA-0.006 [-0.011, -0.002]-0.027 [-0.035, -0.020]12.5 [-2.1, 27.1]
MATH-0.118 [-0.146, -0.089]-0.115 [-0.148, -0.081]
MoreHopQA-0.058 [-0.068, -0.049]-0.051 [-0.061, -0.040]

Most intervals exclude zero. FlashTrace is better on RISE for MQ-Q4 and MQ-Q8 and on MAS for MQ-Q8. The two are not separated on RISE for MQ-Q2 and VT-H10-C1 and on MAS for MQ-Q4. On HotpotQA the Recall interval includes zero, so 48 examples do not support a Recall gain. Our rerun of DT recovers one more example than Table 4, which puts DT level with AttnLRP, and we will correct the table. We will add the intervals and state these cases in the abstract, Sec. 4 and the conclusion.

Reviewer_AW62OpenReview ↗

Thanks for your response

Dear authors,

Could you please reflect the new results in the PDF file? I am more comfortable with the context due to the large amount of reviewed papers.

Reviewer

AuthorsOpenReview ↗

Revised PDF uploaded, with all new results marked in blue

Dear Reviewer AW62,

Thank you for reading our response, and for this suggestion. We have uploaded a revised PDF in which all new results are in place and every change is marked in blue, so they can be read in context.

  • W1 and Q1, the reference choices are in Sec. 4, Table 7, the masking scope is in Appendix A.2, Table 10, and the allocation rules are in Sec. 4, Table 8.
  • W2, the four additional models are in Sec. 4, Table 6.
  • W3, the paired intervals are in Appendix A.1, Table 9, and the cases where FlashTrace is better are now stated in the abstract, Sec. 4 and the conclusion.

We hope this makes the paper easier to assess, and we are happy to clarify anything further.

Hi, according to the new results, I will update my overall score (2 to 3).

Reviewer_JKRSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the input-token attribution for language models. It proposes an efficient yet principled framework. The main novelty lies in the proposed backward propagation rules (D-backward) and their formulation for distributing the output-score differences across input tokens. The design of these backward and distribution rules is the main technical contribution, allowing efficient attribution while ensuring that the contributions sum to the total score difference.

Critical strengths
  1. The proposed method is principled and efficient, which avoids the need to evaluate each input token separately.
  2. The method is applied to open-source models and demonstrates nontrivial improvements in attribution faithfulness and evidence recovery on several benchmarks.
  3. The framework supports both standard attention and linear attention models, and thus can be well suited for wide adoption.
Critical weaknesses
  1. The attribution might not be robust to the choice of reference tokens (EOS in this paper). It would be useful to evaluate the sensitivity to different reference choices. Similarly, although the distribution rules are mathematically principled, alternative rules might also satisfy the conservation property while producing different attribution scores. It is unclear how sensitive the results are to these design choices.

  2. Comparisons with closely related methods such as DeepLIFT and Integrated Gradients are missing, although they are referenced in the manuscript.

Questions for authors

Please see the weakness section.

1 comment
AuthorsOpenReview ↗

Replacement of EOS tokens and comparison with added baselines.

Thank you for your insightful review. The points you raise are important, and addressing your comments has improved the paper.

W1. On the reference, we ran three alternatives. PAD is <|endoftext|>. One row averages scores under five special tokens. Same sign is the share of source tokens with the same sign as under EOS.

ReferenceRISEMASNIAH recallVT recallSame sign (%)
EOS (paper)0.1280.19448.087.8
PAD0.1210.17949.690.584.7
Mean of five special tokens0.1240.19448.289.093.2
Mean embedding0.1650.26242.878.487.7

Token references give close results, and most tokens keep their sign. The mean embedding is worse.

On the distribution rules, we ran two other rules that also sum to the score difference. The asymmetric split is ΔY=ΔPV0+P1ΔV\Delta Y=\Delta P V_0+P_1 \Delta V. The other GDN rule applies Eq. 18 to each product of Eq. 15. The table gives the change from the paper rule.

RuleRISEMASNIAH recallVT recall
Asymmetric attention, Qwen3-8B-0.009-0.025-0.5-3.6
Eq. 18 per GDN product, Qwen3.5-9B0.0000.0000.20.3

The GDN rule changes almost nothing. The asymmetric split lowers RISE and MAS, which is better, and both recalls, which is worse. We will report both and state that the split is a choice.

W2. We ran both. The table compares DT with DeepLIFT with the Rescale rule on the activation functions (Shrikumar et al., 2017) and with IG with 100 steps, under the same reference, target and tests.

MethodRISEMASNIAH recallVT recall
DT0.1280.19448.087.8
DeepLIFT0.3520.52622.313.8
IG0.1610.24136.853.7

IG is better than DT on RISE for three of thirteen tasks, on MAS for five, and on Recall for one of six NIAH tasks, and DT is better on the rest. We will add this comparison to the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.