acceptodds
Under review as a conference paper at ICLR 2027

TraceAttention: Efficient Long-Context Attribution in Reasoning LLMs via Cache-Preserving Evaluation

Abstract

Context attribution explains how input spans influence model responses using internal-state analysis or perturbations. Perturbation-based methods score a fixed response under block masks of the same long context, but physical deletion requires repeated, expensive suffix reconstruction even with prefix caching. We introduce TraceAttention, a training-free evaluator that prefills once and reuses the full-context state by masking only target-to-context attention. This preserves the mask-to-score interface of existing attribution estimators while replacing repeated context reconstruction with target-only replay. The intervention differs from deletion, so we evaluate its utility using held-out physical deletions. For reasoning LLMs, scoring the full reasoning and answer exposes the reasoning's context dependence but increases per-mask work. Dual TraceAttention (DualTA) addresses this cost by selecting answer-relevant reasoning segments for a compact target. Across 15 model-context settings, TraceAttention and DualTA provide geometric-mean incremental speedups of 9.6 and 14.2 over prefix-cached deletion, respectively, while retaining over 90% of the deletion reference on average for both the linear datamodeling score (LDS) and Top-5 removal utility. On Qwen3.5-27B at a mean context length of 205K tokens, DualTA reduces incremental attribution latency from 841.0 to 13.1 s, a 64.2 speedup.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Efficient LLM Inference, Context Attribution, Long Context, State Reuse, Reasoning LLMs

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_6fhLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TraceAttention, a training-free evaluator for context attribution that reuses full-context prefix states and masks target-to-context attention, avoiding repeated context reconstruction under physical deletion. Dual TraceAttention further reduces computation by selecting a compact reasoning target. Experiments evaluate attribution quality using held-out physical deletions and measure latency and memory across multiple models and context settings.

Critical strengths
  1. The efficiency improvement is substantial and addresses a meaningful bottleneck in long-context attribution. Avoiding repeated context reconstruction makes repeated attribution queries considerably cheaper, which could make large-scale model analysis more practical.

  2. The evaluation explicitly acknowledges that attention masking differs from physical deletion and tests attribution quality against held-out physical deletions. Evaluating both quality and computational cost across multiple settings provides useful evidence for the proposed trade-off.

Critical weaknesses
  1. The practical boundary of this trade-off remains insufficiently clear. Quality degradation is task-dependent: for example, Table 6 reports LDS decreasing from 0.749 to 0.579 on SWE-agent trajectories with the full reasoning-and-answer target. Although the paper acknowledges these gaps, it offers limited guidance on when the approximation is reliable enough for diagnosis and when physical-deletion verification remains necessary.

  2. The connection to actionable LLM development remains underexplained. The metrics and case studies demonstrate attribution behavior, but do not directly establish whether the faster attribution helps developers identify or resolve model failures. A clearer account of the intended diagnostic workflow would strengthen the practical significance of the systems contribution.

Questions for authors
  1. Could TraceAttention serve as a screening stage, followed by physical-deletion verification of candidate spans in the original context? A small evaluation reporting total latency, attribution quality, and missed influential spans would help clarify whether this combination offers a useful operating point relative to either method alone. This is a suggestion for validating practical use, rather than a claim that such a combination must improve quality.

  2. What concrete development decisions are the attributions intended to support, and which existing results demonstrate that they provide sufficiently reliable diagnostic evidence? Please also clarify the architectural assumptions relevant to extending the intervention to vision-language or other multimodal models. I regard multimodal experiments as future work; a discussion of applicability and limitations would suffice here.

  3. Appendix A.4 reports that only 46 of 256 Nemotron InfiniteBench examples yield valid evaluation targets because the remaining responses lack the required reasoning–answer delimiter. Could the authors discuss how this filtering affects the representativeness of that setting and provide uncertainty estimates for its reported quality results?

Overall, I lean toward weak acceptance. The efficiency contribution is convincing, while clearer guidance on applicability, practical diagnostic use, and evaluation coverage would strengthen the paper.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 6fhL

W1. Practical use and deletion verification

TraceAttention's selected blocks need not exactly match deletion's ranking to remain useful. Across the 15 settings, TraceAttention recovers 79.1% of deletion's Top-5 block IDs while retaining 96.0% of its physical-removal utility. Different selected sets therefore often have similar measured impact.

This supports using TraceAttention to prioritize sources for investigation. When a development decision depends on the effect of removing a particular source, physical deletion can then verify that candidate directly. Our additional screening experiment in Q1 measures the cost and coverage of this workflow and is now reported in Appendix F.6 (Table 26).

W2. Actionable development use

For a suspected hallucination or prompt-injection failure, a developer can use TraceAttention to investigate which retrieved passages or tool messages influenced the problematic output. The shortlist helps decide which source to inspect, remove, or correct before rerunning the model.

Case C illustrates rapid candidate discovery: TraceAttention locates the relevant code location (B12) and reproduction output (B17) in 0.39 s of query time, versus 11.33 s for deletion. Its complete Top-5 set has lower removal utility (Table 16), motivating physical checks of the candidates before deciding which source to remove.

The systems benefit is making this inspect–test–revise workflow inexpensive enough to repeat across long histories. We have expanded this workflow discussion in Appendix F.7.

Q1. TraceAttention screening followed by physical-deletion verification

Yes. On 32 Qwen3.5 SWE examples, TraceAttention screening plus physical reranking retains 96.0% of exhaustive LOO's mean Top-1 removal utility while reducing attribution compute time by 4.6x.

WorkflowTop-1 recallTop-3 recallTop-5 recallTop-1 utility retentionAttribution time/example
Exhaustive physical LOO100.0%100.0%100.0%100.0%31.35 s
TraceAttention-LOO62.5%59.4%60.0%83.7%0.58 s
TraceAttention-LOO + physical reranking81.3%70.8%60.6%96.0%6.76 s

Recall measures overlap with exhaustive deletion-LOO rankings, with k capped at the available block count. Top-1 utility measures the physical likelihood drop from deleting the selected single block; the verified workflow's retention has a paired-bootstrap 95% CI of [89.2%, 99.6%].

The unchanged shortlist rule selects the top min⁡(n,max⁡(5,⌈0.10n⌉))\min(n,\max(5,\lceil0.10n\rceil)) TraceAttention-ranked blocks, averaging 29.7% of blocks. Verification improves their ordering. In 25/32 examples, the shortlist contains exactly five blocks, so its Top-5 membership stays fixed even when its internal ranking improves.

The 6.76 s includes both stages and excludes initial prefill. Complete-batch wall time, including loading and prefill, decreases from 1,057.3 s to 373.8 s (2.83x). We have added the screening workflow and its evaluation to Appendix F.6 (Table 26).

Q2. Development decisions and multimodal applicability

The workflow above supports concrete decisions such as correcting a misleading retrieved passage or testing removal of an instruction embedded in a tool response. TraceAttention identifies candidates cheaply; physical deletion can check their effect on the original response before the developer changes retrieval or tool handling. Case C provides an existing example of locating the relevant code and tool-output sources.

For multimodal decoders with token-addressable visual keys and values, the same replay mechanism can mask selected visual-token blocks while scoring a text response. The visual prefix is computed once and reused across queries. Information already mixed into other cached visual features remains available, so the same shared-state limitation applies. We have added this discussion to Appendix F.7.

Q3. Nemotron evaluation coverage

The exclusions occur during response parsing, before attribution: 210 responses end without a complete parseable reasoning-and-answer target. For the 46 format-valid responses, TraceAttention's LDS retention is 102.4%, with a paired-bootstrap 95% CI of [97.0%, 110.2%].

Importantly, the aggregate result remains above 90% after removing this setting entirely:

Evaluation scopeTraceAttention mean LDS retentionDualTA mean LDS retention
All 15 settings92.7% [91.9, 93.6]91.5% [90.6, 92.5]
Remaining 14 settings92.0% [91.3, 92.8]90.9% [90.0, 91.8]

Thus, the headline retention result does not depend on the small Nemotron/InfiniteBench subset. Its individual estimate describes the format-valid responses. We have added these uncertainty and exclusion results to Appendix F.1 (Tables 17–18). Appendix A.4 now states the parsing criterion explicitly.

AuthorsOpenReview ↗

Summary of Revisions and Updates for Reviewer 6fhL

Dear Reviewer 6fhL,

Thank you very much for your helpful suggestions on our paper's practical use and evaluation. We wanted to briefly note that we have posted our response above and updated the manuscript (Appendix F) to address each of your points.

To summarize, we tested the two-step workflow you suggested, which uses TraceAttention to quickly shortlist candidate blocks and then checking those candidates with physical deletion, recovering 96.0% of Top-1 removal utility while cutting attribution time by 4.6x on SWE (Appendix F.6, Table 26). We also explained how this workflow supports practical model debugging and multimodal setups (Appendix F.7), and added confidence intervals plus a 14-setting exclusion check for the Nemotron InfiniteBench subset (Appendix F.1, Tables 17–18).

We hope these updates address your concerns and make the practical trade-offs much clearer. Should you have any further questions before the discussion period concludes, we would be glad to discuss them!

Sincerely,
The Authors

Reviewer_AroqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the cost of perturbation-based context attribution for long-context LLMs. Methods like leave-one-out, ContextCite-style sampled masks, and Shapley estimators score a fixed response under many block masks of the context. Physically deleting blocks invalidates the KV cache after the first deleted block, so the suffix has to be recomputed for every mask, even with prefix caching.

TraceAttention prefills the full context once and evaluates each mask by hiding the masked blocks only from the target tokens’ attention to the context. The cached context states and token positions stay fixed. It keeps the mask-to-score interface, so any mask-based estimator can run on top of it.

For reasoning models, the scored target (answer only vs. reasoning plus answer) changes both the attribution signal and the cost of each mask. Dual TraceAttention (DualTA) runs a first stage that picks answer-relevant reasoning segments with a knapsack under a token budget. A second stage then attributes the context using only that compact target.

Because the intervention is not deletion, the authors evaluate against held-out physical deletions using the linear datamodeling score (LDS) and a Top-k removal metric. They use three models (Qwen3.5-4B, Gemma-4-E4B, Nemotron-3-Nano-4B), five dataset settings, a 27B model at up to 205K tokens, and a single-GPU scaling study. They report geometric-mean incremental speedups of 9.6× (TraceAttention) and 14.2× (DualTA) over prefix-cached deletion across 15 settings. Average LDS retention is 92.7% and 91.5%, and Top-5 retention is 96.0% and 94.3%. At a mean context of 205K tokens on Qwen3.5-27B, latency drops from 841.0 s to 13.1 s.

Critical strengths
  1. A simple, practical idea with large and broad efficiency gains. Reusing one full-context state and changing only target-to-context visibility is easy to adopt and estimator-agnostic. The speedup holds across three architectures, five dataset settings, and up to 205K tokens, with peak memory close to prefix-cached deletion (e.g., 36.6 vs. 36.1 GiB at 128K in Table 11). The speedup also grows with context length (3.4× at 1K to 7.8× at 128K against prefix-cached deletion). For anyone running many-mask attribution on long contexts, this is a real improvement.

  2. The evaluation is honest about the intervention’s limits. The authors do not claim TraceAttention equals deletion. They test it against held-out physical deletions, report paired confidence intervals (Table 6), and disclose where retention is lower. They flag that agent histories and hybrid models show larger gaps, and that DualTA depends on an observable reasoning trace.

Critical weaknesses
  1. The fidelity gap to deletion is not analyzed, and the headline averages hide where it is largest. Under TraceAttention, hidden blocks still shape the cached states of all later blocks, and hybrid models also carry information through recurrent states. The paper says so (Sec. 3.1), but it never measures how this affects attribution. The gap is substantial in the settings that motivate the work. LDS retention is 77.3% for Qwen and 82.6% for Nemotron on SWE-agent trajectories, and 79.5% for Qwen on AMA-Bench. The paired LDS confidence intervals in Table 6 exclude zero in every row, including InfiniteBench R⊕A. DualTA drops to 87.1% retention at 99K in Table 10, and to roughly 85% on Gemma InfiniteBench (0.665 vs. 0.780 in Table 7). The “>90% on average” figure is an average of ratios across settings, with no intervals. The reference is also the deletion-fitted estimator, evaluated on held-out deletions. This favors deletion by construction, and no external ground truth is used. I would expect something like gold supporting facts in HotpotQA, or injected-evidence localization.

  2. The efficiency claims against other attribution methods and the DualTA gains are less well supported than the deletion comparison. FlashTrace and AttnTrace are compared only on latency and memory, on a synthetic Llama-3.1-8B workload (Table 11, Fig. 1b). I did not find a quality comparison against them. AttriBoT, which is cited and targets exactly the efficient-LOO problem, and other efficient approximations are not compared at all. The “7.8× faster” and “8.4× faster than AttnTrace” claims therefore compare methods that may not produce equally good attributions. The extra gain from DualTA over TraceAttention (14.2× vs. 9.6× geometric mean) is concentrated in long-reasoning settings. In Table 15 it activates on only a handful of examples elsewhere (e.g., 3/251 for Qwen SWE-agent-trajectories, 2/243 for Nemotron SWE-agent-trajectories). Its configuration (ESM-64, B=512) was tuned on a single InfiniteBench development set with one model (Qwen3.5-4B). On that set, guided selection beats random segment selection by about 0.06 LDS (0.776 vs. 0.714) at similar budget (Table 13).

Questions for authors
  1. How does attribution error depend on a block’s position in the context? Visibility masking should be nearly exact for blocks at the very end of the context, where no later context tokens attend to them. Showing that, and how error grows toward earlier blocks, would explain where the 77–95% retention comes from. Do the low-retention settings (SWE-agent histories, Nemotron and Qwen hybrids) correspond to early-block or recurrent-state leakage?

  2. Can you report quality (e.g., LDS or Top-k against physical deletion, or an external signal such as HotpotQA supporting-fact recall) for FlashTrace, AttnTrace and AttriBoT on at least one dataset? Otherwise, how should readers interpret the cross-method speedups?

  3. The claim that R⊕A gives better attribution than A|R or A_dir rests on LDS values across targets of very different lengths. On InfiniteBench in Table 6, Top-5 for A_dir is 1.3–1.4 vs. 0.20 for R⊕A, so the Top-5 metric points the other way. How much of the LDS difference reflects lower noise from averaging over long targets rather than better attribution?

  4. How sensitive are the DualTA results to B and the Stage 1 ESM-r choice outside the InfiniteBench/Qwen3.5-4B development set? Can you report intervals (e.g., bootstrap over examples) for the retention ratios and speedups in Tables 7 and 8?


Suggestion for future work:

A variant that recomputes the recurrent state or the first few layers under each mask could show how much of the gap in hybrid models is due to leakage.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Aroq (Part 1/2: W1, W2, and Q1)

W1. Uncertainty and the attribution-quality reference

We computed paired-bootstrap 95% confidence intervals for all settings in Tables 7–8 and added them to Appendix F.1 (Tables 17–19). The cross-setting results are:

MethodMean LDS retentionMean Top-5 utility retentionGeometric-mean incremental speedup
TraceAttention92.7% [91.9, 93.6]96.0% [95.4, 96.6]9.6x [9.3, 10.0]
DualTA91.5% [90.6, 92.5]94.3% [93.6, 95.1]14.2x [13.9, 14.6]

Our quality test measures the actual response's dependence on its context. If misleading context produces an incorrect response, faithful attribution should identify that context, even when it differs from the evidence supporting the correct answer. Held-out physical deletion directly tests this response dependence; the deletion-fitted estimator provides the matched reference for measuring retained utility.

We also evaluated HotpotQA gold supporting sources and added the results to Appendix F.2 (Table 21). Across the six model/granularity settings, TraceAttention retains 97.7% of deletion's Recall@5 [96.5%, 98.8%], and DualTA retains 96.6% [95.3%, 97.9%]. The methods therefore remain close under this complementary external evaluation.

W2. Cross-method evidence and DualTA's additional gains

We now report quality and latency together on both HotpotQA and SWE in Appendix F.2 (Table 20; see Q2 in Part 2). On SWE, TraceAttention obtains similar LDS to FlashTrace, higher mean Top-5 removal utility, and approximately half its prefill-inclusive latency.

DualTA's additional gains concentrate in long reasoning because that is precisely the repeated work it removes. Short traces use base TraceAttention through the activation rule.

Guided selection is especially useful under a tight replay budget: ESM-32 with B=128 achieves 0.715 LDS in 1.17 s, compared with 0.714 LDS in 2.00 s for random selection with B=512 (Table 13). Thus, it reaches the same mean LDS with one quarter of the reasoning budget. Q4 (in Part 2) additionally tests the configuration on a different model and dataset (Appendix F.5, Table 25).

Q1. Block position and recurrent-state effects

We tested this directly on 32 Qwen3.5-4B SWE examples, comparing TraceAttention-LOO with physical-deletion LOO for each block:

Context positionExamplesMean scaled error (95% CI)
Early third300.108 [0.072, 0.149]
Middle third320.080 [0.053, 0.110]
Late third320.168 [0.127, 0.211]

Absolute score differences are normalized by each example's largest absolute deletion-LOO score, then averaged within each third and across examples.

The error is largest in the late third. In a recurrent hybrid, even the final context block can enter the cached recurrent memory before target replay, without passing through later context tokens. TraceAttention retains this memory, whereas physical deletion recomputes it. Having fewer later tokens therefore does not eliminate this source of disagreement.

To examine the architecture question, we also compared three attention-only and three recurrent-hybrid reasoning models on matched SWE examples. Mean LDS retention is 101.0% versus 88.8%, respectively. A separate same-family, non-reasoning Granite comparison shows the same direction: 103.4% versus 50.7%. These results support recurrent-state reuse as a contributor to the model-dependent fidelity gaps. The architecture comparison and position analysis are now reported together in Appendix F.3 (Tables 22–23).

AuthorsOpenReview ↗

Response to Reviewer Aroq (Part 2/2: Q2, Q3, and Q4)

Q2. Quality–latency comparison with other attribution methods

We added matched comparisons on real HotpotQA and SWE examples, using the same model, fixed responses, and physical-deletion quality tests.

DatasetMethodLDSTop-5 utility retentionLatency including prefill
HotpotQAPhysical deletion0.823100.0%2.025 s
HotpotQATraceAttention0.78294.9%0.742 s
HotpotQAFlashTrace0.74893.7%0.388 s
HotpotQAAttnTrace0.38136.2%2.320 s
SWEPhysical deletion0.938100.0%7.742 s
SWETraceAttention0.920101.1%2.303 s
SWEFlashTrace0.92298.0%4.561 s
SWEAttnTrace0.70779.1%13.482 s

FlashTrace is faster on short HotpotQA inputs, while TraceAttention achieves higher mean LDS and removal utility. On the longer SWE workload, TraceAttention preserves similar LDS to FlashTrace, improves mean removal utility, and approximately halves latency. This demonstrates the quality–cost advantage on real agent contexts.

All runs use Qwen3-8B BF16 on H200. HotpotQA has 32 shared examples. SWE was sampled from prompts up to 16K tokens for baseline memory compatibility; 30/32 examples completed in all methods, with 28 for Top-5. Timing excludes model loading and response generation.

AttriBoT's proxy-based variants additionally use a smaller model, typically from the same family. We focused these new comparisons on FlashTrace and AttnTrace as single-model quality–cost references. Our deletion baseline already includes exact prefix reuse; AttriBoT's full hierarchy/proxy pipeline was not benchmarked. We have added these comparisons and their evaluation setup to Appendix F.2 (Table 20).

Q3. Reasoning-inclusive scoring versus target length

The Top-5 values average likelihood changes over different output tokens: A_dir scores a separately generated answer, whereas R+A scores the complete reasoning and answer. Their per-token magnitudes therefore do not directly rank the quality of these attribution targets.

Including reasoning also lets us measure context dependence within the reasoning itself. On the same 234 responses, TraceAttention produces nonconstant block attributions for 222 examples when reasoning is scored, compared with 119 when only the final answer is scored: coverage increases from 51% to 95%.

To test whether simply adding tokens provides this benefit, we replaced the reasoning with an equal number of neutral filler tokens on 32 paired examples, keeping the context and final answer fixed:

Scored reasoning contentTraceAttention LDSExamples with nonconstant attributions
Realized reasoning0.84532/32
Equal-token neutral filler0.35213/32

Each row evaluates attributions against held-out physical deletions scored on that row's complete target: reasoning plus answer, or filler plus answer. At the same target length, realized reasoning yields higher LDS and broader attribution coverage. Simply extending the target with filler does not reproduce this benefit. We have added the coverage analysis and filler comparison to Appendix F.4 (Table 24).

Q4. Sensitivity outside the development setting

We tested the parameter grid on all 55 Gemma/AMA examples where default DualTA activated, using the saved responses. The unchanged ESM-64, B=512 configuration retains 95.1% of deletion-reference LDS and 101.7% of Top-5 removal utility.

Stage-1 estimatorReasoning budget BLDS retentionTop-5 utility retentionIncremental attribution time
ESM-6412872.5%78.1%3.38 s
ESM-6425683.0%95.2%3.82 s
ESM-64 — original configuration51295.1%101.7%4.72 s
ESM-3251299.0%100.2%3.69 s
ESM-12851287.0%99.2%5.87 s
Random segment selection51282.4%96.7%2.62 s

Increasing the retained reasoning budget improves LDS in this setting. More selection queries alone do not consistently help, and the original configuration remains effective without retuning. At B=512, ESM-based selection also retains more LDS than random segment selection.

Times include both executed stages and selection overhead on H200; physical-deletion attribution takes 44.40 s on the same examples. We have added these sensitivity results to Appendix F.5 (Table 25). Per-setting confidence intervals for the original Tables 7–8 are reported in Appendix F.1 (Tables 18–19), with cross-setting summaries in W1 (Part 1).

AuthorsOpenReview ↗

Summary of Revisions and Updates for Reviewer Aroq

Dear Reviewer Aroq,

We sincerely appreciate your detailed and constructive feedback on our work. Based on your suggestions, we have posted our two-part response above and incorporated the corresponding new experiments into Appendix F of the revised paper.

Specifically, we added paired-bootstrap confidence intervals across our main tables (Appendix F.1) and evaluated gold supporting-fact recall on HotpotQA (Appendix F.2, Table 21). We also measured how attribution error varies across early, middle, and late context positions on SWE alongside a multi-model architecture comparison (Appendix F.3, Tables 22–23); benchmarked quality and latency against FlashTrace and AttnTrace on HotpotQA and SWE-agent-trajectories (Appendix F.2, Table 20); tested an equal-token neutral filler control for reasoning targets (Appendix F.4, Table 24); and checked DualTA's parameter sensitivity on Gemma/AMA (Appendix F.5, Table 25).

We hope these additions address the questions in your review. If you have any remaining questions or thoughts, we would love to hear from you and discuss them before the discussion period closes.

Best regards,
The Authors

Reviewer_MqLKOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TraceAttention, a training-free framework for efficient long-context attribution that reuses cached context states and evaluates perturbations through target-to-context attention masking. It further introduces Dual TraceAttention (DualTA), which selects answer-relevant reasoning segments to reduce target replay costs.

Critical strengths

1- TraceAttention eliminates repeated context reconstruction, achieving significant speedups over prefix-cached deletion, particularly for long contexts. 2. The experiments cover multiple model architectures, datasets, and context lengths up to 205K tokens.

Critical weaknesses

1- TraceAttention preserves full-context states while masking only target-to-context attention, which is not equivalent to physical deletion (Sec 3.1). The lower attribution-quality retention on some agent-history datasets (Fig 3) raises concerns about reliability in more complex settings. 2- DualTA requires an observable reasoning trace and depends on reasoning segmentation and selection. Its additional efficiency gains come at the cost of attribution fidelity, and its applicability to models without accessible reasoning traces remains unclear.

Questions for authors

1- Under what conditions does target-only attention masking closely approximate physical deletion? What explains the larger attribution-quality gaps on agent-history datasets? 2- Since cached context states already contain information from masked blocks, how does TraceAttention distinguish direct from indirect contributions of those blocks? 3- How does preserving recurrent states in hybrid architectures affect attribution fidelity, particularly when information from masked blocks propagates through these states? 4-How do the speedups change when the initial context prefill is included, particularly for one-off attribution requests without a resident KV cache? 5-Can DualTA be applied when reasoning traces are unavailable or hidden? If not, how does this constrain its practical applicability?

4 comments
AuthorsOpenReview ↗

Response to Reviewer MqLK (Part 1/2: W1 and W2)

W1. Attribution quality on agent histories

Our new SWE comparison shows that TraceAttention achieves similar LDS to FlashTrace, higher mean Top-5 removal utility, and approximately half its prefill-inclusive latency.

MethodLDSTop-5 utility retentionLatency including prefill
Physical deletion0.938100.0%7.742 s
TraceAttention0.920101.1%2.303 s
FlashTrace0.92298.0%4.561 s
AttnTrace0.70779.1%13.482 s

Thus, TraceAttention retains effective removal sets on this agent workload while substantially reducing attribution cost.

The comparison uses Qwen3-8B on H200 with identical fixed responses. We sampled 32 SWE examples with prompts up to 16K tokens; 30 completed in all methods, and 28 support Top-5 evaluation. Utility retention is relative to the matched deletion reference. We have added this comparison to Appendix F.2 (Table 20).

W2. DualTA's quality–efficiency trade-off

DualTA addresses the additional cost of replaying long reasoning traces. At the selected development configuration, it retains 96.3% of full-target TraceAttention's LDS and 95.5% of its Top-5 removal utility while running 4.3x faster, including selection overhead (Appendix C.1, Table 13). Quality is evaluated on the original complete response, so these gains reflect cheaper attribution of the same response. We have also added an evaluation on a different model and dataset in Appendix F.5 (Table 25).

For short traces, the activation rule skips selection and uses full-target TraceAttention.

AuthorsOpenReview ↗

Response to Reviewer MqLK (Part 2/2: Q1, Q2, Q3, Q4, and Q5)

Q1. Conditions for close agreement with physical deletion

Close agreement is expected when direct attention to the masked blocks carries most of their influence on the response. Agent histories can also pass earlier observations into later context states or recurrent memory. Those additional information paths remain available under TraceAttention.

Our matched SWE diagnostic shows that agent histories can nevertheless support high attribution fidelity:

Reasoning-model groupMean LDS retentionMean Top-5 utility retention
Three attention-only models101.0%98.4%
Three recurrent-hybrid models88.8%87.3%

The LDS-retention difference is 12.2 percentage points, with a paired-bootstrap 95% CI of [3.8, 24.0]. This supports a model-dependent explanation for the larger gaps, rather than an inherent failure on agent histories.

The attention-only models are Gemma-4-E4B, Qwen3-4B, and SmolLM3-3B; the hybrids are Falcon-H1R-7B, Nemotron-3-Nano-4B, and Qwen3.5-4B. Results use the same 22 valid examples across all six models, with 20 examples for Top-5. We have added the shared-state discussion and this comparison to Appendix F.3 (Table 22).

Q2. Direct and indirect contributions

TraceAttention measures response-likelihood changes under target-to-context attention masking, without separately estimating direct and indirect contributions. Information already stored in other context states remains available. For example, if a later tool message summarizes an earlier observation, masking the observation still leaves that summary visible.

The attribution estimator chooses which blocks to test together and combines their scores. TraceAttention accelerates each test by reusing the context state; held-out physical deletions then measure how well the resulting attributions predict actual removal effects. We have clarified these information paths in Appendix F.3.

Q3. Preserved recurrent states

Recurrent prefix states can carry information from a masked block into the response even after attention to that block is blocked. The reasoning-model comparison in Q1 measures the associated fidelity gap.

We additionally tested a separate same-family, non-reasoning Granite control:

Non-reasoning modelLDS retentionTop-5 utility retention
Granite-4.0-Micro, attention-only103.4%90.9%
Granite-4.0-H-Micro, recurrent hybrid50.7%44.1%

Both models use direct responses, with 32 shared examples and 30 for Top-5. This control examines the recurrent-state boundary separately from DualTA and reasoning compaction. We have added this control and the recurrent-state discussion to Appendix F.3 (Table 22).

Q4. One-off attribution including initial prefill

The speedup remains substantial when initial prefill is included. At 128K, the native implementation takes 25.3 s with TraceAttention versus 205.6 s with prefix-cached deletion, an 8.1x speedup (Appendix B.5, Table 12).

TraceAttention processes the context once and shares it across all mask queries, so one-off requests also benefit from avoiding repeated suffix reconstruction.

Q5. Applicability without accessible reasoning traces

Base TraceAttention saves repeated context processing. Long reasoning introduces another cost: the reasoning tokens must be replayed for every mask. DualTA was introduced specifically to reduce this remaining work by selecting a shorter reasoning target.

Selection therefore requires accessible reasoning tokens. When no such trace is available, base TraceAttention can attribute the observable response without this selection step. With reasoning disabled on InfiniteBench, base TraceAttention takes 0.35 s versus 68.61 s for deletion, approximately a 196x incremental speedup (Appendix A.4, Table 2). We have clarified the roles of base TraceAttention and DualTA in Appendix F.7.

AuthorsOpenReview ↗

Summary of Revisions and Updates for Reviewer MqLK

Dear Reviewer MqLK,

Thank you again for your time and thoughtful review of our paper. Following your comments, we have posted our detailed responses above and updated the manuscript (see Appendix F) with new experiments and clarifications.

In particular, we added a matched quality-latency comparison against FlashTrace and AttnTrace on SWE-agent-trajectories (Appendix F.2, Table 20), and tested six reasoning models alongside a same-family Granite control to show how recurrent states affect fidelity on agent histories (Appendix F.3, Table 22). We also clarified how TraceAttention handles indirect context information, one-off prefill speedups, and models without visible reasoning traces (Appendix F.3 and F.7).

We hope these new results and explanations resolve the concerns raised in your review. Please let us know if anything remains unclear, we would be very happy to discuss further before the discussion period ends!

Best regards,
The Authors

I thank the authors for their responses. Based on the responses and the discussions, I would like to raise the rating to 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary for the Area Chairs and Program Chairs

Dear Area Chair, Senior Area Chair, and Program Chairs,

Thank you for overseeing the review of TraceAttention. The reviews highlighted three strengths:

  • Substantial speedups with comparable peak memory. Reusing the full-context state avoids repeated context reconstruction, making long-context attribution substantially faster. The single-GPU study also shows peak memory close to prefix-cached deletion.
  • A simple design with broad experimental support. TraceAttention works with existing mask-based attribution estimators, with experiments across multiple architectures, datasets, and contexts up to 205K tokens.
  • Transparent evaluation of attribution quality and cost. The paper tests attributions against held-out physical deletions and reports quality alongside execution cost, making the trade-off explicit.

Building on these strengths, we have answered each review and incorporated additional experiments and analyses into the revised manuscript (Appendix F, Tables 17–26). The key findings are:

1. Real-workload comparisons support the quality–speed advantage (Reviewers Aroq W2/Q2, MqLK W1).

We now compare physical deletion, TraceAttention, FlashTrace, and AttnTrace on HotpotQA and SWE using identical fixed responses (Qwen3-8B, H200). On SWE, TraceAttention achieves similar LDS to FlashTrace (0.920 vs. 0.922), higher mean Top-5 removal utility (101.1% vs. 98.0% of the deletion reference), and approximately half the prefill-inclusive latency (2.303 s vs. 4.561 s). These comparisons directly connect the efficiency gains to attribution quality (Appendix F.2, Table 20).

2. Agent histories support high fidelity, with clear differences across architectures (Reviewers MqLK Q1–Q3, Aroq W1/Q1).

On 22 SWE examples shared across six reasoning models, the three attention-only models average 101.0% deletion-reference LDS retention, compared with 88.8% for the recurrent hybrids (paired-bootstrap 95% CI of the difference: [3.8, 24.0] percentage points). A separate same-family non-reasoning Granite control confirms this boundary (103.4% vs. 50.7% LDS retention), and a 32-example block-position analysis further examines shared-state effects (Appendix F.3, Tables 22–23).

3. External evidence and uncertainty estimates strengthen the main results (Reviewers Aroq W1/Q4, 6fhL Q3).

On HotpotQA gold supporting sources, TraceAttention and DualTA retain 97.7% and 96.6% of deletion's Recall@5. We also provide paired-bootstrap 95% CIs for all 15 settings: mean LDS retention is 92.7% [91.9, 93.6] for TraceAttention and 91.5% [90.6, 92.5] for DualTA. Excluding the small Nemotron/InfiniteBench setting leaves these means above 90% (92.0% and 90.9%), showing the headline result does not depend on that subset (Appendix F.1–F.2, Tables 17–19, 21).

4. Reasoning content matters, and DualTA transfers without retuning (Reviewers Aroq Q3–Q4, MqLK W2/Q4–Q5).

On the same 234 InfiniteBench responses, scoring reasoning increases nonconstant attribution coverage from 51% to 95%; replacing reasoning with equal-length neutral filler drops LDS from 0.845 (32/32 active) to 0.352 (13/32 active). Outside the development set, the unchanged DualTA configuration retains 95.1% LDS and 101.7% Top-5 utility on all 55 activated Gemma/AMA examples with a 9.4x incremental speedup (Appendix F.4–F.5, Tables 24–25). We also clarified standalone prefill-inclusive speedups (8.1x at 128K, Table 12) and using base TraceAttention directly when reasoning traces are unavailable (Appendix F.7).

5. Targeted deletion verification provides a practical diagnostic workflow (Reviewer 6fhL W1–W2/Q1–Q2).

Following Reviewer 6fhL's suggestion, we tested TraceAttention screening followed by physical reranking on 32 SWE examples. It retains 96.0% [89.2, 99.6] of exhaustive deletion-LOO's Top-1 removal utility while reducing attribution compute time by 4.6x, including both stages, and raises Top-1 recall from 62.5% to 81.3%. This gives developers an evaluated way to locate influential sources quickly and verify their removal effects (Appendix F.6–F.7, Table 26).

Together, these additions strengthen the evidence for efficient long-context attribution and clarify its use across architectures and diagnostic tasks. Thank you for considering the revised manuscript and our detailed responses.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.