- Rating
- 3(+1)
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies long-horizon span-to-span attribution, aiming to identify which earlier text influences a later generated span. The authors propose TEMPO, a black-box attribution method that uses probabilistic mask optimization to identify influential source tokens and then combines the learned mask probabilities with word and clause structure to produce a complete attribution ranking without additional model queries. Experiments on four task families compare TEMPO with seven attribution baselines in terms of deletion faithfulness, computational efficiency, and reference-evidence recovery.
Critical strengths
(i) The paper focuses on the problem of tracing how earlier text influences later generated content, which is important for understanding and evaluating language model behavior in long-context and multi-step generation.
(ii) The experiments cover four task families and compare seven attribution baselines, with additional analyses of computational efficiency and decoder design. The paper also evaluates deletion faithfulness and reference-evidence recovery separately, providing a clearer picture of what the attribution methods actually capture.
Critical weaknesses
(i) The experiments do not fully cover the long-horizon span-to-span attribution setting emphasized in the paper. Section 3.1 defines attribution by intervening on earlier source text while keeping the intermediate text and target fixed. This measures dependence under a fixed continuation, but does not capture how changes to the source might affect the target by changing intermediate generated reasoning. Moreover, the main experiments primarily attribute input or supplied reasoning prefixes to generated answers or entire continuations. The paper does not directly evaluate attribution between distant spans within a model's own generated reasoning, nor analyze how attribution performance changes with source–target distance.
(ii) Section 2 does not clearly distinguish the methodological contribution of TEMPO from existing approaches. The paper reviews prior work on mask optimization and hierarchical text explanations, and describes TEMPO as connecting these two directions. It identifies the main difference from TRACE as converting a searched mask into a complete ranking without additional model queries. However, the discussion does not explain how the proposed word- and clause-based decoding differs from earlier methods that use linguistic structure for attribution. Although the paper demonstrates improvements over probability-based ordering, it is unclear what new methodological contribution TEMPO provides beyond adapting mask search and using linguistic structure to organize the resulting ranking.
Questions for authors
(i) Please clarify how the current experiments support the claimed long-horizon span-to-span attribution setting, particularly attribution between distant spans within model-generated reasoning.
(ii) Please explain how TEMPO differs from existing methods that combine mask optimization or linguistic structure for attribution, and what additional methodological contribution the proposed combination provides.
After Rebuttal
4 comments
I appreciate the authors' detailed response and the additional explanations and experimental evidence. However, my main concern about the experimental evaluation remains. Although the fixed-target attribution setting is now better explained, the experiments still do not directly evaluate attribution between distant spans within a model's own generated reasoning or examine how performance changes with source–target distance. These are important aspects of the long-horizon span-to-span attribution setting emphasized in the paper. I appreciate the authors' efforts to address my concerns, but given the remaining gap between the setting emphasized in the paper and the experimental evaluation, I will maintain my original rating.
Clarifications and revisions regarding "long-horizon."
Thank you for clarifying this concern. We have revised the terminology to state our contribution more precisely.
1. Origin of the framing and evaluation choices.
Our use of "long-horizon" followed the framing of FlashTrace (ICML 2026), which we include as a baseline. Our evaluation builds on its retrieval, multi-hop reasoning, and code-generation task settings, while adding misleading-prefix Math and counterfactual MCQ diagnostics. These additions probe error-origin and decisive-value dependence and broaden the comparison between intervention faithfulness and reference-evidence recovery.
2. Targeted terminology revision and the core contribution.
Before this revision, excluding the bibliography, "long-horizon" appeared seven times: six as a modifier in "long-horizon span-to-span attribution" and once as a paragraph heading. We have removed all seven occurrences and reworded one additional sentence using "long horizons". The reference to FlashTrace retains its original published title. Our revised title is "Text-Enhanced Mask Probability Optimization for Span-to-Span Attribution."
Thus, the revision primarily removes a modifier from the span-to-span formulation. The core contribution remains black-box span-to-span attribution through probabilistic mask optimization and query-free text-enhanced decoding into a complete ranking. Source and target spans may be nonadjacent within the model's context window; their selection and the fixed-target intervention objective define the attribution problem.
Section 4.1 now states the standard token-attribution setting explicitly: attributing a fixed generated target to preceding source tokens, with rankings evaluated under a common token-replacement protocol. The revised wording avoids implying a dedicated evaluation of distance-dependent performance. These changes are limited to the title and explanatory text; the formulation, algorithm, experiments, and numerical results are unchanged.
I appreciate the authors' further clarification and the revisions to the terminology. I think these changes are reasonable and help better align the paper's claims with its actual contributions and experimental scope. Given these revisions and the authors' previous responses, I believe my main concerns have been mostly addressed. I raise my rating and would be glad to see the paper accepted.
Response to Reviewer iw3y
Thank you for recognizing the importance of the problem and our separate evaluation of faithfulness and reference-evidence recovery.
W1 / Q1: Token attribution, evidence recovery, and regeneration
The distinction is between model influence under an intervention and textual evidence supporting an answer. Our token attribution measures how replacing selected source tokens changes the likelihood of the same recorded target. Reference-evidence recovery measures agreement with supporting spans. These questions need not have the same answer, as our experiments demonstrate.
Regenerating intermediate reasoning changes the conditioning history and may produce a different answer. Evidence supporting that new answer does not itself measure the source's influence on the original target. Quantifying such influence requires a separately defined outcome and comparison criterion. Our protocol holds the recorded text and target fixed so that all interventions share the same target score, while recomputing hidden states (Section 3.1; Appendix A.1).
The experiments evaluate earlier prompt text or supplied reasoning prefixes against later generated targets; generated-reasoning-to-generated-reasoning attribution and a distance sweep are not separately evaluated.
W2 / Q2: The specific methodological contribution
The contribution is the conversion from fixed-budget mask search to a more faithful complete ranking at essentially no additional cost. We adapt TRACE's visual mask search to text units, position-preserving padding, and a fixed-target necessity objective. The decoder splits word/clause bundles at the searched-set boundary and orders their parts using proposal density. It uses text structure to decode existing search artifacts, preserving the searched set at 20% without additional model scoring.
The same-search ablation directly evaluates this contribution: compared with the anchor-preserving probability ordering, TEMPO increases AUC from 8.54 to 9.50 (+11.2%), with gains on all four tasks, using zero additional model queries and approximately 9 ms of CPU time. The BM25 query-matching control, which releases the anchor constraint, instead lowers AUC to 7.51 (-12.0% versus probability ordering) despite improving evidence recovery. Thus, the benefit comes from the particular decoding procedure; adding text relevance does not automatically improve attribution faithfulness. These results are in Appendix C.8, Table 16.
Following Reviewer 1's aggregation question, the revised means give each eligible input equal weight. The original equal-task comparison was 9.00 to 10.00; task-level results are unchanged. Appendix F also reports relative within-task gains.