acceptodds
Under review as a conference paper at ICLR 2027

Elucidating the Design Space of LM Data Attribution

Abstract

Data attribution methods for Language Models (LMs) are typically evaluated using downstream tasks like fact retrieval. However, existing evaluations vary multiple dataset properties simultaneously, making it impossible to isolate exactly why an attribution algorithm fails. To diagnose these failures, we explore the design space of LLM data attribution, with a primary focus on evaluation. We introduce the Controlled Attribution Benchmark (CATT) for fact tracing, in which we fine-tune LLMs on entirely fictitious knowledge, providing unambiguous ground truth for attribution. This synthetic setup allows us to systematically alter one dataset configuration at a time, such as document redundancy and lexical distractors. With this configurable evaluation, we demonstrate that redundancy and distractors consistently degrade attribution performance. Rather than proposing a new estimator, we examine two design choices that are underexplored: the loss used to compute gradients and how per-token attribution scores are aggregated into a document score. We find that the true causal evidence within a training document is highly localized, but standard methods dilute this evidence into noise by averaging attribution scores across the entire document. We find that replacing standard cross-entropy loss gradients with the negative log-odds gradients surfaces the critical tokens from other uninformative ones. We can then aggregate only the top token scores to improve ground truth recall by 34 to 70 percentage points across multiple baselines on our diagnostic benchmark. The negative log-odds correction also generalizes to FTRACE-TREX, yielding up to a 5.2x increase in counterfactual retraining performance.

Primary area:
interpretability and explainable AI
Keywords:
data attribution, large language models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_EgMJOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how loss functions and token aggregation affect language-model data attribution. It introduces CATT, a controlled benchmark for tracing synthetic facts learned during fine-tuning, and evaluates several existing attribution methods. The main finding is that cross-entropy can suppress signals from confidently predicted tokens, while aggregation over an entire document can dilute useful local signals. Using negative log-odds (NLO) gradients and selecting high-scoring token positions substantially improves retrieval in several settings. The paper also includes an external remove-and-retrain evaluation.

Critical strengths
  • The paper is well written. The motivation is clear, and the token-level analysis makes the failure cases easy to understand. The relationship between cross-entropy and NLO gradients provides a useful explanation for the observed confidence effect.

  • The controlled benchmark and comparisons across multiple attribution methods are valuable. Studying redundancy, distractors, loss choice, and aggregation together helps identify practical design choices that can otherwise be overlooked.

  • The findings could help practitioners audit which fine-tuning documents support a learned fact and choose more effective attribution scores. I also appreciate the external retraining experiment, which evaluates changes in model behavior beyond document retrieval.

Critical weaknesses
  • Long-context practicality remains unclear. Appendix C.2 provides useful runtime measurements, so the computational overhead is acknowledged. However, attribution documents are truncated to 512 tokens, and the Function Binding documents are mostly only a few hundred tokens long. Could the authors provide a small runtime and peak-memory comparison across input lengths, including a longer-context setting, to show how much additional cost token-level scoring introduces? I don't think it hurt the main contribution though, just good to have a sense of how longer context contribute to the computation time.

  • The interpretation of replacing training gradients with NLO needs clarification. The model is trained with cross-entropy, and several methods retain cross-entropy curvature when their training gradients are replaced. This changes the perturbation being scored. The resulting ranking can still be useful, but its relationship to the influence of reweighting the original training documents is less clear. Could the authors state the corresponding perturbation explicitly and clarify when this interpretation is appropriate?

  • Document length explains a substantial part of the distractor result. In Table 14, summing NLO token terms raises EK-FAC recall from 0.18 to 0.99, nearly matching top-10 pooling at 1.00. The authors acknowledge this, but it deserves more emphasis in the main explanation. Please distinguish the gains from NLO, length normalization, and token selection; a length-matched distractor control would help isolate the last component.

  • The external evidence supports the proposed components unevenly. The strongest FTRACE retraining result comes from removing TRAK’s confidence factor QQ, while both dense TRAK variants already use NLO gradients. The top-10 retraining variant also changes the projection, and Table 17 does not show an improvement from top-10 pooling over full aggregation in the matched layerwise retrieval comparison. Please clarify which component each external result validates and qualify the broader claims about token selection accordingly.

Questions for authors
  • Please see above
2 comments
AuthorsOpenReview ↗

Response to Reviewer EgMJ

Thank you for highlighting our writing, the clarity of our token-level analysis, as well as the value of CATT’s controlled comparisons and the external remove-and-retrain evaluation! We address weaknesses in detail below.

Long-context practicality

As suggested, we tested how scoring cost scales with document length by extending 32 training documents from 256 to 2,048 tokens and recording per-document scoring time and peak GPU memory for the same 20 queries on an 80 GB A100. We find that NLO essentially does not change scoring time/memory, while computing per-token scores for top-k\mathit{k} pooling takes 2.1 to 4.4 times as long and costs additional memory overhead for EK-FAC. See the table below for full results.

Cells show scoring time (seconds per document) / peak GPU memory (GB)

MethodScoring256 tokens512 tokens1,024 tokens2,048 tokens
EK-FACCE, all tokens2.87 / 17.72.89 / 17.82.92 / 24.23.16 / 36.9
EK-FACNLO, all tokens2.78 / 17.72.81 / 18.92.87 / 26.23.12 / 41.0
EK-FACCE, top-k7.23 / 22.48.24 / 27.010.81 / 36.214.28 / 54.6
EK-FACNLO, top-k7.06 / 22.48.42 / 27.010.76 / 36.213.63 / 54.6
LoGraCE, all tokens0.31 / 16.20.32 / 20.60.45 / 29.20.73 / 46.6
LoGraNLO, all tokens0.31 / 16.70.32 / 21.60.45 / 31.30.74 / 50.7
LoGraCE, top-k0.96 / 16.20.97 / 20.61.11 / 29.31.59 / 46.6
LoGraNLO, top-k0.93 / 16.80.96 / 21.61.11 / 31.31.59 / 50.7

Interpretation of NLO loss in IF

Thank you for asking about the theoretical interpretation of using NLO training gradients in influence functions. Holding the cross-entropy curvature fixed, the NLO influence score estimates the first-order change in query loss from adding a small weight of the document’s NLO loss to the CE training objective (see equation below).

CE perturbation:∑d′∈DLCE(d′;θ)+ϵLCE(d;θ),NLO perturbation:∑d′∈DLCE(d′;θ)+ϵLnlo(d;θ).\begin{aligned}\text{CE perturbation:}&\sum_{d'\in\mathcal D}\mathcal L_{\text{CE}}(d';\theta)+\epsilon\mathcal L_{\text{CE}}(d;\theta),\\\\\text{NLO perturbation:}&\sum_{d'\in\mathcal D}\mathcal L_{\text{CE}}(d';\theta)+\epsilon\mathcal L_{\text{nlo}}(d;\theta).\end{aligned} We clarify this interpretation in Section 2.3 and provide the full derivation in Appendix A.2.

Separating distractor length effects from token selection

Thank you for pointing this out. As we now clarify in Section 3, full-document averaging dilutes the signal in longer ground-truth documents more than in our short distractors, so top-token pooling partly helps by removing this length penalty. However, top-token pooling also improves recall for several methods over full-document summation, which does not divide by document length (Appendix D, Table 15).

Regarding the suggested length-matched control, a key design constraint is that the distractors must follow the format of the held-out queries, and preserving this match would require artificially long queries for a simple function-output task. Adding filler to the distractors would keep queries short, but introduce additional content that could itself affect attribution.

Generalization of loss and top-token interventions

Thank you and reviewer 4UoY for asking how well each intervention generalizes beyond CATT. To address this, we evaluated three additional attribution methods on FTRACE-TREX and added a second external evaluation, BackdoorLLM [1].

On FTRACE-TREX, with full-document scoring, negative log-odds improved all methods, increasing the accuracy drop after removal and retraining by up to 32 percentage points. With loss held fixed, top-10 pooling yielded small gains for EK-FAC and Grad-Dot, but reduced the drop for Layerwise TRAK (Section 3 and Appendix E of the revision).

To test whether our interventions extend from facts to behaviors, we use BackdoorLLM, where attribution methods must trace a harmful behavior (jailbreaking or refusal) back to poisoned finetuning examples. The NLO loss correction improved attribution across the board for both behaviors, while top-10 pooling helped jailbreak attribution under cross-entropy and gave mixed results for refusals (Section 3 and Appendix F of the revision).

Across these external evaluations, the loss correction improves attribution consistently, while gains from top-token pooling are more variable, being dependent on how the attribution signal is spread within documents. As discussed in the limitations, we leave dynamic thresholding of the top-token budget to future work.

[1] Li et al. BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models.


Thank you again for taking the time to review the paper and providing helpful feedback! Do the above explanations and revisions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?

Reviewer_EgMJOpenReview ↗

Response to the rebuttal.

After carefully reading the response from the author, most of my concerns are resolved. However, I believe my original score already reflect the merits of the paper. Therefore, I would retain my score.

Reviewer_4UoYOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper present CATT, a synthetic benchmark for examining training-data attribution as document redundancy and lexical distractors are varied. It argues that aggregating at document level dilutes the contribution of informative tokens, and proposes to pair negative-log-odds gradients with top-token aggregation. Experiments shows large recall gains on two synthetic tasks, and an external FTRACE-TREX evaluation supports the advantage of negative-log-odds scoring.

Critical strengths

CATT offers a usefull framework for probing attribution failures under controlled dataset variations. Evaluating redundancy and distractors across multiple attribution families gives a more informative analysis than comparing on a single fixed benchmark.

The paper disentangles the effect of loss choice and token aggregation. Figure 4 shows recall going from 0.01 to 0.30 with token selection, to 0.40 with negative log-odds, and to 1.00 when both are combined in the illustrated setting. This gives concrete support to the proposed explanation.

The external evaluation goes beyond document retrieval by removing ranked training examples and retraining the models. Matched removal budgets and paired no-removal controls makes the resulting accuracy differences easier to interpret (Appendix E).

Critical weaknesses

As far as I read the paper and understood it, the experimental design dont fully isolate the factors it is trying to diagnose. Raising redundancy reduces training exposure from 200 epochs to four, and introducing distractors roughly doubles the number of optimizer steps (Table 3). Since attribution depends on the trained model, these shifts make it unclear how much of the degradation comes from dataset properties rather than from optimization and convergence.

Also the external results do not demonstrate that the combined intervention generalize. Negative-log-odds TRAK gives a 47-point accuracy drop, while the layerwise top-token variant gives 25 points (Table 16). The two variants also differ in projection structure, which prevents a clean comparison of aggregation alone. So the strongest external evidence supports the loss correction, not the full proposed combination.

The paper does not adequately seperate recovering documents that explicitly state a fact from identifying the fact’s causal training dependencies. On FTRACE-TREX, removing all annotated proponents lowers accuracy by only six points, whereas the strongest attribution method substantially hurts other facts which share the same relations (Table 16). Although the appendix notes this, it is still unclear how well CATT’s document-recovery metric reflects the causal attribution objective that motivates the paper.

Questions for authors

Do the redundancy and distractor effects hold under controls that equalize training exposure or convergence? This would considerably strengthen the central diagnosis.

Can the external evaluation isolate top-token aggregation while keeping the loss, projection, and module coverage constant?

Is there a remove-and-retrain validation on CATT showing that higher document recall reflects stronger fact-specific causal attribution, rather than mainly recovering explicit factual support?

How well CATT performs against this paper iclr 2026 paper @inproceedings{ICLR2026_8d57f138, author = {Pramanik, Vishal and Maliha, Maisha and Bastian, Nathaniel and Jha, Sumit}, booktitle = {International Conference on Learning Representations}, editor = {C. Vondrick and B. Hariharan and C. Raffel and L. Pinto and D. Yang and A. Faust}, pages = {87956--87994}, title = {Hessian-Enhanced Token Attribution (HETA): Interpreting Autoregressive LLMs}, volume = {2026}, year = {2026} }

I would also recommend the authors to use more recent and SOTA baselines. That would really strengthen the paper.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 4UoY

Thank you for highlighting how CATT’s controlled comparisons help diagnose attribution failures, as well as the clarity of our loss/token interventions and external FTRACE setup! We address questions and weaknesses in detail below.

Validation under matched training exposure

Thank you and reviewer XAmj for pointing out that with a fixed number of optimizer steps, the model sees each document fewer times as documents per fact increase. We therefore retrain Function Bindings models with a fixed 20 epochs per document at varying redundancy, and rescore every method. We find that the same attribution trends from the published models hold in the epoch-matched controls (Figure 9). The decline with redundancy persists: with document-level scores, recall from N=5\mathit{N}=5 to N=50\mathit{N}=50 falls from 0.93 to 0.50 for TRAK and 0.82 to 0.33 for EK-FAC (Appendix B, Table 4). After introducing our fixes, we observe that all methods still remain close to their published-recipe performance. We report the full setup and results in Appendix B.3.

Generalization of loss and top-token interventions

Thank you and reviewer EgMJ for asking how well each intervention generalizes beyond CATT. To address this, we evaluated three additional attribution methods on FTRACE-TREX and added a second external evaluation, BackdoorLLM [1].

On FTRACE-TREX, with full-document scoring, negative log-odds improved all methods, increasing the accuracy drop after removal and retraining by up to 32 percentage points. With loss held fixed, top-10 pooling yielded small gains for EK-FAC and Grad-Dot, but reduced the drop for Layerwise TRAK (Section 3 and Appendix E of the revision).

To test whether our interventions extend from facts to behaviors, we use BackdoorLLM, where attribution methods must trace a harmful behavior (jailbreaking or refusal) back to poisoned finetuning examples. The NLO loss correction improved attribution across the board for both behaviors, while top-10 pooling helped jailbreak attribution under cross-entropy and gave mixed results for refusals (Section 3 and Appendix F of the revision).

Across these external evaluations, the loss correction improves attribution consistently, while improvements from top-token pooling are more variable, being dependent on how the attribution signal is spread within documents. As discussed in the limitations, we leave dynamic thresholding of the top-token budget to future work.

[1] Li et al. BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models

Causal validation of CATT’s ground truth

Thank you and reviewer XAmj for pointing out that documents stating a fact may differ from those that causally teach it. Are CATT’s labelled ground-truth documents truly the most influential training examples? Following your suggestion, we ran remove-and-retrain on CATT settings: for 10 target facts, we removed subsets containing increasing fractions of the ground truth and retrained. We found that target accuracy decreases steadily on a curve as more ground truth is removed. Subsequently, removing each attribution method’s top-ranked documents lands on this curve: the share of ground truth a method removes predicts its retrained accuracy to within 0.03-0.08 per task (correlation 0.94) (Figure 11). We report the full setup and results in Appendix B.5 of the revision.

Adding more SoTA attribution baselines

Following your suggestion, we added GraSS (Hu et al., 2025) and SimpleLoRIF (Reddy et al., 2026). Both follow the same trends as other attribution methods on CATT; full results are in Appendix D of the revision. With their default cross-entropy scores, both are near chance (Recall@N ≤ 0.11); our negative log-odds top-10 scoring raises Function Bindings recall to 0.98 (GraSS) and 0.50 (SimpleLoRIF). In harder settings, GraSS with our fixes ranks among the strongest methods (0.94 with distractors, 0.40 on Rule Routings), while SimpleLoRIF shows more modest improvements, consistent with its aggressive design choices prioritizing efficiency over attribution accuracy, such as compressing each already-projected gradient matrix to rank one.

Comparison with HETA paper

Thank you for pointing us to HETA (Pramanik et al., ICLR 2026). We added a citation in our paper, but we clarify that HETA performs context attribution, identifying which tokens in the current prompt influence a fixed model’s prediction. This is outside the scope of training data attribution, and a direct comparison on CATT would therefore require adapting HETA to rank training documents. Please let us know if you had a specific adaptation in mind that would make this comparison possible!


Thank you again for taking the time to review the paper and providing helpful feedback! Do the above explanations and revisions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?

Dear Reviewer 4UoY,

With just over a day left in the discussion period, we wanted to check in. Thank you again for your thoughtful feedback, which has been very helpful in improving our paper.

Please let us know if you have any remaining questions or concerns.

Thanks again!

Reviewer_XAmjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies training data attribution for fact tracing in LLMs and introduces CATT, a benchmark in which Qwen3.5-4B is fine-tuned with LoRA on fictitious facts (constant functions bound to new vocabulary tokens, and arbitrary routing rules), so that the ground-truth training documents are known and properties such as document redundancy and lexical distractors can be varied. Evaluating ten attribution methods, it finds that redundancy and distractors degrade performance, and uses per-token attribution scores to argue that the evidence in a ground-truth document is concentrated in a few tokens and diluted by averaging over the document. It proposes computing gradients with a token-level negative log-odds loss and averaging only the top-k token scores, which improves Recall@N by 34–70 points on CATT, and reports that the negative log-odds change increases the accuracy drop after removal and retraining on FTRACE-TREX by up to 5.2× for TRAK.

Critical strengths

(S1) Controlled ground truth and a broad comparison. Training on fictitious facts makes the ground-truth documents unambiguous, and ten methods from five families are compared under a common layer-wise protocol with three seeds. The analysis also explains specific failures, for example why TRAK's correction factor ranks distractors first when the ground-truth documents are memorized (App. D.1).

(S2) A simple change that transfers to a counterfactual test. The negative log-odds change needs no new estimator and applies to any gradient-based method. On FTRACE-TREX it raises TRAK's accuracy drop after removal and retraining from about 9% to 47%, and the authors report openly that top-k aggregation does not help there.

Critical weaknesses

(W1) Limited novelty. As the authors state, the paper does not propose a new estimator. The negative log-odds gradient comes from TRAK [1], whose correction factor Q = 1 − p̄ reverts it to cross-entropy, and the related work itself notes that TrackStar has discussed applying the loss conversion per token [2]; averaging the top-k token scores is a simple aggregation choice, and the token-level decomposition used for the diagnosis is taken from [3]. The main analytical takeaway is also close to prior work: the FTRACE paper already identified gradient saturation, the vanishing of gradients for confidently fit examples, as a key obstacle for gradient-based fact tracing [4], which is the mechanism this paper attributes to cross-entropy, yet the connection is not discussed. What remains new is mainly the configurable benchmark and the demonstration that the two choices together recover ground-truth documents in it.

(W2) The gains depend on fact-specific answer tokens. In Function Bindings, each fact's answer is a new vocabulary token that appears only in its ground-truth documents, and under negative log-odds the top-scoring positions are enriched for this answer token at 5.1–11× its base rate (App. D.4), which suggests that the score largely reduces to finding documents containing the answer token. Where the answer is not unique to a fact, the picture changes: in Rule Routings, where every document mentions both answer words, the best gradient method reaches 0.55 Recall@N even with both changes while BM25 reaches 1.00 (App. D.1). It is therefore unclear how much of the headline 34–70 point gain carries over to natural data, where answers are common words.

(W3) The redundancy sweep changes more than one factor. Training runs for a fixed budget of about 1,000 steps, so each document is seen 200 times at N = 1 but only 4 times at N = 50, and the final training loss is higher for larger N (App. B.3, Fig. 8). The drop in attribution attributed to redundancy (Fig. 3) is thus confounded with how often each document is trained on, which conflicts with the paper's claim that the benchmark alters one dataset property at a time.

(W4) Narrow scope. All experiments use a single model (Qwen3.5-4B with LoRA) and single-token answers, and the external validation covers one method (TRAK) on one benchmark (FTRACE-TREX), where top-k aggregation does not help: the top-10 variant produces a smaller accuracy drop than full-document scoring (Fig. 7). The discussion motivates the work with copyright, credit assignment, and tracing emergent misalignment, but the evaluation covers only single-token fact recall, and the choice of k has no validation signal in practice, as the authors note.

[1] S. M. Park, K. Georgiev, A. Ilyas, G. Leclerc, and A. Madry. TRAK: Attributing Model Behavior at Scale. ICML 2023.

[2] T. A. Chang, D. Rajagopal, T. Bolukbasi, L. Dixon, and I. Tenney. Scalable Influence and Fact Tracing for Large Language Model Pretraining. ICLR 2025.

[3] R. Grosse et al. Studying Large Language Model Generalization with Influence Functions. arXiv:2308.03296, 2023.

[4] E. Akyürek, T. Bolukbasi, F. Liu, B. Xiong, I. Tenney, J. Andreas, and K. Guu. Towards Tracing Knowledge in Language Models Back to the Training Data. Findings of EMNLP 2022.

Questions for authors
  • Q1. How does the paper's explanation relate to the gradient saturation identified by Akyürek et al. [4], and what does the token-level view add beyond it?
  • Q2. How do the two changes perform when the answer is a common word shared across documents, for example a Function Bindings variant whose answers are ordinary numerals rather than new tokens?
  • Q3. Does the effect of redundancy remain when the number of passes over each document is held fixed across N, for example by matching epochs rather than optimizer steps?
  • Q4. Do the changes help beyond single-token fact recall, for example for multi-token behaviors implanted by fine-tuning such as a backdoor trigger and its response, and for methods other than TRAK on FTRACE-TREX?
  • Q5. How do the changes affect influence metrics such as the tail-patch score [2], given that CATT measures retrieval of documents that state a fact, which can differ from the documents that most influence a prediction [2]?
3 comments
AuthorsOpenReview ↗

Response to Reviewer XAmj (1/2)

Thank you for the positive comments regarding our benchmark’s controlled ground truth, broad comparison of attribution methods, and our analyses and loss correction! We address questions and weaknesses in detail below.

Novelty and relation to prior work (W1)

We first emphasize that the main contribution of this paper is providing a configurable dataset for fact tracing that uncovers potential limitations and failure modes of current attribution methods. Throughout the paper, we mention that rather than proposing a new estimator, we examine two specific design choices that are underexplored by existing attribution methods. Additionally, we show that they generalize to other models, datasets, and attribution methods in results we have added during the review period. We next explain how these design choices differ from prior work.

TRAK and TrackStar examined log-odds gradients and confidence weighting, but to our knowledge we are the first to systematically test negative log-odds gradients as a remedy in other attribution methods, and to test TRAK extensively without its confidence factor Q. Although prior work used token-level decomposition to visualize influence, we newly show that averaging across a document can dilute the signal from fact-bearing tokens, and we are, to our knowledge, the first to rank documents by their top training-token scores.

Regarding FTRACE’s "gradient saturation", to our understanding, the authors use this term to specifically describe vanishing gradients in the context of facts the model already knows before fine-tuning, which prevent clean attribution to the fine-tuning set (Section 4.1, “Slicing examples” in their paper). This differs from the analysis of confident newly-learned facts that we observed from CATT.

Validation on shared and common-word answers without answer-token shortcuts (W2)

As you point out, fact-specific answer tokens can act as attribution shortcuts. Settings like Rule Routings, where the same answers (Cedar and Birch) appear in every training document, are inherently more challenging. Indeed, most prior attribution methods struggle there. However, our fixes still raise mean Recall@N across the board, for example from 19% to 55% for EK-FAC and from 6% to 32% for Layerwise TracIn, though we agree there is more room for improvement compared to other CATT settings.

Furthermore, FTRACE-TREX provides a setting where more common answer tokens such as English, French, and jazz recur across real facts (e.g., John Stuart Mill’s native language is English). Across the released queries, 98% share their exact answer with another query. In experiments run after submission, we evaluated three more attribution methods (EK-FAC, Layerwise TRAK, Grad-Dot) in FTRACE. Our NLO loss correction still benefits every method, increasing the accuracy drop by 8 to 32 percentage points in the remove-and-retrain experiments. The revision presents these results in Section 3, with full results in Appendix E.

Validation under matched training exposure (W3)

Thanks for pointing out that with a fixed number of optimizer steps, the model sees each document fewer times as documents per fact increase. As suggested by you and reviewer 4UoY, we retrain Function Bindings models with a fixed 20 epochs per document at varying redundancy, and rescore every method. We find that the same attribution trends from the published models hold in the epoch-matched controls (Figure 9). The decline with redundancy persists: with document-level scores, recall from N=5\mathit{N}=5 to N=50\mathit{N}=50 falls from 0.93 to 0.50 for TRAK and 0.82 to 0.33 for EK-FAC (Appendix B, Table 4). After introducing our fixes, we observe that all methods still remain close to their published-recipe performance. We report the full setup and results in Appendix B.3.


Continued in second comment.

AuthorsOpenReview ↗

Response to Reviewer XAmj (2/2)

Generalization beyond single-token fact recall (W4)

As suggested, we evaluate attribution of harmful behaviors using two released BackdoorLLM [1] Llama models, each fine-tuned on data containing poisoned examples: in one, a trigger induces jailbreak responses to harmful prompts; in the other, it induces refusals of benign prompts. We score full, multi-token harmful responses using four attribution methods to identify the poisoned training examples behind each behavior. Negative log-odds raised jailbreak AUROC from 0.57–0.65 to 0.87–0.97 across all methods. On refusal, three methods rose from 0.22–0.63 to 1.00. Top-10 pooling with cross-entropy also improved jailbreak attribution for all four methods. We report the full setup and results in Section 3 and Appendix F of the revision.

[1] Li et al. BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models.

Generalization of top-token pooling (W4)

Thanks for raising a practical concern about how to choose a top token budget without ground-truth attribution labels. On CATT, gains persist across a wide range of k\mathit{k} values (Figure 6), so many smaller token budgets work well. However, we agree that choosing k\mathit{k} without ground-truth labels remains an open problem and leave dynamic thresholding to future work, which we mention in the paper as a limitation.

Causal validation of CATT’s ground truth (Q5)

Thank you and reviewer 4UoY for pointing out that documents stating a fact may differ from those that causally teach it. Are CATT’s labelled ground-truth documents truly the most influential training examples? Following reviewer 4UoY’s suggestion, we ran remove-and-retrain on CATT settings: for 10 target facts, we removed subsets containing increasing fractions of the ground truth and retrained. We found that target accuracy decreases steadily on a curve as more ground truth is removed. Subsequently, removing each attribution method’s top-ranked documents lands on this curve: the share of ground truth a method removes predicts its retrained accuracy to within 0.03-0.08 per task (correlation 0.94) (Figure 11). We report the full setup and results in Appendix B.5 of the revision and also clarify this in Section 2.1.


Thank you again for taking the time to review the paper and providing helpful feedback! Do the above explanations and revisions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?

Dear Reviewer XAmj,

With just over a day left in the discussion period, we wanted to check in. Thank you again for your thoughtful feedback, which has been very helpful in improving our paper.

Please let us know if you have any remaining questions or concerns.

Thanks again!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author response summary

We thank all reviewers for their detailed and constructive feedback. We appreciate their recognition of CATT’s controlled comparisons, broad evaluation, and analyses of attribution failures. Following their suggestions, we have added the following results and clarifications:

  1. Matched training-exposure controls (Appendix B.3, Figure 9). Retraining with a fixed epochs per document preserves the attribution trends observed under the submitted fixed-step training recipe (XAmj, 4UoY).

  2. Causal validation of CATT’s ground truth (Appendix B.5, Figure 11, Section 2.1). Remove-and-retrain experiments show that target accuracy decreases as more ground-truth documents are removed. Removing documents according to attribution rankings closely follows this relationship (correlation 0.94), supporting the connection between ground-truth retrieval and causal influence (XAmj, 4UoY).

  3. Broader external validation (Section 3, Appendices E–F). We expanded FTRACE-TREX to three additional attribution methods and added evaluations on full, multi-token jailbreak responses and refusals with BackdoorLLM datasets/models [1]. The external evaluations consistently support the loss correction, while showing more variable benefits from top-token pooling (XAmj, 4UoY, EgMJ).

  4. Additional recent attribution methods (Appendix D). We added GraSS [2] and SimpleLoRIF [3]. Our fixes raise their Function Bindings Recall@N from at most 0.11 to 0.98 and 0.50, respectively. GraSS also reaches 0.94 with distractors and 0.40 on Rule Routings. SimpleLoRIF’s gains on other settings are more modest, consistent with its focus on efficiency over attribution accuracy (4UoY).

  5. Theoretical and practical clarifications. We explicitly derive the perturbation estimated by NLO influence scores (Section 2.3, Appendix A.2), clarify the distractor-length confound (Section 3), and provide runtime and peak-memory measurements through 2,048 tokens in our response to EgMJ.

The key paper revisions are highlighted in red. As we have not yet received follow-up comments on our responses, we would be grateful for reviewers’ feedback on any remaining concerns so that we can address them before the discussion closes.

[1] Li et al. BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models. NeurIPS 2025.

[2] Hu et al. GraSS: Scalable Data Attribution with Gradient Sparsification and Sparse Projection. NeurIPS 2025.

[3] Reddy et al. Follow the Gradient: Tracing Safety Degradation Back to Training Data. arXiv:2610.37481, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.