acceptodds
Under review as a conference paper at ICLR 2027

STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations

Abstract

Training Data Attribution (TDA) seeks to trace a model's predictions back to its training data. The gold standard for TDA relies on causal interventions, observing how a model changes when data is added or removed, but repeated retraining is computationally challenging for Large Language Models (LLMs). Consequently, most approaches approximate this effect in the parameter space using gradients. However, tracking gradients across billions of parameters is not only prohibitively expensive but relies on local approximations. In this work, we propose a shift: rather than estimating parameter changes, we model the functional effect of training data in the activation space. We introduce STRIDE (Steering-based Training Data Influence Decomposition), a framework that formulates TDA as a sparse recovery problem in the spirit of compressive sensing. STRIDE learns lightweight “steering operators” that mimic the behavioral shift caused by training on data subsets. By measuring how these operators perturb test predictions, we recover individual training example influences via sparse linear decomposition. On LLM pre- training attribution with up to 11.4 million candidate examples, STRIDE achieves the highest Linear Datamodeling Score (LDS). In this setting, attribution takes only 9.93 hours and is an order of magnitude faster than the next best method. We further validate its practical utility through downstream applications, including data selection, data contamination, and qualitative analysis.

Primary area:
interpretability and explainable AI
Keywords:
LLM, interpretability, TDA

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jp9WOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces a cheaper alternative to Training Data Attribution (TDA) named STRIDE, which learns multiple steering operators instead of retraining on multiple subsets of training data. STRIDE builds on the insight that model predictions, achieved by retraining on different subsets of data, can also be achieved by steering the model activations. Based on this, multiple low-rank steering operators are learned, which are built on top the frozen model parameters. These operators are lightweight compared to retraining. After the operators are learned, they are applied on test queries and the resulting loss changes are noted. Then, these changes are approximated/evaluated as per-example contributions. Finally, per-example influence scores are obtained via sparse recovery, which is is basically a ℓ1\ell_1-regularized least square problem. The resulting STRIDE framework, which can be summarized as training steering operators and approximating per-sample influence scores based on them, achieve the highest LDS score among various TDA methods, which are all much more computationally demanding to compute.

Critical strengths
  • The STRIDE framework provides an alternative way to look at the TDA problem. The idea of modeling the retraining on different subsets of training data as steering the model activations is a good motivation and makes the whole setup cheap to run, with just 1000 (number of operators) forward passes per query once the operators are learned, and solving a simple lasso regularized least squares problem.
  • The experiments and ablation studies are unexpectedly detailed and comprehensive, involving multiple model scales, model families and comparison against various baselines.
Critical weaknesses

None

Questions for authors
  • From Table 14 I understand that there are 10001000 steering operators built on top of the intervention layers, but the given subsets per example is only 1010. On the other hand, the pre-training LDS evaluation has a subset fraction α\alpha of 0.50.5. It is unclear to me why dd is selected as a very small number.
  • I am curious on why the LDS scores do not show a constant pattern with increasing depth/model size (e.g. Table 1). Can the authors provide an insight on the relation between the computed LDS score for STRIDE and model size? While larger models should obtain a higher pretraining and validation loss, why don't the LDS scores follow a similar pattern?
Reviewer_PjHxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the Training Data Attribution (TDA) problem, which is to understand what in the training data influenced the model's prediction. The authors propose STRIDE, a framework for TDA that introduces a new formulation of TDA as a sparse recovery problem. STRIDE works by learning a set of steering operators that try to represent behavioral shifts caused by training on data subsets. The evaluation is done on four pre-training settings and STRIDE appears to be competitive with prior techniques for TDA.

Critical strengths

The critical strengths of this paper, in my opinion, are:

  • a new formulation of TDA: the paper frames TDA as sparse recovery over a fixed bank of learned subset-level probes, which is different from the gradient-based methods and the representation-based ones. This is a very interesting contribution in my opinion because it may open new unexplored avenues to improve TDA.
  • strong performance against a wide variety of baselines: STRIDE appears to have the highest LDS at all pre-training scales, and this is measured against gradient-based, representation-based, and unrolling methods. It seams to be cheap and practical to use. The ablations also show that each term contributes to STRIDE effectiveness.
Critical weaknesses
  • operators cover only 1.4% of candidate corpus: according to this table 14, operator training uses 10k iteration times 8 operators times 2 examples, for a total around 160k fidelity examples. At the 1.38B scale each subset holds about 114k examples, so each operator is fitted on aobut 160 of them, and at most 1.4% of the 11.4M candidates are seen during training. Since subsets are uniform random sampled, operators can differ only through those few sampled examples. For the remaining 98%, the score depends only on which random subsets the example was assigned to and not on its content. I think this may undermine the core claim of per-example attribution over millions of candidates.
  • the claim is not sufficiently backed by evidence: for instance, the reported sparsity is imposed and not found by settting \lambda = 0.8 \lambda_max (so Fig.5 and Tab.8 are not evidence that influence is sparse I think). By using K << n, any measurement vector can be fit exactly by some w. So additivity can’t be tested at the subset level. Also the recovery theory does not apply at the stated parameters: K=1000, d=10, k=50, we have that a set of 100 examples sends 1000 edges into 1000 subsets. A random set reaches about 630 distinct subsets, smaller than 833, which is was the (2k, 1/6)-expander condition requires.
Questions for authors
  • at the 1.38B scale, how many distinct candidates does the fidelity loss see during operator training? From tab 14 each operator appears to be fitted on about 160 of its roughly 114k examples, so at most about 1.4% of the 11.4M candidates are ever seen. Please confirm or correct this. If it is correct, could you report LDS and contamination recall separately for candidates the operators saw and candidates they never saw, and explain what mechanism would give the unseen examples meaningful scores?
  • Does your membership matrix M satisfy the (2k, 1/6) expansion condition required by Lemma 1 at K = 1,000, d = 10 and k \approx 50? A random set of 100 examples reaches only about 630 distinct subsets, below the 833 required, and the pairwise-overlap bound from Remark 1 appears to be violated as well. If the condition does not hold, what role does Lemma 1 play in justifying the method?
Reviewer_cfSaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces STRIDE, as an efficient data attribution tool using model activations (i.e. hidden states) instead of gradient-based methods. STRIDE is trained to learn a low-rank steering operator to fit the subset utility. The evaluation shows that STRIDE performs a higher linear datamodel score (LDS) with better efficiency.

Critical strengths

(1) Novelty on Problem Formulation: The authors formulate the TDA problem from the activation-space perspective, which skip the gradient computation.

(2) Efficiency Gain: using the proposed low-rank activation steering operators avoids the model retraining cost and gradient computation overheads, which improves the efficiency of data attribution while preserving a strong LDS correlation.

Critical weaknesses

(1) Lack of Adaptivity: The method only capable to perform an offline scoring given a static model checkpoint, while it is not applicable to do online data attribution/selection.

(2) The architecture is quite complex, with lots of hyperparameter choices not address: firstly, the steering operator fitting directly relies on a predefined subset list A1−AkA_1 - A_k, while the impact from the granularity and partition criterion of these subsets are unsure; How much additive regularization and stability loss are needed has not been ablated. Could the pipeline robust according to various hyperparameter change, including λstab\lambda_{stab}, λadd\lambda_{add} and k?

(3) Limited experiments and evaluations: the "Data selection" experiment only present the greedy rank of a pre-defined set of data, but how the proposed method compares with other baselines in model performance from real instruction finetuning or post-training experiments? Also, how reliable is the presented "LDS" scores?

(4) Over-state the computation overhead of gradient-based methods: the proposed ghost-engine method computes all training samples' gradient alignment scores in one forward-backward pass, which improves the efficiency a lot. It has been widely used currently in gradient-based data attribution and online data selection practices [1, 2, 3].

[1] Data Shapley in One Training Run

[2] GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration

[3] OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration

Questions for authors

(1) How does the proposed method generalizable across various model checkpoints along the training? or do we need to retrain the steering operator for every checkpoints?

(2) How robust is the proposed pipeline? Can you show an ablation on the hyperparameters λstab\lambda_{stab}, λadd\lambda_{add} and subset partition criterion and granularity kk?

(3) Can you present the final performance improvement from data selection experiments across several datasets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.