- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces a cheaper alternative to Training Data Attribution (TDA) named STRIDE, which learns multiple steering operators instead of retraining on multiple subsets of training data. STRIDE builds on the insight that model predictions, achieved by retraining on different subsets of data, can also be achieved by steering the model activations. Based on this, multiple low-rank steering operators are learned, which are built on top the frozen model parameters. These operators are lightweight compared to retraining. After the operators are learned, they are applied on test queries and the resulting loss changes are noted. Then, these changes are approximated/evaluated as per-example contributions. Finally, per-example influence scores are obtained via sparse recovery, which is is basically a -regularized least square problem. The resulting STRIDE framework, which can be summarized as training steering operators and approximating per-sample influence scores based on them, achieve the highest LDS score among various TDA methods, which are all much more computationally demanding to compute.
Critical strengths
- The STRIDE framework provides an alternative way to look at the TDA problem. The idea of modeling the retraining on different subsets of training data as steering the model activations is a good motivation and makes the whole setup cheap to run, with just 1000 (number of operators) forward passes per query once the operators are learned, and solving a simple lasso regularized least squares problem.
- The experiments and ablation studies are unexpectedly detailed and comprehensive, involving multiple model scales, model families and comparison against various baselines.
Critical weaknesses
None
Questions for authors
- From Table 14 I understand that there are steering operators built on top of the intervention layers, but the given subsets per example is only . On the other hand, the pre-training LDS evaluation has a subset fraction of . It is unclear to me why is selected as a very small number.
- I am curious on why the LDS scores do not show a constant pattern with increasing depth/model size (e.g. Table 1). Can the authors provide an insight on the relation between the computed LDS score for STRIDE and model size? While larger models should obtain a higher pretraining and validation loss, why don't the LDS scores follow a similar pattern?