- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Trajectory-based data attribution methods approximate the influence of a training sample at a specific iteration on final model outputs by tracing through the training trajectory from that iteration. This paper decomposes error sources in trajectory-based data attribution into 3 categories: system-level, config-level, and algorithm-level. The config-level error is mainly due to optimizer mismatch, where previous works leverage the SGD assumption. To deal with this, this paper proposes AdamW-influence, which faithfully accounts for AdamW's optimization dynamics and thus improves the Spearman correlation. In addition, this paper shows that the algorithm-level error is due to first-order Taylor approximation governed by the learning rate and the remaining trajectory length. This paper also derives a proxy for per-sample error. Then, these insights are translated into practical guidelines for data selection in both offline and online settings.
Critical strengths
- This paper introduces AdamW-influence, which faithfully accounts for AdamW's optimization dynamics during estimating data attribution and thus improves the Spearman correlation between estimated attributions and the ground-truth values. The improvement in correlation is up to 300%.
- This paper illustrates that algorithm-level error mainly arises from the first-order Taylor expansion that AdamW-influence leverages to estimate attribution. The magnitude of this update-estimation error is governed by both learning rate and trajectory length. In addition, a closed-form error proxy is introduced to approximate per-sample error along the original training trajectory without retraining.
- The proposed offline and online AdamW-influence outperform SGD-influence significantly.
Critical weaknesses
- The proposed AdamW-influence approach can approximate data attribution for models trained with the AdamW optimizer but may not extend to other optimization methods.
- The method requires a substantial computational cost, particularly for large pretrained models. To reduce this cost, SGD-influence uses efficient random projection, while AdamW-influence employs an ensemble of random masking. This masking strategy raises concerns because the estimation error might accumulate with first-order Taylor approximations and random masking. In this paper, the authors apply only 256 dimensions per mask for the Alpaca + Llama 3.2-1B model, which ignores the contribution of massive unselected model parameters.
Questions for authors
Please address my concerns in Weaknesses and:
- Could the authors provide a detail description of how online AdamW-influence work?
- What is the performance of online AdamW-influence compared to GREATS [1]?
[1] Jiachen T Wang, Tong Wu, Dawn Song, Prateek Mittal, and Ruoxi Jia. Greats: Online selection of high-quality data for llm training in every iteration. Advances in Neural Information Processing Systems, 37:131197–131223, 2024
4 comments
Q1: How does online AdamW-influence work?
We thank the reviewer for the question. Online AdamW-influence performs data selection at each training step by sampling a candidate batch, estimating each candidate's influence on validation loss over a K-step look-ahead horizon, and selecting the most beneficial examples for the model update. The influence is estimated by propagating sample perturbations through AdamW's parameter and optimizer-state dynamics. Here, K controls how many subsequent training steps are considered, balancing longer-term influence against accumulated approximation error. We have added a more detailed description of this procedure and the experimental setup in Section 5.2.
Q2: GREATS comparison
We thank the reviewer for the question. Under our unified framework, GREATS corresponds to the immediate-effect setting. We therefore evaluate a GREATS-like baseline to isolate the benefit of look-ahead. This baseline uses K=0 without implementing the greedy selection procedure of GREATS. Online AdamW-influence with a tuned look-ahead horizon achieves better performance on both MNIST+MLP and Tulu3+Llama 3.2-1B.
| Method | MNIST+MLP (Error Rate ↓) | Tulu3+Llama3.2 (Perplexity ↓) |
|---|---|---|
| Standard | 0.1045 ± 0.0017 | 2.2466 ± 0.0461 |
| LESS (same bs as online) | 0.1082 ± 0.0015 | 2.0677 ± 0.0302 |
| Offline AdamW-influence (same bs as online) | 0.1057 ± 0.0020 | 1.9741 ± 0.0251 |
| Online SGD-influence (tuned K) | 0.0952 ± 0.0017 (K=10) | 2.0763 ± 0.0349 (K=2) |
| Online AdamW-influence (K=0, GREATS-like) | 0.0965 ± 0.0016 | 1.8554 ± 0.0163 |
| Online AdamW-influence (tuned K) | 0.0935 ± 0.0011 (K=10) | 1.8470 ± 0.0230 (K=2) |
Thank you for providing additional experimental results and clarifications. Most of my concerns have been adequately addressed; thus, I maintain my rating leaning toward acceptance.
We thank the reviewer for the positive feedback and suggestions!
We sincerely thank the reviewer for the positive feedback, and for recognizing the contributions of AdamW-influence, our analysis of approximation errors, and the improvements in downstream data selection.
We thank the reviewer for raising this concern. Our key message is that attribution methods should faithfully match the optimizer used in the actual training process, rather than that AdamW-influence is a universal solution. As demonstrated in Appendix F.1 (Table 4), when models are trained with SGD, SGD-influence substantially outperforms AdamW-influence, further validating the importance of optimizer alignment. We focus on AdamW because it is widely adopted in modern deep learning and presents non-trivial challenges in deriving how sample perturbations propagate through its moment states.
We thank the reviewer for raising this concern. We would like to clarify why random masking is necessary for AdamW-influence and how we mitigate the resulting information loss.
First, random masking is necessary to preserve AdamW's coordinate-wise optimization dynamics. Unlike SGD-influence, which supports random projection, AdamW-influence explicitly propagates perturbations through coordinate-wise adaptive scaling and optimizer moment states. Random projection mixes parameter coordinates and therefore breaks these dynamics. In contrast, random masking preserves the original coordinates, allowing faithful propagation within the retained dimensions (Appendix D.1).
Second, random masking substantially reduces computational cost. Retaining S out of p parameters reduces the complexity from O(3Tbp2) to O(3TbS2), providing a reduction in computation.
Third, we use an ensemble of independent masks to mitigate information loss and variance. Although each mask retains only 256 dimensions for Llama 3.2-1B, different masks capture complementary subsets of parameters, and their attribution estimates are aggregated. We acknowledge that masking introduces additional approximation error, but our fidelity evaluation against TSLOO ground truth already measures its combined effect with first-order approximation.