- Rating
- 4
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents MAGIC, a data attribution method that uses differentiation through training to predict how changes in training-data weights affect model behavior. The results show substantial improvements in counterfactual prediction over the evaluated baselines across vision and language settings.
Critical strengths
-
The paper is well-written and easy to follow. The motivation is clear, and the experiments make the improvement in prediction quality readily apparent. I also appreciate the distinction between attribution for a particular training run and attribution averaged across training randomness.
-
The approach has a useful role in realistic workflows where practitioners control the training or fine-tuning pipeline. For example, a team investigating a particular failure mode may want to compare many possible data removals before committing to expensive retraining. MAGIC could help screen these interventions and identify promising changes. The aggregate-target experiment is particularly helpful here: targeting a behavior shared by a group of examples makes the approach more practical than computing separate attributions for every individual prediction.
-
The main empirical finding is also valuable beyond these applications. The results suggest that substantially better counterfactual prediction is achievable by accounting more accurately for the training trajectory.
Critical weaknesses
-
The language-model evaluation could better establish the practical scope of the approach. The GPT-2 and Gemma-2B experiments provide encouraging evidence, but leave questions about performance and cost across different fine-tuning settings. I recognize that these experiments are computationally expensive, so an extensive scaling study may be unrealistic. A focused additional setting, broader coverage of evaluation queries, or clearer runtime and memory measurements for the existing LLM experiments would help readers judge when the method is practical.
-
The limits of the additive prediction model deserve further investigation. Although the influence coefficients are computed exactly for the specified training procedure, predicting finite interventions still relies on a first-order approximation. Related or redundant training examples may have joint effects that differ substantially from the sum of their individual effects. The existing targeted-removal experiment is a useful starting point. Extending that analysis to characterize prediction errors for related groups of examples, alongside random removals at matched budgets, would clarify when the approximation remains reliable. This could be addressed through additional evaluation and discussion without requiring a new higher-order method.
Questions for authors
What primarily limits broader LLM evaluation: attribution computation, checkpoint storage, or the counterfactual retraining needed for evaluation? Also, have you observed particular types of jointly removed examples for which the linear predictions become substantially less accurate?
We thank the reviewer for their positive review of our work, as well as the helpful questions and comments which will help us improve the manuscript. In addition to the experiments performed in response to the other reviews, we will make sure to clarify the points below in our next revision. Thanks again!
We appreciate the reviewer's concern here and indeed, our paper does not claim to invent metagradient computation or show how to compute it at scale. Instead, we view the link between Engstrom et al and our work as analogous to the link between the EK-FAC Hessian approximation (George et al., 2015 [1]) powering the EK-FAC influence method (Grosse et al., 2023); or even to the infinitesimal jackknife (Hampel, 1947 [2]; Pregibon, 1981 [3]) powering the now-seminal work of Koh and Liang (2017).
To be more precise, the metagradient provides a computational tool for differentiating through optimization. It was not clear prior to our work (see, e.g., Basu et al., 2020) that even being able to compute such gradients would yield fruitful results for deep learning models, nor that one would be able to achieve near-perfect LDS in a variety of settings. Note that an LDS of 1.0 is not obviously possible: it means that one can perfectly predict the effect of dropping data on a model, without actually retraining the model, which we were very excited to see.
[1] George et al., Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis (2015) [2] Hampel, The Influence Curve and its Role in Robust Estimation (1947) [3] Pregibon, Logistic Regression Diagnostics (1981)
The reviewer is correct in pointing out that the performance of MAGIC seems to degrade with the drop fraction. The reason for this is that for large enough drop fractions, the model behavior itself stops behaving linearly in the dropped data. We were pleasantly surprised that empirically, the performance of MAGIC seems to degrade gracefully in the drop fraction, rather than plummeting at a fixed value, but to some extent a certain amount of degradation is inevitable. Our intuition is that capturing these higher drop fractions would have to come at the cost of near-perfect LDS at the small drop fractions where MAGIC excels, but we are excited for future work to investigate this direction further.
While it is true that the cost of MAGIC increases with the number of test queries, there are two considerations that we believe make MAGIC still a very practical solution:
For a fixed test query, MAGIC is significantly more efficient than any prior method we are aware of. In particular, since we do not require computing any inverse Hessian-vector products or per-sample training gradients, each individual test query in MAGIC runs ~100x faster than, e.g., TRAK. This means that even for tens of test queries, MAGIC provides very competitive runtime along with near-perfect attribution.
The cost scaling can be improved using computational tricks like vectorization: in particular, we only need to train and checkpoint the model once, and then can propagate k test queries simultaneously using
vmapor similar tools.Together, #1 and #2 give us some encouragement that MAGIC can be a competitive attribution solution even in the multi-query case. There is also the opportunity for exciting algorithmic improvements to MAGIC that would allow it to apply to several test queries at once, e.g., by using randomized linear algebra techniques like streaming PCA. This would be significantly outside the scope of our paper though, and we leave it to future work to explore the practical gains from using such techniques.
We sincerely thank the reviewer for flagging these issues and will ensure they are fixed in the potential camera-ready version of our paper. We will also take another careful pass over the paper to ensure that the experimental details are fully documented and there are no similar typos or confusions. Thanks again.