- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Edge-Game Replay (EGR), a framework that replays the training to quantify a sample's influence on the model. This process separates the training into paths such as supervised loss and the effect on shared batch statistics, using Shapley values to measure the contribution of each of these paths to the final loss. The paper claims that this helps quantify influence in models that use batch normalization.
Critical strengths
The paper proposes a novel framework to quantify a data sample's influence. The empirical gains shown in the paper are substantial.
Critical weaknesses
Computationally, this framework is expensive. So the practical usage of this may be limited. Another big weakness is that this framework only applies to architectures with BatchNorm, so it can not be applied as is to more modern architectures. Additionally, while the paper does cite recent trajectory-based data attribution methods such as SOURCE [1] and Simfluence [2], it does not show empirical comparisons of performance.
- Bae, Juhan, et al. "Training data attribution via approximate unrolling." NeurIPS (2024).
- Guu, Kelvin, et al. "Simfluence: Modeling the influence of individual training examples by simulating training runs." (2023).
Questions for authors
- How can this framework be extended to more modern architectures that use LayerNorm, for example?
- Can there be some approximations done to improve the computational overhead?
- How does this method compare to SOURCE and Simfluence?