Representer Point Selection via Tracking Regularized Gradient Trajectory

Understanding the individual contribution of training samples to model predictions is essential for enhancing model transparency and facilitating data-driven debugging. While recent advancements, such as Representer Point Selection and Neural Tangent Kernels, offer efficient frameworks for sample-based attribution, they face significant practical hurdles. Specifically, existing methods often necessitate performance-altering regularization that deviates from the original model, rely on computationally prohibitive Hessian-Vector Products, or fail to faithfully reconstruct the original parameters. Consequently, they explain a surrogate approximation rather than the actual model's predictions. To address these challenges, we propose an alternative derivation for Representer Point Selection via Tracking Regularized Gradient Trajectory (RPS-TrReg), which identifies sample importance by tracing a regularized optimization path toward the original model's parameter state. Unlike previous methods that optimize a surrogate model in isolation, RPS-TrReg incorporates a regularization term to ensure the tracking phase remains strictly anchored to the target model's weights. This enables a high-fidelity decomposition of the target model into a linear combination of training samples without the need for expensive second-order derivatives. Empirically, our results demonstrate that RPS-TrReg consistently outperforms state-of-the-art explanation methods in data debugging tasks across vision and language classification benchmarks.