How Faithful Is Trajectory-Based Data Attribution? Error Sources, Remedies, and Practical Guidelines

Trajectory-based data attribution methods estimate the influence of training samples on model outputs by tracing through the training trajectory. They are increasingly used in applications such as data selection, data valuation, and model diagnosis, but there is a lack of comprehensive error analysis of these methods, raising concerns about method faithfulness and hindering reliable deployment. In this work, we provide the first systematic analysis of error sources in trajectory-based data attribution, together with remedies to mitigate them and practical guidelines for downstream use. We organize the error sources into three categories: config-level, algorithm-level, and system-level. Building on this taxonomy, we make three contributions. First, we identify optimizer mismatch as the dominant config-level error: most existing methods derive trajectory-based attribution under the assumption of SGD, even when the model is trained with the widely used AdamW optimizer. To mitigate this error, we propose AdamW-influence, which faithfully accounts for AdamW's optimization dynamics and improves the Spearman correlation between estimated and ground-truth influence by 10% to over 300% across settings spanning MLP, CNN, GPT-2, and Llama 3.2-1B. Second, after correcting optimizer mismatch, we show that the remaining algorithm-level error due to the first-order Taylor approximation is primarily governed by two factors: the learning rate and the remaining trajectory length from a sample's training step to the final model. The estimated influence is more faithful at smaller learning rates and over shorter trajectories. We further derive a closed-form proxy for per-sample approximation error that can be evaluated along the original training trajectory without retraining. Third, we translate these insights into practical guidelines for downstream attribution-guided data selection. Our empirical results indicate that the attribution faithfulness plays a critical role in the data selection effectiveness: the proposed AdamW-influence significantly outperforms the SGD-based counterpart; while online data selection, where the estimated influence accounts for only a short training trajectory to guide each local update, often outperforms offline data selection, where the estimated influence accounts for the full training trajectory of a reference run. Together, our results provide a systematic framework for diagnosing trajectory-based attribution error and using these methods more reliably in downstream applications.