Cross-Epoch Interactions in Training Data Attribution

Training data attribution predicts how modifying training data affects a model and helps practitioners choose corrections without retraining each candidate. Influence functions approximate retraining through inverse curvature at a unique optimum, although their predictions can differ substantially from deep-network retraining. Unrolled differentiation avoids this assumption by following the recorded optimization steps. Practical approximations reduce cost by shortening trajectories or averaging within training segments. However, these methods do not explicitly quantify how visits in different epochs combine. Propagated visit contributions can reinforce or cancel in the terminal response. We quantify these interactions through the finite-horizon stability dimension (FHSD), the average squared scaled replacement derivative of the final parameters. This quantity decomposes exactly over visit pairs and, under random reshuffling, is tightly bounded by the epoch count times the sum of single-visit energies. Building on this decomposition, we propose PathGram, which propagates each candidate's visit contributions through a shared leading Hessian eigenspace to predict signed loss changes. On randomly reshuffled BERT-tiny and BERT-base, cross-epoch pairs account for about 96% and 94% of the mean squared linear prediction, respectively. In the primary BERT and Swin-T experiments, PathGram achieves normalized mean absolute errors (NMAEs) of 0.071–0.106 and recovers both full-batch BERT oracle top-six sets. We further refine the approximate neighboring path with four gradient measurements. On independent confirmation runs, this refinement reduces NMAE from 0.1034 to 0.0027 for Swin-T feature adaptation and from 0.0741 to 0.0023 for BERT-tiny LoRA.