Influence Functions for Non-Stationary Training Checkpoints via the True Hessian

We generalize influence functions to non-stationary training checkpoints by constructing a local quadratic model of the training objective under a trust-region proximity penalty, which recovers the classical estimator and, in form, the Proximal Bregman Response Function (PBRF). Unlike PBRF, our estimator keeps the true Hessian, including its negative curvature, and requires solving two linear systems in the indefinite Hessian damped by a trust-region parameter $\lambda$. We give a selection rule for $\lambda$ that keeps the damped Hessian invertible with a safety margin and the quadratic model within an estimated trust radius, while controlling how much curvature the damping discards. We solve both systems with MINRES, and give a Lanczos–Taylor approximation that reuses the Krylov basis already computed for $\lambda$-selection. We evaluate on four-layer MLPs trained on UCI Concrete, MNIST, and FashionMNIST. On UCI Concrete, our estimator attains the highest Linear Datamodeling Score (LDS) for a 10-model ensemble and matches the strongest baseline, EKFAC-IF, for a single model, ahead of PBRF and ASTRA-IF. On MNIST it is second to PBRF and ahead of ASTRA-IF, EKFAC-IF, TracIn, and TRAK. On FashionMNIST it ties PBRF and ASTRA-IF and is ahead of EKFAC-IF, TracIn, and TRAK.