Online Inference with Momentum and Nonsmooth Scores

Momentum changes the transient behavior of stochastic gradient algorithms. We ask when these changes preserve statistical inference after Polyak–Ruppert averaging. We develop a framework for fixed stable score filters that includes SGD, fixed preconditioning, scalar and coordinate momentum, and higher-order filters. Under explicit local and trajectory conditions, the filter gain cancels from the first-order influence function. Averaged estimators therefore share the usual sandwich covariance for the same estimating equation and retained observations. The analysis permits nondifferentiable or discontinuous sample scores by imposing differentiability on their population mean and local quadratic-mean continuity. We construct one-pass sandwich intervals, including an online Jacobian estimator for quantile regression that leaves the nonsmooth update unchanged. An integrated $L^2$ process limit also justifies path-based self-normalization without the additional maximal boundary condition used here to establish uniform path convergence. Simulations show lower error for momentum in one least-squares setting with $d=100$, and online quantile coverage near the nominal level across the reported regular settings.