Stochastic Newton without Hessians: Estimation and Inference

Stochastic Newton methods use population curvature, which may remain well defined when sample Hessians are unavailable or undefined. We develop an online stochastic Newton method based on symmetric score secants and Polyak-Ruppert averaging that allows discontinuous sample scores. A spectrally projected recursive average supplies the curvature matrix for subsequent parameter updates. Under stability, iterate rate and score regularity conditions, we derive restrictions on secant second moments, the perturbation radius and the auxiliary batch size under which curvature learning is asymptotically negligible in the Polyak-Ruppert expansion. For quantile regression, shared endpoint observations give an \(O(\delta^-1)\) secant second moment bound, whereas independent endpoints have variance of order \(\delta^-2\) under nondegeneracy, so sharing enlarges the sufficient tuning region for the curvature radius. For direct unbiased primary scores, the averaged estimator retains the usual first-order sandwich limit, and when the spectral projection set contains the population curvature the learned matrix also supports consistent online sandwich estimation. We separately analyze the smoothing bias and direction-dependent covariance of function-value primary updates with auxiliary score access retained, and under additional process conditions a centered functional limit yields iterate-based self-normalized inference. Simulations examine the predicted rates and finite-sample behavior, and an online news application illustrates streaming quantile inference.