From Conservative Certificates to Adaptive Capital: Faster Anytime-Valid Stopping for SGD

Stochastic gradient descent remains the default optimizer in large-scale learning, yet it is almost always terminated by heuristics such as a preset iteration budget or a plateau in the training loss. These rules provide no statistical certificate. Recent trajectory-aware stopping procedures restore validity, but they typically rely on closed-form envelopes of an exponential e-process family. In finite samples those envelopes are often loose, so certification arrives later than necessary and extra gradient steps are wasted. We introduce Stopping-Optimized e-Portfolios for SGD (SOEP-SGD). Instead of collapsing the e-process family into a conservative closed-form bound, SOEP-SGD keeps the family explicit and treats certified stopping as a capital-allocation problem: statistical mass is placed on the variance scales that actually govern when a valid certificate can be issued. We establish anytime-valid, optional-stopping-safe guarantees for convex SGD, in the form of weighted suboptimality bounds, and for smooth nonconvex SGD, in the form of weighted stationarity bounds. Simulations and CIFAR-10 experiments show substantially earlier certified stopping than existing closed-form certificates, without sacrificing validity.