Training with Certificates: Global Optimality Bounds for Neural Networks

How much better could a neural network have been trained on the same data? A flat loss curve, a small gradient, or agreement across restarts does not provide a global bound on the improvement still possible. We develop Neural MERIT, a framework that returns a feasible neural network together with a certified interval $L_k\le p^{\star }(Y)\le U_k$ enclosing the globally optimal empirical objective, so that $U_k-L_k$ bounds the remaining global suboptimality of the returned solution. The central idea is to approach the target training problem $Y$ through a sequence of globally solved or certified surrogate problems $C_k$, and to transport each surrogate's feasible solution and global lower information back to the fixed target. For squared-loss training, we show that exact surrogate minimizers along a straight continuation yield monotonically improving target objectives even for nonconvex neural classes whose global optimizer can change discontinuously. With globally valid directional evidence, the transported certificate obeys an exact quadratic identity; in particular, a zero-gap surrogate certificate at $C$ gives target certificate width $\frac{1}{2}\Vert Y-C{\Vert }^2$. Inexact surrogate solves incur an explicit, vanishing allowance. For exact convex prediction formulations, both certificate endpoints tighten monotonically and meet at the target optimum. Globally solvable neural classes therefore provide bank-free Neural MERIT guarantees, while a Neural Basis Bank of previously certified surrogates can strengthen the same target interval through additional and cooperative evidence. Across 24 independently verified ReLU continuation paths, every final bank-free interval contracts below $4.91\times 10^{-7}$ (median $3.82\times 10^{-7}$); a 64-record basis bank certifies 18 of 24 targets to width $10^{-2}$ before continuation begins. Exact finite-network and frozen-feature experiments further show how the certificate separates optimizer stasis from globally certified training quality.