MINN: A Math-Informed Neural Optimizer with Global First-Order Convergence

We present MINN (Math-Informed Neural Network optimizer), a hybrid learned–classical optimizer for unconstrained nonlinear optimization. Purely learned optimizers can provide fast amortized progress, but often stall or fail under distribution shift; classical methods are more reliable, but may be slower and less adaptive. MINN bridges this gap through a principled two-stage architecture that separates neural proposal from classical certification. At each outer iteration, a coordinate-wise LSTM direction is combined with a steepest-descent direction through a bounded scalar trust mechanism. A descent-preserving projection safeguard and Armijo backtracking line search then produce a safeguarded Stage-1 step with sufficient decrease. The safeguarded point is further refined by an inner loop of learnable L-BFGS-style steps, where curvature-pair weights and step sizes are trainable. An accept/reject monotonicity filter accepts this refinement only if it decreases the objective relative to the safeguarded point, allowing the full method to inherit monotone decrease and global first-order convergence under standard smoothness assumptions. All parameters, including the LSTM weights, trust-head parameters, L-BFGS memory weights, and step-size parameters, are trained end-to-end using an unsupervised amortized loss with an increasing-horizon curriculum. Experiments on least-squares, neural network training, and nonconvex Rastrigin benchmarks show that MINN improves the speed–accuracy trade-off, generalizes across dimensions and distributions, and avoids the stalling and out-of-distribution failures of purely learned optimizers.