Variational Surrogate Jacobians: A Framework for Constrained Gradient Propagation

Many learning systems couple neural networks with discrete or constrained optimisation layers, but the resulting solution maps often have zero or undefined derivatives, preventing useful gradient propagation. We introduce the Variational Surrogate Jacobian (VSJ) framework, which selects the surrogate Jacobian closest to a prior backward rule under a positive-definite metric and has a unique closed form solution when selected active constraints are enforced exactly. To allow constraint enforcement to respond to the local optimisation state, we also introduce adaptive weighting of inequality constraints, which uses primal slacks and dual multipliers to set the enforcement strength. We show that VSJ recovers established backward rules as particular configurations under method-specific local assumptions, and also corrects those that depend directly on the loss. Empirically, VSJ closely reproduces six of them, while validation-selected novel VSJ configurations yield lower mean test regret for every tested prior family on at least one task. As an example, composing VSJ with PEAR, a state-of-the-art method, lowers mean Knapsack regret and increases mean Portfolio cumulative return. These results establish VSJ as a unified geometric framework for constructing, analysing, and designing gradient propagation rules through constrained optimisation layers.