SLED: Sobol-Leverage Designs for the Collocation Batch of Quasi-Newton Physics-Informed Neural Network Training

A *two-phase optimization schedule* for physics-informed neural network training, a first-order *warmup* followed by a quasi-Newton optimizer, achieves more accurate results than single-phase approaches in a growing number of benchmarks. The quasi-Newton phase minimizes the objective on a collocation batch that is usually held fixed, and no pipeline selects that batch from the tangent features of the residual after the warmup. For the network linearized there, every gradient and Broyden-class method on a fixed batch returns the minimum-norm least-squares fit of that batch. The network leaves this linearization, but on all six benchmarks we study the linearized fit still orders the frozen batches by their loss off the batch, and a batch is judged by that loss. We build the batch once, after the warmup. Specifically, the Sobol-leverage design (Sled combines a scrambled-Sobol set with points drawn from the ridge leverage of the residual's tangent features, in a share computed on the candidate pool. The ridge-leverage sampling bound, stated for these features, gives a sufficient batch size for a leverage batch, and the two benchmarks on which leverage points alone fail are the two on which the batch lies below it. On six benchmarks, Sled reaches a median error at or below that of three established frozen batches, the uniform default among them, and of three refresh protocols, up to $60\times$ and $375\times$ below the default, at $2.1-4.2\%$ of the cost of the phase. It reaches the default's error at $16,384$ points with $2-8\times$ fewer points on four of them. Each component alone trails it somewhere: leverage points fail on two benchmarks, and the Sobol set is $32-82\times$ less accurate on two others.