Adaptive Backpropagation for Stable Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) differentiate losses containing spatial or temporal derivatives of network outputs, making backpropagation expensive and potentially unstable. We study a semi-gradient path that preserves residual values while replacing the residual Jacobian with the network-output Jacobian. This reduces update cost and can improve numerical stability, but prolonged use may cause oscillations and limit accuracy. We propose Prediction-guided Adaptive Semi-gradient Switching (PASS), which starts with semi-gradients and switches once to full gradients when successive prediction changes on a fixed probe set lose positive alignment. The criterion requires only output evaluations and no reference solution. Experiments demonstrate improved accuracy across optimizers, architectures, and PINN training techniques, at computational cost comparable to full-gradient training.