Gradient–Update Mismatch: Rethinking Conflict-Free Training of Physics-Informed Neural Networks

Training Physics-Informed Neural Networks (PINNs) requires jointly optimizing physics residual and initial/boundary condition loss terms, which often induce conflicting gradients. Gradient surgery methods mitigate this issue by constructing directions from loss-specific gradients to reduce conflict before optimizer transformation. However, even when the constructed direction is conflict-free, this property may not be preserved after optimizer transformation. Let $a_t$ denote the direction constructed by gradient surgery, $u_t$ the optimizer proposal, and ${\mathcal{C}}_t$ the conflict-free cone induced by the loss-specific gradients. We show that modern optimizers can transform $a_t$ through mechanisms including historical state, adaptive scaling, preconditioning, and decoupled weight decay, so $a_t\in {\mathcal{C}}_t$ does not generally imply $u_t\in {\mathcal{C}}_t$. We refer to this optimizer-induced discrepancy in conflict-freeness between $a_t$ and $u_t$ as **Gradient-Update Mismatch** (GUM). Accordingly, we propose **Gradient-Update Alignment** (GUA), which performs conflict handling on the update actually applied to the parameters. Specifically, after the optimizer transforms $a_t$ into $u_t$, GUA projects $u_t$ onto ${\mathcal{C}}_t$ to obtain the aligned update $p_t$, which is then applied to the parameters. For optimizers with internal state, GUA further aligns the state with targets derived from the applied update. Extensive experiments show that GUM is widespread across diverse optimizers, with conflict rates reaching up to 86.3%. Across all PINN settings, GUA achieves conflict-free applied updates and consistently improves various gradient surgery methods, reducing relative $L_2$ error by up to 98.2%. Data and code are available at [https://anonymous.4open.science/r/GUA-F010](https://anonymous.4open.science/r/GUA-F010).