acceptodds
Under review as a conference paper at ICLR 2027

Conflict-Aware Physics-Informed Neural Networks

Abstract

Due to the mesh-free property and the capability to directly embed physical laws into deep learning models, Physics-Informed Neural Networks (PINNs) have become a promising tool for solving partial differential equations (PDEs). Although these methods have made significant progress, gradients associated with different loss terms can differ substantially in both magnitude and direction, leading to gradient conflicts and difficulties in stable and efficient optimization. While existing balancing methods can mitigate such conflicts to some extent, they typically rely on predefined loss reweighting or gradient manipulation rules that do not explicitly adapt to the evolving conflict state during training. In this paper, we propose Conflict-Aware Physics-Informed Neural Networks (CA-PINNs), which introduces an Adaptive Conflict Modulation mechanism to dynamically adjust the strength of conflict mitigation during training. Specifically, we first quantify the conflict state by considering both directional disagreement and gradient-norm imbalance. Then, we propose Adaptive Conflict Modulation that leverages these two complementary signals to dynamically regulate the constraint radius of the optimization problem, therey balancing conflict resolution with optimization progress. Extensive experiments on five PDE benchmarks demonstrate that our CA-PINNs achieve competitive and consistent predictive accuracy across different physical systems and loss decompositions.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Physics-Informed Neural Networks, Scientific Machine Learning, Multi-Objective Optimization, Gradient Conflict, Adaptive Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_dXJaOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Conflict-Aware Physics-Informed Neural Networks (CA-PINN), a multi-objective optimization algorithm for training PINNs. The method extends existing MTL method, CAGrad, by adjusting the hyperparameter (adaptive conflict modulation) during training based on both direction disagreement and gradient norm imbalance. The experiments show that the proposed method achieves superior performacne across benchmark equations.

Critical strengths

Strengths

  1. The paper is well written and easy to follow.

  2. CA-PINN shows superior performance over existing methods across PDE benchmarks. The experiments are also extensive and carefully controlled, including a comparison with a tuned fixed coefficient, a sensitivity analysis, and computational cost analysis.

Critical weaknesses

Weaknesses

In my opinion, the main contribution of this work is somewhat incremental. The issues of gradient conflict and gradient magnitude imbalance are well-known problems. Gradient conflict and gradient magnitude imbalance are widely recognized problems in PINN training, and the proposed method is essentially another variant of existing MTL methods. As a result, the contribution relies mainly on the empirical results. However, the benchmarks in the paper are already solved well by existing methods. The paper does not show cases where existing methods fail and CA-PINN overcomes their limitations.

Questions for authors
  1. Can the authors provide an example where the existing methods fail but the proposed method succeeds?
Reviewer_x1C7OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

CA-PINN makes CAGrad's correction coefficient time-varying for PINN training. It combines average pairwise gradient angles and a clipped gradient-norm ratio into a conflict score. This score controls the correction radius through a linear mapping and EMA. The paper claims adaptive conflict mitigation and a descent guarantee. It evaluates five forward PDE benchmarks using two- and three-loss settings, ablations, and tuned fixed-CAGrad controls.

Critical strengths

None

Critical weaknesses

1. The methodological contribution is marginal, and the proposed conflict score lacks a sound justification.

Equations (7)–(9) are CAGrad's constrained problem, dual formulation, and update direction [1]. Apart from numerical stabilization, CA-PINN retains the same optimizer and only makes its coefficient cc depend on a heuristic score and EMA. The paper does not establish why this extension constitutes a substantive contribution.

The motivation also overlooks substantial prior analysis. PCGrad [2] and DCGD [3] already examine the joint role of gradient angles and magnitudes. HARMONIC [4] and ConFIG [5] construct updates from PINN gradient geometry. GradVac [6] already tracks pairwise gradient similarities with EMA to adapt correction targets. CA-PINN does not identify a concrete failure of these approaches that its global controller resolves.

More fundamentally, the score discards information needed to assess objective-wise descent. For the mean gradient g0=M−1∑jgjg_0=M^{-1}\sum_j g_j,

gi⊤g0=1M∑j∥gi∥∥gj∥cos⁡θij.g_i^\top g_0=\frac{1}{M}\sum_j\|g_i\|\|g_j\|\cos\theta_{ij}.

The proposed score instead adds an average pairwise angle to an extreme norm ratio. These summaries lose the correspondence between each gradient's magnitude and direction. Consider, with negligible numerical stabilization,

e=(1,0),\quad v=(-1/2,\sqrt{3}/2),\\ G_A=(100e,e,50v),\quad G_B=(50e,e,100v). \end{gathered}$$ Both configurations have $S_{\mathrm{dir}}=4/9$ and $r_{\mathrm{mag}}=100$. Thus, CA-PINN assigns them the same target $c$. Yet $$\min_i g_{i,A}^{\top}g_{0,A}=-25/3<0,\qquad \min_i g_{i,B}^{\top}g_{0,B}=1/3>0.$$ The score cannot distinguish an update that conflicts with an objective from one that improves all objectives to first order. CAGrad's inner solver still uses the full gradients, but this does not justify the additional controller or its score. Section 3.5 does not supply that justification. Its descent proof only assumes $\|d_t-g_{0,t}\|\leq c_{\max}\|g_{0,t}\|$ with $c_{\max}<1$. It holds for any bounded coefficient sequence, including fixed $c$. It never uses the proposed score or EMA. Related mean-objective convergence results already exist in CAGrad [1]. The proof also concerns direct gradient updates, not the Adam updates used experimentally. Unlike theoretically motivated tracking methods such as MoCo [7] and PSMGD [8], the paper does not connect its temporal rule to a defined estimation or optimization problem. --- ### 2. The experiments do not establish that adaptive modulation, rather than hyperparameter selection, explains the reported gains. The method replaces CAGrad's single coefficient with several hyperparameters, including coefficient bounds, initialization, score weight, magnitude threshold, and EMA factor. The sensitivity is substantial. In Appendix Table C.8, changing $\lambda_{\mathrm{dir}}$ from $0.80$ to $0.75$ worsens three-loss Burgers MSE by **5.14 times**. The procedure used to select the shared defaults is unclear. Appendix C.1 tunes fixed CAGrad using "validation loss", but the exact metric and separation from reference-solution evaluation are not adequately explained. I cannot rule out benchmark-dependent hyperparameter selection from the reported protocol. The fixed-CAGrad results are relevant, but they do not isolate the proposed mechanism. A fixed coefficient matched to the adaptive trajectory's mean and a no-EMA control are absent. The main comparisons also omit uncertainty estimates and paired seed results. For example, three-loss Burgers differs from PCGrad by only **0.31%** in reported MSE. Directly relevant PINN baselines, including DCGD [3] and HARMONIC [4], are missing. Reproducibility is a serious concern. I could not locate executable code or a repository link in the provided manuscript. The Reproducibility Statement describes settings but does not identify a code artifact. The two-loss search resolution and the multi-loss inner solver's optimization and stopping rules are unspecified. These choices directly affect results and computation time. The pooled cost totals also require clarification: CA-PINN is $1.26\times$ Adam in total running time but $1.57\times$ in training-only time. The reported evidence does not establish faster optimization under a matched time-to-accuracy comparison. Finally, the manuscript requires more careful editing. Section 2.2 repeats descriptions of existing methods, and Equation (5) introduces undefined $L_{\mathrm{data}}$ and $\lambda_{\mathrm{data}}$. These are secondary concerns, but they further weaken the presentation of an already limited contribution.
Questions for authors

1. Contribution and mathematical justification. What does CA-PINN establish beyond tuning CAGrad's coefficient? Why should the proposed weighted score determine the correction radius, given the counterexample above? What part of Section 3.5 specifically supports the chosen score or EMA, rather than any bounded ctc_t?

2. Hyperparameter selection and temporal controls. How were all controller hyperparameters selected? Was reference-solution error used at any stage? What exactly is the "validation loss" used for fixed CAGrad, and were development and evaluation data separated? Please provide search budgets and matched comparisons against mean-matched fixed cc and no-EMA variants.

3. Missing decisive baselines and statistical evidence. Please add matched comparisons with [3] and [4]. Please also evaluate [9] (LRA + SOAP), using the same backbone and training budget. Report per-seed errors, dispersion, and time-to-accuracy.

4. Code and reproducibility. Where is the executable code for this submission? No repository or code artifact is identified in the manuscript, despite the stated reproducibility provisions. Please provide the exact implementation used for the tables, including the dual solver's grid resolution, iteration count, stopping criteria, numerical stabilization, and timing protocol.


References

[1] Liu et al., "Conflict-Averse Gradient Descent for Multi-task Learning". NeurIPS 2021.

[2] Yu et al., "Gradient Surgery for Multi-Task Learning". NeurIPS 2020.

[3] Hwang et al., "Dual Cone Gradient Descent for Training Physics-Informed Neural Networks". NeurIPS 2024.

[4] Bu et al., "Harmonized Cone for Feasible and Non-conflict Directions in Training Physics-Informed Neural Networks". ICLR 2026.

[5] Liu et al., "ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks". ICLR 2025.

[6] Wang et al., "Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models". ICLR 2021.

[7] Fernando et al., "Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Approach". ICLR 2023.

[8] Xu et al., "PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization". AAAI 2025.

[9] Wang et al., "Gradient Alignment in Physics-Informed Neural Networks: A Second-Order Optimization Perspective". NeurIPS 2025.

Reviewer_c72cOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes adaptive conflict modulation for the training of physics informed neural networks. Their method uses constrained optimization to maximize the improvement of the slowest loss term, while being close to the average gradient. It is shown to be effective on some PDE cases, with some necessary proof for correctness.

Critical strengths
  • The Adaptive Conflict Modulation is intuitive and easy to understand.
  • It achieves the best performance on 4 out of 5 PDEs tested.
Critical weaknesses
  • At the end of "We reveal that gradient conflict in PINN training is jointly manifested in directional disagreement and gradient-norm imbalance." IMHO, both conflicts are well-revealed in the existing literature (e.g. ConFIG and DCGD). It does not qualify as a contribution.
  • CA-PINN needs 5-6 more training hours than Adam and ConFIG.
  • The paper proves the correctness/convergence of the Adaptive Conflict Modulation, but not the convergence rate advantage.
  • As the authors write, the experiments use PDE cases of moderate difficulty. Some suggestions: add wavenumber/spatial frequency for Helmholtz or Heat equations; Burgers with lower viscosity.
Questions for authors
  • Could the authors include another controlled comparison at the same training time/FLOPs?
  • When there are multiple PDEs, are they considered separately when calculating adaptive confliction?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.