acceptodds
Withdrawn from ICLR 2027

AdamFLIP: Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN Training

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Physics-informed neural networks (PINNs) provide a flexible framework for solving forward and inverse problems governed by partial differential equations (PDEs), but standard PINN training typically relies on soft penalty formulations that combine PDE residuals, data mismatch, and initial/boundary conditions using manually chosen weights. This often leads to ill-conditioning, sensitivity to loss weights, and poor constraint satisfaction. In this work, we reformulate PINN training as an equality-constrained optimization problem and propose a novel Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN (AdamFLIP). The key idea is to view the constraint residuals as the output of a controlled dynamical system and to compute the Lagrange multiplier as a system input that drives these residuals toward stable linear contraction dynamics. AdamFLIP then applies Adam-style first- and second-moment adaptation to the resulting feedback-linearized Lagrangian gradient, combining principled constraint handling with the scalability and robustness of adaptive neural-network optimization. We test AdamFLIP on a range of benchmark forward and inverse PDE problems, and it consistently outperforms both the standard soft-constrained PINN and state-of-the-art constrained optimizers. Specifically, on the Navier–Stokes equations benchmark, AdamFLIP reduces relative L_2 error by more than two thirds for the predicted solution compared to the next best method. Our AdamFLIP framework provides an effective and computationally scalable hard constraint optimization method for PINN training.

Primary area:
optimization
Keywords:
partial differential equations, inverse problems, constraint optimization, Physics-informed neural networks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mXDcOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes AdamFLIP, which treats PINN residuals as the outputs of a controlled dynamical system and applies ideas from nonlinear control theory to design a feedback linearization (FL) solver to alleviate the classical gradient pathologies problem in PINNs. Through FL-momentum, the authors introduce an adaptive inertia term lambdatlambda_t for the boundary condition, initial condition, and data loss terms in PINN training, rather than using fixed weights. Comparisons with two similar baseline methods and the FLIP ablation demonstrate that AdamFLIP achieves competitive results on 4 standard PINN forward/inverse problems.

Critical strengths
  1. Gradient pathologies is a significant challenge in PINNs. Therefore, efforts to address this issue can contribute to more robust PINN training, particularly in complex tasks.
  2. Introducing principles from control theory into PINN optimization indeed offers a novel perspective.
Critical weaknesses
  1. Regarding the positioning of the method and lack of critical related work discussion.

My primary concern is whether the proposed method constitutes a sufficiently novel constrained optimization algorithm, as opposed to a relatively straightforward combination of existing FL and Adam-style adaptive updates. This distinction is important because the paper repeatedly emphasizes AdamFLIP as a novel adaptive first-order optimizer for hard-constrained PINN training. However, based on the equation in line 224 (which the authors have not assigned an equation number), suppose that the objective and constraints of AdamFLIP are defined as follows:

f(θ)=Lpde(θ)f(\theta)=\mathcal{L} _ {\mathrm{pde}}(\theta)

h(θ)=[Lic(θ),Lbc(θ)]Th(\theta)=[\mathcal{L} _ {\mathrm{ic}}(\theta),\mathcal{L} _ {\mathrm{bc}}(\theta)]^T

Then, the gradient actually used by AdamFLIP can be written as:

gt=∇Lpde+λic,t∇Lic+λbc,t∇Lbcg_t=\nabla\mathcal{L} _ {\mathrm{pde}}+\lambda _ {\mathrm{ic},t}\nabla\mathcal{L} _ {\mathrm{ic}}+\lambda _ {\mathrm{bc},t}\nabla\mathcal{L} _ {\mathrm{bc}}

In essence, this amounts to introducing two adaptive, dynamically adjusted weights into the PINN loss function, which is similar to the adaptive loss design in many previous works [3,4,5]. Unlike other hard-constraint methods that explicitly incorporate BCs/ICs into the network architecture, AdamFLIP does not inherently guarantee exact satisfaction of BCs/ICs.

Therefore, the authors need to identify precisely what new optimization principle or nontrivial algorithmic property is introduced updates to achieve the "hard constraints", beyond combining an existing FL-based gradient construction with standard Adam. Merely employing Adam-style adaptive moments on an existing constrained optimization direction does not, by itself, establish the claimed level of methodological novelty.

A substantial body of recent work exists whether in optimization methods or adaptive loss functions. For example, optimizers specifically designed for PINNs include DCGD [1] (NeurIPS 2024) and ConFIG [2] (ICLR 2025), etc., while adaptive loss methods include SA-PINN [3] (JCP 2023), BRDR [4] (JCP 2025), and DB-PINN [5] (IJCAI 2025), etc.. Unfortunately, the authors fail to mention any of these closely related baselines in the Related Work section, nor do they discuss how AdamFLIP differs from them.

  1. The experimental evaluation suffers from a serious lack of baseline comparisons and the paper's organization and allocation of space are unreasonable.

As discussed in Point 2, numerous baselines are available from both the optimization and adaptive loss literature. However, excluding FL-PINN, which serves as an ablation, the authors compare their method only against AL-PINN (Neurocomputing 2023) and trSQP-PINN (a rejected/withdrawn ICLR 2025 submission, https://openreview.net/forum?id=GkJCgUmIqA). I do not require the authors to compare against every baseline listed in Point 2. Nevertheless, the current experimental comparisons are insufficient to substantiate the claimed superiority of AdamFLIP over other methods in the context of the state of the field in 2027. In particular, relying on a rejected/withdrawn and still unpublished submission as one of only two principal baselines, while omitting several established and peer-reviewed recent methods, raises concerns about the representativeness of the experimental comparisons and the validity of the claimed state-of-the-art performance.

The authors devote more than one page to introducing PINNs in the Preliminaries section, despite this being well-established knowledge in the field, while relegating the Related Work section to the appendix. Furthermore, the analysis in the main experimental section is overly brief, with the detailed discussion entirely deferred to Appendix D.2, even though there is still unused space on pages 8 and 9 of the main paper. In light of Points 2 and 3 above, I strongly recommend that the authors move the Related Work section into the main paper and substantially condense the Preliminaries section. The theoretical analysis in Section 4.1 could also be moved to the appendix, retaining only one or two paragraphs summarizing the key conclusions in the main paper, thereby providing more space for the experimental section.

Regarding the format, the manuscript appears to use narrative-style citations (e.g., \citet) in latex throughout, including instances where parenthetical citations (e.g., \citep) would be more appropriate. The manuscript also contains numerous incorrect cross-references and instances of overloaded mathematical notation, including but not limited to:

  • Lines 1132–1136 discuss Figure 4 but refer to Fig. 7(a), Fig. 7(b), and Fig. 7(c), even though Figure 7 does not contain any subfigures.
  • The PDE loss term in Equation (4) lacks a squared residual, which is inconsistent with the standard PINN formulation.
  • The symbol κ\kappa is initially used to denote an unknown PDE parameter but is later reused to represent the feedback control gain through K=κIK=\kappa I in Algorithm 2.
  • The abbreviation of Initial conditions (ICs) and boundary conditions (BCs) are repeatedly defined throughout the manuscript.
  • The first entry in the reference list begins directly with the paper title, omitting the author information.
  • Some equations are numbered, while others are not.

References

[1] Y. Hwang and D.-Y. Lim, "Dual Cone Gradient Descent for Training Physics-Informed Neural Networks," in Advances in Neural Information Processing Systems (NeurIPS), vol. 37, 2024.

[2] Q. Liu, M. Chu, and N. Thuerey, "ConFIG: Towards Conflict-Free Training of Physics-Informed Neural Networks," in Proceedings of the 13th International Conference on Learning Representations (ICLR), 2025.

[3] L. D. McClenny and U. M. Braga-Neto, "Self-Adaptive Physics-Informed Neural Networks," Journal of Computational Physics, vol. 474, Art. no. 111722, 2023.

[4] W. Chen, A. A. Howard, and P. Stinis, "Self-Adaptive Weights Based on Balanced Residual Decay Rate for Physics-Informed Neural Networks and Deep Operator Networks," Journal of Computational Physics, vol. 542, Art. no. 114226, 2025.

[5] C. Zhou, J. Chen, Z. Yang, and C. E. Png, "Dual-Balancing for Physics-Informed Neural Networks," in Proceedings of the 34th International Joint Conference on Artificial Intelligence (IJCAI), 2025, pp. 7164–7172.

Questions for authors
  1. What is the precise algorithmic novelty of AdamFLIP beyond combining the existing FL-momentum framework with standard Adam updates?
  2. Compared with other PINN optimizers or adaptive loss-weighting methods, in what sense can AdamFLIP be considered a "hard constraint" method?
  3. Given the authors’ claim that AdamFLIP is “the first adaptive first-order optimizer,” have they considered comparing it against established first-order optimizers, such as ConFIG and DCGD?
Reviewer_vrDBOpenReview ↗
Rating
2(−1)
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Adaptive a novel Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN (AdamFLIP). The method starts from reformulating PINN training as a equality-constrained optimization problem, view it as a controlled dynamical system problem, and then applied Adam adaptive momentum (the main novelty of the paper). A list of seven experiments, including forward and inverse problems, on different type of equations show that the method improves the accuracy apparently in most cases, in particular for Burgers accuracy and Navier–Stokes. The theory for a modified approach is provided.

Critical strengths
  1. Finding a good optimizer for PINNs to reduce its errors, has been a problem of focus and interest, given the popularity of PINNs and the hope to improve PINNs/understand PINNs better. This paper is a natural combination of many recent hot topics.
  2. The choice of the numerical examples, and the way the numerical results are designed and presented are solid, clear and comprehensive. The numerical results also look pretty good overall. It is great that the authors provide the methods for both the forward problems and the inverse problems, and study both in numerical experiments.
Critical weaknesses
  1. The paper's level of novelty is not that high, because essentially it is about combining Feedback Linearization (which is formulated in other works) with a better optimizer, which is the existing Adam adaptive momentum. Also, the the Adam-style adaptive updates and the original multiplier computation uses different metric, which poses a question on whether the introduction of he Adam adaptive momentum to the current natural framework is natural enough.
  2. As the authors mentioned themselves, the theory is on a modified method. So I am curious about how well does that method work on those numerical results. It would be great if the author can add these results. Also, full rank seems to be a restrictive assumption in theory, and may not be satisfied, e.g. under square loss constraints?
Questions for authors
  1. How well does the modified method work on the numerical results?
  2. Can you provide some insights on why using Adam adaptive momentum improves the original results so apparently?
  3. Have you tried some other optimizers when doing this work?
  4. Can you discuss more on for what type of dynamical system (e.g., depending on their nonlinearity or constraint conditioning) , you expect your method works better than other approaches, and when not? Even some intuitive descriptions would help.
  5. Can you relax the full rankedness assumption in the theory?
Reviewer_wxxLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes AdamFLIP for equality-constrained PINN training, treating initial/boundary losses (and physics losses in inverse problems) as constraints. It computes feedback-linearization multipliers and applies Adam-style moment adaptation to the resulting Lagrangian gradient. A separate, metric-consistent AMSGrad-style variant receives a finite-time convergence analysis under regularity assumptions. Experiments cover seven forward/inverse PDE settings, reporting solution errors, constraint losses, parameter estimates, runtimes, and momentum/control-gain ablations.

Critical strengths

The method addresses a useful optimization problem and shows promising empirical results. The main strengths are:

  1. The feedback multiplier provides a concrete mechanism for combining objective reduction with constraint control. Aggregating the losses keeps the multiplier system small, making the approach practical for the tested PINNs.
  2. Table 1 reports the lowest solution relative L2 errors across all seven settings, spanning Burgers, fractional diffusion-wave, heat, and Navier–Stokes equations. The forward/inverse coverage and comparison with FL-PINN help establish that adaptive moments can improve this optimization framework.
  3. The paper reports wall-clock costs and includes momentum/control-gain ablations and an anonymous implementation. These are useful for assessing the accuracy–cost tradeoff; the authors also clearly distinguish the analyzed variant from the original empirical algorithm.
Critical weaknesses

My main reservations concern the applicability of the theory and whether the reported gains can be attributed to the stated algorithm. Three issues currently limit the evidence:

  1. Assumption 1 requires full row rank of the constraint Jacobian on a compact set containing the iterates. However, the constraints include mean-squared initial/boundary losses: for h_j = ||r_j||^2/N, its gradient is 2 J_{r_j}^T r_j/N and is exactly zero whenever that constraint is satisfied. Thus rank degeneracy is structural at feasibility, rather than merely a possible numerical issue. The theorem consequently does not establish convergence for the stated PINN formulation; damping in Algorithm 1 does not resolve the applicability of the undamped theorem for Algorithm 2.
  2. Although the distinction between Algorithms 1 and 2 is acknowledged, the evaluated implementation is also insufficiently tied to Algorithm 1. For example, the released Burgers forward script includes integral feedback and gradient clipping, while the Navier–Stokes script includes multiplier smoothing, clipping, and merit-based backtracking. A precise mapping from Table 1 to the actual updates and training schedules is needed to determine which components produce the improvements and to reproduce the comparisons.
  3. The main comparison lacks reported dispersion, and Table 12 contains a concrete reporting inconsistency: the beta1=0.95, beta2=0.999 row gives 8.48e-3 +/- 5.01e-2 over three seeds. These cannot be the sample mean and standard deviation of three nonnegative errors (the standard deviation is at most sqrt(3) times their mean). Corrected values and per-seed results are needed before drawing strong conclusions from this ablation.
Questions for authors
  1. How can Assumption 1 hold for the squared-loss constraints as feasibility is approached, given that their gradients vanish at zero residual? Can you state precisely what the theorem establishes for the actual PINN formulation, and whether the damped empirical update has a corresponding guarantee?

  2. Which exact code/configuration produced each row of Table 1, including integral feedback, clipping, multiplier smoothing, backtracking, warm-start, and stopping criteria? Please clarify whether these choices are shared across baselines, provide per-seed results, and correct the inconsistent mean/standard deviation in Table 12.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.