- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes AdamFLIP, which treats PINN residuals as the outputs of a controlled dynamical system and applies ideas from nonlinear control theory to design a feedback linearization (FL) solver to alleviate the classical gradient pathologies problem in PINNs. Through FL-momentum, the authors introduce an adaptive inertia term for the boundary condition, initial condition, and data loss terms in PINN training, rather than using fixed weights. Comparisons with two similar baseline methods and the FLIP ablation demonstrate that AdamFLIP achieves competitive results on 4 standard PINN forward/inverse problems.
Critical strengths
- Gradient pathologies is a significant challenge in PINNs. Therefore, efforts to address this issue can contribute to more robust PINN training, particularly in complex tasks.
- Introducing principles from control theory into PINN optimization indeed offers a novel perspective.
Critical weaknesses
- Regarding the positioning of the method and lack of critical related work discussion.
My primary concern is whether the proposed method constitutes a sufficiently novel constrained optimization algorithm, as opposed to a relatively straightforward combination of existing FL and Adam-style adaptive updates. This distinction is important because the paper repeatedly emphasizes AdamFLIP as a novel adaptive first-order optimizer for hard-constrained PINN training. However, based on the equation in line 224 (which the authors have not assigned an equation number), suppose that the objective and constraints of AdamFLIP are defined as follows:
Then, the gradient actually used by AdamFLIP can be written as:
In essence, this amounts to introducing two adaptive, dynamically adjusted weights into the PINN loss function, which is similar to the adaptive loss design in many previous works [3,4,5]. Unlike other hard-constraint methods that explicitly incorporate BCs/ICs into the network architecture, AdamFLIP does not inherently guarantee exact satisfaction of BCs/ICs.
Therefore, the authors need to identify precisely what new optimization principle or nontrivial algorithmic property is introduced updates to achieve the "hard constraints", beyond combining an existing FL-based gradient construction with standard Adam. Merely employing Adam-style adaptive moments on an existing constrained optimization direction does not, by itself, establish the claimed level of methodological novelty.
A substantial body of recent work exists whether in optimization methods or adaptive loss functions. For example, optimizers specifically designed for PINNs include DCGD [1] (NeurIPS 2024) and ConFIG [2] (ICLR 2025), etc., while adaptive loss methods include SA-PINN [3] (JCP 2023), BRDR [4] (JCP 2025), and DB-PINN [5] (IJCAI 2025), etc.. Unfortunately, the authors fail to mention any of these closely related baselines in the Related Work section, nor do they discuss how AdamFLIP differs from them.
- The experimental evaluation suffers from a serious lack of baseline comparisons and the paper's organization and allocation of space are unreasonable.
As discussed in Point 2, numerous baselines are available from both the optimization and adaptive loss literature. However, excluding FL-PINN, which serves as an ablation, the authors compare their method only against AL-PINN (Neurocomputing 2023) and trSQP-PINN (a rejected/withdrawn ICLR 2025 submission, https://openreview.net/forum?id=GkJCgUmIqA). I do not require the authors to compare against every baseline listed in Point 2. Nevertheless, the current experimental comparisons are insufficient to substantiate the claimed superiority of AdamFLIP over other methods in the context of the state of the field in 2027. In particular, relying on a rejected/withdrawn and still unpublished submission as one of only two principal baselines, while omitting several established and peer-reviewed recent methods, raises concerns about the representativeness of the experimental comparisons and the validity of the claimed state-of-the-art performance.
The authors devote more than one page to introducing PINNs in the Preliminaries section, despite this being well-established knowledge in the field, while relegating the Related Work section to the appendix. Furthermore, the analysis in the main experimental section is overly brief, with the detailed discussion entirely deferred to Appendix D.2, even though there is still unused space on pages 8 and 9 of the main paper. In light of Points 2 and 3 above, I strongly recommend that the authors move the Related Work section into the main paper and substantially condense the Preliminaries section. The theoretical analysis in Section 4.1 could also be moved to the appendix, retaining only one or two paragraphs summarizing the key conclusions in the main paper, thereby providing more space for the experimental section.
Regarding the format, the manuscript appears to use narrative-style citations (e.g., \citet) in latex throughout, including instances where parenthetical citations (e.g., \citep) would be more appropriate. The manuscript also contains numerous incorrect cross-references and instances of overloaded mathematical notation, including but not limited to:
- Lines 1132–1136 discuss Figure 4 but refer to Fig. 7(a), Fig. 7(b), and Fig. 7(c), even though Figure 7 does not contain any subfigures.
- The PDE loss term in Equation (4) lacks a squared residual, which is inconsistent with the standard PINN formulation.
- The symbol is initially used to denote an unknown PDE parameter but is later reused to represent the feedback control gain through in Algorithm 2.
- The abbreviation of Initial conditions (ICs) and boundary conditions (BCs) are repeatedly defined throughout the manuscript.
- The first entry in the reference list begins directly with the paper title, omitting the author information.
- Some equations are numbered, while others are not.
References
[1] Y. Hwang and D.-Y. Lim, "Dual Cone Gradient Descent for Training Physics-Informed Neural Networks," in Advances in Neural Information Processing Systems (NeurIPS), vol. 37, 2024.
[2] Q. Liu, M. Chu, and N. Thuerey, "ConFIG: Towards Conflict-Free Training of Physics-Informed Neural Networks," in Proceedings of the 13th International Conference on Learning Representations (ICLR), 2025.
[3] L. D. McClenny and U. M. Braga-Neto, "Self-Adaptive Physics-Informed Neural Networks," Journal of Computational Physics, vol. 474, Art. no. 111722, 2023.
[4] W. Chen, A. A. Howard, and P. Stinis, "Self-Adaptive Weights Based on Balanced Residual Decay Rate for Physics-Informed Neural Networks and Deep Operator Networks," Journal of Computational Physics, vol. 542, Art. no. 114226, 2025.
[5] C. Zhou, J. Chen, Z. Yang, and C. E. Png, "Dual-Balancing for Physics-Informed Neural Networks," in Proceedings of the 34th International Joint Conference on Artificial Intelligence (IJCAI), 2025, pp. 7164–7172.
Questions for authors
- What is the precise algorithmic novelty of AdamFLIP beyond combining the existing FL-momentum framework with standard Adam updates?
- Compared with other PINN optimizers or adaptive loss-weighting methods, in what sense can AdamFLIP be considered a "hard constraint" method?
- Given the authors’ claim that AdamFLIP is “the first adaptive first-order optimizer,” have they considered comparing it against established first-order optimizers, such as ConFIG and DCGD?