- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper produces a proof that second order linear PINNs trained with ReLU (or similar activation functions) do not converge to the true solution even if their residual vanishes. They show that the solutions converge to a "wrong limit" and they extend these results to some other equation classes.
Critical strengths
None.
Critical weaknesses
- The results of this paper are trivial and constitute what I consider easy exercises for a first-year graduate PDE class. They do not rise to the level of significance required by a paper.
- For example: Lemma 3.1 is correct but follows immediately from elementary results. Differentiate a ReLU neural net and note that the second derivative vanishes A.E.. Then trivially if is this neural network then . Theorem 3.2 consists of textbook arguments (from Evans) and these results follow from the observation in Lemma 3.1 combined with routine techniques like density, the trace theorem, and the triangle inequality. Lemma 3.1 and 3.2 already capture the general difficulty of the results in this paper.
- The results of the paper have limited relevance to practical PINN training since ReLU is not a standard activation function in the PINN literature (the original PINN paper by Raissi et al. (2019) uses tanh, which is the adopted standard).
Questions for authors
- The first line of the introduction says "The default activation in contemporary deep learning is piecewise linear". Can you substantiate this claim? PINNs don't typically use ReLU as the activation function (I've always assumed because they have vanishing Hessian), and the broad deep learning community no longer uses ReLU. In 2026 I would argue that ReLU is an antiquated activation function.
- What do the authors consider technically novel about their work? I.e. observations which go beyond using \nabla^2 u = 0 and standard tools from an introductory graduate PDE course? What are the practical applications of this work?
- The proof of Lemma 3.1 is incorrectly labeled as a Theorem in the appendix.
- Why is this classified as "foundation or frontier models, including LLMs".
I really appreciate the detailed response. However, I think the paper still need further improvement. I hope the author can revisit all the reviews and improve their manuscirpt