1. The nonlinear lifting construction relies on a diagonal restriction operator that is not closable in the stated Hilbert spaces.
My main concern relates to the mathematical validity of the nonlinear lifting introduced in Section 4.3.
In lines 327–330 (page 7), the authors introduce the diagonal restriction operator Pi, formally defined by
(Piv)(x)=v(x,…,x),
which maps functions on a product domain to their restriction to the diagonal.
However, this operator is not well-defined on general L2(Ωi) equivalence classes, since the diagonal has measure zero in the product domain. Furthermore, even if Pi is defined as a densely defined operator on smooth functions, it is not closable in the L2 topology.
For example, when i=2, one can construct smooth functions vn such that
\qquad
P^2v_n\to\varphi\neq0
\quad\text{in }L^2(\Omega).$$
Consequently, the $L^2$ adjoint of this densely defined restriction operator has a trivial domain. In particular, the expressions involving $(P^i)^*$ in Theorems 4.6 and 4.9 cannot be justified as Hilbert-space adjoints acting on the nonzero test functions considered in the paper.
A possible remedy would be to work in sufficiently regular Sobolev spaces, where diagonal restriction can be defined as a bounded trace operator. This would allow the nonlinear lifting and its adjoint to be defined rigorously.
However, such a modification would require more than simply changing the functional spaces. In particular, the resulting Rademacher-complexity estimates would involve stronger Sobolev norms of the neural-network outputs, whereas the Koopman-based estimates used in Section 4.4 control $L^2$ norms. Therefore, additional analysis would be needed to recover the proposed architecture-dependent bounds, including their dependence on the determinants of the weight matrices.
Since this construction is central to the paper's treatment of nonlinear differential operators, I consider this a substantial mathematical concern that affects the validity of the main theoretical contribution.
**Additional minor remarks**
**a. Applicability to the Monge–Ampère experiment.**
In Section 5.2, the authors consider a nonlinear operator involving $\log\det(D^2u)$. However, Theorem 4.9 relies on a Taylor expansion of the nonlinear map at zero, where the logarithmic determinant is not defined. Applying the theorem would require an alternative expansion point, together with appropriate domain and regularity assumptions. It is therefore unclear how the stated theoretical result applies to this experiment.
**b. Determinant-based bounds and the experimental architecture.**
The Koopman-based bounds in Section 4.4 involve factors of the form
$$|\det(W_\ell^\top W_\ell)|^{-1/4}.$$
However, the final weight matrix used in the Navier–Stokes experiment has dimensions $3\times64$, so $W_4^\top W_4$ is necessarily singular. Consequently, the determinant-based expression is not finite for this architecture.
Appendix C replaces this quantity with a heuristic based on the positive singular values, but this is not equivalent to the determinant appearing in the theoretical bound. The relationship between the proposed practical regularizer and the theoretical result therefore requires clarification.
Thank you for your review.
On the weakness (solution error of PINNs) and the question (comparison with numerical methods)
We understand the concern to be about the accuracy of PINNs as PDE solvers compared with classical numerical methods. We would like to clarify the scope of the paper. We do not claim that PINNs or VPINNs are more accurate than classical solvers, and we do not propose a new solver. The contribution is theoretical: a generalization bound for neural networks whose loss involves derivatives with respect to the input, i.e. a bound on the gap between the residual loss on the collocation points and its expectation. The analysis covers PINNs and VPINNs, and also other models with differential operators, such as Hamiltonian and Lagrangian neural networks.
The accuracy of PINNs is not guaranteed in the way it is for classical methods. It is therefore important to understand which factors control their error. Our bounds identify several of them: the number of collocation points, the determinants of the weight matrices, the order r of the nonlinearity of the differential operator, and the test function. For example, the bound diverges as the variational model approaches the standard PINN (Remarks 4.10 and 4.11), which suggests an advantage of the variational form in terms of generalization.
Your point also touches a real limitation, which we now state explicitly in Section 6 (marked in red in the revised PDF). Our bound concerns the generalization of the residual, not the error of the solution. A small residual implies a small solution error only if the PDE is stable, and bounding the solution error on the basis of our bound is left for future work.
On soundness
The review does not point to a specific statement or proof. If there is a particular step that you consider incorrect, we would be grateful if you could indicate it, and we will address it. In the revision, we corrected the issues of mathematical rigor raised by the other reviewers: the nonlinear lifting is now defined as a pointwise lift (Section 3, Assumption 3.2), the Koopman-based bounds are re-derived for it (Lemma 4.7, Theorem 4.9), the Monge–Ampère experiment is covered by an expansion at a valid point with an explicit remainder (Appendix D), and the gap between the determinant factor and the regularizer is discussed (Appendix C).