- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies when Physics-Informed Neural Networks (PINNs) can be guaranteed to train successfully using gradient descent. The main idea is to connect NTK positivity to the independence of the physical constraints.
The paper provides a way to check whether PINN constraints produce a positive NTK and then uses that condition to prove convergence of gradient descent for sufficiently wide two-layer PINNs with linear constraints.
Critical strengths
The core theoretical logic appears sound. The authors do not simply assume a positive NTK gap. They characterize the nullspace using the constraint structure and then transfer that positivity to finite width. That is the main theoretical contribution.
I also do not see an obvious hole in the weak-form nullspace argument.
Critical weaknesses
The paper has Proposition C.9 to bridge this with a finite-witness lower bound, so this isn't a logical hole. But the practical strength of the finite-width theorem depends on obtaining a meaningful , not merely establishing rank.
The main theoretical concern is the gap between existential positivity characterization and operational checkability. For strong-form constraints the certificate is a direct finite rank test. For general weak/nonlocal constraints, however, positivity may be witnessed by a sufficiently large finite Taylor/moment truncation, while the paper explicitly notes that no a priori truncation degree is available. Thus it is unclear whether the advertised “checkable necessary-and-sufficient certificate” constitutes a terminating decision procedure, especially for singular designs. Moreover, the finite-width GD theorem requires a quantitative lower bound , whereas structural full rank alone only establishes strict positivity. The authors should clarify exactly what can be computed in finite time and how a usable is obtained for the general classes.
Questions for authors
For the general weak-form and nonlocal settings, does the proposed certificate give a terminating decision procedure for both positive and singular designs? If not, the paper should distinguish a finite witness for positivity from a fully checkable necessary-and-sufficient certificate.
The main results rely on linear constraints, while many practical PINNs contain nonlinear PDE terms. Could the authors clarify which parts of the positivity/certificate framework could potentially extend to nonlinear constraints, and which parts fundamentally fail? The current discussion suggests that a deterministic limiting kernel may fail for genuinely nonlinear rows.
Theorem 4.2 provides explicit width conditions for the finite-width GD guarantee. Could the authors clarify how large these theoretical width requirements are for the PDE problems used in the experiments, and how they compare with the widths actually tested? The experiments explicitly note that they do not verify the constants in Eq. (9).
Thank you for the careful review.
W1. A usable λ0. Theorem 4.2 needs only that λmin(K(0)∞) is positive, which is what the positive-definiteness certificate proves. Corollary 4.3 takes λ0=λmin(K(0)∞), so full rank gives convergence for every width above a threshold explicit in λ0. No lower bound on λmin can hold across designs, because it depends on the PDE, the collocation points, the activation. For a given design it is the smallest eigenvalue of an explicit N×N matrix, which we evaluated numerically for our eleven designs (Q3). As in earlier NTK analyses such as Du et al. (2019), the theorem is a sufficient condition for convergence. It is not meant to prescribe the width or the step size, and in our experiments the loss decreases without satisfying the condition (see Q3).
W2. Checkability for weak and nonlocal rows. For every loss trained in practice the certificate is a terminating decision procedure, for weak and nonlocal rows as well as pointwise ones. In a trained PINN the integrals of weak and nonlocal rows are evaluated by a fixed quadrature rule, so every row is a combination of finitely many point evaluations. For such rows positivity is equivalent to full row rank of the finite coefficient matrix B (Remark C.16, Sections D.3.2 and D.3.3), as it is a rank test per site for pointwise rows (Proposition 3.1, Section D.3.1). Both tests terminate and decide positive and singular designs. Every weak and nonlocal design in our experiments is of this kind. The remark on the truncation degree concerns only continuum functionals with exact integrals, which do not occur in training. There an independent design is certified at some finite moment degree (Proposition C.13, Corollary C.14), and a dependent design by an exact linear relation among the rows (Section D.3.3).
Q1. Yes. For weak and nonlocal rows evaluated by quadrature the finite rank test terminates and decides both positive and singular designs, as for pointwise rows (see W2).
Q2. Nonlinear constraints. Linear PDEs are the main restriction of the paper, and we say so. A nonlinear row is a nonlinear function of finitely many linear observations of the network, and its tangent Gram matrix factors exactly through their kernel at every width (Lemma F.3). The rank certificate still applies to that linear kernel. The deterministic limit fails. The factor multiplying the linear kernel depends on the network values at initialization, so the limiting kernel is random (Proposition F.5) and its smallest eigenvalue has no deterministic positive floor (Remark F.11). Theorem 4.2 transfers a deterministic gap to finite width, so this step has nothing to transfer. Whether a random but positive kernel still gives convergence is open (Remark F.18).
Q3. Size of the width condition. For each of the eleven designs we evaluated the limiting kernel by quadrature and found λmin(K(0)∞) between 1.6×10−7 and 5.2×10−3. With λ0=λmin(K(0)∞), the most favorable admissible choice, we evaluated the three terms of Equation (9) at δ=0.05. The required width lies between 9.3×1016 and 4.5×1031, and the term in λ0−3 dominates in every design. The table lists each design, with N the number of rows.
The largest width we trained is 16,384. The experiments are therefore far outside the regime of Equation (9), which is why we wrote that they do not verify its constants. The condition is sufficient and far from necessary. At widths more than twelve orders of magnitude below it, the runs still show what the theorem concludes. The loss decreases, the kernel drift from initialization shrinks as the width grows, and the smallest eigenvalue of the empirical kernel stays positive. The constants RK, B∗ and L∗ are evaluated on sample points, so the listed widths are lower bounds. The same list is in the README of the anonymous repository.