- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how to infer and correct the unknown error field of a trained physics-informed neural network (PINN) using its PDE residuals and boundary- and initial-condition defects. The PINN parameters remain frozen, and no reference-solution labels are used to fit the correction. The method constructs a continuous correction field and returns
The central idea is to interpret the residuals and condition defects as linear-functional observations of the unknown error field, obtained by applying the PDE or condition operators, rather than as pointwise error labels. For linear PDEs, the error equation is exact. For nonlinear PDEs, the authors linearize around the frozen PINN and reconstruct an approximate Newton correction. Applying these observation operators to a prescribed kernel yields an operator-kernel Gram matrix. A regularized linear system, with scaling across observation blocks, is then solved to reconstruct the continuous correction field. For nonlinear problems, a second stage adjusts the regularization using an estimate of the linearization-remainder magnitude.
Critical strengths
1. A coherent framework for reconstructing a continuous error field from observable defects. The method treats PDE residuals and condition defects as operator observations of the error and constructs their kernel representers. This provides an informative way to improve a frozen PINN without reference-solution labels for correction fitting. The distinction between linearization error and finite-observation reconstruction error, particularly the Newton diagnostics on held-out Burgers instances, also gives concrete insight into the sources of failure.
2. Broad experimental coverage with an independently forced nonlinear validation case. Beyond the canonical PDEs, the paper evaluates held-out parameter instances, an irregular domain, a three-dimensional problem, and a two-dimensional nonlinear problem with independently specified forcing. On the independently forced case, the correction achieves clear field-error improvements over the neural error-field and continued-training baselines at matched computational cost.
Critical weaknesses
The additional contribution and advantages over existing operator-kernel, GP, and defect-correction methods need clearer delineation. The linear error equation is a classical defect-correction identity, and operator-kernel Gram systems and RKHS minimum-norm estimators have established precedents. The direct-kernel comparisons show that zero-mean direct kernel solvers can substantially outperform PINN-based correction on Poisson 2D, Helmholtz 2D, and convection, at similar fitting cost. These findings make it unclear when retaining the PINN as a prior mean is advantageous. The authors should explain the conditions under which this prior is expected to help and identify the substantive contribution of the complete workflow relative to existing operator-GP and defect-correction methods.
Questions for authors
Q1 For linear problems, the error-coordinate model is affinely equivalent to a solution-coordinate model with as its mean. The paper reports several cases where a direct zero-mean kernel solver substantially outperforms PINN-based correction at similar fitting cost. Under what conditions can retaining the PINN as the prior mean be expected to provide an advantage?
Q2 Please explain further how the Stage-2 regularization adjustment controls the nonlinear remainder, particularly the term in Burgers, and whether the shared adjustment can unnecessarily weaken boundary- and initial-condition constraints. Given that Stage-2 does not improve some problems and can increase error, should it be a conditional option selected using observable quantities rather than a default step?
Q3 For Burgers, Allen–Cahn, and the independently forced two-dimensional nonlinear problem, please specify the function spaces and error norms, and explain how the bounded-inverse and local Lipschitz assumptions are satisfied or assessed. Please also clarify the applicability of the RKHS assumptions required by the associated bounds. Where these conditions have not been established for the experiments, please qualify the discussion of Proposition 2 and the related error bounds, clearly separating conditional theoretical interpretations from empirically supported conclusions.
Q4 The paper already includes full-cost comparisons in the supplementary experiments. Could the authors consolidate the timing conventions and report end-to-end costs covering observation construction, kernel-matrix assembly, repeated factorizations during hyperparameter optimization, Stage-2, correction-field evaluation, and the associated certificate computations?
5 comments
Response to Reviewer u5Zr (part 2/4)
Q2. Why can Stage2 weaken boundary or initial conditions?
Your concern is that the nonlinear adjustment can also weaken constraints whose equations are linear. Our new interventions confirm that this can happen and identify which part of the adjustment causes the tradeoff. Stage1 is the first kernel correction fitted to the frozen network's PDE and condition defects. Stage2 refits those same targets with adjusted regularization. The submission applies it to nonlinear problems; based on these ablations, we will present it as optional.
Write the true error as , its spatial derivative as , and the nonlinear Taylor remainder about as . For Burgers, for any finite exponent ,
An amplitude-squared proxy misses this derivative dependence. The submitted Stage2 rule is therefore a heuristic. It increases relative regularization while retaining the network, linearized operator, targets, sites, lengthscales, and kernel matrix before amplitude scaling. It then re-solves the defect system and re-profiles amplitude. Its correction replaces Stage1; it does not add another Newton update. Applying the adjustment to both observation blocks also relaxes boundary/initial enforcement.
To isolate that effect, we used semilinear elliptic and Burgers problems, five seeds, and the early/late checkpoints defined above: 20 base cases, with five independent seeds per fixed problem/checkpoint condition. Four variants apply the fitted Stage1 or prescribed Stage2 regularization level to each of the interior and condition blocks: Stage1, interior-only adjustment, condition-only adjustment, and shared Stage2. Operator coefficients, targets, block scaling, numerical jitter (small diagonal stabilization), and the amplitude-free kernel remain fixed. Burgers also has separate spatial-boundary-only and initial-condition-only interventions. Re-profiled amplitude cancels from the posterior means.
The early Burgers comparison measures field error against the reference solution and condition defects on independent audit sites. Entries are five-seed geometric means; root-mean-square (RMS) defects are in absolute condition units.
| Correction | Field error / Stage1 [95% interval] | Boundary RMS | Initial RMS |
|---|---|---|---|
| Stage1 | 1 | 2.24e-7 | 3.54e-7 |
| Shared Stage2, original rule | 0.9387 [0.9090, 0.9693] | 4.51e-4 | 9.40e-4 |
| New interior-only variant | 0.9548 [0.9246, 0.9860] | 6.72e-8 | 6.05e-8 |
Shared Stage2 reduces field error by 6.1% while substantially weakening condition enforcement. Interior-only adjustment retains a 4.5% reduction with small condition defects. It is a separate variant; this Burgers intervention does not explain the submitted Allen-Cahn degradation.
We tested whether observable defects could select safely among Stage1, shared Stage2, and interior-only adjustment. On 512 interior and 480 condition selection sites, the rule minimizes PDE mean squared error plus 100 times condition mean squared error. Each condition's mean squared error must remain below 1.05 times Stage1's value plus . A separate audit uses 1,024 interior and 960 condition sites. Reference fields are read after selections are committed. The rule still increases true error in 2/5 late Burgers cases; its absolute tolerance can allow large relative condition changes when the baseline is tiny.
We also checked the computed Stage1 correction, denoted . Using the prescribed Burgers proxy factor , the audit-site ratio ranges from 10.85 to 15.27 across early seeds and 23.30 to 35.22 across late seeds. This directly tests the candidate correction, while the true-error remainder remains unknown. At the long-trained nonlinear endpoints, every Stage2/Stage1 error-ratio interval includes one. We will present Stage1 as primary and Stage2 as optional, retaining these failures and avoiding a safety claim for the selection rule.
Response to Reviewer u5Zr (part 3/4)
Q3. Which function spaces make the theoretical argument valid?
The inverse assumption asks whether a small defect produces a controlled solution of the complete linearized problem, including boundary and initial conditions. The nonlinear smoothness assumption then controls the discrepancy between that linearized correction and the true error. The computed kernel functions must also lie in the chosen field space. These requirements depend on the PDE and the norm. Membership of the true error in a reproducing kernel Hilbert space (RKHS), the function space associated with a kernel and its norm, is an additional requirement for the separate reconstruction bounds. It is unnecessary for the Newton error decomposition itself.
For elliptic equations, a weak formulation supplies an appropriate inverse estimate. In the submitted independently forced example, the nonlinear operator acts on a field on as
a(x,y)=0.02[1+0.5\sin(2\pi x)\sin(2\pi y)]\ge0.01.$$ Use $H^1(\Omega)$ for the field and $H^{-1}(\Omega)$ for the interior residual. The former controls square-integrable values and first derivatives; the latter accommodates residuals tested against functions with zero boundary values. Boundary data belong to the global trace space of $H^1$. The linearized energy integrates $a|\nabla v|^2+15p^2v^2$ and controls the zero-boundary $H^1$ norm. Lax-Milgram and bounded extension of boundary data into the domain give the complete inverse estimate. For a cubic coefficient $\gamma\ge0$, let $C_4$ be the embedding constant from $H^1$ into $L^4$. For fields $v,w$ with $H^1$ norms at most $M$, the operator derivative satisfies $$\\|N'(v)-N'(w)\\|_{H^1\to H^{-1}} \le6\gamma C_4^4M\\|v-w\\|_{H^1}.$$ The inverse constant depends on ellipticity, domain, prior norm, and boundary extension. This also covers the new cubic benchmark. It proves a weak formulation; it does not give the original twice-continuously-differentiable setting or bounded point evaluation of arbitrary $H^{-1}$ residuals. For time-dependent Burgers and Dirichlet Allen-Cahn, the space must also control one time derivative and two spatial derivatives. Set $T=1$, let the spatial interval $I$ be $(-1,1)$ for Burgers or $(0,1)$ for Allen-Cahn, and write $Q=I\times(0,T)$. For fixed $q>3$, use the field space $X_q$ and residual space $Y$ defined by $$X_q=W^{1,q}(0,T;L^q(I))\cap L^q(0,T;W^{2,q}(I)),\qquad Y=L^q(Q).$$ Its norm sums the $L^q$ norms of $v,v_x,v_{xx},v_t$. Initial data lie in $W^{2-2/q,q}(I)$ and lateral data in $W^{1-1/(2q),q}(0,T)$, with matching initial-corner values [2]. Allen-Cahn has diffusivity $10^{-4}$ and an initial condition. Linearization at the smooth frozen predictor gives drift/potential $(p,p_x)$ for Burgers and $(0,15p^2-5)$ for Allen-Cahn. A weighted heat-equation inverse and boundary/initial extensions yield bounds depending on diffusion, horizon, coefficients, and extension constants. If $E_0$ is the embedding constant from $X_q$ into bounded functions, the local derivative-Lipschitz constant is at most $2E_0$ for Burgers and $30M_0E_0$ for Allen-Cahn on a convex neighborhood with field amplitude at most $M_0$. This revises the original strong-space claim. Smooth truth and a smooth network do not automatically satisfy the higher corner compatibility required by a classical Newton problem with soft conditions. For Burgers that requirement includes $ee_x=0$ at the initial corners. The finite-$q$ formulation permits the resulting corner loss of higher regularity. General $L^q$ data still do not support bounded point sampling. Our interior-regularity and structured-remainder arguments apply to these smooth PDE data at strictly interior, positive-time sites; they do not prove a general sampling bound or place a trained checkpoint inside a useful Newton neighborhood. Gaussian-RKHS membership likewise remains unverified for the trained errors. For a separately specified Matérn-9/2 kernel with fixed positive lengthscale and amplitude, the two-dimensional RKHS corresponds to restricted $H^{11/2}$ [3,4]. The new sine-mixture Burgers solution is smooth through its positive Cole-Hopf representation. Subtracting a finite tanh network gives an error with an $H^6(\mathbb R^2)$ extension, implying membership. The manufactured Allen-Cahn truth permits the same argument. This is a theoretical specialization, not a nonlinear Matérn comparison experiment. It supplies neither Gaussian membership nor a known RKHS norm; the observed relative-$L^2$ gains remain empirical.
Response to Reviewer u5Zr (part 1/4)
We thank for the reviewer's insightful comments
Q1. When does a trained neural field improve a kernel solve?
The equivalence for linear problems is correct. A physics-informed neural network (PINN) supplies a mean field for a kernel solver of the partial differential equation (PDE). Whether that mean helps depends on what the kernel reconstruction leaves unresolved. A small error in the network alone does not guarantee improvement after correction.
Let u be the true solution and p the frozen network prediction. Let Λ evaluate the PDE and condition operators at the observation sites, d collect their known forcing and boundary/initial values, and S be the fixed linear map from observations to a reconstructed field. The zero-mean solution u0 and the solution retaining the network mean up are
u0=Sd,up=p+S(d−Λp).
This is Gaussian-process conditioning with a prescribed mean [1]. Define the zero-mean error r0=u−Sd and the prior contribution t=p−SΛp. In the chosen evaluation inner product, retaining the prior improves the error norm exactly when
2⟨r0,t⟩>∣t∣2.
The criterion explains why the same prior can help one reconstruction and hurt another. Because it depends on the unknown solution, it cannot select a prior from observable defects alone. Our contribution uses the established defect identity to reconstruct a frozen network's remaining error and separates the ideal linearized correction from what finite observations actually recover.
We designed a new controlled comparison to measure this dependence. On one linear elliptic instance, five paired seeds were evaluated at 32, 128, 500, and 1,000 interior observations, with Gaussian and Matérn-9/2 kernels and fixed lengthscales 0.15/0.30. Each of the 80 designs compares zero, early-network, and late-network means, giving 240 reconstructions. Boundary observations are additional to the interior budgets. Operator, sites, kernel, and regularization stay identical within a design; changing the mean recomputes its defects. Neither separate tuning for each mean nor an oracle prior enters this experiment.
The early network has 2,000 Adam optimizer calls. The late network has 22,300 total calls: 20,000 Adam, 2,000 limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS), and 300 NysNewton-CG (NNCG) calls. L-BFGS estimates curvature from gradient history; NNCG is a Nyström-preconditioned Newton-conjugate-gradient method [5]. These methods train the network before its weights are frozen. The late mean tests whether stronger training makes prior reuse reliably beneficial.
These fixed-parameter comparisons establish dependence on the reconstruction design. Equal numerical lengthscales have different spectral meanings for the two kernels, and the experiment does not rank separately optimized kernel families.
We also tested the choice of initialization for nonlinear iterative kernel solves. Five paired seeds per nonlinear problem share observation sites; each iteration refreshes the linearization and selects damping using physical validation, which scores PDE and condition defects without solution labels, up to 32 iterations. The geometric-mean PINN/zero error ratios are 1.137 [0.675, 1.917] for semilinear elliptic and 1.291 [0.943, 1.767] for Burgers, with paired 95% confidence intervals in brackets. Neither interval establishes an accuracy advantage. Five-seed geometric-mean times including training are 41.60/248.93 seconds for zero/PINN initialization on semilinear elliptic and 18.47/139.00 seconds on Burgers. Thus, these tests support conditional reuse of an available field; they provide no general case for training a network first to accelerate a solve from scratch.