acceptodds
Under review as a conference paper at ICLR 2027

Correcting Trained PINNs from Residuals: Operator-Kernel Inference of the Error Field

Abstract

A trained physics-informed neural network (PINN) retains computable PDE residuals and boundary and initial-condition defects that contain information about its remaining solution error. We use these defects to reconstruct a global correction while keeping the network fixed. The governing equation turns them into operator observations of the unknown error, exactly for linear PDEs and through a frozen linearization for nonlinear PDEs. Applying the observation operators to a kernel produces a finite reconstruction system whose hyperparameters are fitted from the defects. Regularized linear solves then recover the correction without observing the true error or retraining the PINN. Our analysis characterizes the conditional RKHS estimator and separates finite-data reconstruction from nonlinear linearization. Experiments across canonical PDEs, held-out physical instances and irregular geometry show reductions in solution error. On a nonlinear variable-coefficient problem with directly specified forcing, the complete procedure substantially improves the trained field. For this problem, it also achieves lower error than neural error-field fitting and continued training given larger compute allowances. Budget comparisons reveal problem-dependent tradeoffs, while resolved Newton references identify reconstruction as the larger discrepancy on the studied Burgers cases. These results connect observable equation defects to a usable correction of an existing neural field.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
physics-informed neural networks, a posteriori error correction, operator-conditioned kernels, Gaussian processes

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_u5ZrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to infer and correct the unknown error field of a trained physics-informed neural network (PINN) using its PDE residuals and boundary- and initial-condition defects. The PINN parameters remain frozen, and no reference-solution labels are used to fit the correction. The method constructs a continuous correction field e^\hat e and returns

u~=uθ+e^.\tilde u=u_\theta+\hat e.

The central idea is to interpret the residuals and condition defects as linear-functional observations of the unknown error field, obtained by applying the PDE or condition operators, rather than as pointwise error labels. For linear PDEs, the error equation is exact. For nonlinear PDEs, the authors linearize around the frozen PINN uθu_\theta and reconstruct an approximate Newton correction. Applying these observation operators to a prescribed kernel yields an operator-kernel Gram matrix. A regularized linear system, with scaling across observation blocks, is then solved to reconstruct the continuous correction field. For nonlinear problems, a second stage adjusts the regularization using an estimate of the linearization-remainder magnitude.

Critical strengths

1. A coherent framework for reconstructing a continuous error field from observable defects. The method treats PDE residuals and condition defects as operator observations of the error and constructs their kernel representers. This provides an informative way to improve a frozen PINN without reference-solution labels for correction fitting. The distinction between linearization error and finite-observation reconstruction error, particularly the Newton diagnostics on held-out Burgers instances, also gives concrete insight into the sources of failure.

2. Broad experimental coverage with an independently forced nonlinear validation case. Beyond the canonical PDEs, the paper evaluates held-out parameter instances, an irregular domain, a three-dimensional problem, and a two-dimensional nonlinear problem with independently specified forcing. On the independently forced case, the correction achieves clear field-error improvements over the neural error-field and continued-training baselines at matched computational cost.

Critical weaknesses

The additional contribution and advantages over existing operator-kernel, GP, and defect-correction methods need clearer delineation. The linear error equation is a classical defect-correction identity, and operator-kernel Gram systems and RKHS minimum-norm estimators have established precedents. The direct-kernel comparisons show that zero-mean direct kernel solvers can substantially outperform PINN-based correction on Poisson 2D, Helmholtz 2D, and convection, at similar fitting cost. These findings make it unclear when retaining the PINN as a prior mean is advantageous. The authors should explain the conditions under which this prior is expected to help and identify the substantive contribution of the complete workflow relative to existing operator-GP and defect-correction methods.

Questions for authors

Q1 For linear problems, the error-coordinate model is affinely equivalent to a solution-coordinate model with uθu_\theta as its mean. The paper reports several cases where a direct zero-mean kernel solver substantially outperforms PINN-based correction at similar fitting cost. Under what conditions can retaining the PINN as the prior mean be expected to provide an advantage?

Q2 Please explain further how the Stage-2 regularization adjustment controls the nonlinear remainder, particularly the eexe e_x term in Burgers, and whether the shared adjustment can unnecessarily weaken boundary- and initial-condition constraints. Given that Stage-2 does not improve some problems and can increase error, should it be a conditional option selected using observable quantities rather than a default step?

Q3 For Burgers, Allen–Cahn, and the independently forced two-dimensional nonlinear problem, please specify the function spaces and error norms, and explain how the bounded-inverse and local Lipschitz assumptions are satisfied or assessed. Please also clarify the applicability of the RKHS assumptions required by the associated bounds. Where these conditions have not been established for the experiments, please qualify the discussion of Proposition 2 and the related error bounds, clearly separating conditional theoretical interpretations from empirically supported conclusions.

Q4 The paper already includes full-cost comparisons in the supplementary experiments. Could the authors consolidate the timing conventions and report end-to-end costs covering observation construction, kernel-matrix assembly, repeated factorizations during hyperparameter optimization, Stage-2, correction-field evaluation, and the associated certificate computations?

5 comments
AuthorsOpenReview ↗

Response to Reviewer u5Zr (part 1/4)

We thank for the reviewer's insightful comments

Q1. When does a trained neural field improve a kernel solve?

The equivalence for linear problems is correct. A physics-informed neural network (PINN) supplies a mean field for a kernel solver of the partial differential equation (PDE). Whether that mean helps depends on what the kernel reconstruction leaves unresolved. A small error in the network alone does not guarantee improvement after correction.

Let uu be the true solution and pp the frozen network prediction. Let Λ\Lambda evaluate the PDE and condition operators at the observation sites, dd collect their known forcing and boundary/initial values, and SS be the fixed linear map from observations to a reconstructed field. The zero-mean solution u^0\widehat u_0 and the solution retaining the network mean u^p\widehat u_p are

u^0=Sd,u^p=p+S(d−Λp).\widehat u_0=Sd,\qquad \widehat u_p=p+S(d-\Lambda p).

This is Gaussian-process conditioning with a prescribed mean [1]. Define the zero-mean error r0=u−Sdr_0=u-Sd and the prior contribution t=p−SΛpt=p-S\Lambda p. In the chosen evaluation inner product, retaining the prior improves the error norm exactly when

2⟨r0,t⟩>∣t∣2.2\langle r_0,t\rangle>\\|t\\|^2.

The criterion explains why the same prior can help one reconstruction and hurt another. Because it depends on the unknown solution, it cannot select a prior from observable defects alone. Our contribution uses the established defect identity to reconstruct a frozen network's remaining error and separates the ideal linearized correction from what finite observations actually recover.

We designed a new controlled comparison to measure this dependence. On one linear elliptic instance, five paired seeds were evaluated at 32, 128, 500, and 1,000 interior observations, with Gaussian and Matérn-9/2 kernels and fixed lengthscales 0.15/0.30. Each of the 80 designs compares zero, early-network, and late-network means, giving 240 reconstructions. Boundary observations are additional to the interior budgets. Operator, sites, kernel, and regularization stay identical within a design; changing the mean recomputes its defects. Neither separate tuning for each mean nor an oracle prior enters this experiment.

The early network has 2,000 Adam optimizer calls. The late network has 22,300 total calls: 20,000 Adam, 2,000 limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS), and 300 NysNewton-CG (NNCG) calls. L-BFGS estimates curvature from gradient history; NNCG is a Nyström-preconditioned Newton-conjugate-gradient method [5]. These methods train the network before its weights are frozen. The late mean tests whether stronger training makes prior reuse reliably beneficial.

Interior observationsCorrection versus the raw networkComparison with the same-map zero mean
32Every late-prior setting worsens the raw field in 5/5 seedsBeating a poor zero-mean solve need not improve the supplied network
128, 500, 1,000Every early/late setting improves the raw field in 5/5 seedsThe outcome depends on kernel and lengthscale
1,000Both supplied means improve their raw fieldsGaussian favors zero mean; Matérn favors the supplied means at both lengthscales, in 5/5 seeds

These fixed-parameter comparisons establish dependence on the reconstruction design. Equal numerical lengthscales have different spectral meanings for the two kernels, and the experiment does not rank separately optimized kernel families.

We also tested the choice of initialization for nonlinear iterative kernel solves. Five paired seeds per nonlinear problem share observation sites; each iteration refreshes the linearization and selects damping using physical validation, which scores PDE and condition defects without solution labels, up to 32 iterations. The geometric-mean PINN/zero error ratios are 1.137 [0.675, 1.917] for semilinear elliptic and 1.291 [0.943, 1.767] for Burgers, with paired 95% confidence intervals in brackets. Neither interval establishes an accuracy advantage. Five-seed geometric-mean times including training are 41.60/248.93 seconds for zero/PINN initialization on semilinear elliptic and 18.47/139.00 seconds on Burgers. Thus, these tests support conditional reuse of an available field; they provide no general case for training a network first to accelerate a solve from scratch.

AuthorsOpenReview ↗

Response to Reviewer u5Zr (part 2/4)

Q2. Why can Stage2 weaken boundary or initial conditions?

Your concern is that the nonlinear adjustment can also weaken constraints whose equations are linear. Our new interventions confirm that this can happen and identify which part of the adjustment causes the tradeoff. Stage1 is the first kernel correction fitted to the frozen network's PDE and condition defects. Stage2 refits those same targets with adjusted regularization. The submission applies it to nonlinear problems; based on these ablations, we will present it as optional.

Write the true error as e=u−pe=u-p, its spatial derivative as exe_x, and the nonlinear Taylor remainder about pp as Rp(e)R_p(e). For Burgers, for any finite exponent q≥1q\ge1,

Rp(e)=eex,∣Rp(e)∣Lq≤∣e∣∞∣ex∣Lq.R_p(e)=ee_x,\qquad \\|R_p(e)\\|_{L^q}\le\\|e\\|_\infty\\|e_x\\|_{L^q}.

An amplitude-squared proxy misses this derivative dependence. The submitted Stage2 rule is therefore a heuristic. It increases relative regularization while retaining the network, linearized operator, targets, sites, lengthscales, and kernel matrix before amplitude scaling. It then re-solves the defect system and re-profiles amplitude. Its correction replaces Stage1; it does not add another Newton update. Applying the adjustment to both observation blocks also relaxes boundary/initial enforcement.

To isolate that effect, we used semilinear elliptic and Burgers problems, five seeds, and the early/late checkpoints defined above: 20 base cases, with five independent seeds per fixed problem/checkpoint condition. Four variants apply the fitted Stage1 or prescribed Stage2 regularization level to each of the interior and condition blocks: Stage1, interior-only adjustment, condition-only adjustment, and shared Stage2. Operator coefficients, targets, block scaling, numerical jitter (small diagonal stabilization), and the amplitude-free kernel remain fixed. Burgers also has separate spatial-boundary-only and initial-condition-only interventions. Re-profiled amplitude cancels from the posterior means.

The early Burgers comparison measures field error against the reference solution and condition defects on independent audit sites. Entries are five-seed geometric means; root-mean-square (RMS) defects are in absolute condition units.

CorrectionField error / Stage1 [95% interval]Boundary RMSInitial RMS
Stage112.24e-73.54e-7
Shared Stage2, original rule0.9387 [0.9090, 0.9693]4.51e-49.40e-4
New interior-only variant0.9548 [0.9246, 0.9860]6.72e-86.05e-8

Shared Stage2 reduces field error by 6.1% while substantially weakening condition enforcement. Interior-only adjustment retains a 4.5% reduction with small condition defects. It is a separate variant; this Burgers intervention does not explain the submitted Allen-Cahn degradation.

We tested whether observable defects could select safely among Stage1, shared Stage2, and interior-only adjustment. On 512 interior and 480 condition selection sites, the rule minimizes PDE mean squared error plus 100 times condition mean squared error. Each condition's mean squared error must remain below 1.05 times Stage1's value plus 10−1210^{-12}. A separate audit uses 1,024 interior and 960 condition sites. Reference fields are read after selections are committed. The rule still increases true error in 2/5 late Burgers cases; its absolute tolerance can allow large relative condition changes when the baseline is tiny.

We also checked the computed Stage1 correction, denoted e^1\widehat e_1. Using the prescribed Burgers proxy factor CN=0.5C_N=0.5, the audit-site ratio max⁡∣Rp(e^1)∣/(CNmax⁡∣e^1∣2)\max|R_p(\widehat e_1)|/(C_N\max|\widehat e_1|^2) ranges from 10.85 to 15.27 across early seeds and 23.30 to 35.22 across late seeds. This directly tests the candidate correction, while the true-error remainder remains unknown. At the long-trained nonlinear endpoints, every Stage2/Stage1 error-ratio interval includes one. We will present Stage1 as primary and Stage2 as optional, retaining these failures and avoiding a safety claim for the selection rule.

AuthorsOpenReview ↗

Response to Reviewer u5Zr (part 3/4)

Q3. Which function spaces make the theoretical argument valid?

The inverse assumption asks whether a small defect produces a controlled solution of the complete linearized problem, including boundary and initial conditions. The nonlinear smoothness assumption then controls the discrepancy between that linearized correction and the true error. The computed kernel functions must also lie in the chosen field space. These requirements depend on the PDE and the norm. Membership of the true error in a reproducing kernel Hilbert space (RKHS), the function space associated with a kernel and its norm, is an additional requirement for the separate reconstruction bounds. It is unnecessary for the Newton error decomposition itself.

For elliptic equations, a weak formulation supplies an appropriate inverse estimate. In the submitted independently forced example, the nonlinear operator NN acts on a field vv on Ω=(0,1)2\Omega=(0,1)^2 as

a(x,y)=0.02[1+0.5\sin(2\pi x)\sin(2\pi y)]\ge0.01.$$ Use $H^1(\Omega)$ for the field and $H^{-1}(\Omega)$ for the interior residual. The former controls square-integrable values and first derivatives; the latter accommodates residuals tested against functions with zero boundary values. Boundary data belong to the global trace space of $H^1$. The linearized energy integrates $a|\nabla v|^2+15p^2v^2$ and controls the zero-boundary $H^1$ norm. Lax-Milgram and bounded extension of boundary data into the domain give the complete inverse estimate. For a cubic coefficient $\gamma\ge0$, let $C_4$ be the embedding constant from $H^1$ into $L^4$. For fields $v,w$ with $H^1$ norms at most $M$, the operator derivative satisfies $$\\|N'(v)-N'(w)\\|_{H^1\to H^{-1}} \le6\gamma C_4^4M\\|v-w\\|_{H^1}.$$ The inverse constant depends on ellipticity, domain, prior norm, and boundary extension. This also covers the new cubic benchmark. It proves a weak formulation; it does not give the original twice-continuously-differentiable setting or bounded point evaluation of arbitrary $H^{-1}$ residuals. For time-dependent Burgers and Dirichlet Allen-Cahn, the space must also control one time derivative and two spatial derivatives. Set $T=1$, let the spatial interval $I$ be $(-1,1)$ for Burgers or $(0,1)$ for Allen-Cahn, and write $Q=I\times(0,T)$. For fixed $q>3$, use the field space $X_q$ and residual space $Y$ defined by $$X_q=W^{1,q}(0,T;L^q(I))\cap L^q(0,T;W^{2,q}(I)),\qquad Y=L^q(Q).$$ Its norm sums the $L^q$ norms of $v,v_x,v_{xx},v_t$. Initial data lie in $W^{2-2/q,q}(I)$ and lateral data in $W^{1-1/(2q),q}(0,T)$, with matching initial-corner values [2]. Allen-Cahn has diffusivity $10^{-4}$ and an initial condition. Linearization at the smooth frozen predictor gives drift/potential $(p,p_x)$ for Burgers and $(0,15p^2-5)$ for Allen-Cahn. A weighted heat-equation inverse and boundary/initial extensions yield bounds depending on diffusion, horizon, coefficients, and extension constants. If $E_0$ is the embedding constant from $X_q$ into bounded functions, the local derivative-Lipschitz constant is at most $2E_0$ for Burgers and $30M_0E_0$ for Allen-Cahn on a convex neighborhood with field amplitude at most $M_0$. This revises the original strong-space claim. Smooth truth and a smooth network do not automatically satisfy the higher corner compatibility required by a classical Newton problem with soft conditions. For Burgers that requirement includes $ee_x=0$ at the initial corners. The finite-$q$ formulation permits the resulting corner loss of higher regularity. General $L^q$ data still do not support bounded point sampling. Our interior-regularity and structured-remainder arguments apply to these smooth PDE data at strictly interior, positive-time sites; they do not prove a general sampling bound or place a trained checkpoint inside a useful Newton neighborhood. Gaussian-RKHS membership likewise remains unverified for the trained errors. For a separately specified Matérn-9/2 kernel with fixed positive lengthscale and amplitude, the two-dimensional RKHS corresponds to restricted $H^{11/2}$ [3,4]. The new sine-mixture Burgers solution is smooth through its positive Cole-Hopf representation. Subtracting a finite tanh network gives an error with an $H^6(\mathbb R^2)$ extension, implying membership. The manufactured Allen-Cahn truth permits the same argument. This is a theoretical specialization, not a nonlinear Matérn comparison experiment. It supplies neither Gaussian membership nor a known RKHS norm; the observed relative-$L^2$ gains remain empirical.
Reviewer_ig1FOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This works proposes a methodology to overcome the difficulties of training Physics-Informed Neural Networks (PINNs) to completion. Rather than working on improving the training dynamics of PINNs, the authors propose to solve for the pointwise error between the partially trained PINN output and the true solution through RKHS-based scattered data interpolation. The proposed methodology is straightforward to implement and can be applied to both linear and non-linear partial differential equations.

Critical strengths
  • The proposed method is a natural, interesting, and well-motivated combination of related ideas that have appeared in the numerical analysis and SciML literature over the years.
  • The numerical experiments, at least for linear problems, strongly suggests that this method is likely to be a mo
Critical weaknesses
  • The writing and overall presentation make the paper, in my opinion, hard to read and follow. As a result, some of the main ideas, which are for the most part natural and straightforward, become unnecessarily hard to understand and confusing at times (see elaboration below).
  • The theoretical results concerning the algorithm are very limited and do not offer much, if any, practical guidance or guarantees regarding the proposed algorithm, as they rely on a number of assumptions whose satisfaction is unclear. It is in particular unclear to me whether Proposition 2 yields any non-vacuous bound for, most non-linear PDEs solved with, say, a tanh network and an error field recovered with a Gaussian RKHS (see further comments below).
  • The numerical evidence used to demonstrate the efficiency of the proposed method is rather insufficient, as the PINN baselines against which the authors' method is tested are not very meaningful (see further comments below).

Supporting Arguments

Issues with prose and presentation: The readability and accessibility of the manuscript is unnecessarily hurt by odd choices in the prose and overall organization of the text which makes many statements and ideas remarkably more difficult to grasp than they should. For example, there are numerous passages where the authors refer to notation, quantities and terminology as if they were familiar to the reader, despite not being defined until much later in the text. Likewise, some passages include some dense, convoluted prose with non-standard terminology which obscures straightforward ideas. A non-exhaustive list of such passages includes, e.g., the sentences starting on lines 154, 189, 193, 255, or the first two paragraphs of Section 3.5.

Theoretical guarantees are extremely limited: As I understand, the main mathematical guarantee for the proposed method is Proposition 2, concerning the error incurred by approximating the error field induced by a (possibly non-linear) differential operator through its first-order linear approximation (which is exact for a linear operator). While the result is true as a consequence of Taylor's theorem, it does not say anything meaningfully quantitative regarding the reconstruction error one could expect for a "typical" (non-)linear PDE, or how the error is affected by parameters such as the number of collocation points, the choice of kernel, the PDE smoothness etc. As a result, the theoretical results offer close to no insight regarding how to choose such parameters in practice, and no guarantee regarding the ability of the method to recover the actual error field.

Assumptions seem restrictive and are not discussed appropriately: Besides the above mentioned issues, the overall method is theoretically grounded on a number of assumptions that are not obviously satisfied. The authors assume for instance that the linearized boundary-value operator AθA_\theta is boundedly invertible (Proposition 2/Assumption (N3)), or that the error field ee lies in the RKHS (Assumption (A4)). These assumptions are non-trivial and a more fleshed out discussion on their viability would be appreciated. As a matter of fact, it follows from standard RKHS theory that Assumption (A4) generally fails for tanh networks error fields reconstructed using Gaussian RKHS, which as a consequence means the numerical experiments considered in this work are likely beyond the range of applicability of Proposition 2.

Numerical evidence is not compelling: Besides the issue of some necessary assumptions not being satisfied in the experimental setting, it appears to me that while the experiments demonstrate a remarkable benefit of the approach on linear PDEs compared to simply training a PINN for longer, it would have made much more sense to compare the given approach to actually competitive alternatives meant to directly address PINN training failures such as NNCG (Rathore et al., ICML2024), SOAP (Wang et al., NIPS2025), or some variant of ENGD (Müller and Zeinhofer, ICML2023). Lastly, Appendix H seems to suggest that many PDE on which the method performs well would in fact be even more efficiently solved by directly using a kernel collocation approach from the start, without involving training a PINN at all. This further puts into question the actual benefit of the proposed method.

Questions for authors
  1. Could the authors provide some non-trivial conditions for which Assumptions (N3) and (A4) are simultaneously satisfied? Likewise, non-trivial settings under which the quantity CALNC_A L_N in Proposition 2 does not blow up faster than the norm of the error field ∥e∥X\|e\|_X goes to zero would be valuable.
  2. On lines 198-201, the authors state that "The coefficients solve Mc=yMc = y by a factorization rather than by forming M−1M^{-1}", could you explicit which factorization algorithm is used, and how many factorizations are needed in total? In particular, what is the total computational cost of the proposed method, hyperparameter-tuning included? Unless I'm mistaken, the discussion following Algorithm 1 does not make this explicit.
  3. Could the authors, if possible, compare their method against closely related work and/or "state-of-the-art" PINN training methodologies rather than plain Adam, such as Adam+L-BFGS, NNCG, ENGD or SOAP?
4 comments
AuthorsOpenReview ↗

Response to Reviewer ig1F (part 1/4)

Q1. Can the stability and kernel assumptions hold together, with useful constants?

The reviewer's concern about Gaussian-kernel membership is valid, but the inference that this membership is required by Proposition 2 is incorrect. This distinction is explicit in the submitted Appendix A.1, lines 800 to 819. That proposition separates error due to linearizing the partial differential equation (PDE) from error due to reconstructing a correction from finitely many observations. It requires a bounded inverse of the full linearized boundary/initial-value operator on compatible data, a locally Lipschitz derivative of the nonlinear PDE operator, and a correction that can be measured in the chosen norm. It does not require the true error to belong to the Gaussian kernel's function class.

A reproducing kernel Hilbert space, or RKHS, is the function class and norm determined by a kernel. Membership in this space is a substantive restriction. A smooth error from a physics-informed neural network (PINN) with tanh activations need not belong to the Gaussian RKHS. Our separate reconstruction bounds require this membership; the submitted appendix states that it has not been established for the experimental error fields. We will make that limitation explicit in the main text.

To state what Proposition 2 does establish, let uu be the true solution, pp the frozen network, and e=u−pe=u-p its true error. Let δ\delta solve the linearized PDE with right-hand sides given by the negative PDE and boundary/initial defects, and let e^\widehat e be the computed correction. The bound is

\le \\|\delta-\widehat e\\|_X +\frac{C_A L_N}{2}\\|e\\|_X^2.$$ Here $C_A$ bounds the inverse of the linearized PDE together with its boundary/initial conditions; $L_N$ bounds how rapidly the PDE derivative changes near the true and predicted fields. The second term controls the error of the ideal Newton correction. The first term measures how accurately the finite-observation estimator reconstructs that correction. The proposition leaves this first term explicit, so it does not by itself promise improvement or prescribe an observation count. ### A nonlinear example with fixed norms and constants To answer your simultaneous-feasibility question, consider the problem on $(0,1)$ with prescribed endpoint values (Dirichlet conditions) and $$N(v)=-v''+2v^3,\qquad p(x)=0.35+0.4\tanh\bigl(2(x-\tfrac12)\bigr).$$ Use the fixed space $X=C^2([0,1])$ of twice continuously differentiable functions, with $$\\|v\\|_X=\\|v\\|_\infty+\\|v'\\|_\infty+\\|v''\\|_\infty.$$ The data space is $C([0,1])\times\mathbb R^2$, with the sum of the forcing and endpoint norms. The complete linearization is $$\mathcal A_pv=(-v''+6p^2v,\\ v(0),\\ v(1)).$$ Its potential is nonnegative. The maximum principle, the differential equation, and a first-derivative estimate give the nonsharp inverse bound $C_A\le9.75$. On the fixed unit $X$-ball around $p$, the cubic derivative has Lipschitz constant $L_N\le21$. Fix a Gaussian kernel and a nonzero finite sum of kernel functions centered at fixed points, $h=\sum_j a_jk(\cdot,z_j)$. Set $$u_\varepsilon=p+\varepsilon h,\qquad f_\varepsilon=N(u_\varepsilon),\qquad g_{\varepsilon,i}=u_\varepsilon(i),\quad i=0,1.$$ The error $e_\varepsilon=\varepsilon h$ belongs to the Gaussian RKHS by construction. For $0<\varepsilon\\|h\\|_X<1$, all norms, the prior, the kernel, and the constants stay fixed, and $$\frac{\\|e_\varepsilon-\delta_\varepsilon\\|_X}{\\|e_\varepsilon\\|_X} \le\frac{9.75\times21}{2}\\,\varepsilon\\|h\\|_X\longrightarrow0.$$ Thus the constants do not diverge as the error decreases. This is a nonlinear family with a tanh prior satisfying both requirements. The forcing and boundary data vary with $\varepsilon$; it is not a training sequence for one fixed benchmark. Nor does the construction imply that tanh functions generally belong to the Gaussian RKHS.
AuthorsOpenReview ↗

Response to Reviewer ig1F (part 2/4)

What the experiments and theory can each support

For the trained benchmarks, we have not certified Gaussian membership, useful inverse constants, or a sufficiently small error in the theoretical norm. We also give a bound allowing the true error to lie outside the RKHS. It retains the error of approximating that field by an RKHS function, both at the requested output and under the observation operators. This makes the missing approximation requirement explicit; it does not guarantee a good approximant or a universal sampling rate.

The submitted Newton diagnostic illustrates why the two errors should be separated. Using a refined numerical Newton solution and the fixed evaluation-grid L2L^2 norm, the mean linearization ratios on two held-out Burgers instances were 0.0053 and 0.0071; reconstruction ratios were 0.9830 and 0.7677. Each instance used three runs, with all ratios normalized by the original error norm. Reconstruction was the larger discrepancy. These measurements are neither additive percentages nor verification of the theoretical XX-norm bound.

We tested kernel and observation sensitivity separately on one new linear problem. Five seeds, four interior observation budgets (32/128/500/1,000), Gaussian and Matérn-9/2 kernels, and lengthscales 0.15/0.30 give 80 fixed designs. Each compares zero mean with networks saved early and late in training, giving 240 reconstructions. Boundary observations are additional to the interior budgets. Within a design, only the supplied mean and its corresponding defects change. At 32 observations every late-network correction worsens its raw prediction in all five seeds; at 128, 500, and 1,000 all tested learned-mean corrections improve their raw predictions. Their advantage over a zero-mean solve still depends on the kernel. These results document parameter sensitivity; they do not provide a rule selecting parameters from PDE smoothness.

Q2. Which factorization is used, and is hyperparameter tuning included in the cost?

We use Cholesky factorization, followed by triangular solves. The regularized observation matrix is M=KΛΛ+ΣM=K_{\Lambda\Lambda}+\Sigma, where Λ\Lambda collects the differential and condition observations, the first term is their kernel Gram matrix, and Σ\Sigma is the positive regularization matrix. Factoring M=LL⊤M=LL^\top gives the correction coefficients without forming M−1M^{-1}. Every new kernel-parameter objective evaluation requires a factorization of its current matrix.

Stage1 denotes the first fitted kernel correction. Stage2 adjusts regularization and solves the same frozen defect problem again, replacing the first correction. The submission applies this second step to nonlinear problems; based on the new ablations, we will present it as optional. Their costs are reported separately below.

The timing therefore includes the coarse search, all optimization starts, failed attempts, and final reconstruction. To give actual counts, the extended-training studies used limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS), a quasi-Newton optimizer that builds curvature estimates from past gradients, and NysNewton-CG (NNCG), a Nyström-preconditioned Newton-conjugate-gradient optimizer [1]. The latter uses curvature information and preconditioning to address the ill-conditioning of PINN training.

AuthorsOpenReview ↗

Response to Reviewer ig1F (part 3/4)

Network trainingTotal factorizations for 15 final correction fits, including tuning and both applicable stages
Long NNCG5,395
Long normalized L-BFGS4,906

Each row covers the final checkpoints of three problems with five seeds each. With nn observations and NfacN_{\rm fac} factorization attempts, dense factorization work is O(Nfacn3)O(N_{\rm fac}n^3) and storage is O(n2)O(n^2). Observation construction, matrix assembly, and differentiation add measured work. Once the cross-kernel matrix is built, coefficient-based prediction at QQ queries costs O(Qn)O(Qn); auxiliary variance or construction of the equivalent adjoint prediction map requires O(Qn2)O(Qn^2) solve work.

At the long-trained final checkpoints, the five-seed geometric means of the additional Stage1 cost range from 2.50 to 3.71 seconds, including observations, the full kernel search, factorizations, and prediction. For semilinear elliptic after long L-BFGS, training-inclusive totals are 1,998.92 seconds for raw prediction, 2,001.89 seconds with Stage1, and 2,002.09 seconds with both stages and auxiliary variance. Stage1-only accounting subtracts the measured Stage2 cost from the joint workflow. It is a component-based accounting of recorded work, not a fresh end-to-end timing of each possible workflow. Independent reference construction, evaluation, and subsequent numerical checks are reported separately.

The kernel search did not always terminate successfully: among these final fits, its selected finite candidate had an abnormal termination flag in 8/15 NNCG-based fits and 4/15 L-BFGS-based fits. These results and their costs are retained. Successful reconstruction does not establish that the likelihood objective reached a stationary point. Separate checks also found gradient sensitivity at very small numerical regularization. With the fitted systems fixed, two algebraically equivalent prediction calculations changed relative L2L^2 errors by at most 0.6805 parts per million, without changing the checked five-seed confidence-interval conclusions. This checks numerical evaluation of the fitted systems; it does not prove optimization convergence.

Q3. Do the gains survive stronger optimizers, and why not solve directly?

The submitted primary networks and matched-time continuations used Adam. We have now added long L-BFGS and NNCG continuations and a separately tuned SOAP baseline. SOAP means Shampoo with Adam in the preconditioner's eigenbasis: it rotates gradients into curvature-informed coordinates and applies Adam there [3]. Its use for PINN training is studied in [2]. We have not tested the proposed energy natural-gradient alternative.

The purpose is to determine whether correction still helps a field that has already benefited from these optimizers. The new problems are variable-coefficient linear elliptic, independently forced cubic semilinear elliptic, and viscous Burgers. Each uses five seeds. Within each problem/seed pair, the long L-BFGS and NNCG continuations start from identical saved weights after 20,000 Adam, 2,000 L-BFGS, and 300 NNCG calls. NNCG resumes its optimizer state. L-BFGS begins with fresh curvature history and a fixed loss normalization, the reciprocal of the starting physical loss; later extensions preserve that history and scale. Each permits 100,000 additional calls, subject to a 7,200-second NNCG or 1,800-second L-BFGS cap.

The early and late means used in Q1 are the networks at 2,000 Adam calls and at this 22,300-call starting checkpoint. Correction uses 500 interior and 200 boundary/initial observations separate from training. Kernel fitting uses defects alone. Physical validation likewise scores PDE and condition defects without reference-solution labels. Predictions are fixed before evaluation on a common 101×101 grid against analytic or independently refined numerical references.

AuthorsOpenReview ↗

Response to Reviewer ig1F (part 4/4)

Training and evaluated checkpointLinear elliptic gainSemilinear elliptic gainBurgers gain
Long normalized L-BFGS, final74.82× [40.58, 137.95]29.79× [19.65, 45.17]1.254× [1.037, 1.517]
Long NNCG, final16.43× [2.084, 129.63]13.80× [3.459, 55.06]1.175× [1.035, 1.334]
SOAP, best saved by physical validation530.62× [247.77, 1136.40]112.93× [73.19, 174.25]1.133× [0.774, 1.658]

Gain means uncorrected relative L2L^2 error divided by corrected relative L2L^2 error; values above one favor correction. This metric measures the field error relative to the reference solution. Entries are five-seed geometric means. The 95% confidence intervals use paired t statistics on log error ratios, without adjustment for multiple comparisons. L-BFGS improves in 15/15 cases and NNCG in 14/15. Its remaining case worsens by 0.42%, despite a successful kernel-fit return. SOAP improves in all five seeds on each elliptic problem, while its Burgers interval includes one. Long L-BFGS elliptic relative L2L^2 errors decrease from 2.52×10−62.52\times10^{-6} to 3.37×10−83.37\times10^{-8} and from 4.09×10−64.09\times10^{-6} to 1.37×10−71.37\times10^{-7}.

Ten nonlinear NNCG runs reach the threshold of 10−810^{-8} for the largest absolute training-gradient component; the other long runs reach call or time limits. SOAP selection uses physical validation, with all 12,000 tuning calls and the full training/selection workflow charged. These finite-budget results support additional correction gains, particularly for the elliptic problems. They do not establish global convergence of every optimizer. The optional Stage2 regularization adjustment adds no resolved gain at the long nonlinear endpoints: all its paired intervals include one.

Your direct-solver comparison raises a different decision: whether to train a PINN at all. We tested iterative Newton-kernel solves starting from zero or a saved PINN, using five paired seeds per nonlinear problem, shared observation sites, refreshed linearization, physical-validation damping, and a 32-iteration cap. PINN/zero error ratios are 1.137 [0.675, 1.917] for semilinear elliptic and 1.291 [0.943, 1.767] for Burgers. Neither establishes an initialization advantage, and including PINN training favors the zero-start workflow in cost. This is an independently implemented control, not a reproduction of a particular published solver.

Classical solvers were also evaluated on 24 prescribed comparison grids and eight reference-refinement grids across two instances per PDE. On the first instances, prescribed grids reach relative L2L^2 error at most 10−410^{-4} in core CPU times of approximately 0.038/0.171/0.036 seconds, respectively. These are single timings at that accuracy; they are not matched-accuracy speedups over the corrected fields. The evidence supports inexpensive refinement of an available network, but does not establish training a PINN first as a generally faster route from scratch.

We will rewrite the passages identified in the review to define symbols before use and explain the operations in sequence: evaluate defects, formulate the linearized error problem, and reconstruct the correction. The new studies use a rebuilt implementation with retained code, configurations, checkpoints, and logs. The original implementation and raw artifacts are unavailable; the studies supplement the submission and do not reproduce its full benchmark suite.

References

[1] Pratik Rathore, Weimu Lei, Zachary Frangella, Lu Lu, and Madeleine Udell. Challenges in Training PINNs: A Loss Landscape Perspective. ICML 2024, PMLR 235:42159-42191.

[2] Sifan Wang, Ananyae Kumar Bhartari, Bowen Li, and Paris Perdikaris. Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective. NeurIPS 2025.

[3] Nikhil Vyas et al. SOAP: Improving and Stabilizing Shampoo using Adam. arXiv:2409.11321, 2024; version 2, 2025.

Reviewer_djf8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary
  • The paper introduces a post-hoc correction method for trained Physics-Informed Neural Networks (PINNs). The central idea is to use the PDE residuals and boundary/initial-condition defects to reconstruct the remaining solution error through an operator-conditioned kernel method, without further training the original network.
  • For linear PDEs, the method solves the error equation directly through kernel regression. For nonlinear PDEs, it uses a linearization around the trained PINN. Experiments across several PDE benchmarks demonstrate substantial reductions in solution error.

Major strengths/weaknesses and questions are listed below; here are some further minor notes:

  • In general, the B_j's (line 135) do not need to be linear. This is stated without clarifying that it is an assumption/restriction of the governing equation.
  • In lines 123–124, the authors state that the method requires "data". This refers to PDE forcing and boundary/initial-condition data, but could be confused with additional training data.
Critical strengths
  • S1 Mathematical analysis and assumptions The paper provides a thorough mathematical discussion of the proposed method and its underlying assumptions. In particular, the analysis distinguishes between reconstruction error and nonlinear linearization error.
  • S2 Experimental evaluation The method is evaluated across several PDE benchmarks, including nonlinear equations, held-out physical instances and irregular geometries. The experiments demonstrate substantial improvements over the original PINN predictions. The additional comparisons and ablations in the appendix are also valuable.
Critical weaknesses
  • W1 Novelty The paper does not clearly identify which elements go substantially beyond established defect correction, Newton linearization, and operator-kernel PDE solvers.

    • Essentially, the method still solves a PDE, namely the error equation, using an operator-conditioned kernel method. The authors explicitly acknowledge that, in the linear case, their approach is equivalent to an existing kernel PDE solver with the trained PINN as its prior mean. Therefore, this raises the question of when the proposed approach is preferable to directly solving the PDE using conventional numerical methods. A convincing motivation or computational justification for this choice is missing.
    • The nonlinear second stage appears to be one of the more novel algorithmic contributions. However, its benefits do not appear to be statistically significant on Burgers and the new nonlinear benchmark, while it even worsens the results on Allen–Cahn. This raises the question of which novel components actually contribute to the reported improvements.
  • W2 experiments and significance The experimental comparisons do not convincingly establish the computational benefits of the proposed approach.

    • A comparison against a classical full PDE solver seems missing since the method effectively performs a kernel-based PDE solve after training a PINN.
    • "Continued training" gives the impression of letting the network train until convergence. The precise meaning of this remains unclear, since the appendix defines "Matched-time continued training". Assuming that those mean the same thing, the comparison is only conducted against continued training for the additional time that the correction method takes, but starting from deliberately incomplete training runs. I believe that this is not a meaningful comparison. My key point is that PINNs are not (yet) competitive alternatives to classical numerical PDE solvers in terms of computation time. Therefore the speedup only matters at sufficient accuracy, i.e. at fully trained / strongly tuned PINNs. A potentially interesting question might be how much computation time can be saved by stopping training early and using the proposed method to resolve the remaining error.
  • W3 presentation and readability The paper is very dense, poorly readable and not accessible to the general community.

    • The introduction is very dense, poorly readable and not accessible to the general community. The sentence "Automatic differentiation extracts the operator, and its action on both arguments of a kernel produces a Gram matrix coupling scattered interior, boundary and initial observations" (lines 45-47) is a representative example. It combines several technical terms without introduction, uses descriptions like "automatic differentiation extracts the operator" that do not explain the procedure to the reader and uses specific terms like "scattered interior" for no obvious reason.
    • Overall, the introduction reads as unnecessarily technical and insufficiently reader-oriented. I would encourage the authors to substantially revise it, focusing on clearly communicating the underlying ideas and their motivation rather than compressing methodological details into jargon-heavy sentences.
    • Also, the central motivation for the approach and its main benefits are not described clearly in the introduction.
    • Similar criticism extends also to the experiments. Experimental comparisons are often opaque due to unexplained baselines and ablation terminology introduced without clarification. For example, the "neural error field" baseline does not seem to be properly introduced or explained in the experimental setup.
    • Terminology such as "a zero-mean direct kernel solve on the same sites", and the description of uθu_\theta as a prior mean, is used without sufficient introduction, further reducing readability.
Questions for authors
  • Q1 W1 Could the authors clarify which methodological components are genuinely novel compared with established defect correction, Newton linearization and operator-kernel PDE solvers? In particular, what is the contribution of the nonlinear second regularization stage, given the results of the ablation study?
  • Q2 W2 Could the authors clarify under which circumstances the proposed method is preferable to directly solving the PDE using conventional numerical methods? In particular, could they also provide an end-to-end accuracy-versus-computation-time comparison against strongly tuned PINNs (or even classical solvers)?
  • Q3 W3 Could the authors revise the introduction, related work and experimental setup to more clearly explain the motivation, the underlying method and the baselines? In particular, the differences from prior work and the meaning of the experimental comparisons should be understandable without having to reconstruct these details from the appendices.
5 comments
AuthorsOpenReview ↗

Response to Reviewer djf8 (part 1/4)

We thank the reviewer's insightful comments

W1 / Q1. What is the contribution, and which part produces the gains?

The reviewer's concern is whether the method contributes more than a restatement of established correction formulas, especially when its second stage sometimes worsens the result. Our contribution is a specified procedure for reconstructing the remaining error of an existing physics-informed neural network (PINN), together with an analysis and experiments that distinguish the sources of reconstruction error.

The inputs are the trained network and its partial differential equation (PDE), including boundary and initial conditions. We measure where the network violates these equations. Those defects become observations for a kernel reconstruction of the missing correction. The kernel's lengthscales control how observations influence nearby points; regularization controls how closely the reconstruction fits them. Both are fitted from the defects. Adding the resulting correction leaves the network weights unchanged and requires no reference-solution labels for fitting.

The relationship to existing methods

Established approachRelationship to this work
Residual-based and Newton correction of neural-operator predictions [1]We use the established error equation and nonlinear linearization. We study how accurately their correction can be reconstructed from finitely many observations.
Gaussian-process reconstruction with a supplied mean [2]For linear PDEs, our estimator is exactly this estimator when the mean, kernel, observations, and regularization agree. The PINN supplies the starting field, called the prior mean.
Iterative nonlinear kernel solvers [3]These methods update the background solution and linearization repeatedly. Our post-processing keeps the supplied network and linearization fixed, and evaluates the benefit of that reconstruction.

The analysis distinguishes the ideal Newton correction, which solves the complete linearized error equation, from the correction actually reconstructed using finite observations. We tested that distinction in the submitted manuscript by independently solving and refining the same linearized problem. On two held-out Burgers instances, the instance-level mean linearization discrepancies were 0.0053 and 0.0071 times the original error norm; reconstruction discrepancies were 0.9830 and 0.7677, respectively. Each instance had three runs. Reconstruction was the larger discrepancy. These are separate evaluation-grid L2 norm ratios, not additive percentages. The theoretical interpretation assumes the stated inverse and regularity properties; it does not guarantee improvement for every network.

Testing whether the supplied network actually helps

The equivalence to a direct kernel solver raises an experimental question: does retaining the network improve reconstruction, or would a zero starting field work better? To isolate this effect, the new experiments hold the reconstruction method fixed and change only its starting field.

On one new linear elliptic problem, five paired runs use 32, 128, 500, or 1,000 interior observations; Gaussian kernels or Matérn kernels with smoothness parameter 9/2; and fixed lengthscales 0.15 or 0.30. Each of these 80 designs compares zero, an early network saved after 2,000 training calls, and a later network saved after 22,300 calls. These counts cover interior observations, with boundary observations added separately. The training sequence is given under Q2. The equation, observation sites, kernel, and regularization stay identical within a design. Known forcing and condition data remain fixed, while the measured defects change with the starting field. No parameters are separately retuned for each mean.

At 1,000 interior observations, both network-based corrections improve their own uncorrected fields in every run. They also beat the corresponding zero-mean solve with Matérn, whereas Gaussian favors zero mean at both lengthscales. At 32 observations, every later-network setting worsens its uncorrected field in all five runs. Comparing against both the original network and zero mean prevents an advantage over a poor direct solve from being mistaken for successful correction. These fixed-parameter results identify dependence on the reconstruction design; they do not rank individually optimized kernel families.

AuthorsOpenReview ↗

Response to Reviewer djf8 (part 2/4)

What the second regularization step adds

The large gains reported under Q2 use the initial reconstruction alone, called Stage1 in the paper. The second step, Stage2, increases regularization, re-estimates the kernel's overall scale, and solves the same defect problem again. Its output replaces the first correction. The network and linearized equation remain fixed.

This adjustment is intended to tolerate mismatch from linearizing a nonlinear PDE. However, the original rule also makes boundary and initial observations less restrictive. We tested whether these two effects could be separated. On two nonlinear problems, with five runs and two prescribed checkpoints each, we changed regularization on the interior only, on the conditions only, or on both. The network, equation, observations, and kernel lengthscales stayed fixed.

For early Burgers checkpoints, adjusting both blocks reduces field error by 6.1%, but boundary and initial-condition root-mean-square errors rise from approximately 1e-7 to 1e-4 and 1e-3, respectively. Adjusting only the interior retains a 4.5% field-error reduction with both condition errors below 1e-7. These are five-seed geometric means, rather than per-run threshold guarantees. Condition defects are measured at audit points independent of fitting and selection. An improvement in overall field error can therefore conceal poorer condition enforcement.

At the long-trained nonlinear endpoints, every confidence interval for adjusted versus initial reconstruction includes equal error. The submitted Allen-Cahn ablation gives 3.28 times higher error [95% confidence interval: 1.06, 10.1] after adjustment, across seven runs. The amplitude-based rule is a heuristic; Burgers' nonlinear remainder also depends on the spatial derivative of the error. We will present initial reconstruction as the primary result and the adjustment as optional, retaining the negative cases. The new Burgers experiment does not establish the cause of the original Allen-Cahn failure.

W2 / Q2. Does correction remain useful after strong training, and is it worth its cost?

The submitted "continued training" comparison allocated only the additional time taken by correction. It did not mean training to convergence. Your objection is that this cannot establish whether correction remains useful after strong optimization, or whether training should be stopped early. We tested those questions separately and also compared with solving the PDE from scratch.

Correction after much longer, curvature-based optimization

Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) uses past gradients to approximate curvature. NysNewton-CG (NNCG) is a Nyström-preconditioned Newton-conjugate-gradient method: it uses curvature information to compute Newton directions and a low-rank preconditioner to make those computations more effective [4]. We also evaluated SOAP, meaning Shampoo with Adam in the preconditioner's eigenbasis: it rotates gradients into curvature-informed coordinates and applies Adam there [5,6]. These comparisons test whether correction remains useful after methods designed to address poorly conditioned PINN training. The new problems are a variable-coefficient linear elliptic equation, a cubic semilinear elliptic equation with independently specified forcing, and viscous Burgers.

For each PDE and each of five runs, the two long continuations start from identical weights after 20,000 Adam, 2,000 L-BFGS, and 300 NNCG calls. Training and validation points remain fixed. Networks have four hidden layers of width 64, tanh activations, and double-precision arithmetic. NNCG resumes its saved optimizer state. L-BFGS starts fresh curvature history and scales the objective by the fixed reciprocal of the starting PDE-and-condition loss; later extensions preserve that history and scale.

AuthorsOpenReview ↗

Response to Reviewer djf8 (part 3/4)

The L-BFGS and NNCG continuations each allow 100,000 additional calls, with time limits of 7,200 seconds for NNCG and 1,800 seconds for L-BFGS. Ten nonlinear NNCG runs stop when the largest absolute training-gradient component is at most 1e-8; the others stop at call or time limits.

At each terminal checkpoint, we compare the unchanged network with its initial reconstruction. Correction uses 500 interior and 200 condition observations sampled separately from training. Kernel fitting searches 32 coarse parameter settings and refines five starting points using defects alone. Predictions are fixed before evaluation on a common 101×101 grid against analytic or independently refined numerical references. Relative L2 error measures the size of the field error relative to the reference solution; smaller is better.

Training methodPDERelative L2: network → correctedNetwork / corrected error [95% interval]Total time including training: network → corrected (s)
Normalized L-BFGSLinear elliptic2.52e-6 → 3.37e-874.82× [40.58, 137.95]2,031.2 → 2,034.6
Normalized L-BFGSSemilinear elliptic4.09e-6 → 1.37e-729.79× [19.65, 45.17]1,998.9 → 2,001.9
Normalized L-BFGSBurgers6.08e-6 → 4.85e-61.254× [1.037, 1.517]1,501.1 → 1,503.7
NNCGLinear elliptic3.25e-6 → 1.97e-716.43× [2.084, 129.63]7,214.9 → 7,218.3
NNCGSemilinear elliptic4.94e-6 → 3.58e-713.80× [3.459, 55.06]3,851.0 → 3,854.7
NNCGBurgers1.28e-5 → 1.09e-51.175× [1.035, 1.334]2,032.6 → 2,035.2

New numerical comparisons use five independent seeds per fixed problem and setting. Reported means are geometric means. The 95% confidence intervals use paired t statistics on log error ratios, without adjustment for multiple comparisons. Initial reconstruction improves 15/15 L-BFGS and 14/15 NNCG endpoints; the remaining case worsens by 0.42%. The elliptic gains persist when uncorrected errors are already near 1e-6, while Burgers gains are smaller.

The table charges inherited and continued training, validation/checkpoint work, observation construction, every kernel search and factorization, and final mean prediction. Initial-reconstruction time is obtained by removing measured second-step work from the joint workflow. Auxiliary variance, common warmup, reference assessment, and research archival work are separate. The geometric means of per-run added correction costs are 2.50 to 3.71 seconds. These tests quantify correction at the terminal fields; they establish neither global optimizer convergence nor superiority to a further few seconds of training from every endpoint.

Can stopping early save total training time?

Correction also remains useful at the best saved SOAP checkpoint selected by physical validation: the linear and semilinear error-reduction factors are 530.62 [247.77, 1136.40] and 112.93 [73.19, 174.25], respectively. Each improves in all five seeds. The Burgers factor is 1.133 [0.774, 1.658], which does not establish an improvement. Physical validation uses PDE and condition defects, without reference-solution labels.

We then tested your early-stopping strategy at prescribed SOAP training steps 1,000 and 5,000. Each early network is corrected and compared with the complete SOAP run returning that best saved network. Both workflows pay for all 12,000 tuning calls across six candidate settings.

On the linear problem, the two early-stopping rules reduce geometric-mean total cost by 73% to 79%, with lower cost and error in all five runs. On Burgers, they cost less but give 5.19 to 5.95 times higher geometric-mean error. Longer raw Burgers training also beats correction at the original starting checkpoint in all five runs, at greater cost. The linear result demonstrates a saving; the Burgers result shows when further training remains worthwhile for accuracy.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.