acceptodds
Under review as a conference paper at ICLR 2027

Pointwise Detection of Well-Trained Regions in Physics-Informed Neural Networks

Abstract

Physics-informed neural networks (PINNs) rely on residual losses to enforce governing equations, but a small residual at an interior point does not guarantee that the predicted value matches the solution selected by the boundary or initial data. We study this pointwise detection problem: how to identify, during training, which domain points are already well trained using only quantities available to the PINN. We introduce a boundary-seeded detector that grows a well-trained set through local parent–child admissions. A candidate is admitted only when its own residual is small, its parent's residual remains small, and the parent's prediction has not drifted from the value stored at admission. The residual threshold is predicted from PDE-level features by a small structural meta-model, avoiding online estimation of stability constants or parameter-space sampling. The same primitive supports two uses: a conservative coverage-based stopping signal and an adaptive-domain training scheme with stored-value self-distillation. Across multiple PDE families, the detector reduces the false-stop rate from at least 40.9% for aggregate residual and residual–boundary criteria to 6.8%, while the detection-driven frontier improves training on a stiff convection benchmark and Helmholtz.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
physics, PINNs, PDE

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mH55OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper aims to identify which regions of the domain are already well trained during PINN training, since a small PDE residual does not guarantee that the prediction is close to the true solution. The authors propose a "detective" that starts from the boundary and initial points and expands a trusted set to neighboring points, admitting a point only when both its residual and its parent's residual are small and the parent's prediction has stayed stable. The residual threshold is predicted from PDE features by a small meta-model. The trusted set is used either as a stopping criterion or as a guide for training, and experiments show fewer false stops than loss-based criteria and improved accuracy on convection and Helmholtz.

Critical strengths

The paper tackles a well-known but often overlooked issue in PINNs: a small residual alone does not mean the solution is correct. Instead of relying only on the residual, the proposed method connects each interior point back to the boundary and initial conditions, where the true values are known. This idea of checking whether local residual satisfaction is actually anchored to reliable information is simple, intuitive, and well motivated.

Critical weaknesses
  1. The method uses boundary and initial points with known true values as the starting anchors, which effectively limits it to problems with Dirichlet conditions. For Neumann or Robin boundary conditions, the true values on the boundary are not available, so it is unclear how the trusted set would be initialized. A discussion of this limitation would be helpful.

  2. Since the main contribution of the paper is the empirical performance of the proposed method, a more comprehensive evaluation is needed. The training application is evaluated only on a single convection setting and on Helmholtz, where the improvement becomes marginal at the higher wavenumber.

  3. Compared to a vanilla PINN, the proposed method requires additional memory and computation. A fair comparison under matched computational budgets is needed.

  4. The manuscript does not yet seem ready for publication. For example, Dirichlet conditions are referred to as "value-type supervised conditions" (line 125).

Questions for authors
  1. What exactly counts as a "false stop"? Could the authors provide a formal definition?
  2. Could the authors provide a computational cost analysis? Specifically, how much additional wall clock time and memory require compared to a vanilla PINN?
2 comments

Thank you for your comments.

W1. Boundary-condition scope

The detector needs trusted point-value anchors; it does not require every boundary face to be Dirichlet. A nonempty Dirichlet or initial-value subset can initialize the same admission logic, while Neumann/Robin conditions on the other faces remain in the ordinary PINN loss.

As a minimal mixed-boundary sanity check, we tested a manufactured two-dimensional Poisson problem with u∗(x,y)=x2u^*(x,y)=x^2 and −Δu=−2-\Delta u=-2: u(0,y)=0u(0,y)=0 supplies anchors, ∂nu(1,y)=2\partial_n u(1,y)=2, and ∂nu(x,0)=∂nu(x,1)=0\partial_n u(x,0)=\partial_n u(x,1)=0. Pointwise error is normalized by the exact-solution grid RMS.

MetricMean ±\pm sample s.d.
detector coverage99.93%±0.12%99.93\%\pm0.12\%
full-grid relative-L2L_20.00219±0.001340.00219\pm0.00134
admitted-point normalized RMS0.00219±0.001340.00219\pm0.00134
admitted-point Q99Q_{99}0.00379±0.001480.00379\pm0.00148

Thus a trusted-value subset can initialize the current logic in this mixed problem. Pure Neumann/Robin-only data are outside the current method unless an external trusted point value is supplied; that case remains untested. An integral gauge or derivative/Robin functional would require a new functional-anchor analysis. We will replace “value-type supervised conditions” with “Dirichlet boundary and/or initial-value data supplying trusted point values.”

W2. Scope of active-training evidence

The active-training experiments were designed to cover two complementary propagation modes. Convection tests a time-dependent problem in which the trusted frontier advances along the temporal direction. Helmholtz tests a static problem with no temporal ordering, where the same detector must instead grow spatially from the supervised boundary. We will make this experimental-design rationale explicit and, in future work, extend the evaluation to higher-dimensional PDEs and other more challenging equations.

W3. Measured passive overhead and near-equal-wall active control

We isolate passive-detector overhead and report active Detective-only cost separately. On Poisson-2D, Vanilla and passive detection use identical networks, seeds, batches, optimizer, and 2525k updates; JIT is warmed before timing, and the passive observer cannot write to training. The recorded final loss and parameter checksums/norms agree exactly in every pair, consistent with an unchanged trajectory. This paired loop microbenchmark uses one RTX 4090, a 5-by-192 network, 2048/768 interior/boundary batches, 25k steps.

Passive measurementResult
seed 42, baseline →\to passive26.91→28.8326.91\to28.83 s (+7.14%+7.14\%)
seed 43, baseline →\to passive26.60→28.0926.60\to28.09 s (+5.61%+5.61\%)
seed 44, baseline →\to passive26.48→29.1626.48\to29.16 s (+10.14%+10.14\%)
median device / host observer time1.178/0.5841.178/0.584 s
median peak host RSS, passive minus baseline+100.1+100.1 MiB
JAX peak bytes-in-use, passive minus baseline+0.0027+0.0027 MiB

For active use, we ran a fresh three-seed near-equal-wall β=30\beta=30 control. Detective uses 272k/282k/280k updates versus Vanilla's 300k. Both methods share the base PINN loss, architecture, initialization, optimizer, and sample pool, differing only in active-region selection. With AdamW only, Detective obtains 0.01808±0.001740.01808\pm0.00174 versus Vanilla 0.02402±0.000780.02402\pm0.00078, wins 3/33/3 pairs; their reductions average to 24.5%.

W4. Presentation readiness

The revision will use direct scope and guarantee language. We will (i) use “Dirichlet boundary and/or initial-value anchors,” (ii) define all four stopping outcomes, (iii) distinguish the conditional stored-value chain bound from empirical current-set validation, (iv) add the chain corollary and complete first-fire pointwise audit, and (v) add limitations covering derivative-only anchors, dense-grid scaling, and active-training breadth.

Q1. Formal definition of False Stop

Let TT be the first checkpoint at which a criterion fires, with T=∞T=\infty if it does not fire within the shared budget, and let ε=0.005\varepsilon=0.005 be the target relative-L2L_2 error. A Successful Stop has T<∞T<\infty and rel⁡L2(T)≤ε\operatorname{rel}L_2(T)\leq\varepsilon. If the criterion does not fire within the shared budget, it is a Missed Detection when the endpoint error is at most ε\varepsilon, and Correct Non-convergence otherwise. Under these definitions, the submitted full-detector outcomes are 17/3/8/1617/3/8/16 for Successful Stop, False Stop, Missed Detection, and Correct Non-convergence, respectively. We will add these definitions explicitly to the revision.

Q2. Computational cost

See W3. In the paired Poisson-2D measurement, passive detection adds a median 7.14%7.14\% wall-clock overhead, 100.1 MiB of peak host memory, and only 0.0027 MiB of additional JAX peak bytes in use because the detector state is host-resident. One detection cycle scales linearly with the candidates visited over at most SS shell rounds. Passive use otherwise leaves the PINN loss, parameters, and optimizer unchanged.

Thank you for your careful response. However, the proposed method cannot be applied to Neumann and Robin boundary conditions without true data points, so its scope of applicability remains limited. In addition, false stops serve as a key metric in this paper, but they have still not been incorporated into the revised manuscript. Overall, I believe the paper is not yet ready for publication, and I will maintain my score.

Reviewer_1TpfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the problem of identifying well-trained regions during the optimization of physics-informed neural networks (PINNs). To address this issue, the paper introduces a boundary-seeded pointwise detection method.

Critical strengths
  1. An intuitive and relatively lightweight detection mechanism
  2. paper is well-written and easy to follow.
Critical weaknesses
  1. The theoretical analysis relies on a strong local stability assumption that is not verified by the algorithm. A more explicit analysis of error propagation over multiple admission steps would strengthen the theoretical contribution.
  2. The adaptive-training results are promising but limited in scope. For the harder Helmholtz setting with k = 10, the improvement over vanilla PINN is relatively small.
Questions for authors
  1. Can the authors report stopping precision, recall, and false-stop probability conditioned on an actual stopping event, rather than only reporting false stops over all configurations?

  2. How does the error bound in Proposition 1 accumulate along a parent-child chain? Is there any bound on the maximum depth at which the detector remains reliable?

1 comment

Thank you for the constructive questions. We now make the multi-step object precise, directly audit pointwise errors by chain depth, and report the requested stopping metrics.

W1. Local stability and multi-step propagation

Recursively applying Proposition 1 gives an explicit conditional bound that accumulates with path depth; we will add this multi-step derivation and its assumptions. If BB is a uniform per-edge bound and η\eta is the target error, the corresponding conditional maximum depth is ⌊η/B⌋\lfloor\eta/B\rfloor.

This accumulation was also considered in the practical design. In active use, every non-anchor stored value is refreshed by EMA at each detection cycle, so an early local error is not permanently frozen and repeatedly imposed on subsequent training. In passive use, the detector does not modify training and checks only the immediate parent of the current shell; earlier ancestors remain free to update and self-correct.

As a direct edge-level check, we additionally evaluated 2,048 actual Burgers parent--child edges. The 9999th percentile and maximum of ∣ec−ep∣|e_c-e_p| are 1.88×10−41.88\times10^{-4} and 3.74×10−43.74\times10^{-4}, and 99.76% of the sampled edgewise residual maxima remain below the detector threshold.

We also directly audited every first-fire event in a separate canonical pointwise evaluation. Across all seven accurate stops, the normalized RMS error within the admitted set is 0.0003890.000389--0.0026300.002630, and 97.03%--100% of admitted points have normalized pointwise error below 0.0050.005. More importantly, the pooled RMS remains between 0.0016490.001649 and 0.0017550.001755 across path-depth bins from 1--15 to 61--75, showing no deterioration with propagation depth. The same audit includes all three false-stop events, whose pointwise accuracy coverage is 0%--34.34%; thus, the reported results are not selected only from successful cases. The pointwise cutoff is used only for this offline analysis.

W2. Scope of adaptive training

The detector is the contribution; adaptive training is one downstream application. Convection and Helmholtz test temporal and purely spatial propagation, respectively. At Helmholtz k=10k=10, the improvement is smaller than in the lower-frequency setting but remains positive (0.0837→0.07700.0837\to0.0770). To test whether active use remains effective when its additional computation is accounted for, we ran a fresh three-seed near-equal-wall control on Convection β=30\beta=30. Detective step counts were fixed beforehand from timing ratios only; paired end-to-end times differ by at most 3.91%3.91\%. This is frontier-only Detective, with no buffer: both methods share the architecture, initialization, base PINN loss, optimizer, and sample pool, differing only in active-region selection. With AdamW only (no L-BFGS), the three-seed results are:

methodmean wall-clock (s)final relative-L2↓L_2\downarrow
Vanilla PINN111.67±3.01111.67\pm3.010.02402±0.000780.02402\pm0.00078
Detective frontier110.99±0.73110.99\pm0.730.01808±0.00174\mathbf{0.01808\pm0.00174}

Detective wins all three paired runs and reduces error by 24.5%24.5\% on average.

Thus, even under matched wall-clock cost, actively using the detector to select the training region consistently improves performance in this controlled comparison.

Q1. Precision, recall, and conditional false-stop probability

Yes. The submitted full-detector outcomes are S/F/M/C=17/3/8/16S/F/M/C=17/3/8/16. They give:

stopping precision ↑\uparrowstopping recall ↑\uparrowP(F∣stop)↓P(F\mid\mathrm{stop})\downarrow
17/(17+3)=85.0%17/(17+3)=\mathbf{85.0\%}17/(17+8)=68.0%17/(17+8)=\mathbf{68.0\%}3/(17+3)=15.0%3/(17+3)=\mathbf{15.0\%}

Here precision is S/(S+F)S/(S+F), recall is S/(S+M)S/(S+M), and the conditional false-stop probability is F/(S+F)F/(S+F). The submitted 3/44=6.8%3/44=6.8\% is the incidence of False Stops over the complete suite, whereas 3/20=15.0%3/20=15.0\% is the probability that an actual stopping decision is false. We will add these definitions and metrics to the revision.

Q2. Maximum reliable depth

See W1. The maximum reliable depth is conditional on the PDE-specific local stability constants and the target tolerance. If BB is a uniform per-edge error bound and η\eta is the target pointwise error, the resulting conditional depth is ⌊η/B⌋\lfloor\eta/B\rfloor. This is a conservative worst-case bound; in practical use, continued PINN optimization and, in active use, EMA updates allow local errors to be corrected, so propagation is typically much more stable than this theoretical analysis suggests. Consistently, the seven accurate stopping cases show no error-growth trend through the deepest tested depth of 75.

Reviewer_rzDrOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how to identify, during PINN training, regions that are already well trained, using only quantities available to the PINN. Starting from supervised boundary or initial points, the method grows a well-trained set through a parent-child admission rule. A candidate must pass three checks on its own residual, on its parent’s residual, and on the drift of the parent’s prediction from its stored value. The residual threshold is predicted from PDE-level features by a small structural meta-model, calibrated offline on reference PINN runs.

Critical strengths
  • The studied problem is practically meaningful, and the detection mechanism is clear and implementable.
  • The pointwise ablation in the stopping experiments provides useful evidence for the role of each check.
  • The paper is clearly written.
Critical weaknesses

The central claim of this paper is the pointwise identification of well-trained regions, but the supporting theory does not hold for the proposed algorithm itself, and the experiments do not directly verify this claim either.

  • The theory only motivates the method, and no theoretical analysis of the proposed algorithm itself is conducted. Proposition 1 is a one-step bound carried entirely by Assumptions 1 and 2, whereas the algorithm admits points repeatedly along parent-child chains, where the error term ExpE_{x_p} in Eq. (10) accumulates without being tracked. The proposed algorithm therefore remains heuristic and does not have any theoretically guaranteed properties.
  • Assumption 2, which plays the key role, is not shown to hold for the parent-child edges actually constructed by the algorithm, namely shell-adjacent grid pairs at distance δ\delta in the max norm. The authors acknowledge that it does not hold for all PDEs or in every geometric direction, and the constants CstabC_{stab} and CregC_{reg} are never estimated. If they are large, passing the three checks yields no meaningful error bound.
  • The experiments classify entire runs by the final global relative L2L^2 error, which does not show how many points in the admitted set are actually accurate. It is necessary to report the pointwise error distribution within the admitted set, and the full four-class breakdown with stopping times for all baselines, since Table 2 reports only False Stop.
  • The detection mechanism discretizes the whole domain and covers it shell by shell, whose cost grows exponentially with the dimension. All eleven PDE families are one or two dimensional, and applicability to higher-dimensional problems is not discussed.
Questions for authors

Please see Weaknesses.

2 comments

Thank you for identifying the need to make the multi-step behavior explicit.

W1. Repeated admission and accumulated error

Recursively applying Proposition 1 yields an explicit conditional bound that accumulates with path depth; we will add this multi-step derivation and its assumptions to the revision.

This accumulation was also considered in the practical design. In active use, we refresh every non-anchor stored value by EMA at each detection cycle (Eq. 12), so an early local error is not permanently frozen and repeatedly imposed on later training through the buffer loss. In passive use, the detector does not modify training and checks only the immediate parent of the current shell. Once an older point is no longer an immediate frontier parent, its network prediction remains free to update and self-correct; the detector does not constrain the model to an accumulated chain of historical values.

The experiments support this behavior across 44 configurations from 11 PDE families. Moreover, in our new depth-stratified pointwise audit, the pooled admitted-set RMS error over the seven accurate first-stop runs remains between 0.0016490.001649 and 0.0017550.001755 across depth bins from 1 to 75, rather than growing with depth.

W2. Local stability on the constructed edges

Assumption 2 is a local condition over the short parent--child edges actually used by the detector, and our measurements show that its effective constants are small. We evaluated 2,048 actual Burgers parent--child edges and sampled nine points along each edge. The 9999th percentile and maximum of the error change ∣ec−ep∣|e_c-e_p| are only 1.88×10−41.88\times10^{-4} and 3.74×10−43.74\times10^{-4}, while 99.76% of the sampled segment-residual maxima remain below the detector threshold.

More directly, the empirical joint stability coefficient C^edge=∣ec−ep∣/(max⁡segment∣r∣+δ)\widehat C_{\rm edge}=|e_c-e_p|/(\max_{\rm segment}|r|+\delta) has Q99=0.0152Q_{99}=0.0152 and maximum 0.02300.0230. Thus, on the actual constructed edges, the local error variation is controlled with a small coefficient rather than the bound becoming vacuous through large constants.

W3. Pointwise validation and complete stopping outcomes

We audited every full-detector first fire, not selected successes. As a separate pointwise audit, a canonical-batching rerun produced ten first-fire events (seven accurate and three false). On the 101-point-per-coordinate grids, we evaluate the current network at every admitted interior point; ground truth is used only offline.

event classrunsnormalized RMS in Wint\mathcal W_{\rm int}normalized Q99Q_{99}fraction with normalized pointwise error ≤0.005\leq0.005max depth
successful stop70.0003890.000389--0.0026300.0026300.0009230.000923--0.0060170.00601797.03%--100%55--75
false stop30.0080670.008067--0.0192780.0192780.0108800.010880--0.0400420.0400420%--34.34%50--60

Across successful-event depth bins 1--15 through 61--75, pooled RMS stays 0.0016490.001649--0.0017550.001755 and the threshold fraction is 99.08%--100%. Within each run, the deepest occupied bin is no worse than its 1--15 bin.

We recovered the complete outcomes and stopping times using the submitted evaluation records:

criterionS/F/M/CprecisionP(F∣stop)↓P(F\mid\mathrm{stop})\downarrowop. recallmedian stop S/F
residual <10−2<10^{-2}9/30/1/423.1%76.9%90.0%3k/3k
residual <10−2<10^{-2} + boundary <10−3<10^{-3}9/27/1/725.0%75.0%90.0%3k/3k
residual <10−3<10^{-3}13/21/1/938.2%61.8%92.9%8k/4k
residual <10−3<10^{-3} + boundary <10−4<10^{-4}15/18/1/1045.5%54.5%93.8%8k/3.5k
child residual only18/19/0/748.6%51.4%100%17k/22k
full detector17/3/8/1685.0%15.0%68.0%29k/20k

S/F/M/C denote Successful Stop, False Stop, Missed Detection, and Correct Non-convergence. Precision is S/(S+F)S/(S+F), conditional False Stop probability is F/(S+F)F/(S+F), and operational recall is S/(S+M)S/(S+M). The complete submitted results show the intended conservative trade-off.

W4. Extension to high-dimensional domains

The exponential dependence is associated with the submitted dense-grid implementation, rather than with the parent--child detection rule itself. A natural next extension is to construct the detection frontier directly on non-grid collocation samples. This would allow the detector to reuse the sparse samples already employed by high-dimensional PINNs instead of materializing a full tensor grid.

As a preliminary feasibility example, we implemented the same bookkeeping on 50,000 Sobol points connected by a bounded-degree nearest-neighbor graph. The explicit graph arrays require only 20.10 MiB in 5D and 23.34 MiB in 10D, while construction, traversal, and a 10,000-point probe query take 0.499 s and 2.245 s, respectively. Extending the detector to high-dimensional PDEs is an important direction for future work.

Thank you for the rebuttal. The two tables in the response to W3 are especially clear. However, the remaining issues from W1 and W2 (the theory only gives a conditional bound that accumulates with path depth for the algorithm itself, and the constants in Assumption 2 are estimated only on the Burgers equation), together with the results in the new W3 table (3 of the 10 first-fire events are false stops, in which only 0%--34.34% of the admitted set has normalized pointwise error ≤0.005\leq 0.005), still prevent me from accepting the claim of "pointwise detection of well-trained regions''. Taking all of this into account, I raise my rating to 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.