acceptodds
Under review as a conference paper at ICLR 2027

Unbiased Physics Losses Under Model Uncertainty

Abstract

Physics-informed machine learning usually penalizes violations of a chosen governing equation. However, when that equation is misspecified, minimizing the nominal residual can instead increase the error in the predicted solution. To make physics-informed machine learning effective under such misspecification, we propose compatibility training, which uses the smallest residual over a plausible family of equations. At each training step, compatibility training selects the equation in this family that best fits the current prediction and penalizes only deviations from that equation. Our theory establishes a mismatch-dependent bias floor for nominal training and characterizes the physics constraints that remain after the uncertain parameters are adjusted. We derive an exact local curvature identity that shows when the retained physics constraints and observations together identify the true solution. This curvature identity connects solution recovery to the structure and extent of the uncertainty family. Across multiple synthetic PINN benchmarks, PDEBench trajectory and operator tasks, and Darcy operator-learning experiments, compatibility training improves prediction under model mismatch, including in settings with sparse or noisy observations and field-valued uncertainty. In field-valued Darcy flow, compatibility training also reduces the true residual by more than an order of magnitude.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Scientific machine learning, Physics-informed neural networks, Neural operators, Model uncertainty, Model misspecification, Uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_uvHkOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

My read is that this paper tackles training with the wrong physics. The method searches a declared equation family at each step. It then penalizes the residual that parameter adjustment can't explain. The theory shows which useful physics constraints survive that adjustment. The controlled gains look strong, though the family choice does a lot of the work.

Critical strengths

The geometry is the best part for me. The Hessian identity makes the lost and retained constraints explicit. That's a useful way to reason about sparse observations.

Those sparse-data gains are hard to dismiss. With 64 observations and 10% noise, RelL2 falls from 0.3156 to 0.0153 against data-only. I give the paper real credit for that.

Critical weaknesses

I'm struggling to see much method novelty. Classical variable projection does the core work. The added value rests on the analysis and convincing optimization gains.

Choosing the equation family remains a major burden. Missing modes leave mismatch behind, while extra modes can erase useful constraints. I'd worry about how users make that choice from limited measurements.

I still don't know how reliably the actual procedure trains. The key recovery arguments need specific local conditions, and Darcy's inner errors weren't measured. That's a substantial gap between the theory and the experiments.

Questions for authors

Does the advantage survive a stronger inverse solver? I'd want projected updates, separate parameter learning rates, and matched compute. Please include a matched field-inverse Darcy baseline too. What happens when the correction basis misses the actual discrepancy? Please choose the basis and bounds from separate calibration data. That's the practical test I'd find convincing. How often does radial clipping activate in Darcy? Please report inner optimality errors and compare against an accurate constrained solve. I'd also want a ridge sweep. Does extra correction freedom hurt prediction during training? A retrained sweep over correction dimension and collocation count would help. I'd want fresh-point residuals alongside prediction errors.

6 comments

Our contribution is a criterion for correcting equation error while preserving the physics information needed for prediction. Classical variable projection supplies the coefficient fit; our analysis identifies the conditions for solution recovery and quantifies the constraints lost as correction freedom grows. We added controlled comparisons to separate the family's benefit from coefficient optimization, and ablations of solver accuracy, ridge, and correction dimension. A combined Darcy control tests independently learned corrections, accurate solving, and zero ridge together on fresh functions. TPSW W1 details the analysis, and StSM establishes the local recovery condition for nonlinear Fisher–KPP.

1. Stronger inverse solvers and matched compute

Controlled Fisher–KPP comparison at equal training time. To separate the family's benefit from the fitting procedure, we compare profiling with a strengthened inverse solver under a fixed time budget. The inverse uses projected coefficient updates and a separate coefficient learning rate. Every method receives 90 training seconds on an A100 and a cosine schedule spanning the budget. The setting has 64 interior observations plus 256 initial values, all with 10% noise, mismatch/radius 16, λ=0.2\lambda=0.2, and fresh collocation batches. No validation labels are used: the inverse coefficient rate is fixed at 0.02 and the discrepancy-network rate at 0.002. Prediction uses 24,512 disjoint locations; true residuals use 8192 fresh continuous points.

Method, 90-second budgetDisjoint-test RelL2Fresh true-equation residual
Data only0.2648 ± 0.01247.6004 ± 0.7352
Nominal physics0.7630 ± 0.05923.4335 ± 0.7288
Projected inverse0.01306 ± 0.00171(6.82±2.04)×10−5(6.82 \pm 2.04)\times10^{-5}
Compatibility0.01326 ± 0.00178(7.43±1.85)×10−5(7.43 \pm 1.85)\times10^{-5}
Coordinate discrepancy DNN0.08419 ± 0.009540.04469 ± 0.01494

Compatibility gives 20.0-fold lower prediction error than data-only and 6.3-fold lower error than the discrepancy DNN, improving every seed in both comparisons. The strengthened inverse also achieves low errors with the same correction family, and all three tested inverse rates, 0.002, 0.02, and 0.2, retain the family benefit. The controlled comparison supports useful corrected physics under both coefficient-fitting procedures, consistent with Proposition 2's shared globally optimal predictor sets. The paired interval for their prediction difference spans zero. A validation-selected discrepancy DNN given three full 90-second runs reaches 0.06881 ± 0.01444, also above compatibility.

Matched field-inverse Darcy control. To compare coefficient-fitting procedures for spatial corrections, joint training keeps eight persistent coefficients per training function. Basis, norm ball, ridge, FNO, initialization, observations, and residual evaluation match compatibility, which uses the accurate constrained solver. The inverse coefficient rate is selected from 0.001, 0.01, and 0.1 by validation. We compare both equal-epoch and equal-time budgets to measure the effect of training cost.

BudgetMethodRelL2True-equation residual
200 epochsData only0.00984 ± 0.000850.1521 ± 0.0520
200 epochsProjected inverse0.00884 ± 0.000720.00996 ± 0.00116
200 epochsCompatibility0.00875 ± 0.000860.00957 ± 0.00110
180 secondsData only0.00688 ± 0.000560.06341 ± 0.02152
180 secondsProjected inverse0.00742 ± 0.000560.00703 ± 0.00077
180 secondsCompatibility0.00751 ± 0.000760.00699 ± 0.00090

At equal epochs, compatibility improves prediction by 11.1% and lowers true residual 15.9-fold relative to data-only; the paired prediction difference is −0.00110-0.00110 with 95% interval [−0.00151,−0.00068][-0.00151,-0.00068]. At equal time, it lowers true residual 9.1-fold, while data-only has lower prediction error. Both corrected solvers substantially improve equation agreement under both budgets; their prediction differences are unresolved by paired intervals. Timers are synchronized, schedules span the full budget, and per-run training time excludes evaluation and hyperparameter search.

The family improves prediction with either solver in the Fisher–KPP and equal-epoch Darcy controls. Profiling evaluates its loss without a persistent coefficient update or a coefficient learning-rate choice. The frequency initialization ablation in TPSW W2 isolates its additional optimization benefit: a profiled starting frequency enables joint training to reach the same good solution. These studies distinguish useful corrected physics from access to a good coefficient optimum.

2. Independently calibrated basis and bounds

A combined control tests learned corrections with accurate solving and no ridge. We determine which spatial patterns can be corrected and how large those corrections may be using separate calibration data:

  1. Learn the correction patterns. Use 256 separate calibration examples of spatial coefficient corrections. Estimate their average and use principal component analysis (PCA) to find the main patterns around it. Retaining patterns that explain 99% of the variation selects eight directions, each normalized to unit RMS.
  2. Set the allowed correction size. Express another 128 disjoint calibration fields in these directions. Set the radius to the 95th percentile of their coefficient norms, so the bound is determined by calibration examples.
  3. Freeze the family before training. Keep the learned mean, directions, and radius fixed for prediction training and evaluation. The generating basis and target test corrections are never supplied to this procedure.

All methods use the same 1024 training functions and 200 epochs. Compatibility uses an accurate constrained solve and zero ridge. The matched inverse uses the same calibrated family and zero ridge, with its rate selected on 128 validation functions. Frozen models are evaluated on 1024 new physical functions across four seed streams, each at two mismatch amplitudes, giving 2048 test cases. This combined experiment tests the requested controls together; the separate solver and ridge ablations below isolate their individual effects.

Method, independently calibrated familyFresh-test RelL2Fresh true-equation residual
Data only0.01040 ± 0.000760.1665 ± 0.0252
Nominal physics0.01446 ± 0.000830.04301 ± 0.00312
Projected inverse, zero ridge0.00940 ± 0.001110.01157 ± 0.00272
Compatibility, accurate solve, zero ridge0.00922 ± 0.001020.01124 ± 0.00230

Compatibility improves prediction by 11.4% over data-only and 36.2% over nominal physics, and lowers true residual 14.8-fold relative to data-only. Both corrected methods improve over both baselines in every seed average. For compatibility versus data-only, the paired prediction difference is −0.00118-0.00118 with 95% interval [−0.00166,−0.00070][-0.00166,-0.00070], comparing methods within each shared seed. The difference between the two corrected solvers remains unresolved. The combined control establishes prediction gains from a separately learned family with accurate coefficient solves and zero ridge. The matched inverse confirms that the same family also benefits joint training.

The missing-direction control identifies what the family must capture. Truncating the learned family from eight to four modes raises its residual at the true solution from 2.76×10−52.76\times10^{-5} to 0.0279. Increasing the allowed size of the remaining patterns cannot create a missing pattern. The solution-pair calibration experiment in TPSW W4 tests the remedy: learning directions improves prediction when discrepancy lies entirely outside the initial eight-mode basis, even against data-only supplied with the same extra labels.

3. Radial clipping and accurate constrained solves

The prediction gain persists when the approximate coefficient solve is replaced by an accurate one. This solver ablation tests whether clipping error is responsible for the improvement. Radial clipping fits coefficients first and shrinks them uniformly if they exceed the allowed size. The accurate solver instead finds the best fit within that limit. The Darcy inner problem is

min⁡∥α∥≤ρ1n∥Xα−y∥2+τ∥α∥2.\min_{\lVert \alpha\rVert\le\rho}\frac{1}{n}\lVert X\alpha-y\rVert^2+\tau\lVert \alpha\rVert^2.

The columns of XX describe residual corrections, yy is the residual to explain, ρ\rho limits correction size, and τ\tau penalizes large coefficients. With G=XTX/nG=X^TX/n and b=XTy/nb=X^Ty/n, its optimality conditions are

\lVert \alpha\rVert\le\rho,\quad\zeta\ge0,\quad \zeta(\lVert \alpha\rVert-\rho)=0.$$ An unconstrained optimum inside the ball has $\zeta=0$. Otherwise, a one-dimensional search for $\zeta$ places the optimum on the boundary. This accounts for how each correction direction affects the residual; radial clipping rescales all coordinates equally. We compare both with an independent float64 constrained reference. Clipping frequencies below count every training solve; maximum objective gaps come from sampled audits. “Late” means the final 20% of training. These are pooled numerical diagnostics, not seed-level performance estimates. | Radius and protocol | Fraction of solves clipped | Largest audited clipping objective gap | Largest late gap | | --- | ---: | ---: | ---: | | $\rho=1$, 1024 functions, 200 epochs | 0.0974% of 1,638,400 | 70.8 | $1.83\times10^{-12}$ | | $\rho=0.25$, 512 functions, 80 epochs | 96.71% | $8.21\times10^3$ | 0.00645 |

For radius 1, 1152 audited problems give accurate-solver mean KKT residual 7.47×10−77.47\times10^{-7} and maximum objective discrepancy 5.47×10−85.47\times10^{-8}. The KKT residual measures violation of constrained optimality. Radial clipping gives mean KKT residual 1.24 overall and 3.99×10−73.99\times10^{-7} late in training.

Retraining with the accurate solver gives RelL2 0.00877 ± 0.00094, versus 0.00868 ± 0.00078 for clipping and 0.00975 ± 0.00087 for data-only. It retains a 10.1% prediction gain and 14.4-fold true-residual reduction over data-only. At the frequently active radius 0.25, accurate and radial errors are 0.02129 ± 0.00124 and 0.02343 ± 0.00354. The gain therefore persists after replacing the approximate inner solve: useful corrected physics remains effective when the coefficient fit is computed accurately.

The calibrated zero-ridge experiment also exercises active bounds: 9.39% of 1,638,400 solves lie on the boundary. Its 1152 audited solves have mean KKT residual 1.23×10−61.23\times10^{-6} and maximum objective discrepancy, normalized by one plus reference residual, of 3.92×10−83.92\times10^{-8}.

Appendix D.11 connects these diagnostics to the ideal loss: for τ>0\tau\gt 0, a feasible KKT residual κ\kappa bounds inner-value error by κ2/(4τ)\kappa^2/(4\tau), and ridge changes the unregularized profile by at most τρ2\tau\rho^2. The two terms quantify solver error and regularization error separately. Zero-ridge solves are checked directly against the reference. We now report these diagnostics and use the accurate solver in the new constrained-solve controls.

4. Ridge ablation, including zero

A one-factor ablation tests whether ridge is needed for the gain. We retrain at each ridge value, including zero, while holding initialization, architecture, 512 training functions, 80 epochs, mismatch amplitude 1, eight modes, radius 1, and the accurate solver fixed. This isolates the effect of the ridge penalty from the correction family and solver accuracy. Modes have unit maximum absolute value, fixing the scale of τ\tau. All nine tested values are shown.

Method / ridge τ\tauRelL2True-equation residual
Data only0.02262 ± 0.001020.6533 ± 0.0599
Nominal physics0.02690 ± 0.003060.09804 ± 0.01575
Compatibility, 00.02079 ± 0.001320.04765 ± 0.00623
Compatibility, 10−610^{-6}0.02079 ± 0.001320.04762 ± 0.00626
Compatibility, 10−410^{-4}0.02079 ± 0.001320.04761 ± 0.00627
Compatibility, 10−210^{-2}0.02080 ± 0.001320.04760 ± 0.00629
Compatibility, 10−110^{-1}0.02085 ± 0.001260.04763 ± 0.00605
Compatibility, 10.02185 ± 0.001160.05595 ± 0.00576
Compatibility, 100.02412 ± 0.001030.07920 ± 0.00617
Compatibility, 1000.02491 ± 0.000990.08681 ± 0.00632
Compatibility, 10410^40.02561 ± 0.001000.08965 ± 0.00429

Zero ridge retains an 8.1% prediction gain and 13.7-fold true-residual reduction over data-only; the paired prediction difference is −0.00183-0.00183 with 95% interval [−0.00321,−0.00045][-0.00321,-0.00045]. Prediction is essentially unchanged through 10−210^{-2}. Large ridge suppresses the correction and reduces its benefit. The matched ablation shows that corrected physics improves prediction without ridge and remains effective across small ridge values. The observed gain does not require fine tuning this penalty.

Reviewer_TPSWOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary
  • The paper investigates problems in physics-informed learning when the governing equations are uncertain or misspecified. Instead of assuming one fixed governing equation, which might lead to biases in cases of misspecification, it proposes compatibility training. The proposed method minimizes the residual over a chosen family of governing equations.
  • The authors provide an extensive theoretical analysis of the resulting loss, including prediction bias, recovery guarantees, and identifiability conditions.
  • Experiments on PINNs and neural operators show improvements under misspecification.

Major strengths/weaknesses and questions are listed below; here are some further minor notes:

  • clarity of presentation the paper might be difficult to follow for readers unfamiliar with the relevant literature. Several concepts, such as profiling, global profile, nuisance family, and correction patterns, are used without further explanation. Some terminology also appears newly introduced for established mathematical concepts, making it harder than necessary to recognize connections to classical methods.
  • title seems overstated Unbiased might suggest statistical unbiasedness, which is not established, especially since the empirical minibatch objective is generally biased.
  • practical computation The main text provides little information about actually solving the inner minimization. The highly interesting details concerning closed-form solutions or grid searches are provided exclusively in the appendix. A brief overview in the main text would improve accessibility and clarity.
Critical strengths
  • S1 Important and practically relevant problem The central limitation addressed by the paper is compelling: physics-informed learning often requires specifying exact governing equations, which might be uncertain or incorrect. This can actively harm predictive quality of the model, while failing silently since the monitored residual might indicate successful training.
  • S2 Careful and thorough theoretical analysis The authors provide mathematically rigorous results characterizing the consequences of the proposed minimization over an admissible family of governing equations. In particular, Theorem 4 and Corollaries 5-8 provide a useful interpretation of the remaining constraints/guidance after optimizing over the nuisance parameters.
  • S3 Encouraging empirical results Experiments cover several PDEs, PINNs and neural operators, including sparse and noisy observations and spatially varying coefficients. The evaluation demonstrates that the proposed optimization over a suitably chosen family of admissible governing equations can improve predictions compared with enforcing nominal (misspecified) laws.
Critical weaknesses
  • W1 Limited novelty and insufficient positioning against related work There is substantial overlap with existing work. This concerns classical methods, but also the optimization principle (classical variable projection) applied to PDE problems by Dong & Wang (2023) and the correction of misspecified physics in PINNs investigated by Zou et al. (2024). I appreciate that both connections have been acknowledged by the authors. The main contribution of the paper therefore appears to be the theoretical analysis rather than purely methodolocial. However, the abstract and stated contributions seem to suggest a much broader methodological advance, which might not be supported sufficiently by the novelty of the underlying approach. Moreover, much of this analysis relies on established arguments (projection, perturbation, identifiability), making the precise theoretical advance difficult to assess. Given that this assessment is crucial for novelty, the differentiation to prior work should play a substantial part in the main text instead of only appearing briefly and in the Appendix (B).

  • W2 Lack of experimental isolation Especially given the novelty concerns and the identity established in Proposition 2, stating that compatibility training and matched inverse PINNs have identical global optimizers, a careful experimental study is needed to distinguish improvements due to profiling specifically from those due to allowing corrections to the governing equations generally. However, the current direct comparisons differ in several important aspects: optimization procedure (profiling versus joint AdamW updates), constraint handling (clipping versus sigmoid parameterization), and parameter sharing (minibatch-specific versus persistent parameters). These differences make it difficult to isolate the source of the reported improvements. Furthermore, the field-valued Darcy experiment lacks a matched inverse baseline (acknowledged by the authors), and not direct comparison with Zou et al. (2024) is provided.

  • W3 Consistency across collocation points The practical implementation uses minibatches of collocation points. Consequently, different batches might select different equations from the admissible family, and no consistency wrt. a single governing equation over space and time is guaranteed. While the authors acknowledge that minibatch profiling leads to a different optimization objective compared with minimization over the full population, the practical implications of this difference are not investigated.

  • W4 Limited guidance for practical application For practical application, the method relies on choosing a suitable uncertainty family that captures the relevant model discrepancy while preserving the informative constraints. The tradeoff is analyzed theoretically, but this does not provide constructive guidance for selecting suitable correction directions. Several experiments use carefully designed uncertainty families that correspond to the synthetically induced mismatch. It remains unclear how such families could be constructed in general, when model mismatches are unknown.

Questions for authors
  • Q1 W1 Novelty Can the authors clarify the specific theoretical and methodological contributions beyond the cited prior work?
  • Q2 W2 Experiments Can the authors provide more controlled comparisons to isolate the benefits of profiling?
  • Q3 W3 Consistency How does minibatch profiling affect consistency wrt. a single governing equation across the full domain?
  • Q4 W4 Practical applicability How can uncertainty families be constructed and validated when the true model discrepancy is unknown?
6 comments

Our contribution is a criterion for choosing physics corrections that preserve solution recovery. The new controlled experiments separate the correction family's predictive benefit from profiling's optimization benefit: finding a good coefficient fit when local updates are sensitive to initialization. Fixed-reference and fresh-point tests verify that the gains persist with shared equation coefficients.

W1/Q1. What the analysis adds to prior work

We give a quantitative basis for choosing corrections that remove equation error while preserving the information needed to recover the solution. The analysis explains why correction is needed, when it enables recovery, and what information each added correction removes.

Dong and Wang (2023; https://www.math.purdue.edu/~sdong/PDF/InversePDE_JCP2023.pdf) use variable projection in random-weight neural inverse-PDE solvers, eliminating physical parameters or network coefficients. Zou et al. (2024; https://arxiv.org/abs/2310.10776) add neural discrepancy functions to misspecified PINNs and quantify their uncertainty. APHYNITY (Yin et al., 2021; https://arxiv.org/abs/2010.04456) selects a minimal-norm dynamics augmentation and studies uniqueness of that decomposition. Building on these approaches, our analysis connects equation correction to prediction accuracy through three results:

  1. Why a wrong equation can force a wrong prediction. Even at the true solution, an incorrect equation can produce a physics gradient that pushes the prediction away from truth. When this gradient is nonzero, Theorem 1 gives a positive lower bound on the error of every nearby nominal stationary solution, under its exact-data and smoothness assumptions. Better optimization of that objective cannot remove this error floor.
  2. When correcting the equation still allows recovery. After the coefficients are fitted, the remaining physics must detect prediction errors that the observations miss. Corollary 5 gives local recovery when the data and remaining physics together uniformly penalize every prediction change, under its exact-data and regularity assumptions. Theorem 4 identifies that remaining physics through D2C(u⋆)=2A∗QAD^2\mathcal{C}(u^\star)=2A^*QA at a regular interior compatible truth: AA maps prediction changes to residual changes, and QQ keeps the part that coefficient adjustment cannot cancel. The criterion concerns predicted fields and requires no convexity in the network weights.
  3. How much information is lost by allowing more correction. A new correction direction can cancel residuals caused by incorrect predictions as well as by equation error. Corollary 7 measures this loss exactly: for regular nested correction spaces with the same state derivative AA, the curvature removed along a prediction error hh is 2∥ΠEAh∥22\lVert \Pi_{\mathcal{E}}Ah\rVert^2, where E\mathcal{E} contains the added directions. This quantifies how much less strongly the physics loss penalizes that error after the family is expanded.

A simple example shows why the recovery criterion matters. Suppose observations determine only xx, and the residual is (x+a,y)(x+a,y). Fitting aa leaves the penalty y2y^2: data determine xx, and physics determines yy. If we also allow a correction bb to obtain (x+a,y+b)(x+a,y+b), fitting aa and bb makes the penalty zero for any prediction, leaving yy undetermined. Both families contain the true equation, but only the first supplies the information missing from the observations.

The third result gives a practical distinction between correction size and correction direction. Wider bounds allow larger corrections in existing directions and can cover more equation error without changing the local projector at an interior truth. Adding directions can remove useful constraints even with exact population residuals. More collocation points improve residual estimation, but cannot restore information that the correction family has removed. An invertible change of coefficient coordinates also preserves this local information: a learned basis can retain it by spanning the same correction space without reproducing the original coordinates.

Proposition 2 extends these recovery criteria to matched joint inverse learning through the equality of globally optimal predictor sets. The criteria therefore guide correction-family design in both formulations. The new Fisher–KPP proof turns the recovery condition into a concrete observation-sensitivity test, and the retrained dimension ablations test the predicted loss of constraints. We have foregrounded these contributions in the abstract, Contributions, and main-text related work.

W2/Q2. Controlled experiments separating family and optimizer effects

We added controlled experiments and an initialization ablation to separate the correction family's benefit from the effect of coefficient optimization. The first experiment compares profiling and joint inverse training with the family, constraints, and sharing held fixed. Both use direct coefficient bounds, one trajectory-wide coefficient vector, and the same fixed set of 4096 residual points. Predictor, initialization, 4096 observations, mismatch 16, λ=0.02\lambda=0.02, and 2500 updates are matched. Radius 32 places truth inside the affine families; radius 64 does so for the non-affine family. Inverse and discrepancy-network rates are validation-selected from three candidates.

MethodBurgersFisher–KPPNon-affine reaction
Data only0.01133 ± 0.003170.00647 ± 0.000840.00647 ± 0.00084
Nominal physics0.01713 ± 0.001590.1226 ± 0.00270.03801 ± 0.00136
Projected inverse0.00397 ± 0.000870.00271 ± 0.000280.00276 ± 0.00021
Compatibility0.00416 ± 0.000820.00278 ± 0.000230.00276 ± 0.00021
Coordinate discrepancy DNN0.00795 ± 0.001070.00421 ± 0.000610.00367 ± 0.00048

Across all three tasks, both solvers improve over data-only and nominal physics, with similar mean errors. The matched comparison establishes that the correction family supplies useful physics under both optimization procedures. This supports the scope of our recovery and family-design criterion, which applies to both formulations through Proposition 2's equivalence of globally optimal predictors under matched objectives. Profiling evaluates the corrected loss directly, eliminating the persistent coefficient update and its learning-rate choice. The next ablation isolates when coefficient optimization produces an additional benefit.

Compatibility also improves over the coordinate discrepancy DNN in every seed of every task, with paired 95% intervals below zero. This baseline follows Zou et al.'s correction architecture: coordinate inputs and two width-50 tanh hidden layers. It isolates their deterministic correction component under matched predictors and budgets; their Bayesian/ensemble uncertainty procedure is not part of this comparison. The non-affine profile evaluates 25 candidates, while joint inverse uses the continuous interval.

The second experiment isolates the role of finding a good coefficient optimum. We introduce uncertain forcing frequency through 2sin⁡(ωt)cos⁡x2\sin(\omega t)\cos x in Fisher–KPP, making the coefficient objective nonconvex. All physics methods share a 500-step data-only warm-up within 2500 total updates. Nominal frequencies are half or twice the truth. Joint coefficient rates and physics weights are selected on held-out observations. Profiling searches 161 frequencies with two local refinements.

Method4096 observations256 observations, 1% noise64 observations, 10% noise
Data only0.00652 ± 0.000890.1667 ± 0.01950.3213 ± 0.0121
Nominal physics0.1416 ± 0.00050.1763 ± 0.00260.2007 ± 0.0030
Joint from nominal frequency0.08045 ± 0.016710.1126 ± 0.02820.1252 ± 0.0097
Compatibility0.00206 ± 0.000300.00573 ± 0.002020.08862 ± 0.05390
Joint initialized by profiling0.00206 ± 0.000310.00573 ± 0.001990.07948 ± 0.04964
True frequency supplied0.00206 ± 0.000310.00565 ± 0.002000.01501 ± 0.00331

Profiling gives 39-fold and 20-fold lower mean error than the selected joint solver in the dense and 256-observation settings, improving every seed average. Initializing joint training with the warm-up profile brings its mean errors to the same level. An initialization-only ablation fixes the joint coefficient-rate multiplier at 0.1 and λ=0.02\lambda=0.02: changing only the starting frequency from nominal to the warm-up profile lowers dense RelL2 from 0.12963 ± 0.00038 to 0.00206 ± 0.00031. This directly identifies access to a good coefficient optimum as the source of the improvement. Profiling is effective both as a training procedure and as an initializer for joint training in these settings.

With 64 observations and 10% noise, the variability is larger and the profiling-versus-joint difference is unresolved. The frequency experiments match update counts; the separate equal-time PINN and Darcy comparisons below assess performance under matched training time.

The uvHk response provides those equal-time comparisons and the matched field-inverse Darcy baseline. Together, these experiments identify the family's prediction benefit, test it against stronger inverse solvers, and isolate an additional optimization benefit when coefficient fitting has multiple local minima.

W3/Q3. Consistency with one equation across the domain

The prediction gain persists with a common reference set, and one fitted equation transfers to fresh locations with little additional residual. First, we train on one fixed domain-wide reference set, with one coefficient vector shared across all its points at each fit. This removes batch-specific equation selection. Compatibility gives RelL2 0.02826 ± 0.00458, projected inverse 0.02774 ± 0.00474, and data-only 0.3170 ± 0.0137.

Second, we directly measure how well coefficients fitted on one batch apply elsewhere. We freeze four trained predictors, fit 24 independent batches per predictor and batch size, and transfer each fitted equation unchanged to 32,768 fresh points. Comparing its residual there with a refit on those fresh points measures the cost of keeping one equation fixed.

Points used to fit the equationAcross-batch SD of reaction rateExcess fresh residual over a fresh-point refit
320.004186(3.96±1.54)×10−6(3.96 \pm 1.54)\times10^{-6}
1280.002241(9.38±3.59)×10−7(9.38 \pm 3.59)\times10^{-7}
5120.001032(2.24±0.64)×10−7(2.24 \pm 0.64)\times10^{-7}
40960.000413(3.06±1.06)×10−8(3.06 \pm 1.06)\times10^{-8}

At the actual 4096-point batch size, keeping the fitted coefficients unchanged adds only 0.059% to the fresh-refit residual. Both coefficient variability and the transfer gap shrink as the fitting batch grows. A separate 65,536-point check gives residuals 5.15995×10−55.15995\times10^{-5} with the original 4096-point coefficient fit and 5.15806×10−55.15806\times10^{-5} after refitting. The reaction-rate column averages within-predictor batch SDs; the transfer column gives mean ± SD across four predictors, after averaging each predictor's 24 refits.

These tests directly assess the practical effect of the distinction

\le\min_\mu\mathbb E_B\widehat R_B(u,\mu).$$ The left side allows coefficients to be selected per batch; the right requires one coefficient choice across the domain. The transfer diagnostic measures the effect of keeping one fitted equation at new locations. The shared-equation property at truth is exact: if the family contains the true equation, the same true coefficients give zero residual on every batch. Exact unregularized profiling therefore preserves zero physics penalty at the true solution under minibatching. For trained predictors, the tests above measure agreement with one fitted equation beyond its fitting points. For Darcy, the sharing unit is one physical input: each coefficient field has one spatial correction vector shared across its locations.
Reviewer_StSMOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper addresses model misspecification in physics-informed machine learning, where enforcing an incorrect governing equation degrade prediction accuracy. The authors propose "compatibility training", which tries to minimize the physics residual over a prescribed family of plausible equations rather than enforcing a single nominal model.

Critical strengths

There is clear motivation for this line of work and the experiments demonstrate improvements across several benchmark problems.

Critical weaknesses
  1. Overall methodology: The analysis relies on generic Hilbert-space optimization arguments without establishing well-posedness, stability, or convergence of the PDE under study. In particular, the finite-collocation analysis does not establish that minimizing the empirical residual yields accurate PDE solutions.

  2. The recovery results rely on strong convexity (Theorem 2) or uniform positive definiteness of the combined Hessian (Corollary 5, Eq. 14). These assumptions may fail near bifurcations, singular linearizations, or nonunique solution branches, and are not verified for the PDEs considered. This substantially limits the significance and applicability of the theoretical guarantees (and is omitted from the "contributions" subsection).

The combination of 1) and 2) make the methodology a heuristic. I think more work needs to be done to demonstrate the broad applicability of the work.

Questions for authors

The framework is very general, which is a strength and a weakness.

  • Can you establish coercivity condition in Eq. (14) for any of the nonlinear PDE examples considered?
  • What happens to therecovery guarantees as the linearized PDE operator approaches singularity, particularly near bifurcations?
5 comments

We establish Eq. (14) for nonlinear Fisher–KPP and local recovery at an exact Bratu bifurcation. Our criterion specifies which observations complement corrected physics so that their combined constraints detect every local prediction error. We verify this condition, use it to bound solution error under finite collocation, and test it with trained predictors.

1. Q1: establishing Eq. (14) for a nonlinear PDE

The local proof has three steps.

  1. Use stability at fixed coefficients. With the same initial and boundary conditions, a small residual change implies a small solution change.
  2. Identify what coefficient adjustment leaves unresolved. At first order, adjusting diffusion and reaction can explain two independent patterns of solution change without increasing the physics loss.
  3. Use observations to resolve those two patterns. If their effects on the observed values are distinguishable, observations and physics together bound the full prediction error.

We make these steps explicit for the submitted equation

\qquad x\in[0,2\pi],\quad t\in[0,1].$$ With periodic boundaries, $D\gt 0$, and the submitted smooth initial condition in $(0,1)$, parabolic well-posedness and the maximum principle give a unique finite-time solution $0\lt u^\star\lt 1$. Perturbations preserving these conditions lie in $$X=\lbrace h\in H^1_tL^2_x\cap L^2_tH^2_{\mathrm{per},x}:h(0)=0\rbrace.$$ This norm controls the prediction error and its time and second spatial derivatives. Define $$Ah=h_t-D^\star h_{xx}-\alpha^\star(1-2u^\star)h, \qquad Jv=-v_Du^\star_{xx}-v_\alpha u^\star(1-u^\star).$$ Here $Ah$ is the residual change caused by a prediction error; $Jv$ is the change caused by adjusting diffusion and reaction. Thus $Q=I-\Pi_{\operatorname{Range}(J)}$ retains the residual change that coefficient adjustment cannot explain. Let $Bh=(h(z_i)/\sqrt{N})_i$ record interior observation errors, where $N$ counts all supervised values. Initial observations contribute zero because $h(0)=0$. Parabolic stability gives a bounded inverse $A^{-1}:L^2\to X$, with constant $C_E$. Set $S=A^{-1}J$: its two columns are, up to sign, the changes in the solution caused by changing diffusion and reaction. Point observations are bounded on this space in one spatial dimension. The observation condition is $s=\sigma_{\min}(BS)\gt 0$: no nonzero combination of these two solution changes can leave all observed values unchanged. To prove the bound, split any error as $h=Sc+w$, where $w=A^{-1}QAh$. The part $Sc$ can be explained by coefficient adjustment because $ASc=Jc$. The remaining part is controlled by physics: $\lVert w\rVert_X\le C_E\lVert QAh\rVert$. Observations then control $Sc$: since $BSc=Bh-Bw$ and $s\gt 0$, its coefficients satisfy $s\lVert c\rVert\le\lVert Bh\rVert+\lVert B\rVert\lVert w\rVert_X$. Combining these two bounds gives $$\lVert h\rVert_X\le a\lVert Bh\rVert+b\lVert QAh\rVert, \qquad a=\frac{\lVert S\rVert}{s},\quad b=C_E\left(1+\frac{\lVert S\rVert\lVert B\rVert}{s}\right).$$ Theorem 4 and weighted Cauchy–Schwarz now give Eq. (14): $$D^2\mathcal J_{\mathrm{comp}}(u^\star)[h,h] =2\lVert Bh\rVert^2+2\lambda\lVert QAh\rVert^2 \ge\underbrace{\frac{2}{a^2+b^2/\lambda}}_{\gamma\gt 0}\lVert h\rVert_X^2.$$ Thus the combined loss penalizes every local prediction error by at least a fixed multiple of its squared size. The two columns of $J$ are independent for this initial condition, so the rank condition also characterizes when this Hessian bound holds: if $BS$ loses rank, some nonzero $h=Sc$ is invisible to both terms. Eq. (14) is therefore reduced to a directly checkable observation test for this nonlinear PDE. The observation test holds by construction and in the benchmark. Two sensors at $x=0$ and $x=\pi/2$ at a common small positive time suffice. They distinguish the effects because diffusion changes the two values with opposite signs, while reaction changes them with the same sign. Formally, $BS=tK+O(t^2)$, where the rows of $K$ are $(-u_{0,xx},-u_0(1-u_0))/\sqrt{2}$, and $\det K\lt 0$ (numerically, $-0.0914561$). At $t=0.5$, sensitivity integration gives $s=0.1192$ on grids of 64, 128, and 256 points, with independent finite differences agreeing within $5\times10^{-8}$ relative error. The actual benchmark observation draws also satisfy the test: $s=0.05602$–$0.06086$ with 64 interior observations across all four seeds.

For an interior true coefficient, continuity gives local quadratic growth and recovery through Corollary 5. If the truth lies on a coefficient bound, the bounded profile is at least the unrestricted profile, and both vanish at the truth; the same quadratic-growth bound transfers. Sparse observations suffice because the PDE supplies the other constraints. This establishes recovery through the combined data–physics condition without requiring the data loss alone to be strongly convex. We have foregrounded this result, its local scope, and its distinction from Theorem 2's data-convexity assumption in the Contributions and main-text recovery discussion.

Training under the proved initial and boundary conditions. Every method enforces the exact initial condition and periodicity in its architecture. The 64 noiseless interior observations, initialization, mismatch 16, radius 32, λ=0.2\lambda=0.2, 4096 fixed collocation points, and 2500 updates are matched. The inverse coefficient rate is validation-selected. True residuals use 8192 fresh continuous points.

MethodRelL2Fresh true-equation residual
Data only0.05339 ± 0.002560.5293 ± 0.1351
Nominal physics0.5632 ± 0.120826.1883 ± 4.4362
Projected inverse0.03236 ± 0.002770.00780 ± 0.00189
Compatibility0.03099 ± 0.005350.00382 ± 0.00088

Compatibility reduces prediction error by 42.0% and true residual by 139-fold relative to data-only, improving prediction in every seed. This connects the PDE proof to training within the admissible prediction class used by that proof.

2. Connecting finite collocation to solution accuracy

Finite collocation connects to solution accuracy in two steps. First, Theorem 3 bounds the gap between the sampled and full-domain compatibility losses. Second, the PDE estimate above converts a small full-domain data–physics loss into a small solution error. We establish this stability step after coefficient adjustment, following the stability-based PINN analysis of Mishra and Molinaro (2022; https://arxiv.org/abs/2007.01138).

In the compatible noiseless setting, three errors enter the bound:

  • δc\delta_c: a uniform bound on the gap between sampled and full-domain compatibility losses.
  • ηin\eta_{\mathrm{in}}: a uniform bound on the excess loss from approximate feasible coefficient solves.
  • εout\varepsilon_{\mathrm{out}}: predictor optimization error plus the best approximation error achievable by the predictor class, measured in the objective.

For predictors within the recovery neighborhood, comparing objective values gives

\le\sqrt{\frac{8\lambda\delta_c+4\lambda\eta_{\mathrm{in}}+4\varepsilon_{\mathrm{out}}}{\gamma}}.$$ The bound says how accuracy improves: better domain coverage reduces $\delta_c$, a more accurate coefficient solve reduces $\eta_{\mathrm{in}}$, and better predictor fitting or approximation reduces $\varepsilon_{\mathrm{out}}$. The positive constant $\gamma$ measures how strongly observations and retained physics detect prediction errors; we established it above. The proof compares the same full-domain objective gap from both sides: sampling and optimization bound it from above, while local quadratic growth from Eq. (14) bounds it from below by a multiple of the squared solution error. Under Theorem 3's bounded-residual and covering assumptions on the predictor family, sending the three errors to zero therefore gives local solution convergence. Collocation ablation. We retrained on nested fixed point sets, holding observations, architecture, initialization, and 2500-update budgets fixed. Each fitted coefficient vector was transferred unchanged to 32,768 independent continuous points. | Collocation points | RelL2 | Fresh residual of fitted equation | Fresh true-equation residual | | --- | ---: | ---: | ---: | | 128 | 0.03698 ± 0.00105 | 0.1220 ± 0.0357 | 0.1158 ± 0.0294 | | 512 | 0.03249 ± 0.00190 | 0.03789 ± 0.00513 | 0.03653 ± 0.00475 | | 1024 | 0.03154 ± 0.00235 | 0.02477 ± 0.00380 | 0.02270 ± 0.00277 | | 4096 | 0.03099 ± 0.00535 | 0.00367 ± 0.00037 | 0.00358 ± 0.00054 | Increasing from 128 to 4096 points reduces the fresh residual of the fitted equation 33.3-fold and improves prediction in all four seeds, with the same observations and update budget. The 128-point training residual is already $2.06\times10^{-8}$; the additional points improve equation agreement across the domain, as measured by the fresh residual decreasing from 0.1220 to 0.00367. This tests the sampling contribution in the bound: wider collocation coverage improves agreement with one fitted equation at locations never used to fit it.

3. Q2: recovery at singularities and bifurcations

The combined recovery condition can hold at an exact PDE bifurcation. Near a singularity, some solution changes produce little or no change in the PDE residual. Eq. (14) specifies the information observations must supply to resolve these changes. For a unit critical direction hϵh_\epsilon with ∥Ahϵ∥≤ϵ\lVert Ah_\epsilon\rVert\le\epsilon,

γ≤2∥Bhϵ∥2+2λϵ2.\gamma\le2\lVert Bh_\epsilon\rVert^2+2\lambda\epsilon^2.

As ϵ→0\epsilon\to0, this bound requires observations to remain sensitive to the critical directions for a uniform positive recovery margin. When observations resolve those directions and retained physics controls the rest, the combined margin can stay positive. The Bratu example below establishes this at the exact singularity.

Consider the Dirichlet Bratu equation u′′+μeu=0u''+\mu e^u=0 on (0,1)(0,1). At its fold, μc≃3.513830719\mu_c\simeq3.513830719, the self-adjoint linearization has kernel spanned by a positive function ϕ\phi. To identify what corrected physics leaves unresolved, take QAh=0QAh=0, so Ah=JcAh=Jc with J=eu⋆J=e^{u^\star}. Taking the inner product with ϕ\phi gives 0=c⟨ϕ,J⟩0=c\langle\phi,J\rangle. Since ⟨ϕ,J⟩>0\langle\phi,J\rangle\gt 0, this forces c=0c=0 and ker⁡(QA)=span⁡{ϕ}\ker(QA)=\operatorname{span}\lbrace \phi\rbrace.

Only one direction remains to be measured. Because ϕ\phi is nonzero at every interior point, an interior sensor detects it; elliptic stability controls all remaining directions. This gives γ>0\gamma\gt 0 at the fold. The coefficient fit also remains regular because J∗J>0J^*J\gt 0.

The following deterministic calculation uses four fixed interior sensors, λ=0.02\lambda=0.02, and ∥h∥2=∫(h2+hxx2)\lVert h\rVert^2=\int(h^2+h_{xx}^2). Ranges span the two solution branches, not random runs.

Bratu parameterCurvature from the known PDE aloneCombined compatibility curvature γ\gamma
μ=3.0\mu=3.00.00872–0.013720.01694–0.02461
μ=3.5\mu=3.50.000304–0.0003380.02193–0.02365
μ=3.5138\mu=3.5138(7.10–7.14)×10−7(7.10\text{–}7.14)\times10^{-7}0.02279–0.02287
Exact fold0 analytically0.02283

The combined margin remains positive through the exact fold; doubling resolution changes the reported margins by at most 0.031%. In neural training at the fold with three interior observations and 1% noise, compatibility gives RelL2 0.00095 ± 0.00070, versus 0.01890 ± 0.00703 for data-only. True-coefficient and joint-inverse controls both give approximately 0.00095. The criterion thus identifies observations that preserve local recovery at a singular PDE, and the training experiment achieves accurate predictions with uncertain coefficients and noisy observations.

Comments on the paper

1 comment

We thank the reviewers for their detailed questions. Our central contribution is a criterion for correcting an inaccurate equation while keeping enough physics to recover the solution. We show why enforcing a wrong equation can force prediction error, when corrected physics and observations together determine the solution locally, and exactly which constraints are lost when correction freedom is increased. These results guide the choice of correction family and observations in both profiled and joint inverse training.

We added controlled comparisons and targeted ablations to test this criterion and distinguish the sources of improvement:

  • PDE-specific recovery: we derive Eq. (14) for nonlinear Fisher–KPP from PDE stability and a two-column observation test, bound solution error in terms of sampling and optimization errors, and establish local recovery at an exact Bratu bifurcation.
  • Controlled comparisons: matching constraints, parameter sharing, predictors, and update budgets establishes the correction family's benefit with both profiling and stronger joint inverse solvers. A separate initialization ablation isolates profiling's additional benefit in finding a good coefficient optimum; equal-time controls test performance under matched training time.
  • Practical family construction: we learn correction patterns and their allowed size from separate calibration data, then fix them before training. A combined Darcy control uses this learned family, accurate constrained solving, and zero ridge. On fresh test functions, compatibility improves prediction by 11.4% over data-only and 36.2% over nominal physics, with 14.8-fold lower true-equation residual relative to data-only.
  • Mechanism tests: separate solver and ridge ablations show that the gains persist with accurate coefficient fitting and no ridge penalty. Retrained family-dimension × collocation-count ablations test the predicted tradeoff between correcting equation error and retaining useful physics constraints.

Theorem numbers refer to the submission. RelL2 is relative L2 prediction error; residuals are mean squared residuals. Performance results report mean ± sample standard deviation across four independent training seeds, averaging benchmark conditions within each seed first. Reported 95% intervals are pointwise paired Student-t intervals over those four seed averages. Diagnostic batches and test functions do not increase the number of independent training runs. Deterministic and pooled diagnostics are labeled separately.

Manuscript correction: PDEBench coefficient convention

The official PDEBench Burgers generator (https://github.com/pdebench/PDEBench/blob/main/pdebench/data_gen/data_gen_NLE/BurgersEq/burgers_multi_solution_Hydra.py) uses diffusion ν/π\nu/\pi, so dataset label ν=0.02\nu=0.02 means physical diffusion 0.02/π0.02/\pi. We have corrected the manuscript's coefficient labels and true-residual diagnostic; prediction errors are unchanged. Nominal labels δ=1,16\delta=1,16 correspond to physical mismatch factors πδ\pi\delta. At label 16, truth lies outside the radius-16 family, so that result is evidence for prediction improvement under an approximate family. The revision distinguishes the corrected diagnostics from the original residual table.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.