- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
My read is that this paper tackles training with the wrong physics. The method searches a declared equation family at each step. It then penalizes the residual that parameter adjustment can't explain. The theory shows which useful physics constraints survive that adjustment. The controlled gains look strong, though the family choice does a lot of the work.
Critical strengths
The geometry is the best part for me. The Hessian identity makes the lost and retained constraints explicit. That's a useful way to reason about sparse observations.
Those sparse-data gains are hard to dismiss. With 64 observations and 10% noise, RelL2 falls from 0.3156 to 0.0153 against data-only. I give the paper real credit for that.
Critical weaknesses
I'm struggling to see much method novelty. Classical variable projection does the core work. The added value rests on the analysis and convincing optimization gains.
Choosing the equation family remains a major burden. Missing modes leave mismatch behind, while extra modes can erase useful constraints. I'd worry about how users make that choice from limited measurements.
I still don't know how reliably the actual procedure trains. The key recovery arguments need specific local conditions, and Darcy's inner errors weren't measured. That's a substantial gap between the theory and the experiments.
Questions for authors
Does the advantage survive a stronger inverse solver? I'd want projected updates, separate parameter learning rates, and matched compute. Please include a matched field-inverse Darcy baseline too. What happens when the correction basis misses the actual discrepancy? Please choose the basis and bounds from separate calibration data. That's the practical test I'd find convincing. How often does radial clipping activate in Darcy? Please report inner optimality errors and compare against an accurate constrained solve. I'd also want a ridge sweep. Does extra correction freedom hurt prediction during training? A retrained sweep over correction dimension and collocation count would help. I'd want fresh-point residuals alongside prediction errors.
6 comments
2. Independently calibrated basis and bounds
A combined control tests learned corrections with accurate solving and no ridge. We determine which spatial patterns can be corrected and how large those corrections may be using separate calibration data:
- Learn the correction patterns. Use 256 separate calibration examples of spatial coefficient corrections. Estimate their average and use principal component analysis (PCA) to find the main patterns around it. Retaining patterns that explain 99% of the variation selects eight directions, each normalized to unit RMS.
- Set the allowed correction size. Express another 128 disjoint calibration fields in these directions. Set the radius to the 95th percentile of their coefficient norms, so the bound is determined by calibration examples.
- Freeze the family before training. Keep the learned mean, directions, and radius fixed for prediction training and evaluation. The generating basis and target test corrections are never supplied to this procedure.
All methods use the same 1024 training functions and 200 epochs. Compatibility uses an accurate constrained solve and zero ridge. The matched inverse uses the same calibrated family and zero ridge, with its rate selected on 128 validation functions. Frozen models are evaluated on 1024 new physical functions across four seed streams, each at two mismatch amplitudes, giving 2048 test cases. This combined experiment tests the requested controls together; the separate solver and ridge ablations below isolate their individual effects.
| Method, independently calibrated family | Fresh-test RelL2 | Fresh true-equation residual |
|---|---|---|
| Data only | 0.01040 ± 0.00076 | 0.1665 ± 0.0252 |
| Nominal physics | 0.01446 ± 0.00083 | 0.04301 ± 0.00312 |
| Projected inverse, zero ridge | 0.00940 ± 0.00111 | 0.01157 ± 0.00272 |
| Compatibility, accurate solve, zero ridge | 0.00922 ± 0.00102 | 0.01124 ± 0.00230 |
Compatibility improves prediction by 11.4% over data-only and 36.2% over nominal physics, and lowers true residual 14.8-fold relative to data-only. Both corrected methods improve over both baselines in every seed average. For compatibility versus data-only, the paired prediction difference is with 95% interval , comparing methods within each shared seed. The difference between the two corrected solvers remains unresolved. The combined control establishes prediction gains from a separately learned family with accurate coefficient solves and zero ridge. The matched inverse confirms that the same family also benefits joint training.
The missing-direction control identifies what the family must capture. Truncating the learned family from eight to four modes raises its residual at the true solution from to 0.0279. Increasing the allowed size of the remaining patterns cannot create a missing pattern. The solution-pair calibration experiment in TPSW W4 tests the remedy: learning directions improves prediction when discrepancy lies entirely outside the initial eight-mode basis, even against data-only supplied with the same extra labels.
3. Radial clipping and accurate constrained solves
The prediction gain persists when the approximate coefficient solve is replaced by an accurate one. This solver ablation tests whether clipping error is responsible for the improvement. Radial clipping fits coefficients first and shrinks them uniformly if they exceed the allowed size. The accurate solver instead finds the best fit within that limit. The Darcy inner problem is
The columns of describe residual corrections, is the residual to explain, limits correction size, and penalizes large coefficients. With and , its optimality conditions are
\lVert \alpha\rVert\le\rho,\quad\zeta\ge0,\quad \zeta(\lVert \alpha\rVert-\rho)=0.$$ An unconstrained optimum inside the ball has $\zeta=0$. Otherwise, a one-dimensional search for $\zeta$ places the optimum on the boundary. This accounts for how each correction direction affects the residual; radial clipping rescales all coordinates equally. We compare both with an independent float64 constrained reference. Clipping frequencies below count every training solve; maximum objective gaps come from sampled audits. “Late” means the final 20% of training. These are pooled numerical diagnostics, not seed-level performance estimates. | Radius and protocol | Fraction of solves clipped | Largest audited clipping objective gap | Largest late gap | | --- | ---: | ---: | ---: | | $\rho=1$, 1024 functions, 200 epochs | 0.0974% of 1,638,400 | 70.8 | $1.83\times10^{-12}$ | | $\rho=0.25$, 512 functions, 80 epochs | 96.71% | $8.21\times10^3$ | 0.00645 |For radius 1, 1152 audited problems give accurate-solver mean KKT residual and maximum objective discrepancy . The KKT residual measures violation of constrained optimality. Radial clipping gives mean KKT residual 1.24 overall and late in training.
Retraining with the accurate solver gives RelL2 0.00877 ± 0.00094, versus 0.00868 ± 0.00078 for clipping and 0.00975 ± 0.00087 for data-only. It retains a 10.1% prediction gain and 14.4-fold true-residual reduction over data-only. At the frequently active radius 0.25, accurate and radial errors are 0.02129 ± 0.00124 and 0.02343 ± 0.00354. The gain therefore persists after replacing the approximate inner solve: useful corrected physics remains effective when the coefficient fit is computed accurately.
The calibrated zero-ridge experiment also exercises active bounds: 9.39% of 1,638,400 solves lie on the boundary. Its 1152 audited solves have mean KKT residual and maximum objective discrepancy, normalized by one plus reference residual, of .
Appendix D.11 connects these diagnostics to the ideal loss: for , a feasible KKT residual bounds inner-value error by , and ridge changes the unregularized profile by at most . The two terms quantify solver error and regularization error separately. Zero-ridge solves are checked directly against the reference. We now report these diagnostics and use the accurate solver in the new constrained-solve controls.
4. Ridge ablation, including zero
A one-factor ablation tests whether ridge is needed for the gain. We retrain at each ridge value, including zero, while holding initialization, architecture, 512 training functions, 80 epochs, mismatch amplitude 1, eight modes, radius 1, and the accurate solver fixed. This isolates the effect of the ridge penalty from the correction family and solver accuracy. Modes have unit maximum absolute value, fixing the scale of . All nine tested values are shown.
| Method / ridge | RelL2 | True-equation residual |
|---|---|---|
| Data only | 0.02262 ± 0.00102 | 0.6533 ± 0.0599 |
| Nominal physics | 0.02690 ± 0.00306 | 0.09804 ± 0.01575 |
| Compatibility, 0 | 0.02079 ± 0.00132 | 0.04765 ± 0.00623 |
| Compatibility, | 0.02079 ± 0.00132 | 0.04762 ± 0.00626 |
| Compatibility, | 0.02079 ± 0.00132 | 0.04761 ± 0.00627 |
| Compatibility, | 0.02080 ± 0.00132 | 0.04760 ± 0.00629 |
| Compatibility, | 0.02085 ± 0.00126 | 0.04763 ± 0.00605 |
| Compatibility, 1 | 0.02185 ± 0.00116 | 0.05595 ± 0.00576 |
| Compatibility, 10 | 0.02412 ± 0.00103 | 0.07920 ± 0.00617 |
| Compatibility, 100 | 0.02491 ± 0.00099 | 0.08681 ± 0.00632 |
| Compatibility, | 0.02561 ± 0.00100 | 0.08965 ± 0.00429 |
Zero ridge retains an 8.1% prediction gain and 13.7-fold true-residual reduction over data-only; the paired prediction difference is with 95% interval . Prediction is essentially unchanged through . Large ridge suppresses the correction and reduces its benefit. The matched ablation shows that corrected physics improves prediction without ridge and remains effective across small ridge values. The observed gain does not require fine tuning this penalty.
Our contribution is a criterion for correcting equation error while preserving the physics information needed for prediction. Classical variable projection supplies the coefficient fit; our analysis identifies the conditions for solution recovery and quantifies the constraints lost as correction freedom grows. We added controlled comparisons to separate the family's benefit from coefficient optimization, and ablations of solver accuracy, ridge, and correction dimension. A combined Darcy control tests independently learned corrections, accurate solving, and zero ridge together on fresh functions. TPSW W1 details the analysis, and StSM establishes the local recovery condition for nonlinear Fisher–KPP.
1. Stronger inverse solvers and matched compute
Controlled Fisher–KPP comparison at equal training time. To separate the family's benefit from the fitting procedure, we compare profiling with a strengthened inverse solver under a fixed time budget. The inverse uses projected coefficient updates and a separate coefficient learning rate. Every method receives 90 training seconds on an A100 and a cosine schedule spanning the budget. The setting has 64 interior observations plus 256 initial values, all with 10% noise, mismatch/radius 16, λ=0.2, and fresh collocation batches. No validation labels are used: the inverse coefficient rate is fixed at 0.02 and the discrepancy-network rate at 0.002. Prediction uses 24,512 disjoint locations; true residuals use 8192 fresh continuous points.
Compatibility gives 20.0-fold lower prediction error than data-only and 6.3-fold lower error than the discrepancy DNN, improving every seed in both comparisons. The strengthened inverse also achieves low errors with the same correction family, and all three tested inverse rates, 0.002, 0.02, and 0.2, retain the family benefit. The controlled comparison supports useful corrected physics under both coefficient-fitting procedures, consistent with Proposition 2's shared globally optimal predictor sets. The paired interval for their prediction difference spans zero. A validation-selected discrepancy DNN given three full 90-second runs reaches 0.06881 ± 0.01444, also above compatibility.
Matched field-inverse Darcy control. To compare coefficient-fitting procedures for spatial corrections, joint training keeps eight persistent coefficients per training function. Basis, norm ball, ridge, FNO, initialization, observations, and residual evaluation match compatibility, which uses the accurate constrained solver. The inverse coefficient rate is selected from 0.001, 0.01, and 0.1 by validation. We compare both equal-epoch and equal-time budgets to measure the effect of training cost.
At equal epochs, compatibility improves prediction by 11.1% and lowers true residual 15.9-fold relative to data-only; the paired prediction difference is −0.00110 with 95% interval [−0.00151,−0.00068]. At equal time, it lowers true residual 9.1-fold, while data-only has lower prediction error. Both corrected solvers substantially improve equation agreement under both budgets; their prediction differences are unresolved by paired intervals. Timers are synchronized, schedules span the full budget, and per-run training time excludes evaluation and hyperparameter search.
The family improves prediction with either solver in the Fisher–KPP and equal-epoch Darcy controls. Profiling evaluates its loss without a persistent coefficient update or a coefficient learning-rate choice. The frequency initialization ablation in TPSW W2 isolates its additional optimization benefit: a profiled starting frequency enables joint training to reach the same good solution. These studies distinguish useful corrected physics from access to a good coefficient optimum.