- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1(−1)
- Soundness
- 2
Read review
Paper summary
The paper compares a physics residual built from the known governing equation against equally tuned generic regularizers for neural PDE surrogates, across five 1D PDE families (heat and advection-diffusion as linear controls, Burgers, KdV, and Allen-Cahn as nonlinear). At matched tuning budget, the physics residual beats the best generic regularizer for the three nonlinear families, and this advantage survives across a roughly order-of-magnitude range of capacities while the two linear families collapse toward or below parity. A proposition identifies the mechanism as a closure term from spectral truncation that the naive coarse-grid residual omits, and shows the benefit reverses once the model's grid under-resolves the nonlinear dynamics. A separate experiment finds that pretraining across families and in-context conditioning do not beat a Fourier Neural Operator trained from scratch in the tested regime. Validation includes a pre-registered set of four falsifiable hypotheses, two of which are reported as falsified, and an independent verification of the refinement rate for the closure term's bound.
Critical strengths
- Proposition 1 identifies the mechanism as a closure term from spectral truncation that the coarse-grid residual omits. Appendix C.1 states this term is not new, naming it as the spectral viscosity commutator in Tadmor (1989), the dealiasing term in Canuto et al. (2006) and Hesthaven et al. (2007), and the sub-grid stress of large-eddy simulation in Sagaut (2006). The paper's stated contribution is applying this identity to a learned penalty.
Critical weaknesses
-
The main text frames the spectral-truncation closure term as the paper's mechanistic finding without stating it is a known classical quantity. Appendix C.1 concedes that τN is not new — it appears under established names in Tadmor (1989), Canuto et al. (2006) and Hesthaven et al. (2007), and Sagaut (2006) — and restricts the contribution to applying this known identity to a learned penalty. The main text and the appendix make different claims about what is novel, and a reader of the main text receives the stronger claim without the qualification.
-
The paper invokes pre-registration as its direct answer to McGreivy and Hakim (2024)'s critique of selective reporting, but the pre-registration record reports only that two of four hypotheses were falsified, without naming which two or stating what they predicted. A pre-registration that does not specify the falsified predictions cannot function as evidence against selective reporting: the reader cannot assess whether the falsifications were peripheral or bore directly on the comparison methodology.
-
Section 3.2 and Appendix C.1 verify the paper's assumption that coefficients are known exactly only against spatial under-resolution. Coefficient misspecification, the realistic case for most deployed systems, is left as an open extension of the paper's own question, when known physics helps.
Questions for authors
-
Proposition 1 (page 13) states τN(u) = −½ ∂x PN[(QNu)(u + PNu)] for the Burgers flux, but the definition τN = FN(PNu) − PNF(u) and the identity (PNu)² − u² = −(QNu)(u + PNu) given in the same appendix together give a positive leading coefficient. The downstream bound takes a norm and is sign-independent, so this does not affect the paper's conclusions, but which sign is correct?
-
Is a test of robustness to coefficient misspecification planned, training with a residual built from an incorrect viscosity, wave speed, or reaction coefficient, to see whether the benefit degrades gracefully or collapses the way under-resolution does?
-
What is the 95% confidence interval for the Burgers relevance-gradient result at M=16, matching the table already given for KdV and Allen-Cahn?
Response to Reviewer 4Xn3
Thank you. All three weaknesses are addressed in the revision, and the misspecification experiment you asked about is now Section 3.4.
W1: novelty of the closure term. Agreed. τ_J is the standard Galerkin-truncation (subgrid) term. The main text now says so where it is introduced, cites Tadmor, Canuto et al., Hesthaven et al. and Sagaut there, and states that we use it as a diagnostic and claim no novelty for it. Main text and appendix now make the same claim.
Q1: sign. You are right: the correct sign is +½ ∂ₓ P_J[(Q_J u)(u + P_J u)] under τ_J = F_J(P_J u) − P_J F(u). Corrected, and checked numerically to round-off (Fig. 16).
W2: pre-registration. The record is now in the supplementary material (
PGWM_flip_preregistration.md, verbatim with one identifying phrase removed), and Appendix E gives each hypothesis with its decision rule and outcome. Two clarifications the submission did not make. It is a dated private document written before the first run, not an entry in a public registry. And the registered comparison is with data-only training; after a first run in which the physics arm won at ρ=1, we gave the data-only arm its own regularization grid before rerunning, which is the "best generic regularizer" comparison of the submission. Outcomes in the present runs are the same under both comparisons:So the two falsified hypotheses are H-flip-1 and H-flip-3, and neither is peripheral: both are predictions about the headline comparison. The loss-scale control (general response), added in revision and marked as post hoc, changes the reading of H-flip-1: against the scale-matched baseline the dense end is inert (0.98–1.17), which is what the record predicted. H-flip-3 is still not supported as stated, because the advantage decays smoothly (1.3–1.6 at ρ=0.25, above ρ*).
W3 / Q2: coefficient misspecification. We ran it (Section 3.4, Tables 1, 15, 16): the data keep the true coefficients and the residual uses one coefficient multiplied by m from 0.5 to 2, at ρ=0.12. Ratios are reference error / residual-model error; m = 0.9 / 1.1 shown, with the correct equation (m=1) for comparison.
Direct answer to Q2: it collapses; it does not degrade gracefully. At the standard weights (λ ≥ 0.01, which already weights the residual 400× the data loss) a 10% error turns a 1.5–2.6× gain into a 2–100× loss. With the weight tuned down to 1e-5, validation picks the smallest weight for every wrong coefficient except the Allen–Cahn diffusion term (negligible at ε = 0.002): the constraint is effectively switched off, a 10% error in viscosity or dispersion becomes neutral against the unscaled generic arm, and errors in flux, reaction or linear-family coefficients remain harmful. In no case is the gain of the correct equation retained. Errors below 10% were not tested, and we state that the method assumes coefficients known to well within that.
Q3: Burgers interval at M=16. In the rerun, with M=16 Burgers trajectories, the from-scratch / transfer error ratio is 1.35 [1.18, 1.52] for sequential pre-training on another family, 1.06 [1.00, 1.11] for joint training on another family, and 1.19 [1.06, 1.28] for sequential pre-training on Burgers' own linear part (Table 17 gives intervals for all targets and M). The design differs from the submission (one source family, not four). In this design the negative-transfer result holds for Allen–Cahn and not for Burgers; see our response to Reviewer RLw4, W1.