acceptodds
Under review as a conference paper at ICLR 2027

Does Known Physics Help Neural PDE Models? Only When It's Non-Redundant and Resolvable

Abstract

Structural priors - equation residual penalties, cross-family pretraining and conditioning, and discretization-invariant architectures are the default prescription for neural PDE surrogates, yet they are rarely tested against baselines with comparable tuning. Under a fixed architecture family and evaluation protocol, we ask which priors beat a strong baseline: a from-scratch neural operator with multi-step pushforward training. Only one does, and we characterize it mechanistically. A known-equation residual outperforms the best generic regularizer (weight decay, input noise, or data-only training, each tuned at matched budget) at every observation density, including full supervision. The advantage is capacity-robust only for nonlinear operators: as model width grows it persists for Burgers, KdV, and Allen-Cahn while collapsing to or below parity for linear families (heat, advection-diffusion), which serve as controls. We explain this dissociation with a measured quantity, the non-redundant content of the one-step map provably zero iff the operator is linear, which predicts which families retain a durable advantage. We further prove and verify the boundary of the effect: on under-resolved grids the enforced residual equals the spectral-truncation closure term, a bias irreducible by data or capacity, and the penalty reverses sign and harms nonlinear models in proportion to the unresolved tail energy. A pre-registered hypothesis that the benefit is sparsity-gated is falsified and reported as such. In the same regime, cross-family pretraining and in-context conditioning fail to beat the from-scratch baseline across three backbones and all model widths tested. The result is a falsification-tested account of when encoding known physics pays: nonlinear dynamics and resolvable evaluation.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
physics-informed machine learning, neural operators, PDE surrogates, equation residual penalties, scientific machine learning, structural priors, strong baselines, negative results, spectral truncation, resolution invariance, cross-family transfer, in-context learning for PDEs

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4Xn3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1(−1)
Soundness
2
Read review
Paper summary

The paper compares a physics residual built from the known governing equation against equally tuned generic regularizers for neural PDE surrogates, across five 1D PDE families (heat and advection-diffusion as linear controls, Burgers, KdV, and Allen-Cahn as nonlinear). At matched tuning budget, the physics residual beats the best generic regularizer for the three nonlinear families, and this advantage survives across a roughly order-of-magnitude range of capacities while the two linear families collapse toward or below parity. A proposition identifies the mechanism as a closure term from spectral truncation that the naive coarse-grid residual omits, and shows the benefit reverses once the model's grid under-resolves the nonlinear dynamics. A separate experiment finds that pretraining across families and in-context conditioning do not beat a Fourier Neural Operator trained from scratch in the tested regime. Validation includes a pre-registered set of four falsifiable hypotheses, two of which are reported as falsified, and an independent verification of the refinement rate for the closure term's bound.

Critical strengths
  1. Proposition 1 identifies the mechanism as a closure term from spectral truncation that the coarse-grid residual omits. Appendix C.1 states this term is not new, naming it as the spectral viscosity commutator in Tadmor (1989), the dealiasing term in Canuto et al. (2006) and Hesthaven et al. (2007), and the sub-grid stress of large-eddy simulation in Sagaut (2006). The paper's stated contribution is applying this identity to a learned penalty.
Critical weaknesses
  1. The main text frames the spectral-truncation closure term as the paper's mechanistic finding without stating it is a known classical quantity. Appendix C.1 concedes that τN is not new — it appears under established names in Tadmor (1989), Canuto et al. (2006) and Hesthaven et al. (2007), and Sagaut (2006) — and restricts the contribution to applying this known identity to a learned penalty. The main text and the appendix make different claims about what is novel, and a reader of the main text receives the stronger claim without the qualification.

  2. The paper invokes pre-registration as its direct answer to McGreivy and Hakim (2024)'s critique of selective reporting, but the pre-registration record reports only that two of four hypotheses were falsified, without naming which two or stating what they predicted. A pre-registration that does not specify the falsified predictions cannot function as evidence against selective reporting: the reader cannot assess whether the falsifications were peripheral or bore directly on the comparison methodology.

  3. Section 3.2 and Appendix C.1 verify the paper's assumption that coefficients are known exactly only against spatial under-resolution. Coefficient misspecification, the realistic case for most deployed systems, is left as an open extension of the paper's own question, when known physics helps.

Questions for authors
  1. Proposition 1 (page 13) states τN(u) = −½ ∂x PN[(QNu)(u + PNu)] for the Burgers flux, but the definition τN = FN(PNu) − PNF(u) and the identity (PNu)² − u² = −(QNu)(u + PNu) given in the same appendix together give a positive leading coefficient. The downstream bound takes a norm and is sign-independent, so this does not affect the paper's conclusions, but which sign is correct?

  2. Is a test of robustness to coefficient misspecification planned, training with a residual built from an incorrect viscosity, wave speed, or reaction coefficient, to see whether the benefit degrades gracefully or collapses the way under-resolution does?

  3. What is the 95% confidence interval for the Burgers relevance-gradient result at M=16, matching the table already given for KdV and Allen-Cahn?

1 comment

Response to Reviewer 4Xn3

Thank you. All three weaknesses are addressed in the revision, and the misspecification experiment you asked about is now Section 3.4.

W1: novelty of the closure term. Agreed. τ_J is the standard Galerkin-truncation (subgrid) term. The main text now says so where it is introduced, cites Tadmor, Canuto et al., Hesthaven et al. and Sagaut there, and states that we use it as a diagnostic and claim no novelty for it. Main text and appendix now make the same claim.

Q1: sign. You are right: the correct sign is +½ ∂ₓ P_J[(Q_J u)(u + P_J u)] under τ_J = F_J(P_J u) − P_J F(u). Corrected, and checked numerically to round-off (Fig. 16).

W2: pre-registration. The record is now in the supplementary material (PGWM_flip_preregistration.md, verbatim with one identifying phrase removed), and Appendix E gives each hypothesis with its decision rule and outcome. Two clarifications the submission did not make. It is a dated private document written before the first run, not an entry in a public registry. And the registered comparison is with data-only training; after a first run in which the physics arm won at ρ=1, we gave the data-only arm its own regularization grid before rerunning, which is the "best generic regularizer" comparison of the submission. Outcomes in the present runs are the same under both comparisons:

  • H-flip-1, the sign flip (no advantage at ρ=1, an advantage at the sparsest ρ): falsified at the dense end. The physics arm wins at ρ=0.03 as predicted (4.1–12.7×) but also at ρ=1 (Burgers 1.50 [1.40, 1.60], KdV 1.78 [1.59, 2.01], Allen–Cahn 1.43 [1.25, 1.69]).
  • H-flip-2, the advantage is non-increasing in ρ: supported in all five families.
  • H-flip-3, a sign change within a factor of two of ρ* = 2m/N = 0.156: falsified, because the advantage never changes sign.
  • H-flip-4, the crossover moves with bandwidth across families: untestable on this suite, since the families share the same bandwidth.

So the two falsified hypotheses are H-flip-1 and H-flip-3, and neither is peripheral: both are predictions about the headline comparison. The loss-scale control (general response), added in revision and marked as post hoc, changes the reading of H-flip-1: against the scale-matched baseline the dense end is inert (0.98–1.17), which is what the record predicted. H-flip-3 is still not supported as stated, because the advantage decays smoothly (1.3–1.6 at ρ=0.25, above ρ*).

W3 / Q2: coefficient misspecification. We ran it (Section 3.4, Tables 1, 15, 16): the data keep the true coefficients and the residual uses one coefficient multiplied by m from 0.5 to 2, at ρ=0.12. Ratios are reference error / residual-model error; m = 0.9 / 1.1 shown, with the correct equation (m=1) for comparison.

Coefficientm=1, λ ≥ 0.01, vs scale-matchedm=0.9 / 1.1, λ ≥ 0.01, vs scale-matchedλ ≥ 1e-5, vs scale-matchedλ ≥ 1e-5, vs generic (unscaled)
Heat ν1.520.01 / 0.010.16 / 0.160.37 / 0.38
Adv.–diff. c1.730.01 / 0.010.12 / 0.120.17 / 0.17
Burgers ν2.230.49 / 0.480.94 / 0.901.10 / 1.05
Burgers flux2.230.09 / 0.090.62 / 0.600.73 / 0.70
KdV δ2.550.29 / 0.290.82 / 0.861.02 / 1.07
KdV flux2.550.11 / 0.110.71 / 0.640.88 / 0.80
Allen–Cahn reaction1.940.09 / 0.090.56 / 0.590.91 / 0.96
Allen–Cahn ε1.941.92 / 1.662.28 / 1.893.70 / 3.07

Direct answer to Q2: it collapses; it does not degrade gracefully. At the standard weights (λ ≥ 0.01, which already weights the residual 400× the data loss) a 10% error turns a 1.5–2.6× gain into a 2–100× loss. With the weight tuned down to 1e-5, validation picks the smallest weight for every wrong coefficient except the Allen–Cahn diffusion term (negligible at ε = 0.002): the constraint is effectively switched off, a 10% error in viscosity or dispersion becomes neutral against the unscaled generic arm, and errors in flux, reaction or linear-family coefficients remain harmful. In no case is the gain of the correct equation retained. Errors below 10% were not tested, and we state that the method assumes coefficients known to well within that.

Q3: Burgers interval at M=16. In the rerun, with M=16 Burgers trajectories, the from-scratch / transfer error ratio is 1.35 [1.18, 1.52] for sequential pre-training on another family, 1.06 [1.00, 1.11] for joint training on another family, and 1.19 [1.06, 1.28] for sequential pre-training on Burgers' own linear part (Table 17 gives intervals for all targets and M). The design differs from the submission (one source family, not four). In this design the negative-transfer result holds for Allen–Cahn and not for Burgers; see our response to Reviewer RLw4, W1.

Reviewer_HGwqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks when known-equation residual penalties improve neural PDE prediction beyond generic regularization and a strong from-scratch baseline. It compares linear and nonlinear one-dimensional PDE families, varies observation density and model width, and studies how under-resolution changes the usefulness of a residual constraint. It also reports negative results for cross-family transfer in a limited source-diversity regime. The topic is valuable, but the main experimental protocol is inconsistent between the text and appendix, and the theoretical closure statement contains a counterexample. These issues prevent a reliable assessment of the central empirical attribution and the scope of the proposed explanation.

Critical strengths

S1. The work targets a meaningful attribution question

Comparing a physics penalty against validation-selected generic regularization, rather than only an unregularized model, is a useful experimental goal (Section 2.2, p. 2). The observation-density and capacity sweeps with heat and advection–diffusion controls could help distinguish finite-capacity effects from behavior specific to the nonlinear families (Figure 1 and Section 3.1, p. 3), provided the actual protocol is clarified.

S2. The study includes boundaries and negative outcomes

The under-resolution experiment tests when a nominally helpful residual becomes harmful (Section 3.2, p. 4; Figure 3, p. 5). Appendix E (p. 16) records the rejected dense-supervision and bandwidth-crossover hypotheses instead of presenting the final result as the originally expected outcome. The text also scopes the transfer findings to low source diversity (pp. 5–6), which is preferable to a universal claim about PDE pretraining.

Critical weaknesses

W1. The central strong-baseline protocol is internally inconsistent

Section 2.1 (p. 2) describes the baseline as autoregressive multi-step pushforward training evaluated by held-out rollout error. However, the configuration table in Appendix D (p. 15) labels both the headline and capacity experiments as K=1 and one-step training, and Appendix F (p. 16) explicitly labels the reported quantity held-out one-step relative L2 error. The full-supervision numbers there match the central ratios in Section 3.1 (p. 3), so the discrepancy concerns the main evidence, not an unrelated diagnostic. Until it is clear which runs and metrics produced Figure 1, the claim of improvement beyond the stated strong multi-step baseline is not established.

W2. The theoretical dichotomies are stronger than the derivations support

Proposition 1 (Appendix C.1, p. 13) states that the quadratic closure term vanishes if and only if Q_N u=0. Consider u(x)=sin(kx), with k above the resolved cutoff. Then P_N u=0 while Q_N u is nonzero. For Burgers, the derivative of u squared contains only frequency 2k; both it and the linear viscous term vanish after resolved projection. Consequently F_N(P_N u)−P_N F(u)=0 despite a nonzero unresolved field. This refutes the stated necessity, though not the possibility of harmful closure in the experiments. Proposition 2 (p. 14) likewise asserts an unconditional equivalence between zero measured non-redundant content and linear F without specifying the input distribution or conditions on the finite-time solution map. These issues limit the claimed general mechanism.

Overall assessment

2 / 4. Weak rejection. The scientific question and several reported phenomena are worthwhile, but I cannot currently verify the central claim against the advertised baseline, and the theory requires correction and narrower conditions. This recommendation does not imply that every empirical observation is invalid. A reconciled protocol, results under the actual multi-step baseline and corrected statements could change the assessment.

Questions for authors

Q1. Which training horizon and evaluation metric produced the headline and capacity results in Figure 1 (p. 3)? Please reconcile Section 2.1 (p. 2), Appendix D's K=1 one-step rows (p. 15), and Appendix F's one-step metric (p. 16), identifying the actual configurations and logs. Do the central ratios persist with the stated multi-step pushforward baseline and rollout evaluation?

Q2. How should Proposition 1 and Corollary 1 (p. 13) be corrected in light of the high-frequency sine counterexample? Which conclusions require only that nonlinear closure can be nonzero, rather than that it must be nonzero whenever unresolved modes exist? Please separate the valid bound and empirical failure regime from the false equivalence.

Q3. Under what assumptions on the input ensemble and the finite-time map is Proposition 2 (p. 14) intended to hold? The reported correlation across five families can support an empirical predictor within this suite; what additional argument justifies the stated if-and-only-if result, and how much of the capacity interpretation depends on it?

1 comment

Response to Reviewer HGwq

Thank you for the careful reading. Both the protocol inconsistency and the counterexample are correct, and both are fixed.

W1 / Q1: the protocol inconsistency. Appendices D and F of the submission were the accurate description: the submitted headline and capacity results (Fig. 1) were trained with K=1 and reported one-step error, and the multi-step description in Section 2.1 did not apply to them. In the revision every experiment is rerun from one code base under both protocols. Appendix D lists K, metric and budget for each experiment, no result mixes the two, and the commands and per-seed errors of these runs are in the supplementary material. At ρ=1, width 32, three dataset draws × five seeds:

K=1, one-step, vs generic gridK=4, rollout, vs generic gridK=4, rollout, vs scale-matchedK=4, rollout, scale-invariant Adam, vs data-only
Heat3.94 [3.41, 4.56]3.07 [2.79, 3.42]1.09 [0.94, 1.27]1.00 [0.98, 1.01]
Adv.–diff.2.04 [1.92, 2.16]1.71 [1.56, 1.90]1.17 [1.08, 1.30]1.01 [1.00, 1.03]
Burgers2.31 [2.08, 2.51]1.50 [1.40, 1.60]1.05 [1.01, 1.09]0.99 [0.98, 1.00]
KdV2.15 [2.07, 2.26]1.78 [1.59, 2.01]1.04 [1.01, 1.07]1.01 [1.00, 1.01]
Allen–Cahn4.04 [3.18, 5.02]1.43 [1.25, 1.69]1.04 [0.98, 1.08]0.96 [0.90, 1.01]

Direct answer to Q1: yes. The advantage over the stated baselines persists under multi-step training and the rollout metric, in every family, and is smaller than the one-step figure (Burgers: 3.4× submitted, 2.3× in the one-step rerun, 1.5× under K=4 and rollout). The capacity sweep under the same protocol reproduces the trend for Burgers (1.3 → 2.1), KdV (1.2 → 2.0) and advection–diffusion (1.6 → 1.06 [0.80, 1.27]); heat stays between 1.9 and 2.9 and does not collapse.

The last two columns are the new loss-scale control (general response). They show that at full supervision this advantage is matched by rescaling the data loss, so we now locate the equation-specific advantage at sparse labels, where it is 2.6–11× at ρ=0.03 and, for Burgers and KdV, grows with width.

W2 / Q2: Proposition 1 (Proposition 2 in the revision). Your counterexample is correct and is now part of the statement. Corrected:

  • τ_J(u) = F_J(P_J u) − P_J F(u) = +½ ∂ₓ P_J[(Q_J u)(u + P_J u)] for the quadratic flux (the submitted sign was wrong);
  • ‖τ_J(u)‖{H⁻¹} ≤ ½ ‖Q_J u‖{L²} ‖u + P_J u‖_{L^∞};
  • a non-zero tail is necessary but not sufficient: for u = sin(kx), k > J, the tail is non-zero and τ_J(u) = 0, whereas for u = sin(Jx) + sin((J+1)x), τ_J(u) = −½ sin x. Hence τ_J ≡ 0 as a map iff the nonlinear flux is absent, and no pointwise equivalence holds.

Corollary 1 now bounds the residual of the projected truth by the endpoint-averaged closure plus an explicit h²/12 · sup‖v‴‖ time-discretization term.

On what depends on what: the resolution-boundary conclusion needs only that the closure can be non-zero, together with the bound. Its size on our ensembles is measured, not inferred (Fig. 16 checks the identity and the corollary numerically), and the failure regime is measured directly (Fig. 5, Tables 12–13): under-resolved, the penalized model reproduces the error of the Crank–Nicolson solver of the unclosed coarse system. Nothing in the revision uses the false equivalence.

Q3: Proposition 2 (Proposition 3 in the revision). The "iff" was false as stated; ρ_nr depends on the ensemble. Corrected: (i) linear F_J implies ρ_nr = 0 for every ensemble and step; (ii) if the ensemble's support contains an open set of the resolved space and ρ_nr = 0 for all h in an interval (0, h₀), then F_J is linear; (iii) neither hypothesis can be dropped (Allen–Cahn on the constants ±a has ρ_nr = 0). We keep ρ_nr only as a descriptive statistic (Appendix C.2) and have removed the five-family correlation. None of the capacity interpretation depends on it now: redundancy is defined against the labels (Proposition 1), which the experiments test directly. On notation: we write τ_J for J retained modes where the submission wrote τ_N.

Reviewer_RLw4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The submission compares a Crank–Nicolson PDE-residual penalty against the best of a small generic-regularizer grid on five 1D periodic families. It claims:

  1. The residual wins at every observation density.
  2. Across a width sweep, the advantage persists only for nonlinear operators. They attribute this to non-redundancy: ρ_nr > 0 iff the operator is nonlinear.
  3. The advantage reverses under under-resolution, explained by the closure term τ_N.
  4. Cross-family transfer gives negative transfer, which they attribute to "basin compatibility" rather than source relevance.
Critical strengths
  • Matched-budget comparisons, pre-registration, and falsified hypotheses reported honestly.
  • The under-resolution failure is correctly identified and verified: ‖τ_N‖ ∝ ‖Q_N u‖, matching the predicted refinement rate. The authors credit Tadmor and the LES literature.
Critical weaknesses
  1. The transfer conclusion conflates the kind of relationship with relevance. The "pde_related" source is other PDE families used through sequential pretrain-then-finetune or in-context conditioning. That is the weakest form of relevance, delivered by the protocol most prone to interference. It omits the most relevant source class: mechanism decompositions of the target itself (heat or viscous-only for Burgers, Burgers-type advection for KdV) trained jointly with the target and given separate heads. Ma et al. (ICLR 2026) show this class helps. Their D.6 ablation also shows that the wrong decomposition (reaction-only) hurts by up to 64%, which is direct evidence that relevance matters once it is defined structurally. The "basin compatibility" story may therefore be an artifact of the sequential protocol. Tripling the fine-tuning epochs does not rule this out; a joint-training arm would. The relevance gradient needs:

    • a "target decomposition" source, and
    • joint training versus pretrain-finetune as an explicit factor.
  2. Missing related work. Ma et al. (2026) inject physics knowledge at the data level instead of the loss level. They jointly train a shared-backbone neural operator on the target PDE plus cheap simulations of its decomposed basic form (diffusion-only for FitzHugh–Nagumo, the inviscid advective form for NS, the linear form for KS). This improves data efficiency, rollout, and OOD generalization on 1D/2D/3D problems and ScalarFlow. That is the obvious second way to bring physics into a neural operator, and the submission's framing ("when known physics helps neural PDE models") requires comparing against it, or at least positioning relative to it.

  3. The "linear physics is redundant at capacity" claim is tested on the wrong systems. The ρ_nr argument is only tested where the whole operator is linear (heat, advection–diffusion). The claim that matters is about partial physics inside a nonlinear target, and there Ma et al. point the other way: linear basic forms (diffusion, linear KS) carry non-redundant value for nonlinear targets, albeit at 2–16 trajectories. The decisive and cheap experiment is a split residual on Burgers, KdV, and Allen–Cahn: penalize only the linear part L(u) versus only the nonlinear part B(u). If Prop. 2's reading is right, the L-only residual should go to parity as width grows while the B-only residual keeps the advantage. Without this, Prop. 2 is close to a tautology (linear translation-invariant maps are exactly the Fourier multipliers), and the n=5 Pearson correlation adds nothing.

  4. The two papers disagree mechanistically about resolution. Here, enforcing an equation on coarse fields imports τ_N as irreducible bias. In Ma et al., data from a different PDE (the basic form), and data from coarsened simulations (App. D.1), both help. The likely reconciliation is that a separate output head absorbs the mismatch between the auxiliary dynamics and the target, while a residual penalty acting on the shared output cannot. That suggests the natural remedy: apply the residual to an auxiliary head, so the shared trunk receives the physics signal without being forced to satisfy a biased constraint. That is a stronger fix than "resolve the grid or model the closure," which the paper suggests but never tests.

. The generic baselines are too weak. Weight decay at two values plus a single input-noise level does not represent label-free priors. Physics-motivated data augmentation, namely Lie-point symmetry augmentation (Brandstetter et al. 2022) and cost-matched coarse-simulation mixing, was used as a baseline in Ma et al. and is the right comparison class. Also report the error of the CN integrator alone: at fixed coefficients on a resolved grid, the penalized model may simply be relearning the CN step.

Questions for authors
  1. Does the negative-transfer result survive joint training on decompositions of the target with separate heads?
  2. Under the split residual (L-only versus B-only), which part carries the capacity-robust advantage?
  3. Does Fig. 1 hold with K-step training, rollout error, and a sweep over trajectory count?
  4. Does a residual applied to an auxiliary head remain beneficial under under-resolution?
3 comments

Response to Reviewer RLw4 (1/2)

Thank you for the concrete suggestions. The split residual, the auxiliary-head residual, the source × delivery transfer design and the Crank–Nicolson baseline in the revision are the experiments you proposed, and the split residual is what led us to the loss-scale control described in the general response.

W1 / Q1: transfer with target decompositions and joint training. We ran the factorial: auxiliary source (the target's own linear part; another family; for Allen–Cahn also reaction-only) × delivery (joint with separate heads; sequential pre-train then fine-tune) × M ∈ {2, 4, 8, 16} target trajectories. Ratios are from-scratch error / arm error (Table 17 has all rows with intervals, including KdV).

TargetSource, deliveryM=2M=4M=8M=16
Burgerslinear part, joint1.031.231.070.97
Burgerslinear part, sequential1.311.221.171.19
Burgersother family, sequential1.441.361.331.35
Allen–Cahnlinear part, joint0.870.700.680.74
Allen–Cahnlinear part, sequential1.921.972.071.61
Allen–Cahnreaction-only, sequential4.192.842.371.81
Allen–Cahnother family, joint0.870.500.290.22

Direct answer to Q1: the negative-transfer result survives only for cross-family sources on Allen–Cahn (sequential 0.69–0.89, joint 0.22–0.87). With decompositions of the target it does not: the target's own linear part helps all three targets when delivered sequentially (1.1–1.4 for Burgers and KdV, 1.6–2.1 for Allen–Cahn), which agrees with Ma et al. in the low-data regime they study. So relevance defined structurally matters, as you argued, and our claim was too broad; it is now scoped to Allen–Cahn. Joint training is not the missing ingredient in our setting: it beats sequential in 2 of 16 Burgers/KdV cells, and for Allen–Cahn it is 2–3× worse. One difference from their ablation: for Allen–Cahn the reaction-only source helps (1.8–4.2 sequential). We have dropped the basin-compatibility interpretation and draw no mechanistic conclusion about transfer. The in-context conditioning experiment is retained unchanged in Appendix A.4 and nothing in the main text rests on it.

W2: Ma et al. (ICLR 2026). Now cited and discussed (Sections 3.5 and 5). Their route injects physics through data simulated from a simplified form of the PDE; ours through a soft residual on the full equation. The table above is our closest comparison. We did not reproduce their cost-matched protocol and say so in the limitations.

Response to Reviewer RLw4 (2/2)

W3 / Q2: split residual. We ran it in two forms across widths 8–128 on Burgers, KdV and Allen–Cahn. In the well-posed form (penalize only L, or only B, with the other part supplied from the labels so the truth satisfies the constraint) both halves give the same error as the full residual, and the same as the rescaled data loss: all ratios against the scale-matched baseline are 0.87–1.14. In the literal form (an incomplete equation) both are harmful (0.00–0.36). Direct answer to Q2: at full supervision neither part carries a capacity-robust advantage; the loss scale does. Under sparse labels a capacity-robust advantage is present for Burgers and KdV with the full residual (1.7 → 3.3 and 1.7 → 4.0 from width 8 to 128 at ρ=0.12). We also agree with your remark on Proposition 2: ρ_nr is now a descriptive statistic with a corrected statement (Appendix C, Proposition 3), the five-point correlation is removed, and the capacity reading no longer rests on it.

W4 / Q4: residual on an auxiliary head. We ran it on the Burgers and KdV resolution sweeps, with the prediction head trained on data only. With λ ≥ 0.01 (the residual weighted at least 400× the data loss) the arm is worse than data-only training. With λ down to 1e-5 it is neutral at full supervision (1.0–1.1 against the generic arm) and gives a small gain at ρ=0.12 on every grid (1.1–1.3). The residual on the prediction itself gives 1.5–4.0 in the same cells. Direct answer to Q4: the auxiliary head passes a little of the physics signal through the trunk at sparse sensors, but it does not rescue the under-resolved full-supervision case.

Baselines.

  • Crank–Nicolson integrator alone. At N=64 it is 20× (Burgers, advection–diffusion) to 5000× (Allen–Cahn) more accurate than the best learned model, one-step and over 39 steps, so on a resolved grid the penalized models are optimization-limited and not at the solver's floor. On under-resolved grids your reading is exactly right: the penalized model reproduces the error of the Crank–Nicolson solver of the unclosed coarse system (Burgers, N=24: 1.49e-3 vs 1.41e-3). We now say explicitly that the paper is about what a surrogate learns from limited labels, not about replacing a solver when the equation is known and resolved.
  • Symmetry augmentation (sign/reflection symmetries; Galilean boosts for Burgers and KdV). It improves on the generic grid by 1.0–2.2× and is weaker than loss rescaling in 48 of 60 cells. Under the scale-invariant optimizer it is the only arm that improves on data-only training at full supervision (1.10–1.33). The residual beats it by 2.5–6× at ρ=0.03 (heat, advection–diffusion, Burgers, Allen–Cahn; Galilean boosts need full-field labels, so KdV has no usable augmentation at sparse sensors).
  • Loss-scale-matched data-only training and a scale-invariant optimizer. Added throughout (general response).
  • Cost-matched coarse-simulation mixing. Not run; listed as a limitation.

Q3: K-step training, rollout metric, trajectory count. All done. With K=4 and rollout error the residual beats the generic grid in every family at every density (1.4–3.1× at ρ=1), and the equation-specific part of that gain is at sparse labels (general response, part 2). Trajectory count: at ρ=1 the ratio against the scale-matched baseline stays within 0.76–1.19 from 2 to 64 trajectories; at ρ=0.12 the gain is largest with few trajectories (Burgers 7.2× with 2 trajectories, 2.2× with 64).

Reviewer_RLw4OpenReview ↗

all questions addressed

All my questions were addressed and hence I will improve the score.

Comments on the paper

3 comments

General response (1/3)

We thank the reviewers. The revision keeps the paper's question, its design and the conclusion in its title (known physics helps only when it is non-redundant and resolvable), and makes both conditions precise. It follows the structure of the submission; the additions are the experiments and corrections the reviews asked for.

What the reviews led us to do.

  • Protocol (HGwq W1). Every experiment is now run with the multi-step training and rollout metric that Section 2.1 describes, and also one-step, from one code base. Appendix D lists K, metric and budget for each experiment.
  • Stronger baselines (RLw4). We added exact symmetry augmentation, the Crank–Nicolson integrator alone, and data-only training at the loss scale the residual induces.
  • Requested experiments. Split residual and auxiliary-head residual (RLw4 W3, W4), source × delivery transfer with target decompositions and joint training (RLw4 W1), coefficient misspecification (4Xn3 W3), trajectory-count sweep (RLw4 Q3).
  • Theory (HGwq W2, Q2, Q3; 4Xn3 Q1). Both propositions are corrected, including the sign.
  • Presentation (4Xn3 W1, W2). The closure term is stated as classical in the main text; Appendix E gives each pre-registered hypothesis with its prediction and outcome, and the record itself is now in the supplementary material.
  • Scope. The geometry experiment from the appendix is now a two-dimensional test of the main result on two backbones (Section 3.6).
  • Reproducibility. The supplementary material now contains the code with its numerical self-tests and per-seed results.

What is reproduced under the corrected protocol.

  • The residual beats the best generic regularizer in every family at every observation density, including full supervision: 1.4–3.1× with K=4 and rollout error, 2.0–4.0× one-step (Fig. 1a).
  • Against the generic arm the capacity sweep again separates the families: from width 8 to 128 the advantage grows from 1.3 to 2.1 for Burgers and 1.2 to 2.0 for KdV, and falls to parity for advection–diffusion, 1.06 [0.80, 1.27] (Fig. 3a).
  • The resolution boundary: under-resolved, the residual-penalized model is worse than the data-only model, and we now show exactly why: it reproduces the error of the Crank–Nicolson solver of the unclosed coarse system (Burgers, N=24: 1.49e-3 vs 1.41e-3).
  • Multi-step training lowers the baseline's rollout error by 27–64%; the local CNN beats the FNO on unseen obstacle layouts (by 4×).
  • The outcomes of the pre-registered hypotheses, against data-only training and against generic regularizers: H-flip-1 falsified at the dense end, H-flip-2 supported, H-flip-3 falsified, H-flip-4 untestable.

General response (2/3)

What changes in the claims. One new control, added because the baselines were judged too weak, changes how three results are read. On fully observed targets the Crank–Nicolson residual of a prediction is, to first order, (I/h − DF/2) applied to the prediction error (new Proposition 1), so the penalty is the data loss multiplied by about λ/h² and is redundant with the labels. Adam with fixed ε is not invariant to that scale. A data-only baseline with the data loss multiplied by λ/h² ("scale-matched") therefore separates the equation from the loss scale, and a scale-invariant optimizer (clipping off, ε = 1e-12) tests the mechanism.

  1. Where the advantage is non-redundant. At full supervision the scale-matched baseline matches the residual: 0.98–1.07 for K=1 and 1.03–1.17 for K=4, 0.96–1.01 under the scale-invariant optimizer, at every width from 8 to 128 and from 2 to 64 trajectories (and 0.93–1.18 / 0.92–1.04 in a rerun with exact float32 products, Appendix D.1). The advantage over generic regularizers at ρ=1 is real, but it is not specific to the equation. The equation-specific advantage is the part that grows as labels become sparse (three dataset draws × five seeds; all intervals exclude 1 at ρ ≤ 0.25):
Familyρ=0.030.060.120.250.51.0
Heat2.611.691.471.291.141.09
Adv.–diff.3.102.091.481.301.191.17
Burgers11.343.902.071.501.161.05
KdV9.553.842.321.591.211.04
Allen–Cahn8.004.022.791.541.161.04

The gain does not rely on the dense inputs of our training windows. Trained from the initial condition only (one dense state per trajectory, a 39-step unroll, sparse sensors afterwards; Appendix A.2), the ratio is 2.0 / 2.5 / 2.4 / 1.7 at ρ=0.12 and 6.5 / 19.6 / 16.1 / 5.5 at ρ=0.03 for advection–diffusion, Burgers, KdV and Allen–Cahn. Heat is the exception: 1.4 at ρ=0.03, and at ρ=0.12 the scale-matched baseline is better (0.44). At full supervision in that setting the scale-matched baseline is as good as the residual or better in every family (0.22–0.94). The residual arm selects the largest weight in its grid there, so the cases where it trails may reflect tuning.

This is the sense of "non-redundant" in the title, now with a definition and a test. It also changes the reading of one pre-registered hypothesis. H-flip-1 predicted no advantage at full supervision; it is falsified against data-only and generic-regularizer baselines, as we reported, and the dense end is inert once the loss scale is controlled. Appendix E gives both, and marks the second as post hoc. 2. Capacity. The growth with width at ρ=1 is matched by the scale-matched baseline (ratio 1.00 at the median, Fig. 3b). A capacity-robust advantage does survive the control, at sparse labels and for the two advective families: at ρ=0.12 it grows from 1.7 to 3.3 for Burgers and 1.7 to 4.0 for KdV (Fig. 3c). It is flat for the linear families and non-monotone for Allen–Cahn, so we state it for Burgers and KdV and not as a linear/nonlinear dichotomy. 3. Transfer. With the source × delivery design Reviewer RLw4 asked for, cross-family data do not beat from-scratch training for Allen–Cahn (0.2–0.9) but give Burgers and KdV a modest gain (1.1–1.5). We no longer interpret transfer through basin compatibility (Section 3.5).

General response (3/3)

New: two dimensions, obstacles, a second backbone (Section 3.6). The comparison of point 1 on 2D advection–diffusion around random obstacles, unseen layouts at test time, for an FNO and the same network with 5×5 periodic convolutions:

Sensor cells (of 4096)409625664164
FNO0.99 [0.98, 1.01]1.362.213.986.60
CNN1.02 [0.65, 1.69]1.53 [0.95, 2.46]3.656.989.82

(Residual vs scale-matched baseline, one dataset draw × five seeds; intervals exclude 1 wherever none is shown.) Under the scale-invariant optimizer the data-loss scales coincide on both backbones (1.00). For the FNO all arms then coincide at full supervision (1.00–1.01) and the 16-sensor gain stays at 4.3. For the CNN the scale-invariant optimizer is stable only at a lower learning rate (3e-4; it diverges at 1e-3); there the residual keeps a small lead at full supervision, 1.11 [1.10, 1.13], which the loss scale does not explain, against 7.7 [7.3, 8.4] at 16 sensors. The equation is resolved by construction here, so this does not test the resolution boundary.

Where things are.

SubmissionRevision
§3.1 residual vs generic priors, Fig. 1§3.1 (same comparison, K=4 and K=1; loss-scale control added), Figs. 1–3
Fig. 2 Burgers rolloutFig. 4
§3.2 resolution boundary, Prop. 1§3.3, Prop. 2 and Cor. 1 corrected
(none)§3.4 coefficient misspecification (new)
§3.3 and App. A.2–A.3 transfer§3.5 (source × delivery); in-context conditioning retained unchanged in App. A.4
App. A.4 geometry§3.6, rerun and extended
App. C.2 ρ_nr, Prop. 2App. C, corrected (Prop. 3)
App. D configuration tableApp. D, with K and metric for every experiment
App. E pre-registrationApp. E, each hypothesis with decision rule and outcome; record in the supplement
App. F per-condition tablesApp. F

Limitations we state. The main campaign uses one backbone (FNO) and 1D periodic problems; the 2D experiment covers one linear equation, one dataset draw and no under-resolved or misspecified case. Five seeds throughout (three in the short-budget check), a single dataset draw outside the 1D observation-density experiment, and no cost-matched coarse-simulation baseline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.