acceptodds
Under review as a conference paper at ICLR 2027

Physical Robustness under Equation Corruption

Abstract

Robustness in machine learning usually means invariance, which is evaluated by perturbing an input while the correct label stays fixed. This evaluation setup fails for PDE surrogates: changing a coefficient or a boundary condition moves the physical solution, and changing the mesh or the sensor changes how the solution is measured. Physical robustness should instead be evaluated by how well the prediction tracks changes in the correct output, in both direction and magnitude. Nominal accuracy does not guarantee response fidelity: under a Burgers viscosity shift, an FNO with held-out field error below 2% still mispredicted the response by half the true response magnitude. To make physical robustness measurable and trainable, we decompose changed-target error into nominal error and sensitivity mismatch, introduce diagnostics that grade the response at held-out severities, and evaluate three training interventions on six specification-shift tasks. Our evaluation identifies effective training interventions for different perturbations, PDEs, and backbones: response alignment cut the response mismatch on public multi-viscosity Burgers data roughly sixfold and led on coefficient and boundary shifts, whereas stress augmentation cut evaluation-mesh and initial-condition error by 21.5% and 16.2% over a matched changed-label baseline. Adversarial training, a standard approach to robustness in machine learning, instead increased evaluation-mesh and initial-condition error by roughly 28%. Based on these results, we argue that a surrogate should be judged by how faithfully the prediction tracks the governing system, not by how little the prediction moves when the physics changes.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Scientific machine learning, Neural operators, Operator learning, PDE surrogates, Physical robustness, Sensitivity analysis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_w7ipOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Dear Area Chairs,

I do not believe I have sufficient expertise to review this paper, as I do not have any expertise in the field of Neural PDEs / PDE surrogates. I would greatly appreciate guidance on how to proceed with this review.

Critical strengths

N/A

Critical weaknesses

N/A

Questions for authors

N/A

1 comment

Thank you for your assessment. We have restructured the introduction and main exposition so that readers without prior familiarity with neural PDEs can understand the motivation, core idea, and practical value before encountering the mathematical framework. We begin with a concrete viscosity example, introduce the necessary terminology as it arises, and use that example to guide the explanation of the problem and our training method.

Let us briefly introduce our work in plain terms. A PDE surrogate is a neural network used to approximate a physical simulation. Changing a condition such as fluid viscosity changes the correct solution. Our goal is to teach the model how that solution changes. PhySA therefore trains the predicted difference between two conditions to match the true difference, alongside the usual training on the solutions themselves. It measures the mismatch relative to the size of the true change, so that small physical responses receive appropriate attention.

The theory identifies conditions under which learning these changes, together with an accurate prediction at the reference condition, also limits prediction error between the physical settings used in training. The practical motivation is to support tasks that compare physical settings or infer an unknown parameter from observations.

Reviewer_YciBOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper defines the robustness property of neural PDE surrogates, e.g., Fourier Neural Operators (FNO). Model robustness is formulated based on the expected supremum of the L2L_2 error of the predicted solution compared to the true solution under perturbed configurations, denoted as the physical robust risk. Because this supremum cannot be computed directly, it is approximated by evaluating a grid of possible values in the perturbation set, denoted as the severity-grid estimator.

The paper provide theoretical justifications for why nominal accuracy (as used in prior PDE surrogate evaluations) is insufficient in the sense that small nominal error does not guarantee a correct physical response, whereas a model with a larger nominal error may still correctly capture changes in the solution.

The paper extend the notion of physical robust risk by proposing three metrics: EAF, SAE, and SNRE. The experimental results show that models achieving low L2L_2 error can still exhibit high Sensitivity Alignment Error (SAE).

Critical strengths
  1. Evaluating neural operator architectures and PDE foundation models are important topics worth investigating.
Critical weaknesses
  1. No evaluation of SOTA: The paper does not mention Transolver++, which is among the current state of the art, and fails to examine FNO variants from 2024 onward (e.g., CNO, Poseidon, Transolver).

  2. No code provided: The authors do not provide code to reproduce their results.

  3. Writing quality, repetition, and phrasing issues:

    • The introduction is overly repetitive, dwelling excessively on the problem without clearly explaining the proposed solution. It is difficult to understand the actual solution when reading from line 139 to the end of the introduction.
    • The text relies heavily on constant contrast framing and choppy sentences, making it difficult to read smoothly.
    • The paper exhibits noticeable stylistic quirks and unpolished phrasing:
      • Overuse of colons (:) as substitutes for em-dashes.
      • Peculiar or vague word choices (e.g., "training interventions", "governing system", ... why just say it PDEs).
      • Informal language in formal technical contexts (e.g., line 182: "an evaluation mesh must be fine enough").
  4. Unclear mathematical formulation (line 221): In the equation at line 221, what does o(∥δ∥Z)o(\|\delta\|_Z) mean? Why does the map need to be Fréchet differentiable, and why is it necessary to introduce a Banach chart around zz?

  5. Selective reporting of proposed metrics: The paper proposes three metrics (EAF, SAE, and SNRE), but only reports results for a single metric (SAE) in the main text.

  6. Metric not translated into an actionable method: Introducing a new metric is one thing, but this metric is not turned into an actionable solution. Why wasn't the metric used to design a better loss function that improves the quality of neural PDE surrogates and establishes a new state of the art for neural PDEs?

Questions for authors

No question, as I will not change my score. Paper need substantial revision, which is not possible to complete during the rebuttal period.

10 comments

Thank you for the detailed comments. The submitted paper introduces a response-training objective, connects its attained loss to an error bound, and demonstrates improvements in PDE surrogate predictions. Section 3 defines PhySA, which trains the change between two predictions to match the physical change between their reference solutions. Theorem 1 and Corollary 1 connect this training target to accuracy throughout the interval between supervised conditions under a curvature bound. We have also completed evaluations of all four requested model families: CNO, Poseidon, Transolver, and Transolver++, including fine-tuning the official pretrained Poseidon-T. On public Burgers, PhySA improves held-out field error and SAE in every matched seed for each family. The additional experiments demonstrate benefits beyond tuned response and sensitivity objectives, including better physical-parameter recovery at comparable field accuracy. The responses below identify the submitted method and evidence, present the additional evaluations, and specify the reproducibility materials and presentation changes.

1. Requested SOTA evaluations: CNO, Poseidon, Transolver, and Transolver++

PhySA improves the requested contemporary models under matched comparisons that isolate the contribution of response supervision. We evaluate CNO, Transolver, and Transolver++ task adaptations, together with fine-tuning of the official pretrained Poseidon-T. Each baseline already learns from the correct solutions at both nominal and changed physical conditions; adding PhySA explicitly supervises the change between those solutions. Holding the backbone and supervision data fixed tests the benefit of this training objective on each requested architecture. The Burgers experiments use the same input information, reference trajectories, severity splits, and evaluation metrics within each matched comparison. Both sides receive 600 training epochs, since validation errors continued to improve beyond the original 150-epoch budget. All entries below are five-seed means; each cell shows changed-label baseline → baseline with PhySA.

Backbone and task, 600 epochsHeld-out relative L2L^2 ↓SAE ↓EAFSNRE ↓
CNO adaptation, public Burgers0.2772 → 0.135014.630 → 1.71414.866 → 2.10514.781 → 3.425
Transolver adaptation, public Burgers0.1106 → 0.07583.082 → 1.4433.548 → 2.0082.512 → 1.284
Transolver++ adaptation, public Burgers0.1096 → 0.09273.238 → 2.0513.665 → 2.5452.867 → 1.918
Pretrained Poseidon-T, public Burgers0.0962 → 0.07261.982 → 0.7362.451 → 1.3271.995 → 0.840
Pretrained Poseidon-T, narrow Burgers viscosity shifts0.0093 → 0.00580.188 → 0.0471.086 → 1.0070.377 → 0.295

The CNO and attention rows use implementations adapted to the one-dimensional space–time prediction task. The CNO implementation uses band-limited resampling; the attention implementations use structured-grid slices, with adaptive temperatures and Gumbel slicing for Transolver++. The latter adaptation uses deterministic evaluation. Poseidon-T loads the official pretrained trunk and uses its fine-tuning optimizer; an input/output adapter maps the Burgers space–time grid to the model's resolution and back, with losses and evaluation computed on the original trajectory grid. The experiment documentation now specifies these adapters and settings in Appendix M.5 and the supplementary code.

For CNO and the attention models, PhySA is selected using nominal/seen validation data within the 600-epoch runs. Poseidon-T retains the floor and weight selected at 150 epochs when extending training to 600 epochs. Held-out results are not used for selection.

On public Burgers, field error and SAE improve in all five seeds for each of the four requested model families. The pretrained Poseidon result further demonstrates the method's value at high forward accuracy: the baseline reaches 0.93% field error on the narrow viscosity task with SAE 0.188, and response alignment reduces these to 0.58% and 0.047. At the same 600-epoch fine-tuning budget, PhySA improves both the prediction accuracy and physical-response fidelity of a pretrained PDE foundation model.

The requested Transolver families also improve in their native two-dimensional setup. We additionally evaluate the upstream Transolver (https://github.com/thuml/Transolver) and Transolver++ (https://github.com/thuml/Transolver_plus) architectures on Darcy forcing shifts, using eight layers, width 128, 64 slices, positional features, and 500 epochs. The paired control and PhySA use identical field supervision, including the upstream gradient term. The response coefficient 0.1 is selected on validation data. Results are mean ± standard deviation over five seeds.

Native Darcy implementationHeld-out relative L2L^2 ↓SAE ↓EAFSNRE ↓
Transolver, paired field control0.0236 ± 0.00120.0762 ± 0.00460.98850.1659
Transolver + PhySA0.0227 ± 0.00230.0707 ± 0.00470.98670.1590
Transolver++, paired field control0.0571 ± 0.00170.1712 ± 0.00290.97170.4064
Transolver++ + PhySA0.0519 ± 0.00690.1495 ± 0.01660.98160.3694

Field error and SAE improve in four of five seeds for each native architecture, with mean improvements shown above. Transolver++ retains its stochastic slicing in this implementation. Evaluation uses common random draws for matched comparisons; increasing from eight to 64 independent draws preserves the conclusion, with field error 0.05712 → 0.05178 and SAE 0.17103 → 0.14916. The native two-dimensional checks extend the evidence beyond the Burgers task adaptations.

The complete task tables also quantify settings that emphasize response accuracy. On narrow viscosity shifts, the Transolver++ adaptation reduces SAE from 0.578 to 0.356, with field error changing from 0.0363 to 0.0375. Reporting both quantities distinguishes the effect at this setting from the simultaneous field and response improvements on public Burgers and native Darcy. The joint improvements across all four requested families establish the value of response supervision on contemporary architectures, including an already accurate pretrained model. The revision brings the CNO and Poseidon discussion into the main text, adds Transolver and Transolver++, and presents the full task tables with their implementation and validation-selection protocols in Section 4.4 and Appendix M.5.

2. Reproducible code and experiment records

A reproducibility package covering both the submitted and additional experiments has been prepared. It contains the experiment source, data-generation and public-data preparation commands, model and objective implementations, training configurations, seed/severity records, and analysis scripts. The diagnostic implementation is also printed in Appendix F, Listing 1 of the submitted manuscript.

The package records the nominal/seen/held-out splits, checkpoint and hyperparameter selection criteria, training and diagnostic floors, public-data viscosity convention, upstream model provenance, and stochastic-evaluation settings. Machine-readable completed-run results allow the reported tables to be regenerated without retraining. The anonymized supplementary package accompanies the revision and documents the commands needed to reproduce the experiments.

3. Introduction, repetition, terminology, and presentation

The central contribution is a trainable criterion for predicting physical changes, supported by an error bound and measured improvements. The revised introduction now states this connection explicitly:

We make this requirement actionable with physical sensitivity alignment (PhySA). Given solutions at a reference and a changed condition, PhySA trains the difference between the two predictions to match the difference between their reference solutions. The mismatch is scaled by the physical-response magnitude, so a small physical change receives supervision appropriate to its size. This objective uses paired solution fields without parameter-derivative labels. Our anchor theorem relates the attained response loss to prediction accuracy between supervised conditions under a curvature bound. Controlled experiments test the value of the response term, its normalization, and its effect on parameter recovery.

The revised introduction presents the method, guarantee, and empirical benefit before the detailed perturbation taxonomy. It uses “PDE specification” and “training objectives” consistently, consolidates repeated explanations, and connects the motivation, method, and evidence in continuous prose.

The term “PDE specification” has a precise role in the formulation. It includes the domain, equation, coefficients, forcing, initial and boundary data, physical parameters, and discretization and observation maps. A robustness claim must specify which of these components changes and how the resulting outputs are compared. This scope lets the framework treat changes in the physical solution and changes in its representation or measurement within one evaluation protocol. We have made this definition explicit at first use.

For the phrase “an evaluation mesh must be fine enough,” we now state the actual requirement: the reference and comparison discretizations must resolve the evaluated field to a specified tolerance in the comparison norm. The relevant grid, solver settings, and resolution checks are given with the experimental protocol. This ties the robustness evaluation to a resolved physical reference and a common comparison norm, with the same admissibility conditions for every model.

4. Meaning of the remainder, Fréchet differentiability, and the Banach chart

The formulation separates error at a reference condition from error in the model's response to a physical change. Let e(z)=A(z)−S(z)e(z)=\mathcal A(z)-\mathcal S(z) denote prediction error. In the local expansion, o(∣δ∣Z)o(\\|\delta\\|_{\mathcal Z}) is the remainder after the first-order term. If rz(δ)r_z(\delta) denotes that remainder, then

\frac{\\|r_z(\delta)\\|_{\mathcal Y}} {\\|\delta\\|_{\mathcal Z}}=0.$$ Thus, the remainder becomes negligible relative to the perturbation size. Fréchet differentiability supplies the expansion $$e(z+\delta) =e(z)+\big(D\mathcal A(z)-D\mathcal S(z)\big)[\delta] +r_z(\delta).$$ In words, the error after a small physical change consists of the existing error plus the mismatch in how the model and the true solution respond, up to the stated remainder. Fréchet differentiability is the assumption that justifies this first-order expansion. For scalar viscosity, it is ordinary differentiation; the general formulation also covers function-valued coefficients, initial conditions, and boundary data. A Banach chart specifies local coordinates in which admissible increments and their derivatives are defined. For example, log-viscosity provides a coordinate for varying a positive viscosity. These choices let the same analysis cover scalar and functional perturbations along admissible continuous paths. Discrete mesh changes are evaluated by finite comparisons in a common output space. Theorem 1 and Corollary 1 show how supervising finitely many physical responses controls predictions throughout the interval they cover. The anchors are the physical conditions at which responses are supervised, such as several training viscosities. For a twice continuously differentiable error path with curvature bounded by $M$, nominal error at most $\epsilon_{\rm nom}$, maximum anchor-response mismatch $\epsilon_{\rm pair}$, and largest anchor gap $h$, Theorem 1 gives $$\sup_{t\text{ between the outer anchors}}\\|e(\gamma(t))\\| \le \epsilon_{\rm nom}+\epsilon_{\rm pair}+\frac{Mh^2}{8}.$$ The three terms have concrete roles. The nominal error fixes the accuracy at the starting condition. The anchor-response mismatch limits how much that error can change at the supervised conditions. The final term bounds how far the error can depart from the interpolation between neighboring anchors; $M$ limits how sharply the error curve can bend, and $h$ measures the widest unsupervised gap. For example, when viscosity varies between two neighboring training values, the theorem controls prediction error at every intermediate viscosity under the curvature assumption. For fixed $M$, halving the largest gap reduces the interpolation allowance by a factor of four. Corollary 1 makes the training connection explicit: $\epsilon_{\rm pair}\le R_{\rm resp}\sqrt q$, where $q$ is the maximum attained unweighted PhySA response loss over the anchors and $R_{\rm resp}$ is their largest floor-stabilized reference-response scale. Reducing this maximum loss tightens the response term in the continuous error bound. Nominal field training, response training, and anchor spacing therefore control distinct, explicit terms in the guarantee. A numerical certificate over the covered interval combines the measured nominal error and maximum anchor loss with a valid curvature bound. Held-out and dense-grid experiments additionally evaluate the learned responses at unsupervised conditions. We now use the viscosity example to introduce the theorem and collect the general coordinate assumptions in the appendix. ### 5. Reporting EAF and SNRE alongside SAE The submitted Appendix I, Tables 11–16, reports all three diagnostics for applicable tasks, and EAF also appears in the main text. Figure 2 reports EAF 1.24 and SAE 0.77 for the parametric model, versus EAF 0.99 and SAE 0.10 for PhySA, and Section 4.2 reports a held-out EAF of 65.15 for the clean-trained FNO. We now report all applicable response diagnostics alongside field error in Table 1 so the evidence is visible in one place. The three diagnostics answer distinct questions about the same prediction. EAF asks whether the predicted change has the correct magnitude; SAE asks whether it matches the true change in both direction and magnitude; SNRE measures the remaining changed-target error relative to the physical-response scale. Equal response magnitudes can point in different directions, so EAF and SAE together distinguish amplification from response fidelity. Field error and SNRE then assess the accuracy of the resulting solution. When the true response determines the denominator, a matched response has EAF 1; if a floor is active, the matched reference is $\\|\Delta\mathcal S\\|/\eta$.
Reviewer_Xjp7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines how PDE surrogates behave when the physical problem changes. Unlike standard label-preserving perturbations, changing a PDE coefficient, initial condition, or boundary condition generally changes the correct solution. The paper argues that a surrogate should therefore be judged not only by its field error but also by whether it predicts the direction and magnitude of the resulting change. It introduces the response diagnostics SAE, EAF, and SNRE, derives bounds relating prediction error to sensitivity mismatch, and evaluates three training approaches (PASA, PhySA, and ISF) on six Burgers and Darcy shift tasks. Some settings show improvements in both response fidelity and field accuracy, while others show clear trade-offs.

Critical strengths

S1. The distinction between field accuracy and response fidelity is worth examining. The Burgers example makes it concrete: a surrogate can have low field error yet poorly predict how the solution changes.

S2. The reported experiments are carefully controlled. The matched seeds, sampling controls, and detailed tables make the comparisons easier to assess.

S3. Some gains extend beyond SAE to field and physical-quantity errors. The paper also reports cases where the added losses hurt performance.

Critical weaknesses

W1. The benefit over existing sensitivity-supervised methods is not clear.

SC-FNO (Behroozi et al., ICLR 2025) already uses sensitivity supervision for PDE surrogates and discusses obtaining sensitivity targets by finite differences. This paper explains the difference between parameter sensitivities and finite-response matching, but does not compare the resulting training methods directly. Given how closely related their objectives are, a matched SC-FNO comparison is needed to establish what the proposed training losses add in practice.

W2. A simple ablation of PhySA's response normalization is missing.

Let the prediction errors at the nominal and perturbed specifications be

e0=A(z)−S(z),eδ=A(zδ)−S(zδ),e_0=A(z)-S(z),\qquad e_{\delta}=A(z_{\delta})-S(z_{\delta}),

and let ΔS=S(zδ)−S(z)\Delta S=S(z_{\delta})-S(z). The response-matching term in PhySA, for a nominal–perturbed pair, can be written as

=\frac{\|e_{\delta}-e_0\|^2} {\max\left(\|\Delta S\|^2,\epsilon_{\mathrm{SA}}^2\right)}.$$ A natural baseline is the same difference loss without response normalization: $$\mathcal{L}_{\mathrm{diff}}=\|e_{\delta}-e_0\|^2.$$ This would show whether the normalization itself helps, rather than just adding supervision on response differences. The existing weighted-parametric baseline matches the sampling proportions of nominal and perturbed examples; it does not test this normalization. The comparison should keep the other loss terms and sampling procedure fixed, with reasonably tuned weights for both objectives. **W3. The downstream value of response fidelity remains untested.** Parameter inversion, design, and control motivate the proposed diagnostics, but none of these tasks is evaluated. Table 18 reports physical-quantity errors, which is useful, but it does not show whether a lower SAE leads to a better inverse estimate or design decision. This matters because PhySA directly optimizes a loss closely related to SAE, and field accuracy sometimes gets worse. A representative downstream task would help establish when this trade-off is worthwhile. **W4. The trade-off between field accuracy and response fidelity needs a clearer analysis.** On the Burgers viscosity task with FNO, PhySA lowers SAE from 0.5006 to 0.1860, but held-out relative $L^2$ error rises from 0.0181 to 0.0292, and SNRE also worsens (Table 11). On Public Burgers, the field-error gains do not pass the paper's multiple-comparison threshold ($q=0.0814$). Results are mainly reported for fixed loss weights. A weight sweep and a field-error-versus-SAE Pareto plot would clarify whether the method offers a better trade-off or simply places more emphasis on response accuracy.
Questions for authors

See the points raised under Weaknesses.

7 comments

Thank you for the detailed comparisons you propose. Our contribution connects a measurable physical-response error to a practical training objective and a uniform prediction-error bound between supervised conditions. PhySA directly supervises the error in a finite physical change relative to its magnitude, using paired solution fields. The same response discrepancy enters the diagnostics, the learning objective, and the anchor analysis; Theorem 1 and Corollary 1 relate the attained loss to accuracy throughout the covered interval under a curvature bound.

The additional matched experiments address all four questions: PhySA improves response accuracy beyond tuned sensitivity supervision; normalization outperforms tuned raw difference matching; response improvements lead to better parameter recovery at comparable field accuracy; and validation selects settings that improve field accuracy and response fidelity together.

W1. PhySA improves response accuracy beyond tuned sensitivity supervision

Adding PhySA to tuned sensitivity supervision improves response accuracy on both tested backbones. This directly tests its additional value: the model already receives local sensitivity targets, and PhySA adds supervision on the physical change between two conditions.

SC-FNO (https://arxiv.org/abs/2505.08740) supervises local Jacobians and can obtain those targets by finite differences. PhySA directly supervises a different quantity: the relative error of the actual change between two solutions. Finite differences determine how a derivative target is computed; they do not change SC-FNO's derivative-matching objective. PhySA uses the paired solutions themselves to train the model's finite response.

Let e(z)=A(z)−S(z)e(z)=\mathcal A(z)-\mathcal S(z). For any nominal–changed pair,

=\big[\mathcal A(z_\delta)-\mathcal A(z)\big] -\big[\mathcal S(z_\delta)-\mathcal S(z)\big].$$ This exact identity identifies the response discrepancy to control together with nominal error to maintain accuracy after a physical change. PhySA penalizes its squared norm relative to the squared physical-response magnitude, with a floor to keep the denominator positive. Two predictions and two reference solutions provide the complete response-training signal at finite perturbations. This makes stored solver outputs and public solution pairs usable without parameter-derivative labels, model derivatives with respect to physical parameters, or a differentiable PDE residual. For smooth response curves, the finite response error equals the derivative mismatch accumulated along the path between the two conditions. PhySA directly supervises this accumulated change. Its normalization also gives each pair a weight determined by that pair's physical response. This distinction remains visible even for a small increment $z_\delta=z+hv$: first-order expansion gives $$\mathcal L_{\rm SA}(z,z+hv) \approx \frac{\\|(D\mathcal A(z)-D\mathcal S(z))[v]\\|^2} {\\|D\mathcal S(z)[v]\\|^2},$$ when $D\mathcal S(z)[v]\ne0$ and the physical response exceeds the training floor. The local form therefore measures sensitivity error relative to the true directional sensitivity, whereas SC-FNO's squared Jacobian loss measures absolute sensitivity error. W2 tests the practical value of this scaling directly. For a fixed physical pair and fixed predictions, the PhySA loss also stays unchanged when viscosity is expressed as $\nu$ or $\log\nu$, because it depends on the endpoint outputs. PhySA directly trains the response discrepancy in a uniform prediction-error bound. Theorem 1 and Corollary 1 connect the attained PhySA loss to accuracy between response anchors, the physical conditions at which responses are supervised. For a twice continuously differentiable error path with curvature bounded by $M$ and largest anchor gap $h$, $$\begin{aligned} \sup_{t\text{ between the outer anchors}}\\|e(\gamma(t))\\| &\le \epsilon_{\rm nom}+\epsilon_{\rm pair}+\frac{Mh^2}{8} \\\\ &\le \epsilon_{\rm nom}+R_{\rm resp}\sqrt q+\frac{Mh^2}{8}, \end{aligned}$$ where $q$ is the maximum attained PhySA loss at the anchors and $R_{\rm resp}$ is their largest floor-stabilized reference-response scale. The bound has a direct interpretation: accurate predictions at the starting condition, accurate changes to the anchors, and sufficiently close anchors control error throughout the covered interval under the curvature assumption. PhySA directly trains the response term in this bound; halving the largest anchor gap reduces the interpolation term by a factor of four for fixed $M$.

Matched comparison. We use the same backbone, nominal and changed field labels, paired sampling, field/PDE terms, optimizer, 150 training epochs, and five matched seeds. The SC sensitivity loss uses model automatic differentiation and local solver finite differences. On ConvResNet1D, we apply the same SC loss to that backbone. Hyperparameters are selected using nominal/seen validation data, with the common score

=\frac{L_{\rm field,val}}{L_{\rm field,val}^{\rm parametric}} +\frac{\mathrm{SAE}_{\rm seen,val}}{\mathrm{SAE}_{\rm seen,val}^{\rm parametric}}.$$ The field term averages nominal and seen-severity validation errors. No held-out severity is used for checkpoint or hyperparameter selection. Each entry below is held-out relative $L^2$ / SAE, averaged across seeds; both are lower-is-better. These are additional matched runs, separate from the submitted table. | Training objective | Burgers viscosity, FNO | Burgers viscosity, ConvResNet1D | | --- | ---: | ---: | | Parametric training with changed labels | 0.0181 / 0.473 | 0.0474 / 0.943 | | SC sensitivity supervision | 0.0082 / 0.090 | 0.0353 / 0.335 | | PhySA | 0.0112 / 0.134 | 0.0354 / 0.184 | | SC sensitivity supervision + PhySA | 0.0083 / 0.081 | 0.0342 / 0.225 | Adding PhySA to SC reduces SAE in all five seeds on both backbones, with mean paired reductions of 10.6% on FNO and 32.6% on ConvResNet1D. The derivative-loss coefficient remains fixed at its selected value in both combination comparisons. On FNO, SAE improves from 0.090 to 0.081 while field error remains close, 0.0082 versus 0.0083. On ConvResNet1D, the combined objective improves both field error and SAE. PhySA alone also reduces ConvResNet1D's SAE from 0.335 to 0.184 at nearly identical mean field error, 0.0353 versus 0.0354. The combined comparisons isolate the additional benefit after the sensitivity loss has already been tuned: directly supervising finite responses improves a model that already learns local derivatives. The additional benefit persists with accurate local sensitivity targets on public data. We construct local finite-difference targets for public Burgers using a reference solver with the dataset's physical diffusivity convention, $\nu_{\rm physical}=\mathrm{Nu}/\pi$, and the same initial conditions, observation times, and spatial averaging as the released trajectories. Its aggregate discrepancy from the released anchor fields is at most 0.254%; halving the finite-difference step changes the derivative targets by at most $2.4\times10^{-6}$ in relative norm. This provides accurate local targets independently of the spacing between released public-data anchors. Seen validation selects the SC coefficient 0.01 from nine values spanning $10^{-6}$ to $100$. We then keep that coefficient, derivative targets, field/PDE terms, sampling, optimizer, and 150-epoch budget fixed and sweep only the added response coefficient. The following is the complete added-weight ablation, with mean ± standard deviation over five seeds on a fixed initial-condition split. Field-only validation selects the SC-only point in this experiment; the added-weight rows measure the effect of finite-response supervision with the tuned SC configuration fixed. | Added PhySA weight | Held-out relative $L^2$ ↓ | SAE ↓ | EAF | SNRE ↓ | | --- | ---: | ---: | ---: | ---: | | 0: accurate local SC baseline | 0.0925 ± 0.0100 | 2.2948 ± 0.4843 | 2.9394 ± 0.4657 | 2.3571 ± 0.4869 | | $5\times10^{-5}$ | 0.0866 ± 0.0046 | 1.6964 ± 0.1425 | 2.3410 ± 0.1288 | 1.8826 ± 0.1304 | | $5\times10^{-4}$ | 0.0895 ± 0.0039 | 0.9630 ± 0.2085 | 1.5750 ± 0.2063 | 1.3936 ± 0.1820 | | $5\times10^{-3}$ | 0.1040 ± 0.0064 | 0.7030 ± 0.0696 | 1.2397 ± 0.0704 | 1.6291 ± 0.1049 | At $5\times10^{-4}$, SAE falls from 2.2948 to 0.9630 and SNRE from 2.3571 to 1.3936, with both improvements occurring in every seed. Mean held-out field error decreases from 0.0925 to 0.0895, with mixed seedwise changes in field error. Thus, the additional response improvement persists with accurate local targets and a fixed, tuned derivative-loss coefficient. The full sweep reports the field accuracy associated with each response improvement.

Positioning against related methods. DINO (https://arxiv.org/abs/2206.10745) and SC-FNO supervise local Jacobians. sTCL (https://arxiv.org/abs/2609.32797) obtains derivative supervision through a known differentiable PDE residual and its tangent actions. Reference Neural Operators (https://arxiv.org/abs/2405.17509) predict solution differences conditioned on a reference solution and a geometric deformation, while solver-integrated adversarial training (https://arxiv.org/abs/2510.18989) uses changed solver targets to optimize robustness. Our contribution connects a specific learning target, the relative error of a finite physical response, to an objective built from paired solution fields, a bound between response anchors, and evaluation on held-out physical changes. The matched comparisons establish its value beyond tuned sensitivity supervision, and W3 tests the resulting benefit in parameter recovery.

W2. Response normalization improves on tuned raw difference matching

Normalization improves response fidelity beyond a tuned unnormalized difference loss on both backbones. Its role is simple: the importance of a response error depends on the size of the physical change it is supposed to predict.

For illustration, consider true response magnitudes of 1 and 10, both above the training floor. An absolute response error of 1 is a 100% error in the first case and a 10% error in the second. The raw squared loss assigns both cases the same penalty, 1. The normalized loss assigns penalties of 1 and 0.01. Multiplying the raw loss by any single coefficient keeps the two penalties equal; normalization gives each pair a weight appropriate to its own physical change. The coefficient controls the overall strength of response supervision, while normalization controls how that supervision is distributed across pairs. The floor limits the weight when a physical response is very small.

The requested ablation compares

\qquad \mathcal L_{\rm SA} =\frac{\\|e_\delta-e_0\\|^2} {\max\\{\\|\Delta\mathcal S\\|^2,\varepsilon_{\rm SA}^2\\}}.$$ The raw and normalized objectives use identical nominal/changed field terms, PDE penalty, batches, model, optimizer, and training budget. We also include a paired control with response coefficient zero. The raw loss is tuned over ten weights, $\\{0.03,0.1,0.3,1,3,10,30,100,300,1000\\}$, using the same validation criterion as in W1. The normalized loss sweeps both its coefficient and training floor. The resulting five-seed means are: | Backbone | Objective | Held-out relative $L^2$ ↓ | SAE ↓ | | --- | --- | ---: | ---: | | FNO | Paired field/PDE control, no response loss | 0.0180 | 0.436 | | FNO | Tuned raw difference | 0.0113 | 0.185 | | FNO | Tuned normalized response | **0.0112** | **0.134** | | ConvResNet1D | Paired field/PDE control, no response loss | 0.0450 | 0.930 | | ConvResNet1D | Tuned raw difference | 0.0392 | 0.377 | | ConvResNet1D | Tuned normalized response | **0.0354** | **0.184** | Relative to the tuned raw loss, normalization lowers SAE by 27.6% on FNO and 51.1% on ConvResNet1D, computed as the mean paired percentage reduction. FNO retains nearly identical field accuracy, and ConvResNet1D improves both metrics. Because paired sampling and field/PDE supervision are held fixed, these comparisons isolate the response objective and its normalization. Two controls strengthen this conclusion. First, fixing the training floor to $10^{-3}$ in standardized mean-square units and tuning only seven response weights still reduces SAE relative to the ten-weight raw sweep by 27.6% on FNO and 28.2% on ConvResNet1D. Thus, the normalization benefit persists with fewer candidate configurations. Second, for every one of the ten tested raw-loss weights on each backbone, a tested normalized setting achieves lower mean field error and lower mean SAE. Normalization therefore improves the attainable combinations of field and response accuracy within these measured sweeps. Independent public-data control. A separate fixed-split experiment retains the submitted training floor, uses the native PDEBench diffusivity convention, and evaluates seven raw weights spanning 0.01–10,000. The normalized weight 0.02 is selected by seen-validation SAE; the raw weight 10 is selected by seen-validation field error. The normalized point has lower mean held-out field error and SAE than every raw point in that sweep. The table gives the strongest raw field-error choice, with five-seed mean ± standard deviation. | Public Burgers objective | Held-out relative $L^2$ ↓ | SAE ↓ | EAF | SNRE ↓ | | --- | ---: | ---: | ---: | ---: | | Paired field/PDE control | 0.2536 ± 0.0387 | 2.7062 ± 0.6269 | 3.1158 | 2.6602 | | Raw difference, $\lambda=10$ | 0.1830 ± 0.0284 | 1.6865 ± 0.1832 | 2.1813 | 1.6867 | | Normalized response, $\lambda=0.02$ | **0.1727 ± 0.0062** | **0.6262 ± 0.0749** | **1.0501** | **1.4428** |

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.