acceptodds
Under review as a conference paper at ICLR 2027

Learning When to Trust PDE Coupling Priors in Neural Operators

Abstract

Coupled neural operators must decide how strongly to use interactions suggested by the governing equations. We study this structural decision under finite data using controlled interventions that vary cross-field coupling while holding the operator backbone and evaluation fixed. On Active Matter and a turbulent radiative layer, fixed PDE coupling priors exhibit different data-scaling behavior, and leave-one-pathway-out interventions show that the predictive effect of individual interactions can change sharply with the data regime. Motivated by this observation, we factor each cross-field interaction into a learned operator and a scalar pathway strength. A short warmup provides pathway-specific evidence to initialize these strengths before joint training. On scarce Active Matter data, the resulting model reduces relative error from for dense coupling and for the fixed PDE graph to . Across Active Matter and the turbulent radiative layer, the same method improves on the strongest fixed structural baseline in all four scarce and medium regimes and remains within 0.08 percentage points of the best fixed choice in both abundant regimes. With the method frozen before evaluation on Rayleigh–B\'enard convection, it improves scarce-data error from to and matches the strongest fixed baseline with abundant data. These results support calibrating PDE coupling structure at the level of individual pathways using target-regime evidence.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operators, scientific machine learning, physics-informed machine learning, coupled PDEs, structural priors, multiphysics learning, adaptive inductive bias, operator learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_tBGrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines how cross-field coupling structure affects neural-operator learning at different data scales. Structural interventions compare PDE-derived graphs with dense, uncoupled, and mismatched alternatives. A proposed initialization procedure uses short pathway-specific probes to set interaction strengths before joint training. The experiments report improvements in several regimes and evaluate the frozen procedure on an additional physical system.

Critical strengths
  1. The structural controls are a useful part of the study. Holding the edge budget fixed makes mismatched graphs more informative than an arbitrary architecture comparison. The pathway-removal results also show why equally sized graph changes can have very different predictive consequences. This is a concrete observation that could guide model design.

  2. The method is accompanied by relevant ablations, including uninformed gates and frozen probe initialization. Reserving a further system for evaluation after the procedure was fixed adds evidence beyond the development settings. I also appreciated the explicit reporting of a stronger external baseline in one regime.

Critical weaknesses
  1. The main interpretation goes beyond what the comparisons identify. The prior-fidelity measure in Section 3.2 compares the PDE graph with deliberately mismatched graphs. Increasing separation can reflect growing harm from a wrong structure, without showing increasing benefit from a correct prior over a flexible model. The paper itself reports a different trend for the latter comparison. There is a related issue with “trust”: the adaptive method considers the full candidate set, and a gate can be compensated by rescaling its kernel. Useful predictions do not make the gate a uniquely interpretable measure of reliance on PDE knowledge. I see stronger evidence for an initialization strategy and a misspecification study.

  2. The reliability and cost of the additional method benefit need a fairer comparison. The proposed procedure uses auxiliary probe training beyond the baselines' main optimization budget, while the low-overhead statement emphasizes the final run. A baseline with comparable auxiliary training or validation-guided initialization would be more informative. Evaluation also relies on few independent test configurations, and some reported improvements are small relative to training variability. The stronger scarce-data results are encouraging, but they do not settle whether the full adaptive procedure is consistently preferable to simpler initialization choices.

Questions for authors
  1. Can the scaling conclusions distinguish PDE-versus-Dense benefit from PDE-versus-mismatched-graph separation? Which claimed findings require each contrast?
  2. Is there any normalization or functional contribution measure that makes gate values interpretable as trust? If not, how should their physical interpretation be limited?
  3. What is the complete cost per regime, including the self-only model, all pathway probes, validation, and final training?
  4. Were equally budgeted training or initialization controls run? Existing results would help determine whether the auxiliary evidence, rather than additional optimization opportunity, explains the gain.
  5. Can paired per-seed and per-test-configuration differences be reported against the strongest fixed baseline? Any existing alternative-split results would help assess sensitivity to the small test set.
4 comments
AuthorsOpenReview ↗

Response to Reviewer tBGr (1/2)

We thank the reviewer for a careful review. Its main suggestion is to read the paper as a misspecification study together with an initialization strategy, and the revision now presents it that way.

W1: What the comparisons identify, and the trust reading

"The main interpretation goes beyond what the comparisons identify. The prior-fidelity measure in Section 3.2 compares the PDE graph with deliberately mismatched graphs. [...]"

The revision separates the two contrasts that the review distinguishes (Section 3.2, Table 13):

  • Benefit of a correct prior (PDE graph against dense coupling): it falls with data on both systems, 4.11 -> 2.13% and 7.96 -> 2.62% of the Dense error.
  • Cost of a wrong prior (PDE graph against equally sparse mismatched graphs): it rises on Active Matter, 1.70 -> 16.04% of the PDE error, and falls on the turbulent radiative layer, 9.49 -> 3.54%.

The span S(N)S(N) is the second contrast normalized by the Dense error, and the Fig. 2 caption now says that the panel plots S(N)S(N) only. The claim of the paper is the one these comparisons support: the amount of data alone does not determine the cost of a wrong prior.

On the gate: it multiplies a linear spectral weight, so the kernel can absorb its scale. The revision describes it as an evidence-initialized pathway scale (Fig. 1 caption, Section 4.1) and presents the method as an initialization strategy. A new control supports that reading (Appendix T, Table 16). We removed the gate and initialized each operator at its probe-based scale. This gate-free arm matches M3 in all four scarce and medium cells of AM and TRL, on the same three seeds: 6.445% against 6.468% (AM-scarce), 3.981% against 3.972% (AM-medium), 10.48% against 10.46% (TRL-scarce) and 8.51% against 8.47% (TRL-medium).

W2: Reliability and cost of the gain

"The reliability and cost of the additional method benefit need a fairer comparison. [...]"

Cost. The probe is seven 2,000-step runs per regime, which is 70% of a full run in raw steps and 30% in full-model-equivalents (Section 4.5, Appendix N). Section 4.5 now limits the no-overhead statement to the final run.

Comparable controls. Three controls separate the probe evidence from extra optimization. The second and third are new (Appendix T, Tables 15 and 16).

  1. Same gates and same budget, without probe evidence (M2): 6.89% on AM-scarce and 10.76% on TRL-scarce, which is no better than the fixed structures (Table 7).
  2. Validation-guided structure selection with the probe's step budget. Nine candidate structures get a short run each, and the one with the best validation error is trained in full. Over five seeds it reaches 7.40% on AM-scarce against 6.44% for M3, and 0.0878 on RBC-scarce against 0.0833. M3 is lower on every seed in both cells. A search over all 64 subgraphs gives 6.61%, 6.82% and 7.56% on three AM-scarce seeds, against 6.50%, 6.46% and 6.45% for M3.
  3. A simpler initialization with the same evidence: the gate-free arm above matches M3 in all four cells.

Reliability. The revision reports four checks.

  • AM-scarce: the seed-paired reduction is 0.298 pp (95% CI [0.010, 0.586], 3/3 seeds; Appendix Q, Table 11), against a seed standard deviation of 0.04.
  • RBC-scarce, where the method was frozen before evaluation: the seed-paired reduction against dense coupling is 0.0041 (95% CI [0.0021, 0.0060], 3/3 seeds; Table 17).
  • Against dense coupling, M3 is lower on every seed in all four scarce and medium cells of AM and TRL (by 8.4%, 5.0%, 9.5% and 3.4%), and all four paired intervals exclude zero (Table 17).
  • Over the 21 held-out AM configurations the mean paired change is -0.30 pp (95% CI [-0.86, +0.12]), and M3 removes the two largest failures of the fixed graph (3.9 and 3.6 pp; Section 5.6). On TRL-scarce, M3 improves 7 of 9 held-out configurations (Table 14).

On AM-medium the gain is 0.09 pp on 3/3 seeds. TRL-medium is a tie, and the two abundant cells are 0.07 and 0.08 pp behind the PDE graph. The abstract and Section 5.2 now say so.

AuthorsOpenReview ↗

Response to Reviewer tBGr (2/2)

Q1: Which contrast each claim needs

"Can the scaling conclusions distinguish PDE-versus-Dense benefit from PDE-versus-mismatched-graph separation? Which claimed findings require each contrast?"

Yes, and each claim now names its contrast (Section 3.2, Table 13).

  • A correct prior loses value as data grows: this rests on the PDE-versus-Dense contrast.
  • The cost of a wrong prior depends on the system: this rests on the PDE-versus-mismatched contrast.
  • The harm of a wrong graph follows the pathway it removes: this rests on the leave-one-pathway-out interventions (Section 3.3).

Q2: Can gate values be read as trust?

"Is there any normalization or functional contribution measure that makes gate values interpretable as trust? If not, how should their physical interpretation be limited?"

We found no normalization that does this, because the kernel can absorb the scale. On Active Matter the D->U gate ends at 0.27, while removing that pathway costs 64% at abundant data (Appendix F, Table 5). The revision limits the interpretation where the gate is defined: geg_e is an initial pathway scale set from probe evidence (Fig. 1 caption, Section 4.1). The functional importance of a pathway is measured by its removal effect (Section 3.3).

Q3: Complete cost per regime

"What is the complete cost per regime, including the self-only model, all pathway probes, validation, and final training?"

Per regime the method runs one self-field model and six single-pathway probes, 2,000 steps each. This is 14,000 raw steps, or 70% of the 20,000-step full run, and 30% in full-model-equivalents. One 20,000-step final run follows. It takes 1,111 s on one RTX 3090, against 1,276 s for dense coupling (Appendix N). Validation is not a separate stage: each probe is scored on the validation split within its 2,000 steps, and checkpoint selection uses the validation split for every arm.

Q4: Equally budgeted controls

"Were equally budgeted training or initialization controls run? [...]"

Yes. The three controls under W2 cover this (Table 7; Appendix T, Tables 15 and 16).

  • Equal budget without evidence (M2) shows no gain.
  • Structure selection with the probe's step budget and the same validation split is behind M3 on every seed, on AM-scarce (five seeds) and on RBC-scarce (five seeds).
  • A simpler initialization with the same probe evidence, the gate-free arm, matches M3 in all four scarce and medium cells of AM and TRL.

The auxiliary evidence therefore explains the gain.

Q5: Paired differences per seed and per test configuration

"Can paired per-seed and per-test-configuration differences be reported against the strongest fixed baseline? [...]"

Both are reported. Per seed, Appendix Q (Table 11) compares M3 with the PDE graph, the strongest fixed baseline on AM-scarce: 6.468% against 6.766%, a reduction of 0.298 pp (95% CI [0.010, 0.586], 3/3 seeds). On RBC-scarce the strongest fixed baseline in the paper is dense coupling, and the paired reduction is 0.0041 (95% CI [0.0021, 0.0060], 3/3 seeds; Appendix T, Table 17). Per test configuration, Section 5.6 reports the paired changes over the 21 held-out AM configurations, and Table 14 lists all nine TRL configurations one by one. The splits are fixed for all arms, and the 21 and 9 held-out configurations are the sensitivity evidence.

If the revision addresses your concerns, we would appreciate it if you would consider updating your score.

AuthorsOpenReview ↗

Follow-up to Reviewer tBGr

We would like to follow up on our rebuttal before the author-reviewer discussion closes on Nov. 18 at 11:59 p.m. AoE. The rebuttal answers each of your comments in detail. In particular, it covers the reliability and the cost of the gain. The revision adds seed-paired tests on AM-scarce (a reduction of 0.298 pp, 95% CI [0.010, 0.586]) and on RBC-scarce (0.0041, 95% CI [0.0021, 0.0060]). It also adds the equally budgeted control that you asked for. A selection baseline with the probe's step budget reaches 7.40% on AM-scarce against 6.44% for M3, and M3 is lower on every seed (Appendix T, Tables 15 and 17). The rebuttal also covers the reading of the gate, which the paper now describes as an evidence-initialized pathway scale (Section 4.1). We hope these additions give a fuller picture of the work.

We would also like to highlight the main result of the paper. PDE-derived coupling structure is already a common inductive bias in neural operators for multi-field systems, and such models are usually compared with an unconstrained baseline. Our design holds the backbone, the edge budget and the evaluation fixed, and each cross-field pathway can be imposed, removed or replaced. It shows that the value of a coupling graph comes from individual pathways. One pathway per system dominates and grows with data (Active Matter D->U: +10.6% -> +64.4%; turbulent radiative layer U->R: -0.8% -> +31.4%), and the harm of a wrong graph follows which pathway it removes (Spearman 1.00 on TRL). The method that follows from this study was frozen before it was tested on Rayleigh-Benard convection, where it is 4.7% below the best fixed structure at scarce data. These results are, to the best of our knowledge, new in the literature on physics-guided neural operators. Your review notes that holding the edge budget fixed makes mismatched graphs more informative than an arbitrary architecture comparison, and we hope the new controls complete this picture.

We would be very grateful if you could read the rebuttal before the discussion closes and tell us what you think. If it resolves your concerns, we kindly ask you to consider raising your score. If anything remains open, please let us know and we will answer right away.

Thank you for your time and consideration. We look forward to your reply.

Reviewer_tBGrOpenReview ↗

Official Comment by Reviewer tBGr

Thanks to the author for the rebuttal. After reading the rebuttal carefully, I will maintain the score.

Reviewer_dD1FOpenReview ↗
Rating
2(−1)
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how strongly a coupled neural operator should use the cross field interactions suggested by the governing equations when data are limited. The authors build a controlled spectral operator in which each cross field pathway can be imposed, removed or replaced while the backbone, the edge budget and the evaluation are held fixed, and use it to measure how the value of a fixed PDE coupling graph changes with training set size on Active Matter and a turbulent radiative layer. They find that the value does not follow data quantity. On Active Matter the advantage of the correct graph over mismatched graphs grows with data, from about 1.7 to 15.7 percentage points, while on the turbulent radiative layer it shrinks from 8.2 to 3.5. A pathway removal analysis shows that individual pathways differ by an order of magnitude and change with the data regime. The authors then factor each pathway into a learned spectral operator and a scalar strength, initialize the strengths from a 2000 step warmup probe and refine them during training, and report gains over the strongest fixed structure in four scarce and medium regimes, a tie at abundant data, and a prospective transfer to Rayleigh Benard convection.

Critical strengths
  1. The controlled design is the main contribution. Letting each pathway be switched independently, with the edge count equal to the parameter budget, makes the interaction information a genuine independent variable. Most work compares a physics guided model with an unconstrained baseline and cannot attribute the gap to structure itself. This design can, and it is what makes the negative result credible.

  2. The pathway level finding is useful beyond this paper. One pathway dominates and grows with data, inert pathways exist inside a physically valid graph, and the harm of a mismatched graph is predicted by which pathway is removed rather than by how many. That is a concrete lesson for anyone designing structural priors, and the prospective test on a third system supports it.

Critical weaknesses
  1. The gains are small and untested. The largest is 0.33 points on Active Matter scarce, and the turbulent radiative layer medium result is 8.48 against 8.47, which is a tie. No paired significance tests are reported, and the abstract counts all four scarce and medium regimes as improvements. The method claim is therefore weaker than the framing suggests.

  2. The scope is narrow. All three systems have three field groups and six candidate pathways, and all come from the same data family. The probe costs one short run per pathway, so the approach does not obviously scale to systems with many fields. It is also unclear that joint training is needed, since frozen gates are better on Active Matter and only 0.08 points worse on the turbulent radiative layer.

Questions for authors
  1. Could you add paired tests, or mark clearly which differences fall within seed noise, especially the medium regime on the turbulent radiative layer?

  2. What does joint training add over probe initialization with a fixed soft mask, given that the frozen variant is better on Active Matter?

  3. On the turbulent radiative layer scarce regime, removing all coupling gives 10.49 and the method gives 10.36. Is the benefit mainly avoiding a bad prior rather than learning a good one?

  4. How would the probe cost scale to a system with around ten field groups, and is there a cheaper screening variant?

3 comments
AuthorsOpenReview ↗

Response to Reviewer dD1F (1/2)

We thank the reviewer for a careful review, and for identifying the controlled design as the main contribution of the paper. The revision keeps that design at the center and adds the statistical tests that the review asks for.

W1: Size of the gains and paired tests

"The gains are small and untested. The largest is 0.33 points on Active Matter scarce, and the turbulent radiative layer medium result is 8.48 against 8.47, which is a tie. [...]"

The revision adds paired tests and corrects the abstract. Appendix T adds two more paired tests (Table 17).

  • AM-scarce: 6.468% for M3 against 6.766% for the PDE graph, a seed-paired reduction of 0.298 pp (95% CI [0.010, 0.586], 3/3 seeds; Appendix Q, Table 11).
  • RBC-scarce, where the method was frozen before evaluation: 0.0831 against 0.0872 for dense coupling, a seed-paired reduction of 0.0041 (95% CI [0.0021, 0.0060], 3/3 seeds).
  • AM-medium: 3.97% against 4.06%, lower on 3/3 seeds, with a seed standard deviation of 0.004.
  • TRL-scarce: M3 improves on the fixed graph in 7 of 9 held-out configurations (median -1.1%; Table 14).
  • Dense coupling is the default when no structure is assumed. Against it, M3 is lower on every seed in all four scarce and medium cells of AM and TRL, by 8.4%, 5.0%, 9.5% and 3.4%. All four paired intervals exclude zero.

TRL-medium is a tie. Section 5.2 and the Fig. 4 caption now say so, and the abstract no longer counts four improvements.

The gains should also be read against what a user can know in advance. The best fixed structure changes from cell to cell: the PDE graph on Active Matter, no coupling on TRL-scarce, and dense coupling on Rayleigh-Benard. M3 is ahead of the best fixed structure in four cells, ties it in two, and is at most 0.08 pp behind in the other two. Every fixed structure loses more than that somewhere (dense coupling by 1.07 pp on TRL-scarce, the PDE graph by 6.4% on RBC-abundant).

A user could also try to pick the structure by validation. We ran this control (Appendix T, Table 15). A selection baseline spends the probe's step budget on short runs of nine candidate structures and trains the best one in full. Over five seeds it reaches 7.40% on AM-scarce against 6.44% for M3, and 0.0878 on RBC-scarce against 0.0833. M3 is lower on every seed in both cells.

W2: Scope, probe cost and joint training

"The scope is narrow. All three systems have three field groups and six candidate pathways, and all come from the same data family. [...]"

The three systems have three field groups and six candidate pathways each, and all come from The Well. Table 1 and Section 6 now state this scope. Within it, the third system was held out: the method, the probe and the training configuration were frozen before any Rayleigh-Benard run (Section 5.5, Appendix M). Probe cost is answered under Q4 and joint training under Q2, in the second part of this reply.

AuthorsOpenReview ↗

Response to Reviewer dD1F (2/2)

Q1: Paired tests and seed noise

"Could you add paired tests, or mark clearly which differences fall within seed noise, especially the medium regime on the turbulent radiative layer?"

Both are done. The paired tests are listed under W1: AM-scarce against the PDE graph (Appendix Q, Table 11), RBC-scarce against dense coupling, and the four scarce and medium cells against dense coupling (Appendix T, Table 17). TRL-medium is marked as a tie in Section 5.2 and in the Fig. 4 caption. The two abundant cells are reported as 0.07 and 0.08 pp behind the PDE graph, in the abstract and in Section 5.2.

Q2: What joint training adds over a fixed soft mask

"What does joint training add over probe initialization with a fixed soft mask, given that the frozen variant is better on Active Matter?"

The probe initialization carries most of the gain. On AM-scarce, frozen gates give 6.39 +/- 0.03 and trained gates 6.44 +/- 0.04, which is within seed spread. On TRL-scarce, trained gates are lower: 10.36 against 10.44. Uniform initialization gives 6.89 and 10.76 (Table 8). A further arm removes the gate and initializes each operator at its probe-based scale. It matches M3 in all four scarce and medium cells of AM and TRL (6.445% against 6.468% on AM-scarce; Appendix T, Table 16).

Joint training lets the model correct the probe where it is wrong. On Active Matter the probe ranks D->U low, yet D->U has the largest removal effect. During training its gate climbs from 0.12 to 0.27 (Appendix I, Fig. 6). We therefore keep the gates trainable: it adds no cost and it guards against a misranked pathway.

Q3: TRL-scarce: avoiding a bad prior or learning a good one?

"On the turbulent radiative layer scarce regime, removing all coupling gives 10.49 and the method gives 10.36. Is the benefit mainly avoiding a bad prior rather than learning a good one?"

In this cell coupling mildly hurts the fixed structures: no coupling (10.49) beats the PDE graph (10.64) and dense coupling (11.56; Section 3.2). M3 starts from the dense candidate set and reaches 10.36 +/- 0.18, at or below all three. So it avoids the cost of a bad prior without giving up the cross-field pathways, which the abundant regime needs (no coupling is 9.25 there, against 7.14 for M3). Against the fixed PDE graph, M3 improves 7 of 9 held-out configurations (Table 14).

Q4: Probe cost for about ten field groups

"How would the probe cost scale to a system with around ten field groups, and is there a cheaper screening variant?"

The probe needs one 2,000-step run per candidate pathway, so its cost grows linearly with the number of candidates. For the six pathways studied here the total is 30% of a full run in full-model-equivalents (Section 4.5). Two cheaper variants follow from our results. The first is a shorter probe. In runs of 1,560 steps repeated over five seeds (Appendix T), the top pathway on Active Matter is the same in every seed, so a shorter probe already identifies it. The second is a smaller candidate set, for example the pathways that the equations nominate. Section 6 now states that systems with many fields will need such screening.

If the revision addresses your concerns, we would appreciate it if you would consider updating your score.

AuthorsOpenReview ↗

Follow-up to Reviewer dD1F

We would like to follow up on our rebuttal before the author-reviewer discussion closes on Nov. 18 at 11:59 p.m. AoE. The rebuttal answers each of your comments in detail. In particular, it covers the size of the gains. In the new seed-paired tests M3 is lower than dense coupling on every seed in all four scarce and medium cells of AM and TRL, by 8.4%, 5.0%, 9.5% and 3.4%, and all four intervals exclude zero (Appendix T, Table 17). The rebuttal also covers the probe cost, which is now reported in raw steps and in full-model-equivalents (Section 4.5, Appendix N), and the scope of the study (Section 6). We hope these additions give a fuller picture of the work.

We would also like to highlight the main result of the paper. PDE-derived coupling structure is already a common inductive bias in neural operators for multi-field systems, and such models are usually compared with an unconstrained baseline. Our design holds the backbone, the edge budget and the evaluation fixed, and each cross-field pathway can be imposed, removed or replaced. It shows that the value of a coupling graph comes from individual pathways. One pathway per system dominates and grows with data (Active Matter D->U: +10.6% -> +64.4%; turbulent radiative layer U->R: -0.8% -> +31.4%), and the harm of a wrong graph follows which pathway it removes (Spearman 1.00 on TRL). The method that follows from this study was frozen before it was tested on Rayleigh-Benard convection, where it is 4.7% below the best fixed structure at scarce data. These results are, to the best of our knowledge, new in the literature on physics-guided neural operators. Your review notes that this design can attribute the gap to structure itself, and we hope the new controls complete this picture.

We noticed your rating dropped from 3 to 2 before our response was posted, and we apologize for the delay. We hope our response adequately addresses your points and restores your confidence in our work.

We would be very grateful if you could read the rebuttal before the discussion closes and tell us what you think. If it resolves your concerns, we kindly ask you to consider raising your score. If anything remains open, please let us know and we will answer right away.

Thank you for your time and consideration. We look forward to your reply.

Reviewer_24V4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies when and how strongly neural operators should use coupling structures suggested by governing PDEs. The authors investigate this question through controlled comparisons of coupling graphs and pathway-removal experiments, and propose learnable pathway gates initialized using evidence from short training runs. They evaluate the approach on one-step prediction in active matter, turbulent radiative layers, and Rayleigh–Bénard convection across different training-data regimes.

Critical strengths
  1. The paper has a clear motivation about when and how strongly PDE-derived coupling structures should be incorporated into neural operators under limited data, and it provides a detailed analysis through controlled graph comparisons and ablation experiments.
  2. The method they design is easy to understand withouy large computing cost.
Critical weaknesses
  1. The analysis identifies which pathways matter, but provides a limited explanation of why their predictive value changes across systems and data regimes. Pathway experiments establish sensitivity to particular connections, but do not clearly distinguish the roles of physical coupling, data variability, and optimization effects.
Questions for authors
  1. The reported correlations support using isolated-pathway probes for initialization. What motivated this design over measuring a pathway’s contribution in the presence of other pathways, for example through short leave-one-out probes? How might redundancy or complementarity among pathways affect the resulting initialization?
  2. How stable are the probe utilities and pathway rankings across random seeds or validation configurations? Since min–max normalization maps the lowest and highest utilities to fixed gate values even when their difference is small, how sensitive is the initialization to uncertainty in these estimates?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 24V4 (1/2)

We thank the reviewer for the careful review and for the questions on the probe design. We ran both checks that the questions suggest. The results are below and in Appendices H and T.

W1: Why pathway value changes across systems and regimes

"The analysis identifies which pathways matter, but provides a limited explanation of why their predictive value changes across systems and data regimes. [...]"

The paper gives a pathway-level account, and the revision makes it more explicit (Sections 3.2 and 3.3, Table 13).

Physical coupling. In each system the dominant pathway carries the main physical mechanism. On Active Matter, D->U is the active-stress forcing of the velocity, and removing it costs +10.6% at scarce data and +64.4% at abundant data. On the turbulent radiative layer, U->R is the advection of velocity information into the radiation field, and its removal effect moves from -0.8% to +31.4%. Pathways with little physical content stay inert: U->C never exceeds +2.1%, because the concentration field is nearly constant.

Data. The revision separates two effects of data. The benefit of the correct graph over dense coupling falls with data on both systems. The cost of a wrong graph rises on Active Matter and falls on the turbulent radiative layer, and it follows the pathway that the wrong graph removes (Spearman 1.00 on TRL; Section 3.3).

Capacity and optimization. The controls rule out the simpler explanations. The edge count fixes the parameter budget, and a width-matched dense model differs by at most 0.08 pp, so capacity does not drive the differences (Section 3.1). Training sets are nested and share the same test configurations. The central trend also holds for multi-step rollouts up to horizon 10 (Appendix K).

These interventions identify which pathway matters and when. A quantitative split of the three factors is beyond them.

Q1: Isolated probes or leave-one-out probes?

"What motivated this design over measuring a pathway's contribution in the presence of other pathways, for example through short leave-one-out probes? [...]"

We ran the short leave-one-out probe (Appendix H). The PDE graph and each graph with one pathway removed were trained for 2,000 steps, with three seeds, at scarce data. The pathways were then ranked by the error increase that their removal causes. On both systems this ranking equals the ranking by converged removal effect IeI_e (Spearman 1.00, with 4 pathways on Active Matter and 5 on the turbulent radiative layer). So a short probe is enough to recover the converged ordering.

The two probes have different uses. A leave-one-out probe needs a reference graph, and it scores only the pathways of that graph. The isolated probe needs no graph and scores all six candidate pathways, with six independent runs and no trained full model (Section 4.2). M3 starts from the dense candidate set, so it uses the isolated probe. Its ranking agrees with the converged ranking at Spearman +0.80, +0.80, +0.60 and +0.80 in the four AM and TRL cells. The two alternatives that we tested reach +0.40 at best and are negative in two cells each (Appendix H, Table 6).

On redundancy and complementarity: an isolated probe can misjudge a pathway whose value depends on the others. D->U on Active Matter is such a case. The isolated probe ranks it low, and its removal effect is the largest. The short leave-one-out probe ranks D->U first, so it captures this dependence. Inside M3 the trainable gates play the same role. During joint training the D->U gate climbs from 0.12 to 0.27 (Appendix I, Fig. 6). Appendix H now reports the leave-one-out probe as a second-stage check for cases where a reference graph is available.

AuthorsOpenReview ↗

Response to Reviewer 24V4 (2/2)

Q2: Stability of the utilities and the min-max map

"How stable are the probe utilities and pathway rankings across random seeds or validation configurations? [...]"

We repeated short single-pathway runs over five seeds (1,560 steps each, at scarce data; Appendix T). Four results bear on the question.

  • Across seeds. On Active Matter, U->D has the highest utility in all five seeds. Its utility is about 3.8 pp of validation error in every seed (3.72 to 3.81), and no other pathway exceeds 0.02 pp. On Rayleigh-Benard, U->P and U->B take the top two places in all five seeds.
  • Against the converged ranking. The probe ranking agrees with the converged ranking in all four AM and TRL cells (+0.60 to +0.80; Table 6).
  • Across training sets. On Rayleigh-Benard the six utilities have the same order at scarce and at abundant data, although the two regimes use different training sets (Table 10).
  • Effect on the final error. The final error depends little on the exact gate values. Freezing the probe-initialized gates gives 6.39 on AM-scarce against 6.44 for trained gates, and 10.44 against 10.36 on TRL-scarce (Table 8).

On the min-max map: it sends the lowest and the highest utility to 0.12 and 0.88 whatever their gap (Section 4.3), so the initialization depends on the order of the utilities. The five-seed runs show what this means in practice. On Active Matter the map gives U->D an initial scale of 0.88 in every seed, and the other five pathways stay between 0.12 and 0.18. The uncertainty in the utilities therefore moves the initial scales by at most 0.06. When a small gap is stretched by the map, joint training can move the gates back, as it does for D->U.

If the revision addresses your concerns, we would appreciate it if you would consider updating your score.

AuthorsOpenReview ↗

Follow-up to Reviewer 24V4

We would like to follow up on our rebuttal before the author-reviewer discussion closes on Nov. 18 at 11:59 p.m. AoE. The rebuttal answers each of your comments in detail. In particular, it covers the leave-one-out probes that you asked about. They rank the pathways in the same order as the converged removal effects on both AM and TRL (Spearman 1.00). The rebuttal also covers the stability of the utilities. Over five seeds the top pathway on AM is the same in every seed, with an initial scale of 0.88, and the other five pathways stay between 0.12 and 0.18 (Appendices H and T). We hope these additions give a fuller picture of the work.

We would also like to highlight the main result of the paper. PDE-derived coupling structure is already a common inductive bias in neural operators for multi-field systems, and such models are usually compared with an unconstrained baseline. Our design holds the backbone, the edge budget and the evaluation fixed, and each cross-field pathway can be imposed, removed or replaced. It shows that the value of a coupling graph comes from individual pathways. One pathway per system dominates and grows with data (Active Matter D->U: +10.6% -> +64.4%; turbulent radiative layer U->R: -0.8% -> +31.4%), and the harm of a wrong graph follows which pathway it removes (Spearman 1.00 on TRL). The method that follows from this study was frozen before it was tested on Rayleigh-Benard convection, where it is 4.7% below the best fixed structure at scarce data. These results are, to the best of our knowledge, new in the literature on physics-guided neural operators. Your review notes the clear motivation of the paper and its controlled graph comparisons, and we hope the new controls complete this picture.

We would be very grateful if you could read the rebuttal before the discussion closes and tell us what you think. If it resolves your concerns, we kindly ask you to consider raising your score. If anything remains open, please let us know and we will answer right away.

Thank you for your time and consideration. We look forward to your reply.

Reviewer_HD7yOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how much a typed FNO should rely on PDE-derived cross-field coupling (Active Matter, TRL-2D, with Rayleigh–Bénard held out as a prospective test). It intervenes on the coupling graph: PDE graph vs. Dense vs. NoCross, mismatched graphs with the same edge count, and removing one pathway at a time. Their findings include that the PDE prior's advantage grows with data on AM but shrinks on TRL, and pathway importance is highly uneven. It then proposes M3, which scales each cross-field spectral block by a sigmoid gate g_e. The gates are initialized from a 2k-step single-pathway probe and trained jointly with the blocks. M3 matches or beats the best fixed structure in most regimes and transfers to RBC.

Critical strengths
  • The edge count fixes the parameter budget, and the width-matched Dense control rules out capacity effects. Leave-one-out and same-budget mismatches isolate which interactions matter, not just how many.
  • Graph-edit distance doesn't predict harm. On TRL, all five mismatched graphs sit one edge from the PDE graph, yet their harm tracks the removed pathway's removal effect. That is a useful finding.
Critical weaknesses
  1. The gate is redundant with the kernel, so the "factorization" is really an initialization and optimization effect. K_e is a linear spectral weight, so g_e·K_e describes exactly the same functions as K_e alone. The claim that the gate separates "how much a pathway is used" from "what it computes" therefore has no content at the level of what the model can represent. What g_e changes is:

    • each pathway's effective initial scale and learning rate;
    • under AdamW weight decay on both factors, an implicit group-sparsity penalty on each pathway (the product of two decayed factors acts roughly like an L1 penalty on the block norm).

    The paper's own data support this reading. The probe ranks AM D→U low, its gate ends at only 0.27, yet removing it costs 64%. K_e has simply absorbed the scale, so the gates are not interpretable as "trust."

    Missing baselines that would test this directly:

    • Dense with each K_e's initialization scaled by m_e, with no gate;
    • per-pathway group-lasso or tuned weight decay.

    If either matches M3, the contribution is "warm-start each pathway's scale from a short probe."

  2. The probes cost one self-field run plus six single-pathway runs at 2k steps each: 14k extra steps on top of the 20k-step full run, about 70% more training. They also use the validation set to choose structure. The fixed baselines get neither the extra compute nor that validation signal.

    • The fair comparison is structure selection at the same budget: choose among Dense, PDE and NoCross (or greedily among the 2⁶ subgraphs) using short-run validation, then train the winner.
    • "Lightweight" and "no measurable compute overhead" describe the final run only.
  3. The test sets are too small for the claimed effects.

    • Every system tests on 2 configurations, with 2 more for validation.
    • TRL-scarce trains on 2 configurations; RBC-scarce trains on 1.
    • The reported s.d. covers training seeds only. It ignores configuration-sampling variance, which almost certainly dominates.
    • Most claimed gains are small: TRL-medium 8.47 vs 8.48, AM-abundant 3.28 vs 3.21 (M3 is worse), TRL-scarce 10.36±0.18 vs 10.49.
    • "Improves on the strongest fixed baseline in all four scarce and medium regimes" includes a 0.01 pp tie.

    Please report intervals that account for configurations (bootstrap over test configs and frames, or cross-validation over configuration splits). Without them, only the AM-scarce result clearly stands.

Questions for authors
  • Does Dense, with each K_e's initialization scaled by the probe utility m_e and no gate, match M3?
  • What happens with structure selection at matched compute, i.e., short-run validation over subgraphs followed by full training of the selected graph?
  • Do the M3 gains survive configuration-level confidence intervals?
  • Does a constraint-aware graph (adding U→P and B→P) change the RBC ranking and the AM/TRL span trends?
4 comments
AuthorsOpenReview ↗

Response to Reviewer HD7y (1/2)

We thank the reviewer for a detailed technical review. The controls it proposes are now run, the configuration-level analysis is added, and the paper's description of the gate is rewritten.

W1: Is the gate redundant with the kernel?

"The gate is redundant with the kernel, so the "factorization" is really an initialization and optimization effect. [...]"

The revision adopts this reading. The gate multiplies a linear spectral weight, so it changes the initial scale and the optimization path of a pathway and leaves the function class unchanged. The revision describes it as an evidence-initialized pathway scale (Fig. 1 caption, Section 4.1). Your D->U example is the paper's own evidence: the gate ends at 0.27 while removing the pathway costs 64% (Appendix F, Table 5).

We ran the baseline you propose (Appendix T, Table 16). The gate is removed and each KeK_e is initialized at its probe-based scale mem_e. This arm matches M3 in all four scarce and medium cells of AM and TRL, on the same three seeds:

CellGate-free armM3
AM-scarce6.445%6.468%
AM-medium3.981%3.972%
TRL-scarce10.48%10.46%
TRL-medium8.51%8.47%

The contribution is the probe-based warm start of each pathway's scale, and the gate is one way to apply it.

The paper adds one thing to that statement: the warm start has to be informed. M2 uses the same gates with a uniform initialization, and it gives 6.89% on AM-scarce and 10.76% on TRL-scarce, which is no better than the fixed structures (Table 7).

M2 also addresses the weight-decay point. M2 has the same factored form and the same AdamW weight decay as M3 (Appendix D), so any implicit group-sparsity penalty acts on both. M2 shows no gain and M3 does. The gate-free arm has no factored form and it matches M3. The gain therefore comes from the probe initialization.

W2: Probe cost and a matched-compute comparison

"The probes cost one self-field run plus six single-pathway runs at 2k steps each: 14k extra steps on top of the 20k-step full run, about 70% more training. [...]"

Your count is the one the revision reports: seven 2,000-step runs per regime, which is 70% of a full run in raw steps and 30% in full-model-equivalents (Section 4.5, Appendix N). Section 4.5 now limits the no-overhead statement to the final run.

We ran the comparison you describe, in two forms (Appendix T, Table 15).

In the first form, nine candidate structures get a 1,560-step run each: the PDE graph, dense coupling, no coupling and the six single-pathway graphs. Together this is the probe's step budget. The structure with the best validation error is then trained for 20,000 steps. Over five seeds this baseline reaches 7.40% on AM-scarce against 6.44% for M3, and 0.0878 on RBC-scarce against 0.0833. M3 is lower on every seed in both cells.

In the second form, all 64 subgraphs enter a successive-halving search with at least the same step budget (64 candidates at 100 steps, 16 at 300, 4 at 800). On three AM-scarce seeds it gives 6.61%, 6.82% and 7.56%, against 6.50%, 6.46% and 6.45% for M3. M3 is lower on every seed.

In these cells, extra compute and a validation signal do not reproduce the gain.

On the validation split: the probe uses it only to set the initial scales. Checkpoint selection uses it for every arm, the fixed baselines included.

AuthorsOpenReview ↗

Response to Reviewer HD7y (2/2)

W3: Test-set size and configuration-level intervals

"The test sets are too small for the claimed effects. [...]"

The test sets are larger than Appendix B of the submission stated, and the revision gives the correct sizes. They are 21 held-out configurations on Active Matter and 9 on the turbulent radiative layer. Only Rayleigh-Benard has 2. The validation sets are 16, 9 and 2 configurations (Section 3.1, Appendix B).

The revision adds the configuration-level analysis you ask for (Section 5.6, Appendices Q and S), and Appendix T adds two seed-paired tests (Table 17):

  • AM-scarce, seeds: a paired reduction of 0.298 pp (95% CI [0.010, 0.586], 3/3 seeds; Table 11).
  • AM-scarce, 21 test configurations: a mean paired change of -0.30 pp (95% CI [-0.86, +0.12]). The gain sits in the hardest configurations, where M3 removes the two largest failures of the fixed graph (3.9 and 3.6 pp).
  • TRL-scarce, 9 test configurations: M3 improves 7 of 9, with a median change of -1.1% (Table 14).
  • RBC-scarce, seeds: a paired reduction of 0.0041 against dense coupling (95% CI [0.0021, 0.0060], 3/3 seeds). The method was frozen before this system was run.
  • Dense coupling as the reference: M3 is lower on every seed in all four scarce and medium cells of AM and TRL, by 8.4%, 5.0%, 9.5% and 3.4%. All four paired intervals exclude zero.

In the small cells, TRL-medium is a tie and AM-abundant is 0.07 pp behind the PDE graph. Section 5.2, the Fig. 4 caption and the abstract now say so.

Q1: Gate-free scaled initialization

"Does Dense, with each K_e's initialization scaled by the probe utility m_e and no gate, match M3?"

Yes, in all four scarce and medium cells of AM and TRL. The differences are between 0.01 and 0.04 pp (table under W1; Appendix T, Table 16).

Q2: Structure selection at matched compute

"What happens with structure selection at matched compute, i.e., short-run validation over subgraphs followed by full training of the selected graph?"

We ran it on AM-scarce and RBC-scarce, and it stays behind M3 in both. Selection among nine structures gives 7.40% against 6.44% on AM-scarce and 0.0878 against 0.0833 on RBC-scarce, with M3 lower on 5/5 seeds in each. Selection over all 64 subgraphs gives 6.61%, 6.82% and 7.56% on three AM-scarce seeds, against 6.50%, 6.46% and 6.45% for M3 (see W2; Appendix T, Table 15).

Q3: Configuration-level confidence intervals

"Do the M3 gains survive configuration-level confidence intervals?"

On AM-scarce the seed-paired interval excludes zero. Over the 21 configurations the mean change has the same size (-0.30 pp), with an interval of [-0.86, +0.12], and the improvement is concentrated in the hardest configurations. On TRL-scarce, 7 of 9 configurations improve. Rayleigh-Benard has two test configurations, and Table 12 reports them separately. Its seed-paired interval at scarce data excludes zero.

Q4: A constraint-aware graph on Rayleigh-Benard

"Does a constraint-aware graph (adding U->P and B->P) change the RBC ranking and the AM/TRL span trends?"

We trained this graph (Appendix T): the PDE graph plus U->P and B->P, at scarce data, with three seeds. It reaches 0.0853. This is better than the PDE graph (0.0880) and dense coupling (0.0872), so the constraint edges help a fixed structure. M3 remains the lowest at 0.0831.

This agrees with what M3 learns. On Rayleigh-Benard all six directed pathways are candidates (Table 1, Table 10), and M3 gives U->P a high strength (0.89 at scarce data, 0.50 at abundant data) and B->P a low one (0.24 and 0.21).

If the revision addresses your concerns, we would appreciate it if you would consider updating your score.

AuthorsOpenReview ↗

Follow-up to Reviewer HD7y

We would like to follow up on our rebuttal before the author-reviewer discussion closes on Nov. 18 at 11:59 p.m. AoE. The rebuttal answers each of your comments in detail. In particular, it covers the gate-free arm that you asked for (Appendix T, Table 16) and the structure selection at matched compute. Selection among nine structures reaches 7.40% on AM-scarce against 6.44% for M3, and selection over all 64 subgraphs stays behind M3 on all three seeds (Table 15). The rebuttal also covers the configuration-level intervals (Section 5.6, Tables 12 and 14) and the constraint-aware graph on Rayleigh-Benard, which reaches 0.0853 against 0.0831 for M3. We hope these additions give a fuller picture of the work.

We would also like to highlight the main result of the paper. PDE-derived coupling structure is already a common inductive bias in neural operators for multi-field systems, and such models are usually compared with an unconstrained baseline. Our design holds the backbone, the edge budget and the evaluation fixed, and each cross-field pathway can be imposed, removed or replaced. It shows that the value of a coupling graph comes from individual pathways. One pathway per system dominates and grows with data (Active Matter D->U: +10.6% -> +64.4%; turbulent radiative layer U->R: -0.8% -> +31.4%), and the harm of a wrong graph follows which pathway it removes (Spearman 1.00 on TRL). The method that follows from this study was frozen before it was tested on Rayleigh-Benard convection, where it is 4.7% below the best fixed structure at scarce data. These results are, to the best of our knowledge, new in the literature on physics-guided neural operators. Your review calls the pathway-level harm result a useful finding, and we hope the new controls complete this picture.

We would be very grateful if you could read the rebuttal before the discussion closes and tell us what you think. If it resolves your concerns, we kindly ask you to consider raising your score. If anything remains open, please let us know and we will answer right away.

Thank you for your time and consideration. We look forward to your reply.

Reviewer_HD7yOpenReview ↗

pathway-intervention study is useful, but the method claims are not strongly supported.

The new experiments are useful, but they narrow the contribution instead of supporting it. The gate-free arm and M2 show that M3's gain comes entirely from setting each pathway's initial scale with a short probe. The gate and its "trust" reading add nothing, and on AM the frozen probe scales (6.39) do as well as joint training (6.44). The matched-compute selection baseline (7.40%) does worse than simply using the PDE graph (6.77%). That shows short-run selection is noisy, not that M3's gain can't be bought with extra compute, and a comparison that trains the fixed baselines for the same total steps is still missing. The authors' own configuration-level analysis gives an AM-scarce interval of [−0.86, +0.12], which includes zero, with the gain concentrated in two configurations.

The pathway-intervention study is still valuable, but the method claims are not supported strongly enough to justify a higher score.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response to All Reviewers (1/3)

We thank all four reviewers for their careful reading. We are glad that the reviews agree on the core of the paper. The controlled design holds the edge budget and the capacity fixed, so the interaction information is the only thing that changes (Reviewers tBGr, dD1F, HD7y). The pathway-level findings are useful beyond this paper (Reviewers dD1F, HD7y). The method was frozen before it was tested on a third system (Reviewer tBGr), and its motivation is clear (Reviewer 24V4).

The reviews share three questions: how reliable the gains of the method are, whether the probe makes the comparison unfair, and what the gate is. The revision answers each with new analyses, and this response summarizes them. Part 3 lists the additional controls that we ran for this response. Each reviewer also receives a point-by-point reply. Section, table and appendix numbers refer to the revised PDF, where changes are marked in blue.

1. What the paper shows

The paper has two parts. The first is a controlled study. Each cross-field pathway can be imposed, removed or replaced while the backbone, the edge budget and the evaluation stay fixed. This lets us ask how the value of a PDE coupling graph changes with the amount of data. The second part is a method that follows from the study. M3 gives every candidate pathway a scalar strength and initializes it from a 2,000-step probe. Following the reviewers' suggestion, the revision describes this scalar as an evidence-initialized pathway scale (Fig. 1 caption, Section 4.1).

2. Main results

QuestionResultWhere
Does the value of a coupling prior follow the amount of data?No. The benefit of the correct graph over dense coupling falls with data on both systems (4.11 -> 2.13% and 7.96 -> 2.62% of the Dense error). The cost of an equally sized wrong graph rises on Active Matter (1.70 -> 16.04% of the PDE error) and falls on the turbulent radiative layer (9.49 -> 3.54%).Section 3.2, Table 13
Where does the difference come from?From individual pathways. One pathway per system dominates and grows with data (AM D->U: +10.6% -> +64.4%; TRL U->R: -0.8% -> +31.4%), while other valid pathways stay inert. The harm of a wrong graph follows which pathway it removes (Spearman 1.00 on TRL).Section 3.3, Fig. 3
Does probe evidence help?M3 against the best fixed structure: 6.44 vs 6.77 (AM-scarce), 3.97 vs 4.06 (AM-medium), 10.36 vs 10.49 (TRL-scarce), 8.47 vs 8.48 (TRL-medium), and 0.07-0.08 pp behind at abundant data. The same gates without probe evidence (M2) give 6.89 and 10.76 in the two scarce cells.Section 5.2, Tables 2 and 7
Does the frozen method transfer?On Rayleigh-Benard convection, M3 reaches 0.0831 +/- 0.0009 against 0.0872 for the best fixed structure at scarce data (4.7% lower), and matches it at abundant data (0.0751 vs 0.0753).Section 5.5, Table 3
AuthorsOpenReview ↗

General Response to All Reviewers (2/3)

3. Shared concerns

How reliable are the gains? (Reviewers tBGr, dD1F, HD7y) The best fixed structure changes from cell to cell: the PDE graph on Active Matter, no coupling on TRL-scarce, and dense coupling on Rayleigh-Benard. A user cannot know this in advance. M3 is ahead of the best fixed structure in four cells (AM-scarce, AM-medium, TRL-scarce, RBC-scarce), ties it in two, and is at most 0.08 pp behind in the other two. Every fixed structure loses more than that somewhere: dense coupling is 1.07 pp behind on TRL-scarce, and the PDE graph is 6.4% behind on RBC-abundant. Four checks support the gains.

  • AM-scarce, paired seeds: 6.468% against 6.766% for the PDE graph, a reduction of 0.298 pp (95% CI [0.010, 0.586], 3/3 seeds; Appendix Q, Table 11).
  • RBC-scarce, paired seeds: the method was frozen before this system was run. M3 reaches 0.0831 against 0.0872 for dense coupling, a reduction of 0.0041 (95% CI [0.0021, 0.0060], 3/3 seeds; Appendix T, Table 17).
  • Dense coupling as the reference: M3 is lower on every seed in all four scarce and medium cells of AM and TRL, by 8.4%, 5.0%, 9.5% and 3.4%. All four paired intervals exclude zero (Table 17).
  • Held-out configurations: over the 21 AM configurations the mean paired change is -0.30 pp (95% CI [-0.86, +0.12]), and M3 removes the two largest failures of the fixed graph (3.9 and 3.6 pp; Section 5.6). On TRL-scarce, M3 improves on the fixed graph in 7 of 9 configurations (Table 14).

The abstract now reports the paired result and the abundant-data gaps, and Section 5.2 reports the TRL-medium tie.

Is the comparison fair, given the probe? (Reviewers tBGr, HD7y) The revision reports the full cost: seven 2,000-step runs per regime, which is 70% of a full run in raw steps and 30% in full-model-equivalents (Section 4.5, Appendix N). We also ran the control that the reviews ask for (Appendix T, Table 15). A selection baseline spends the probe's step budget on short runs of nine candidate structures, keeps the one with the best validation error, and trains it in full. Over five seeds it reaches 7.40% on AM-scarce against 6.44% for M3, and 0.0878 on RBC-scarce against 0.0833. M3 is lower on every seed in both cells. A search over all 64 subgraphs gives 6.61%, 6.82% and 7.56% on three AM-scarce seeds, against 6.50%, 6.46% and 6.45% for M3. In these cells, extra compute and a validation signal do not reproduce the gain.

What is the gate? (Reviewers tBGr, HD7y) The gate multiplies a linear spectral weight. It changes the initial scale and the optimization path of a pathway, and it leaves the function class unchanged. The revision describes it as an evidence-initialized pathway scale (Fig. 1 caption, Section 4.1). Three results show that the probe evidence carries the gain. First, a gate-free arm that initializes each operator at its probe-based scale matches M3 in all four scarce and medium cells of AM and TRL (Appendix T, Table 16). Second, M2 has the same gates and the same weight decay as M3 but a uniform initialization, and it shows no gain (6.89 and 10.76, against 6.44 and 10.36 for M3). Third, freezing the probe-initialized gates gives 6.39 on AM-scarce and 10.44 on TRL-scarce, close to M3 (Table 8).

Is the probe a stable signal? (Reviewer 24V4) We ran two checks (Appendices H and T). Short leave-one-out probes rank the pathways of the PDE graph in the same order as the converged removal effects, on both AM and TRL (Spearman 1.00). Short single-pathway runs repeated over five seeds give the same top pathway in every seed on AM. Its initial scale is 0.88 in all five seeds, and the other five pathways stay between 0.12 and 0.18.

Scope (Reviewers dD1F, 24V4) All three systems have three field groups and six candidate pathways (Table 1). Section 6 states this scope and what a system with many fields would need. The central scaling result also holds for multi-step rollouts up to horizon 10 (Appendix K).

AuthorsOpenReview ↗

General Response to All Reviewers (3/3)

4. Additional controls

Each control answers a specific request in the reviews. All errors are test relative L2. They are reported in Appendix T of the revised PDF (Tables 15 to 17) and in Appendix H.

ControlResultRequested by
Structure selection with the probe's step budget. Nine candidates (PDE graph, dense, no coupling, six single-pathway graphs) get 1,560 steps each. The best one by validation is trained for 20,000 steps. Five seeds.AM-scarce: 7.40% against 6.44% for M3. RBC-scarce: 0.0878 against 0.0833. M3 is lower on 5/5 seeds in both cells.tBGr, HD7y
Selection over all 64 subgraphs by successive halving, with at least the same step budget. Three seeds.AM-scarce: 6.61%, 6.82%, 7.56% against 6.50%, 6.46%, 6.45% for M3. M3 is lower on 3/3 seeds.HD7y
Gate-free arm. Each operator is initialized at its probe-based scale and the gate is removed. Same three seeds as M3.AM-scarce 6.445% against 6.468% for M3. AM-medium 3.981% against 3.972%. TRL-scarce 10.48% against 10.46%. TRL-medium 8.51% against 8.47%.tBGr, HD7y
Short leave-one-out probes. The PDE graph and each graph with one pathway removed are trained for 2,000 steps. Three seeds, scarce data.The ranking of the pathways equals the ranking by converged removal effect on AM (4 pathways) and on TRL (5 pathways). Spearman 1.00 on both.24V4
Probe stability. Short single-pathway runs (1,560 steps) are repeated over five seeds.AM: U->D has the highest utility in 5/5 seeds, and the min-max map gives it 0.88 in every seed. The other five pathways stay between 0.12 and 0.18. RBC: U->P and U->B take the top two places in 5/5 seeds.24V4
Constraint-aware fixed graph on RBC-scarce (PDE graph plus U->P and B->P). Three seeds.0.0853, against 0.0880 for the PDE graph, 0.0872 for dense coupling and 0.0831 for M3.HD7y
Seed-paired test on RBC-scarce against dense coupling.Reduction of 0.0041 (95% CI [0.0021, 0.0060], 3/3 seeds).dD1F, tBGr, HD7y
Seed-paired tests against dense coupling in the four scarce and medium cells of AM and TRL.M3 is lower by 8.4%, 5.0%, 9.5% and 3.4%. 3/3 seeds in every cell, and all four intervals exclude zero.dD1F, tBGr, HD7y

5. Summary of changes in the revised PDF

ChangeWhere
Benefit and harm reported as two contrasts; each claim names its contrastAbstract, Section 3.2, Table 13
Gate described as an evidence-initialized pathway scaleFig. 1 caption, Section 4.1
Seed-paired test on AM-scarceSection 5.6, Appendix Q, Table 11
Configuration-level results on 21 AM and 9 TRL test configurationsSection 5.6, Tables 12 and 14
Probe cost in raw steps and in full-model-equivalentsSection 4.5, Appendix N
Abstract reports the paired result and the abundant-data gapsAbstract, Section 5.2, Fig. 4 caption
Split sizes and seed counts stated for every cellSection 3.1, Appendix B, Table 2 caption
Scope and scaling to many fieldsSection 6
Additional controls listed aboveAppendix T, Tables 15 to 17; Appendix H

Please let us know if you have any further questions.

AuthorsOpenReview ↗

Summary of the Rebuttal for the Area Chair and Reviewers

We thank the Area Chair and the reviewers for their time and valuable feedback. As the discussion period is ending, we thought it could be useful to summarize the current state of the rebuttal.

We would first like to highlight the main result of the paper. PDE-derived coupling structure is already a common inductive bias in neural operators for multi-field systems, and such models are usually compared with an unconstrained baseline. Before this work there was, to the best of our knowledge, no controlled test of when such a prior helps. Our paper provides this test. The design holds the backbone, the edge budget and the evaluation fixed, and each cross-field pathway can be imposed, removed or replaced. It shows that the value of a coupling graph comes from individual pathways. One pathway per system dominates and grows with data (Active Matter D->U: +10.6% -> +64.4%; turbulent radiative layer U->R: -0.8% -> +31.4%), and the harm of a wrong graph follows which pathway it removes (Spearman 1.00 on TRL). The method that follows from this study was frozen before it was tested on Rayleigh-Benard convection, where it is 4.7% below the best fixed structure at scarce data. Three of the four reviews list the controlled design as a strength of the paper, and one of them calls it the main contribution.

We initially received scores 2, 3, 2, 2. Based on reviewer feedback, we added the following to our manuscript:

  • New seed-paired tests showing that M3 is lower than the fixed PDE graph on AM-scarce by 0.298 pp (95% CI [0.010, 0.586]) and lower than dense coupling on RBC-scarce by 0.0041 (95% CI [0.0021, 0.0060]). Against dense coupling, M3 is lower on every seed in all four scarce and medium cells of AM and TRL (Appendices Q and T, Tables 11 and 17).
  • A new structure-selection baseline at the probe's step budget. Selection among nine structures reaches 7.40% on AM-scarce against 6.44% for M3, and 0.0878 on RBC-scarce against 0.0833, with M3 lower on 5/5 seeds in both cells. Selection over all 64 subgraphs also stays behind M3 on 3/3 seeds (Appendix T, Table 15).
  • New probe checks showing that short leave-one-out probes rank the pathways in the same order as the converged removal effects on AM and TRL (Spearman 1.00), and that the top pathway on AM is the same in 5/5 seeds (Appendices H and T).
  • A gate-free arm with probe-based initialization, and a description of the gate as an evidence-initialized pathway scale (Section 4.1, Appendix T, Table 16).
  • A configuration-level analysis on 21 AM and 9 TRL test configurations (Section 5.6, Tables 12 and 14).
  • A constraint-aware fixed graph on Rayleigh-Benard (Appendix T).
  • A full account of the probe cost in raw steps and in full-model-equivalents (Section 4.5, Appendix N), and a discussion of the scope of the study (Section 6).

During the discussion phase, we provided detailed responses to the concerns raised by all four reviewers and revised the paper accordingly. The rating changes and current status of the four reviewers are as follows:

ReviewerOriginal RatingAfter Discussion
tBGr22 (No Reply)
dD1F32 (Changed on Nov. 17, before our response; No Reply)
24V422 (No Reply)
HD7y22 (No Reply)

Reviewer dD1F's rating changed from 3 to 2 on Nov. 17, a few hours before our rebuttal was posted. The text of the review stayed the same, and we have asked the reviewer whether a further concern led to this change.

Although the reviewers have not replied yet, our responses have addressed all of their comments. Given this, we hope the Area Chair can take our experiments and clarifications into account. We also kindly ask the reviewers to read our responses and to update their scores if their concerns are resolved.

We are extremely grateful for the feedback given by the reviewers, and sincerely believe their recommendations improved the paper. We are happy to answer any question about the paper or the rebuttal.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.