acceptodds
Under review as a conference paper at ICLR 2027

Executable Neural Operators: Routing Local Physical Mechanisms in Continuous Spatial Dynamics

Abstract

While Neural Operators and Physics-Informed Neural Networks (PINNs) have accelerated the simulation of continuous spatial data, they predominantly rely on global implicit mappings. This monolithic representation can entangle local physical rules, rendering models brittle to non-stationary mutations in complex media, such as sudden changes in boundary conditions or unobserved subsurface scatterers. To address this limitation, we propose the Executable Neural Operator (ENO), a novel architecture that represents localized physical mechanisms within continuous fields. By leveraging a unified bipartite graph embedding, ENO tokenizes large-scale continuous spatial data into a routed graph. Instead of global spectral mixing, ENO employs Dynamic Operator Routing (DOR) to assign localized neural programs to simulate wave propagation and energy transfer. During medium property mutations, our Targeted Mechanism Revision (TMR) protocol provides gradient isolation under explicit locality and parameter-ownership conditions. We extensively evaluate ENO on high-contrast 3D magnetotelluric dynamics. Results demonstrate that ENO achieves competitive zero-shot accuracy on unseen out-of-distribution geometries while suppressing the physics violation rate and reducing background prediction drift, establishing a theoretical and empirical basis for mechanism discovery in non-stationary physical systems.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Neural Operators, Physics-Informed Machine Learning, Dynamic Routing, Causal Disentanglement, Non-stationary PDEs, Continual Learning, Magnetotellurics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LbD1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Executable Neural Operators (ENO), which replaces global parameter sharing with bipartite tokenization, dynamic routing over localized graph-neural mechanisms, and Targeted Mechanism Revision (TMR) enforcing a gradient firewall during online adaptation. Theoretically, the authors provide conditional guarantees on Sobolev approximation, finite covering of local Green's operators, architecture-conditioned error isolation, and causal identifiability. Experiments on 3D magnetotelluric inversion show that ENO achieves accuracy parity with FNO on zero-shot OOD generalization while substantially reducing physics violations.

Critical strengths
  1. The paper addresses a well-motivated problem: updating a model to accommodate a localized change can disrupt predictions on the original background input. The proposed “gradient firewall” tackles this through explicit restrictions on parameter updates, offering a concrete architectural approach to reducing interference.

  2. The adaptation experiments include useful controls. Table 7 compares update policies from the same checkpoint, including routed TMR, rotated expert selection, updates to all mechanisms, and full-parameter updates. These comparisons help isolate the effect of selecting which parameters to adapt. Background prediction drift is also a valuable metric because it directly tests the model’s ability to retain its previous behavior.

Critical weaknesses
  1. The evaluation remains small in scale. The main results use an (8^3) grid, 48 training samples, and ten online adaptation steps. The (12^3) experiments in the appendix provide an additional check, but both settings are still far smaller than realistic field-scale geophysical problems. It remains unclear whether the accuracy, computational efficiency, and retention benefits persist at substantially larger scales.

  2. The zero-shot results do not demonstrate an accuracy advantage over FNO. ENO has a slightly higher mean NMSE of 0.471, compared with 0.4623 for FNO. Given the reported uncertainty, these results are better interpreted as comparable accuracy. The case for ENO therefore rests mainly on its adaptation and retention benefits, rather than improved zero-shot prediction.

Questions for authors
  1. How should the mechanism-library size (K) be chosen as the coefficient class (\mathcal{A}_R) becomes more complex, for example through additional interfaces or larger conductivity contrasts? Is there a computable lower bound or a practical selection criterion for (K) at a given approximation tolerance, rather than choosing it through validation alone?

  2. How does TMR handle cases where the same expert is used in both the anomaly and background regions? Updating that expert could affect background predictions through shared parameters or subsequent message passing. Could the authors clarify how the evaluated implementation enforces the parameter-ownership and dependency conditions required for isolation, and what accounts for the remaining nonzero background drift?

1 comment

Most of my concerns have been addressed. That said, I am willing to raise my score to 3, though I would not object to the paper being rejected.

Reviewer_X6XgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes the Executable Neural Operator (ENO) for learning solution operators of parametric partial differential equations in heterogeneous media. It pursues two goals: zero-shot generalisation to unseen coefficient compositions, and localised online adaptation to a non-stationary coefficient mutation without loss of accuracy on the unchanged background. Global spectral mixing is replaced by four components. A bipartite graph tokenisation samples observation nodes on the known input field and mechanism anchors by farthest-point sampling or physics-informed clustering, connects them within a support radius R, and encodes each anchor through a local integral transform that retains zeroth and first moments (§3.1). A library of K local graph-neural mechanisms is selected by hard, coefficient-conditioned routing, implemented as Gumbel-softmax top-1 with a straight-through surrogate (§3.2, Eq. 5). Targeted Mechanism Revision (TMR) fixes the routes, freezes the shared encoder, the decoder and the non-selected mechanisms, and updates only the parameters owned by the mechanism or mechanisms activated in the anomaly (§3.3). A bilevel episodic meta-objective couples a localised PDE residual with fast adaptation on synthetic anomaly episodes and a background-retention penalty (§3.4). Four conditional results are given: a Sobolev approximation bound for tokenisation (Theorem 1), a finite ε-covering of local Green’s operators that motivates a finite mechanism library (Theorem 2), an error-isolation statement that combines a hyperbolic cone-of-influence argument with a computation-graph dependency argument (Theorem 3), and an identifiability result relating regimes to experts (Theorem 4).

The empirical study uses a three-dimensional frequency-domain magnetotelluric forward problem solved in SimPEG at 1 Hz on an 8×8×8 mesh with 500 m cells, together with a 12×12×12 domain at the same cell width, 48 and 64 training samples respectively, and three seeds. ENO is compared with FNO, DeepONet, PINN, MoE-FNO, TFNO, U-NO, an adapted GAOT and an adapted EqGINO. The stated results are accuracy parity with FNO on compositional out-of-distribution conductivity fields (NMSE 0.471 against 0.4623), a lower physics violation rate, and reduced background drift after ten supervised adaptation steps to a localised anomaly (0.068 against 1.028 for FNO).

Critical strengths
  1. The problem is formulated at a useful level of abstraction. Distinguishing an operator that is accurate on average from one that can be patched locally without perturbing the rest of the field identifies a practical obstacle in surrogate deployment, and treating adaptation as an ownership problem over parameters under a fixed computation graph is a workable way to address it (§1, §3.3).

  2. The ablation matrix contains the controls that comparable studies often omit. Table 7 includes an “all mechanisms only” row and a “rotated selection” row. Rotated selection holds the number of routed mechanisms fixed while permuting their assignments, and it degrades anomaly recovery to 0.580, which separates the effect of mechanism specialisation from the effect of freezing alone.

  3. The “no TMR (same weights)” row is an appropriate control. Reusing a single checkpoint isolates the adaptation protocol from pretraining quality, and this row supports most of the causal claim that the paper makes (Tables 3 and 5).

  4. The authors state the limits of their theoretical results explicitly. The closing passages of §3.1, §3.3 and §3.4 delimit what the theorems do not imply, and Appendix D.7 lists counterexamples to the identifiability theorem, including coefficient-conditioned side channels, disjoint supports and episode-specific parameter states. Reporting these qualifications is preferable to leaving them for the reader to discover.

  5. Parameter efficiency is a substantive result that the text does not report. ENO reaches FNO-comparable out-of-distribution error with 25,536 parameters against FNO’s 592,428 at 8³, and with 61,249 against 5,313,772 at 12³ (Appendix E, Table 9).

  6. The 12³ study changes the domain extent rather than the resolution, and the caption states this. Reporting the confound instead of presenting the larger mesh as a resolution gain is appropriate practice (Table 9 caption).

Critical weaknesses
  1. The physical-consistency claims are not adequately supported by the reported metrics.

PVR is not clearly defined, and its ranking is inconsistent with field accuracy. For example, at (12^3), EqGINO achieves an NMSE of 0.9995, close to a zero predictor, yet its PVR is comparable to ENO's, while FNO achieves a much better NMSE of 0.2146 but a PVR of 0.992. Moreover, removing the physics loss changes PVR from 0.224 to 0.9585, suggesting that the metric is strongly influenced by the training objective. The claimed order-of-magnitude improvement is also overstated: the reported reductions are approximately 4.4–5.4 times.

  1. The baseline comparisons and performance claims are insufficiently justified.

Several baselines report NMSE values near 1.0, suggesting that they failed to learn meaningful predictors under the evaluated configurations. Without evidence of adequate tuning and convergence, these results cannot support broad claims about architectural superiority. The comparison with FNO is also mixed: ENO achieves better relative L2 error but slightly worse NMSE. In addition, the most relevant architectural baseline, MoE-FNO, is neither properly cited nor sufficiently discussed. A more controlled comparison is needed to isolate the benefits of ENO's architecture.

  1. The theoretical results do not adequately cover the experimental setting.

The assumptions of the four theorems do not appear to hold for important aspects of the evaluated problem. In particular, the coefficient class used in the experiments is not precompact in (L^\infty), contrary to Theorem 2's assumption, while Theorem 3 explicitly does not apply to the frequency-domain magnetotelluric system. Theorem 4 also assumes the absence of a regime-dependent side channel, which the proposed routing mechanism does not satisfy. These issues primarily concern the scope of the theoretical guarantees rather than the correctness of the derivations.

  1. The adaptation protocol relies on supervision that may be unavailable in the motivating application.

The TMR adaptation procedure uses ground-truth field values inside the anomalous region. However, in practical magnetotelluric inversion, subsurface fields are unknown and observations are typically collected at the surface. The paper does not adequately explain how the required adaptation signal would be obtained in this setting. Furthermore, the reported drift comparison is not matched for the number of updated parameters, making it difficult to attribute the improvement specifically to the routing mechanism.

  1. The experimental evidence and reproducibility are limited relative to the strength of the claims.

The evaluation uses only three training seeds and eight samples for each of the OOD and mutation tests, limiting the reliability of the reported uncertainty estimates. Claims of definitive mechanism identification are particularly strong given an ARI of 0.482. Important implementation details and hyperparameters are also missing, and no accessible code or persistent artifact is identified. Finally, some claims in the abstract and main text, including zero background drift and state-of-the-art generalization, are inconsistent with the reported results.

Questions for authors
  1. What is the exact definition of PVR, including the threshold, the normalisation, the mask, and whether the quantity is evaluated from Eq. 14 at the predicted field? Would the authors report the PVR of a zero predictor as a reference row? At present EqGINO with NMSE 0.9995 attains 0.2385 while FNO with NMSE 0.2146 attains 0.992, which suggests that the metric rewards low-amplitude or spectrally smooth predictions rather than accuracy.
  2. Was PINO, that is, FNO trained with the same Eq. 11 residual and the same λ, trained and evaluated? Since removing the physics loss moves PVR from 0.224 to 0.9585 with little change in NMSE, how much of the gap to FNO is attributable to routing and how much to the objective?
  3. Were GAOT, EqGINO, DeepONet and TFNO trained to convergence, including a learning-rate sweep, an adequate epoch budget and comparable capacity? Their NMSE is 1.000 ± 0.003, which is the value of a zero predictor. If they were limited by capacity or budget, the comparison reduces to ENO against FNO and MoE-FNO, and the assessment would change accordingly.
  4. Are the adaptation observations S_mut restricted to quantities observable in the magnetotelluric setting, such as surface fields at stations, or do they include ground-truth subsurface fields inside Ω_mut? If the latter, what deployment path do the authors have in mind?
  5. Given the argument in Appendix C.5 that piecewise-constant coefficients with interfaces are not precompact in L∞, does the data generator satisfy the hypothesis of Theorem 2? If not, which result should a reader retain as support for dynamic operator routing, and what observation would have counted against the design?
  6. How were K = 3 and M = 33 selected, and how does performance behave in their immediate neighbourhood? The reported variants bracket the operating point at 23 and 43 anchors and at K = 1 and K = 5, but the default values are not stated.
  7. Is background drift an absolute or a relative change, and is it normalised by the pre-adaptation background prediction? Table 6 suggests that the baselines also have much higher background error in absolute terms, which affects how the column should be read.
Reviewer_xvjxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Executable Neural Operators (ENO), a modular architecture for adapting PDE surrogates to localized changes in material properties while preserving previously learned predictions. It combines bipartite graph tokenization, Dynamic Operator Routing (DOR) over a library of localized neural mechanisms, and Targeted Mechanism Revision (TMR) to update selected mechanism parameters. Physics-informed losses and episodic meta-training encourage physical consistency and reduce forgetting. The paper also provides conditional theoretical results concerning approximation, mechanism representation, isolation, and identifiability.

Validation uses 3D frequency-domain electromagnetic simulations on 838^3 and 12312^3 grids with varying conductivity distributions. ENO reports competitive forward-prediction accuracy relative to FNO, improved adaptation to localized conductivity changes, and substantially less drift in previous predictions under the evaluated adaptation protocols. Ablation studies examine the contributions of routing, physics losses, meta-training, and selective parameter updates.

Critical strengths
  1. A meaningful adaptation problem with promising empirical results. The paper evaluates both adaptation to localized material changes and retention of predictions on unchanged inputs. On the 838^3 benchmark, ENO + TMR reduces anomaly NMSE from 0.4520.452 to 0.2850.285, with reported background drift of 0.0680.068. This joint evaluation addresses an important practical requirement for reusable PDE surrogates and provides encouraging evidence that the proposed framework can accommodate changes while limiting disruption to earlier predictions.

  2. Controlled evidence for selective revision within ENO. Table 7 compares adaptation policies from the same checkpoint using the same learning rate and ten update steps. Routed TMR achieves anomaly NMSE/background drift of 0.285/0.0680.285/0.068, compared with 0.410/0.2800.410/0.280 for full-parameter updates and 0.292/0.1850.292/0.185 for updating all mechanisms. These controls strengthen the evidence that selective mechanism updates contribute to retention within the proposed architecture, beyond merely freezing shared encoder and decoder parameters.

Critical weaknesses
  1. The interpretation as physical mechanism discovery exceeds the demonstrated results. Theorem 4 and Appendix D.7 explicitly acknowledge that the identifiability result does not establish causal identification for unrestricted, coefficient-conditioned ENO. Nevertheless, Figure 3 interprets agreement between routing assignments and mutation labels as confirmation of causal identifiability. Such agreement, together with the rotated-selection control, supports learned specialization but does not establish correspondence to independently identifiable physical mechanisms. This gap is consequential because mechanism discovery is central to the paper’s claimed scientific contribution.

  2. The added value over alternative adaptation methods remains insufficiently established. The same-checkpoint controls support selective revision within ENO, but the comparisons across architectures do not separate mechanism organization from physics supervision, additional episodic meta-training, and restrictions on parameter updates. In particular, no matched alternative operator with comparable adaptation training and a restricted-update strategy is reported. Consequently, the experiments leave unresolved whether ENO’s particular construction offers an advantage over other methods designed for adaptation and retention. This limits the evidence for the method’s broader contribution without negating its demonstrated within-model benefits.

Questions for authors
  1. Physical mechanism identification. Section 3.4 and Appendix D.7 acknowledge that the identifiability theorem does not establish causal identification for unrestricted ENO. What additional evidence supports Figure 3’s interpretation of routing alignment as causal identifiability, beyond specialization by conductivity or spatial region?

  2. Baseline adaptation protocol. For FNO and MoE-FNO, what losses are used during adaptation, and which parameters are updated? Given the additional episodic training specified for ENO, how should the reported comparisons distinguish architectural benefits from differences in preparation for adaptation?

  3. Suggested additional experiment—generality and connection to theory. All current experiments concern frequency-domain electromagnetic simulations. Could the authors evaluate a qualitatively different PDE family, preferably a time-domain wave problem with localized coefficient changes that directly connects to Theorem 3? Measuring both accuracy on the mutated solution and retention on unchanged inputs would help establish whether the proposed adaptation strategy extends beyond the current application. If such evidence is unavailable, please narrow the claims according to the evaluated setting.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.