acceptodds
Under review as a conference paper at ICLR 2027

SymOnet: Expression-driven Foundation Model for Multi-operator Learning with Data-free Fine-tuning

Abstract

Operator foundation models have advanced PDE solving through large-scale pre-training, enabling shared representations and transfer across physical systems. However, training and adapting these models typically require substantial high-fidelity solution data, which are expensive to obtain through numerical simulations or observations. To reduce this dependence during downstream adaptation, we introduce the Symbolic expression-driven Operator network foundation model (SymOnet) for data-free fine-tuning guided by governing equations, initial conditions, and boundary conditions. SymOnet learns a shared operator representation across heterogeneous PDE families through a token-query architecture that maps discretized input functions and space–time coordinates to solution values. Multi-PDE pre-training builds this representation, while downstream adaptation optimizes only lightweight adapters using expression-driven losses evaluated through automatic differentiation, keeping the shared backbone frozen. We establish a family-wise approximation theorem showing that a shared query basis can support operator approximation across a family of PDEs. At comparable model sizes, SymOnet outperforms all evaluated baselines on all 12 tasks under expression-driven fine-tuning, while also attaining the lowest L2RE on 9 tasks under data-driven fine-tuning. Additional experiments on three-dimensional compressible Navier–Stokes dynamics demonstrate transfer from two-dimensional pre-training to both fine-tuning regimes.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Neural Operators, PDE Foundation Models, Multi-operator Learning, Physics-informed Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_67BYOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SymOnet, a transformer-based network capable of predicting the PDE solutions at arbitrary space/time query points. The model is pretrained on variety of 2d datasets. It is tested on several downstream tasks, with some of them not seen during the pretraining. The backbone is frozen downstream, while only light adapters are trained. They use physics-based, as well as data-based loss functions for finetuning.

Critical strengths
  • The problem the authors tackle is very relevant for the community, as adapting the pretrained PDE models using only physics loss is a core issue.
  • The design of the architecture makes it possible to differentiate the model at arbitrary query point.
  • A lot of different baselines are tested in their experiments.
  • The ablation studies are well-designed.
  • The 2d to 3d transfer is very valiuable experiment that shows physics transfer between different number of dimensions
  • The appendix is very well-written and detailed.
Critical weaknesses
  • Comparisons against the pretrained models, such as Poseidon, DPOT or Walrus in physics-informed settings are not fair ones, given the design of those models. Those models are not designed to be differentiable with respect to query points. Their default design is suitable for data-driven downstream tasks. Finite-difference scheme used in their finetuning may lead to poor estimates of the partial derivatives, particularly the temporal derivatives, which could explain their weak downstream performance. I suggest the authors to report SymOnet finetuning performance with the same finite-difference approach, so that we can separate the effect of the architecture and pre-training from the derivative estimator.
  • 8 of the 12 tasks in Table 2 are included in the pre-training. I believe that one of the points of pretrained models in this field is to use them on unseen tasks, so the pre-training comparisons are not very useful. Regarding the out-of-training tasks, the CNS task is an unseen combination of seen parameters of already seen tasks (very similar data distribution). This leaves us with 3 out-of-training tasks. This number of tasks is not enough to judge the performance of a pre-trained model/pipeline. I suggest the authors to run additional experiments, such as some of the multi-scale Navier-Stokes, Euler, MHD or wave experiments from Walrus or Poseidon works. The 2d to 3d transfer is a useful experiment that suggests that the SymOnet may perform well to the new, unseen and generally difficult downstream tasks.
  • The previous concerns also lead to the conclusion that the data-free framing is overstated, since in majority of the table 2 experiments, the model has already seen the data during the pretraining.
  • The authors set Tpre=1,Tout=5T_{pre} = 1, T_{out} = 5, meaning that only 5 rollouts are applied. It is unclear how much the trajectories are actually moving in this time frame. For SWE in particular, in Table 5, we observe that the frozen model reaches ∼0.02\sim 0.02 error in zero-shot settings, despite never seeing a single sample from this dataset. The authors should include the trivial persistence baseline ut+1=utu_{t+1} = u_{t} and state the L2RE for this baseline for all the OOD experiments. Moreover, the authors should report the errors at longer rollouts, and show how the errors behave in time for the SymOnet against the best-performing baselines.
  • It is unclear to me whether the data- and the expression-driven experiments are tested on different time windows. The use of 10 history frames for data-driven models suggests that the testing time-frames for two different regimes differ.
  • The authors do not show actual samples and predictions anywhere in the paper or the appendix. It is hard to judge how difficult the tasks are, since no visualization is provided.
Questions for authors
  1. For the grid-based baselines, how exactly is the temporal derivative computed?

  2. What are the zero-shot errors of the baselines (without any downstream training)?

  3. What is the error of the frozen pre-trained SymOnet (no downstream training) on the 8 pre-training tasks from the main table?

  4. Could the authors confirm that CNS (1, 1e-8), DR, SWE and CFDBench were excluded from pretraining for SymOnet and other baselines?

  5. For SWE, what is the time step between the evaluated frames?

Reviewer_ZYrFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SymOnet, a foundation model for PDEs designed to adapt to unseen equations without requiring additional high-fidelity simulation data. The approach relies on pretraining a neural operator on solution data from multiple PDEs, followed by fine-tuning in which most network parameters are frozen and only a small set of lightweight adapters is optimized. The framework introduces a unified objective combining data-driven supervision and physics-informed losses based on the governing equations and initial and boundary conditions, allowing downstream adaptation using either simulation data or exclusively physical constraints, similarly to PINNs.

The effectiveness of the proposed framework is demonstrated through experiments in which SymOnet is pretrained on datasets from the original FNO paper and PDEBench, and subsequently evaluated on a range of unseen downstream tasks, including Navier–Stokes and shallow-water equations. SymOnet achieves the best performance among the evaluated baselines on all 12 tasks under expression-driven fine-tuning and on 9 out of 12 tasks under data-driven fine-tuning. Additional ablation studies demonstrate the benefits of pretraining over training from scratch.

Critical strengths
  1. The paper tackles the problem of the high cost of obtaining simulation data, and proposes a solution under the form of a procedure involving a foundation model which can be fine-tuned to a completely new setting or even PDE without the need of supplemental data, simply by fine-tuning to a physics loss. The paper provides a relatively thorough experimental analysis of the different architectural components and fine-tuning strategies. The ablations demonstrate that removing or replacing individual components consistently degrades performance, while combining all three adapter groups yields the strongest results.

  2. The proposed framework demonstrates consistently strong performance across 12 downstream tasks, including unseen parameter regimes, PDE families, and geometries. In particular, SymOnet outperforms all considered baselines under expression-driven fine-tuning and achieves the lowest error on 9 out of 12 tasks under data-driven fine-tuning.

Critical weaknesses
  1. The authors fail to establish an actual computational trade-off between generating additional computational data and simply fine-tuning the model. They do report that physics-informed fine-tuning requires approximately 1.5 times the wall-clock time and twice the peak GPU memory of data-driven fine-tuning, with experiments conducted on eight A800 GPUs over 200 epochs. However, absolute training times are not provided, nor is there a comparison against the cost of generating a limited number of high-fidelity simulations followed by conventional data-driven fine-tuning.

  2. The authors provide only limited evidence of generalization to 3D problems and complex geometries (only the ones in PDEBench which are pretty simple), while generalization to unseen complex geometries would be a fundamental necessity in fields such as inverse design.

Questions for authors
  1. Could the authors provide a more in-depth analysis of the trade-off between fine-tuning, generating supplemental data, and how these would affect performance ?

  2. In connection to the previous question: the framework introduces a unified loss that combines data-driven and physics-informed objectives, but the main experiments consider these two regimes separately. Have the authors investigated hybrid fine-tuning using a limited number of simulation trajectories together with PDE constraints? Could such a strategy achieve better accuracy or reduce the computational cost of adaptation?

  3. Pre-trained foundation models are of paramount importance for industrial-scale inverse design, where the geometries can be very complex and simulations very costly tp generate. The pretraining datasets considered in this work are primarily based on Navier–Stokes equations on relatively simple geometries. Have the authors considered whether pretraining on simpler PDEs, such as Poisson or diffusion equations, but over a much broader range of complex geometries could improve downstream generalization?

Reviewer_ZqKdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents SymOnet, a multi-PDE operator foundation model for adaptation without high-fidelity data. It freezes a pretrained backbone and updates adapters using delicately designed losses. Experiments cover 12 tasks, OOD PDE families and parameter regimes, model/data scaling, 3D transfer, and component ablations. The results are promising and extensive.

Critical strengths

[+]: This paper studies an important question, i.e., adapting pretrained operator models when downstream solution data are unavailable or expensive.

[+]: Broad evaluation includes unseen parameter regimes, unseen PDE families and etc.. Ablations compare architectural replacements and adapter subsets; the paper also reports the remaining accuracy gap to full-model fine-tuning.

[+]: The paper provides substantial implementation and benchmark details in the appendices.

[+]: This paper is generally well-organized.

Critical weaknesses

[-]: The method is a thoughtful combination of multi-PDE pretraining, parameter-efficient adaptation, and physics-informed losses, but its conceptual novelty over prior physics-informed operator adaptation is moderate.

[-]: The approximation theorem is conditional and based on universal approximation. It does not establish that the proposed pretraining procedure learns the required shared representation. The local adapter analysis does not provide a practical convergence or generalization guarantee.

[-]: The baselines use different methods to compute PDE derivatives and update different parts of the model. Therefore, the results show that SymOnet performs better overall, but do not reveal which design choice gives it the advantage.

[-]: The residual-to-solution analysis does not cover shock-forming nonlinear systems such as the SWE setting without additional stability or entropy assumptions.

[-]: This paper is highly specialized and theoretically dense, such that the reading barrier is prohibitively high for readers unfamiliar with the background. While ensuring the writing remains sufficiently clear and accessible, I recommend that the authors provide more background introduction and visual results to help readers understand.

Questions for authors

See weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.