acceptodds
Under review as a conference paper at ICLR 2027

Cross-Domain Pretraining for Neural CFD Surrogates

Abstract

Neural surrogates for computational fluid dynamics (CFD) have the potential to greatly enhance engineering innovation through accelerating simulation. However, the primary limitation for neural surrogates is the lack of generalization to geometries and applications beyond the training set, which is significant given the diversity of engineering scenarios. Currently, this is addressed by generating a new dataset for a specific application; however, this requires running costly numerical solvers. In this work, we take a step toward addressing this by studying neural surrogates trained across different geometries, boundary conditions, and fidelities. We find that cross-domain pretraining improves zero- and few-shot performance on held-out datasets relative to both training from scratch and transferring from domain-specific experts. In particular, finetuning a pretrained, cross-domain model can achieve 2-3x lower errors at the same sample size and use 8x fewer samples to achieve the same error, compared to training from scratch. This benefit is architecture agnostic and improves with model size and pretraining dataset diversity. Furthermore, we study how and why cross-domain pretraining works in CFD surrogates, and find that simply pooling steady-state datasets is both sufficient and effective. Given the high cost of generating CFD data, leveraging existing datasets through cross-domain pretraining will likely be a valuable strategy as future surrogates expand to tackle new problems and use cases.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural surrogates, computational fluid dynamics, PDEs

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_yNpUOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper works on a practical and critical limitation of current existing CFD surrogates, the limited generalization issue. Authors propose a cross domain pretraining method (by pooling several existing datasets) that can reduce additional CFD simulation data needed to adapt the pretrained model to a new domain. The results show that cross-domain pretraining improves zero-shot and few-shot performance on held-out datasets compared with training from scratch and transferring from domain-specific experts. The authors further study sample efficiency during fine-tuning and how transfer performance changes with pretraining dataset diversity and model size. Authors also plan to release a unified collection publicly available CFD datasets, which could help the community to keep working on the CFD surrogates generalization.

Critical strengths
  1. This paper addresses a practical limitation of current CFD surrogates, the cross domain pretraining method is intuitive and straightforward to adopt across existing surrogate architectures, while the performance improvements are good over training from scratch when there are limited target domain data and training budget.
Critical weaknesses
  1. It is still unclear how well the method generalizes across different Re numbers independently of geometry and other dataset differences, considering the main contribution is to help generalize CFD surrogates. Current work apply only to steady-state and time-averaged flows, whether the method can be extended to unsteady predictions still uncertain and can be discussed.
Questions for authors
  1. Could authors discuss more about how to select datasets for cross-domain pretraining when a new target domain has few or no CFD simulatios yet in practical scenario? When adding a dataset might reduce transfer performance? Are there any general criteria?
  2. Could the authors provide training and validation curves for joint and single-domain pretraining to show how cross-domain pretraining affects convergence? And curves for target-domain training would help check whether the scratch baselines have converged or undertrained.
2 comments

Thank you for taking the time to review our manuscript. We have incorporated your feedback into a revised version, but feel free to let us know if there are any remaining questions or concerns.

Reynolds Number

Thank you for pointing this out. One way to examine generalization to different Reynolds numbers is by considering a single dataset, which prevents exploiting geometric/dataset differences as a proxy for the Reynolds number. Our study has results for this setting, where the Emmi Wing dataset has a range of Reynolds numbers (5-20e6) for randomly generated wing geometries. This is due to different Mach numbers and length-scales for each wing, which result in a random sweep of Reynolds numbers. We find that expert and joint models trained on Emmi Wing do not have issues predicting flows for different Reynolds numbers, since the necessary information is provided to the model (inlet velocity + length-scale is provided, and density/viscosity is held constant). However, we agree that this should be better discussed and that generalizing across large ranges in the Reynolds number is not investigated (i.e., 1e3 vs. 1e6). We discuss this in the revised conclusion section (Line 507).

Apart from this setting, it does not seem possible to test different Reynolds numbers independent of geometric or dataset differences. Within a single dataset, the geometry is restricted to a parametric family and we can vary the Reynolds number without confounding factors (which is done for Emmi Wing mentioned above). Combining datasets would unfortunately re-introduce dataset differences (discretization, base geometry, etc.) that serve as a proxy for flow regimes. We further verify this by performing an ablation study where the Reynolds number is provided as extra input during cross-domain pretraining, and find that it does not contribute extra information (discussed in Appendix C.6, Table 17):

DrivAerNetEmmi-WingWindsorMLDouble-DeltaDrivAerMLSuperWing
w/o Re0.17300.04090.07460.08900.13160.0790
w/ Re0.17300.04130.07400.08900.13180.0783

Table 17: Pretraining validation error of the joint model with and without the Reynolds number Re.

Extensions to Transient Systems

Thank you for this suggestion. We have discussed potential extensions to transient systems in the revised conclusion (around line 503). Predicting mesh-based, time-dependent systems is a very interesting direction, however, current datasets don’t exist to be able to examine cross-domain pretraining in this setting. Despite this, we would expect that cross-domain pretraining would also benefit transient systems, since there is also the added benefit of being able to use context frames to infer different regimes.

Dataset Selection

Thank you for the question. In general, we observe that pretraining on datasets closer to the downstream task is the most beneficial. For example, experts that are closer to the target domain do better in zero/few-shot evaluations.

For zero-shot generalization, there are cases where adding datasets can reduce performance. For example, if we add a dataset that is significantly different from the target domain, then models may overfit to this during pretraining. We have found that few-shot samples during finetuning serve an important role to align pretrained models to a new domain; without this, models cannot adapt to arbitrary shifts in the target regime.

Training Curves

Thank you for the suggestion. We have provided training and validation curves for both pretraining and finetuning in Figures 10, 11, 12, 13 in Appendix C.7.

In general, we find that all models converge during pretraining and finetuning. There are no significant differences in the convergence between models, although the cross-domain model may be a little more unstable during pretraining. During finetuning, the validation error of the scratch models also tends to converge before the training error, likely due to overfitting at low sample counts.

Thanks for the experiments, explanation and additional training info. My questions are addressed.

Reviewer_YpJrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper presents a study of pretraining transformer based surrogates for steady state external aerodynamic CFD simulations across multiple existing datasets. The authors use up to six datasets to train a joint model without supplying specific dataset indices and compare the downstream performance in zero-shot and few shot tasks across four held-out datasets. They find that pretraining across multiple datasets enables better zero-shot performance as well as serving as a better initialization compared to no-pretraining, indicated by the lower errors across the different few-shot settings. They additionally report that a domain expert of a specific dataset of the pretraining corpus always beats the joint model in its own domain, and supplying simulation context does not help in steady state settings in contrast to existing transient pretraining approaches.

Critical strengths
  • The authors show that pretraining helps to reduce zero shot and few shot error on downstream datasets and this holds across all tested architectures.

  • The subject is very relevant to a lot of people wanting to train and use CFD surrogates without much compute to generate their own datasets. Therefore, releasing pretrained models can serve as a starting point for subsequent work in data limited regimes.

Critical weaknesses
  • What confused me the most was the normalization strategy. Non-dimensionalization is the right approach, but then sthe dataset-specific scaling uses ground-truth statistics of the target set, which likely inflates the zero-shot results (Table 2 vs. Table 10). The approach presented in C.2. is the cleaner approach in my eyes. Given that the presented approach in the main paper incurs data leakage, which is mentioned in the limitations, I think for a fair picture (of how this approach would perform in real world settings), the results from C.2. should be reported in the main paper. Furthermore, no discussion on the other dimensionless quantity of Re is given.

  • The performance of the method is hard to assess based on the metrics provided. 1. The authors show that expert models usually outperform finetuned joint models in their own domain. It would be important to show the performance of expert models trained on the full datasets of the held out sets in Table 2 and Figure 3. This would give an intuition of how far off the finetuned models are at the different sample sizes from the lower bound. 2. Engineering metrics (lift/drag coefficients, relative errors and R2 scores) are not reported. These are crucial metrics for several downstream applications. 3. There have been other attempts at pretraining neural surrogates for irregular meshes with geometric pretraining [1, 2]. These are not discussed. Placed in current literature, it is unclear to me how much of the pretraining value comes from simply feeding more geometries, vs. the actual physical solutions of those. This would be an important insight and these approaches should therefore be compared.

References

[1] Zhang et al. (2025). From Cheap Geometry to Expensive Physics: Elevating Neural Operators via Latent Shape Pretraining. arXiv:2509.25788.

[2] Wu et al. (2026). GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Training. arXiv:2602.20399.

Questions for authors
  • Why is Re not fed as a simulation parameter? According to non-dimenzionalization theory this is the other parameter governing fluid flows besides Ma. Thinking about this, how would you define a global reference length and handle geometry scaling/handle global normalization of Re?
  • Is the zero shot gap between Table 2 and Table 10 due to the leaked statistics?
  • How would the results change in the main table if you compute the downstream normalization stats from the 2, 4 or 8 finetuning samples?
  • Given enough finetuning compute, at what amount of samples, does a randomly initialized expert model reach lower errors than a finetuned joint model? I think this is an essential factor for many people that would consider using your pretrained model.
  • What caught my eye in Table 3 is the fact that almost all Joint (N=6) most models come relatively close to the expert models on their own dataset, except for the DrivAerML dataset, where the error of the Joint model is almost 3x the error of the domain expert. Why do you think that is?
  • What are the standard deviations for the presented results? Multiple sources of randomness are in the pipeline (model initialization, few shot sample selection, or query sampling). To get a sense of the approach it is important to know the results hold up against potential variablility and this could actually be a strongpoint of pretraining in my opinion. (stabilizing models at small finetuning sample sizes)
6 comments

Thank you for taking the time to write a detailed and thorough review. We have incorporated your feedback into our revised manuscript, but feel free to let us know if there are any additional concerns or questions.

Normalization Strategy

We also agree that the proposed global normalization should be discussed in the main body, which we do in the revised “Normalization” section in Section 3. Additionally, we have revised the main results of the paper to use the proposed global normalization to remove the dependence on finetuning statistics. In particular, we have re-run the results in Tables 2/3, and Figures 3/4 with models pretrained and finetuned using global normalization. Additional details about normalization are also further discussed in Appendix C.1, including the prior method that assumed access to downstream statistics.

Overall, the main claims of the paper still hold:

  • Pretraining errors are the same, since changes in normalization can be accommodated by learning a different scale/shift during pretraining. (see revised Table. 7):

  • Few-shot errors are not affected, since models can adapt to changes in normalization during finetuning (e.g., adapting from the pretraining distribution to a finetuning one). For example, the S=8 results are mostly the same when comparing global normalization to the prior method (can compare revised Table 2 to Table 8) :

NormalizationAhmedMLSubmarineShift-CCAHiLiftAeroML
Full0.090.090.110.33
Global0.090.090.110.34

Comparing Table 2 to 8: Joint model errors using full vs. global normalization, in the few-shot setting (S=8)

  • Zero-shot errors increase on 3 out of 4 datasets, when comparing global normalization to the prior method. However, this trend also affects the expert models, and a similar increase can be seen for the best/worst experts, for example.
ModelNormalizationAhmedMLSubmarineShift-CCAHiLiftAeroML
JointFull0.310.520.520.90
Global0.500.640.550.75

Comparing Table 2 to 8: Error of the joint model using full vs. global normalization, in the zero-shot setting

ModelNormalizationAhmedMLSubmarineShift-CCAHiLiftAeroML
Best ExpertFull0.380.590.610.86
Global0.510.670.650.82

Comparing Table 2 to 8: Error of the best expert using full vs. global normalization, in the zero-shot setting

Reynolds Number

Thank you for this suggestion. We originally did not include the Reynolds number as input, since we believe that its information can be inferred from other inputs. For example, the inlet velocity is represented by the Ma number or is constant within a dataset and is thus inferred from the geometric family. The reference density and viscosity is also constant within a dataset and is implicitly represented by the geometric family. The length scale is implicitly encoded by the combination of the geometric family and the normalized point cloud.

To validate this, we ran an additional study where we provided the Reynolds number as additional input to the joint model during pretraining, and we observed no difference in the pretraining loss. This result and the accompanying discussion are given in Table 17 and Appendix C.6 in the revised manuscript:

DrivAerNetEmmi-WingWindsorMLDouble-DeltaDrivAerMLSuperWing
w/o Re0.17300.04090.07460.08900.13160.0790
w/ Re0.17300.04130.07400.08900.13180.0783

Table 17: Pretraining validation error of the joint model with and without the Reynolds number Re.

Expert Models on the Held-out Sets

Thank you for the suggestion. We have trained “oracle” expert models on the full held-out sets, at the full budget (100K steps). We have reported this in Figure 3 as an upper bound on the finetuning performance of models. We find that finetuning a joint model can approach the oracle models in many of the downstream datasets. When comparing the finetuned joint model (64 samples, 4K steps) to the oracle (100-1000 samples, 100K steps):

ModelAhmedMLSubmarineShift-CCAHiLiftAeroML
Joint0.0470.0410.0680.179
Oracle0.0360.0240.0610.067

*Comparing Finetuned, Joint model to Oracle

Engineering Metrics

Thank you for providing this feedback. We have evaluated the drag/lift by plotting the C_d/C_l of the joint model vs. a randomly initialized model during finetuning. These plots are generated for the zero-shot and few-shot setting for the AhmedML and HiLiftAeroML datasets, shown in Figure 14/15 and Appendix C.8. R2 scores for these scatter plots are also reported.

We believe that the relative error that is mentioned should be the same as the NRMSE that we use. We apologize if this was ambiguous, and clarify that NRMSE/Relative L2 are common names for the same metric, in Line 236 of the revised manuscript.

Geometric Pretraining

Thank you for this comment. We have expanded our related works section (around Line 115) to include the provided works [1, 2], as well as a newer work [3] that also investigates geometric pretraining. We have also added some discussion about how this work fits into this landscape.

When examining prior geometric pretraining studies, the benefits are tangible, yet small. During few-shot evaluation, geometric pretraining demonstrates an error reduction between 10-20% (on average) when compared to a random initialization. In contrast, cross-domain pretraining generally results in around 50% error reduction in the few-shot setting, likely due to the use of labeled data. Furthermore, cross-domain pretraining allows for zero-shot evaluation, which is not possible with geometric pretraining. Lastly, prior works on geometric pretraining evaluate the few-shot setting at larger sample counts, usually with 16 or more finetuning samples. We hypothesize that this is due to the discrepancy between the pretraining and finetuning objective; in our case we are able to obtain accurate surrogates with as few as 2 finetuning samples.

Overall, examining geometric pretraining and situating these results within our contributions suggests that training with labeled data is more effective, though it requires more costly simulations. However, these simulations already exist and CFD datasets will continue to be generated, so we can leverage them instead of building a bespoke, and often more complex, geometric pretraining pipeline.

  1. https://arxiv.org/abs/2509.25788
  2. https://arxiv.org/abs/2602.20399
  3. https://arxiv.org/abs/2610.03363

Zero-Shot Gap

Yes, it is likely that normalizing the finetuning set to be zero-mean/unit variance unintentionally benefited zero-shot errors. We have since remedied this by reporting zero-shot errors with the global normalization strategy, which does assume access to finetuning statistics.

However, all models suffer this gap, and therefore the findings are unaffected. In particular, the joint model is still the best-performing model on all 4 finetuning sets during zero-shot evaluation.

Computing Normalization Statistics from Few-Shot Samples

Thank you for this suggestion. We have run an additional set of studies where the finetuning statistics are computed using only 2/4/8 samples during few-shot evaluation. This is detailed in Appendix C.1 and Tables 8,9 in the revised manuscript. Surprisingly, computing the normalization statistics from few-shot samples does not affect few-shot errors. We hypothesize that errors in these statistics can be absorbed during finetuning. For example, if the pretraining set has zero-mean and unit-variance, but the finetuning means and variances are slightly biased due to estimation errors, the model can compensate by learning a constant shift and scale during finetuning.

Convergence between Joint model and Scratch model

This is an interesting question, and we interpreted it as: if the joint model and scratch model are both finetuned to a large budget (“enough finetuning compute”), at what number of samples would they meet? This is difficult to test, since the budget for fully training models is around 50-100K steps, which takes a few days of compute. Running this test at multiple sample sizes would be infeasible; however, we did test finetuning the joint and scratch model using 100K steps and the full finetuning dataset. Across the four downstream datasets, the joint and scratch models converged to the same error, suggesting that if given a large enough downstream dataset and enough compute, then there is no benefit to pretraining.

ModelAhmedMLSubmarineShift-CCAHiLiftAeroML
Joint0.0350.0230.0610.066
Scratch0.0360.0240.0610.067

*Comparing Joint vs. Scratch Models, both trained to 100K steps in the full downstream dataset

Standard Deviations

Thank you for the comment. We have added standard deviations into the main results (Table 2/18, Figures 3,4). These were computed by randomly sampling the few-shot examples over three seeds. Furthermore, queries are sampled differently for each seed. Unfortunately, we were not able to use different model initializations during pretraining, as re-training these models would be prohibitively costly.

Table 18 (which is the main Table 2 w/ error bars), and the newly added Figure 13 give an idea of the spread with respect to the number of finetuning samples. As you mention, the joint model can have a lower spread at low sample counts (S=2), which is particularly evident in AhmedML experiments. The spread also decreases as the number of finetuning samples grows. This is further discussed in Appendix C.9 in the updated manuscript.

DrivAerML Pretraining Loss

Thank you for pointing this out; we observed this as well. One likely explanation is that much of the error reduction on DrivAerML occurs late in pretraining, during the slow-convergence/lazy regime. As shown in the pretraining loss curves (Figures 10 and 11), the DrivAerML training loss continues to decrease from 0.5 to 0.25 over 90K steps. The joint model does not receive this much dedicated training on DrivAerML. We hypothesize that this extended training is needed to learn the dataset’s finer-scale features, which arise from its complex geometry and high-fidelity DDES simulations.

Reviewer_gMKyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies cross-domain pretraining for neural surrogates for CFD modeling. Specifically, the authors propose combining 6 datasets covering different geometries and flow conditions to pretrain a single model, then evaluate it on 4 unseen datasets. The results show that, the pretrained model consistently outperforms training from scratch and domain-specific models in zero-shot and few-shot settings. It achieves 2–3× lower errors and requires 8–16× fewer training samples to reach similar accuracy.

Critical strengths
  • Strong practical value: Reduces the amount of expensive CFD simulation data needed for new applications.
  • Convincing experiments: Evaluates 3 architectures across 4 unseen datasets, with consistent improvements.
  • Good analysis: Studies the effects of model size, dataset diversity, normalization, and domain ambiguity.
Critical weaknesses
  • Limited novelty: The main method is simply pooling datasets for pretraining, without introducing a new learning technique.
Questions for authors
  • Main experiments assume access to target-domain normalization statistics, although Appendix C.2 partly addresses this. I think this content should have been moved to the main content, since it is essential to the method.
2 comments

Thank you for taking the time to read and review our paper. We have incorporated your remarks into our revised manuscript, and they have helped improve the quality of our work. Please find our answers to your questions and concerns below, and feel free to let us know if there is anything else we can help with.

Limited Novelty

Thank you for the comment. While our study relies on existing architectures, our study presents, to the best of our knowledge, the following novel contributions:

  1. This is the first time that pretraining across CFD datasets has been thoroughly studied and has shown large gains in sample efficiency when finetuning on held-out datasets. This is a crucial problem in CFD applications due to the high cost of generating data, and we present a novel approach towards addressing this issue.

  2. To make this possible, we introduced a method for addressing normalization in zero/few-shot CFD settings, which has not been previously explored. Furthermore, substantial effort was made to align datasets and create a standard, machine-learning ready format for CFD data, which has not been done before.

  3. Unlike prior findings in literature, we show that cross-domain models in the steady-state CFD setting do not rely on context. This is a novel finding, and contradicts the precedent from time-dependent PDE foundation models.

We have highlighted some of these points in the revised manuscript, such as the normalization strategy discussed below.

Target-Domain Normalization

Thank you for bringing this point to our attention. We also agree that content about finetuning normalization should be discussed in the main body. We have revised the “Normalization” section in Section 3 to reflect this. Furthermore, we have revised the main results of the paper to use the proposed global normalization and remove the dependence on target-domain statistics. In particular, we have re-run the results in Tables 2/3, and Figures 3/4 with models pretrained and finetuned using global normalization. Additional details about normalization are also further discussed in Appendix C.1, including the prior strategy that assumes access to downstream statistics.

Thank you for the response. I have decided to maintain my positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.