acceptodds
Under review as a conference paper at ICLR 2027

Scaling 3D Physics Surrogates: Low-Fidelity Pretraining for High-Fidelity Transfer

Abstract

Training neural surrogates for high-fidelity simulation requires expensive labels, limiting the scale of available supervision. We investigate low-fidelity simulation as a reusable source of pretraining data for unstructured 3D physics. Our approach is simple yet effective: using open-source fluid and solid-mechanics solvers, we generate approximate responses on source geometries constructed independently of downstream datasets, pretrain standard surrogate backbones, and finetune them on high-fidelity targets. Across unstructured downstream tasks spanning fluid fields, deformable-body responses, and physical systems whose governing equations are absent from pretraining, low-fidelity pretraining improves over training from scratch and competing pretraining baselines. Our pretraining can reduce high-fidelity data requirements by up to 77% at matched accuracy. Transfer remains effective when pretraining uses a single fixed physical condition and varies only geometry, demonstrating that this restricted source distribution already provides useful supervision for downstream adaptation.We further study scaling with pretraining-corpus size, model capacity, and physical-condition coverage, observing task-dependent improvements in downstream transfer. These results demonstrate low-fidelity simulation as an effective source of large-scale, reusable supervision for high-fidelity surrogates on unstructured 3D geometry.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Physics pretraining, Physics Simulation, Neural surrogates, Transfer learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_x3c4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper explores the use of low-fidelity simulations as supervised pretraining data for neural surrogates, and demonstrates improved downstream fine-tuning accuracy on unseen engineering benchmarks and cross-operator tasks compared with training from scratch and geometry-based GeoPT pretraining. The paper further studies the effects of source-corpus scale, model capacity, downstream data availability, and physical-condition diversity.

Critical strengths

strengths:

  1. The paper constructs sizable fluid and solid-mechanics simulation corpora for studying pretraining and transfer of neural surrogates. The curated datasets and associated pipeline could be useful to the community for future research on surrogate pretraining and transfer learning.

  2. The paper evaluates its pretraining approach across a reasonably broad set of downstream settings, including cross-operator transfer from CFD to acoustics and electrostatics and a separate extension from solid-mechanics pretraining to deformable-body benchmarks. This breadth helps characterize where the proposed pretraining strategy transfers successfully and where its benefits are less consistent.

Critical weaknesses

weaknesses:

There are two variables in the proposed pretraining scheme: 1. low-fidelity 2. supervised pretraining with curated dataset, and my main concern is that the current experiments don't fully justify the merits on both factors.

  1. a low-fidelity vs. high-fidelity pretraining comparison is missing. It would be valuable to validate the promise of low-fidelity-based pretraining on whether it would help the surrogate achieve better downstream adaptation with similar data generation time, or similar downstream accuracy with less data generation time.

  2. Although the datasets are used for supervised pretraining, there isn't a baseline of model with supervised pretraining with alternative datasets. A justifiable starting point would be collecting off-the-shelf datasets such as DrivAerNet++ [1], AhmedML [2], and HiLiftAeroML[3] just to name a few, besides ones mentioned in this paper, and use a mixture of them as the supervised pretraining baseline. This comparison would help clarify if the curated dataset is more valuable than off-the-shelf public datasets in downstream task transfer as in the current setup

[1] Elrefaie et al., DrivAerNet++: High-Fidelity Computational Fluid Dynamics & Deep Learning Benchmarks

[2] Ashton et al., AhmedML: High-Fidelity Computational Fluid Dynamics Dataset for Incompressible, Low-Speed Bluff Body Aerodynamics

[3] Ashton et al., HiLiftAeroML: A High-Fidelity Computational Fluid Dynamics Dataset for High-Lift Aircraft Aerodynamics

Questions for authors

See W1 and W2 for requests for additional evidence that will isolate and validate the core contributions.

5 comments
AuthorsOpenReview ↗

(1/1) Response to Reviewer x3c4's comments

We are grateful for the reviewer’s valuable comments and insightful suggestions. Our detailed responses to the concern are as follows, and please refer to the updated PDF, with all changes highlighted in blue.

Low-Fidelity Pretraining versus Public CFD Pretraining: Generation Cost and Downstream Transfer (W1, W2)

Thank you for these closely related suggestions. We address them jointly by examining both the time required to construct the pretraining corpus and the downstream performance of pretraining on existing public CFD datasets.

Generation scale within a fixed simulation time. Our low-fidelity pipeline generates each case in approximately 542 seconds on one A100 GPU, requiring approximately 1,406 GPU-hours for the 9,336-case corpus. For comparison, a typical DrivAerML simulation requires approximately 40 hours on 1,536 CPU cores. Over the same cumulative simulation wall-clock time, this setup would produce only approximately 35 high-fidelity cases, which is scarce to support pretraining. Although the hardware differs, this comparison illustrates the substantial practical barrier to constructing a large and diverse high-fidelity pretraining corpus.

Comparison with large-scale public-data pretraining. To evaluate whether existing public CFD datasets offer a stronger alternative, we additionally compare against the CD-CFD [1] pretrained checkpoint using the same AB-UPT-24M backbone on Aircraft, Car-Crash, and NASA-CRM as DrivAerML is part of the CD-CFD pretraining corpus. CD-CFD draws on six high-fidelity public CFD datasets (e.g., DrivAerML, DrivAerNet++) whose original simulation generation required substantial computational resources. Nevertheless, our low-fidelity initialization demonstrates stronger downstream adaptation compared with large-scale high-fidelity public-data pretraining.

Pretraining MethodAircraftCar-CrashNASA-CRM
Scratch6.46 ± 0.1619.81 ± 4.618.30 ± 0.28
High-Fidelity Pretraining (CD-CFD)6.71 ± 0.3835.70 ± 0.028.21 ± 0.15
Low-Fidelity Pretraining (Ours)4.82 ± 0.0815.69 ± 0.017.76 ± 2.03

Together, these results support low-fidelity simulation as a practical source of scalable, reusable pretraining supervision: it enables broad source-data coverage without the generation burden of high-fidelity corpora, while retaining substantial downstream transfer value. We have incorporated this discussion into Section 6.5 of the revised paper.

Summary

We sincerely thank you for your thoughtful and constructive review. We greatly appreciate your recognition of our curated simulation corpora and pipeline, and we hope that our new experiments comparing our low-fidelity curated corpus vs. off-the-shelf high-fidelity public CFD dataset mixtures on pretraining fully address your concerns.

Reference

[1] Anthony Zhou, et al. "Cross-Domain Pretraining for Steady-State Neural CFD Surrogates."

Dear Reviewer,

As the discussion period is closing soon, we would like to kindly follow up.

For your W2 (a supervised-pretraining baseline on off-the-shelf public datasets), we compared against the CD-CFD checkpoint, pretrained on a mixture of six high-fidelity public CFD datasets including DrivAerML and DrivAerNet++, with the same AB-UPT-24M backbone. Our low-fidelity pretraining transfers better on all three held-out benchmarks. This also resolves W1: high-fidelity pretraining at a much larger generation cost brings weaker downstream adaptation than our corpus.

Could you let us know whether this addresses your concerns, and if so, whether you would consider updating your rating? We are happy to provide any detail further.

Dear Reviewer,

We understand this is a busy period, and we would like to send a last brief note before the discussion closes.

Your review identified two questions to justify, and our additional experiments along with the revision now test both directly:

  • W2 (a supervised baseline pretrained on off-the-shelf datasets): we compared against the CD-CFD checkpoint, pretrained on six high-fidelity CFD datasets including DrivAerNet++, with the same AB-UPT-24M backbone. Our low-fidelity corpus transfers better on all three held-out benchmarks (Table 7).
  • W1 (low vs. high fidelity at comparable generation time): the generation budget of our full corpus would produce only about 35 DrivAerML-scale high-fidelity cases. The CD-CFD comparison above futher proves that our low-fidelity pretraining yields better downstream accuracy at a much lower generation cost.

In addition, following other reviewers, we pretrained on our own geometries with a geometry-only objective, which shows that the benefit actually comes from the simulated physical responses.

Your reply on whether these address your concerns would be very helpful to us and to the AC. Thank you so much for your time.

Reviewer_8QXNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies low-fidelity simulation pretraining for high-fidelity 3D physics prediction. Separate fluid and solid models are pretrained and then fine-tuned on downstream tasks. The authors report improved accuracy across several tasks and estimate reductions of 34–77% in high-fidelity training data relative to selected same-depth Scratch baselines.

Critical strengths

The motivation is clear and practically relevant. The experiments are broad and provide useful evidence for transfer across different physical tasks. Reporting cases with limited gains also helps establish the scope of the approach.

Critical weaknesses
  1. Most reduced-data configurations are single runs using one fixed subset ordering, leaving uncertainty in the estimated savings unquantified. The same-depth Scratch comparison is reasonable; a best validation-selected Scratch reference would additionally clarify practical savings.

  2. The cross-operator results are interesting, but the role of physical supervision remains unclear. The comparison with GeoPT does not fully separate the effect of the supervision targets from other differences between the pretraining schemes.

  3. Table 4 shows competitive results against other architectures, but does not include a randomly initialized version of the same model under the same setup. This makes it difficult to attribute its strong performance specifically to pretraining.

Questions for authors
  1. How sensitive are the savings to subsets and seeds, and how do they change against the best validation-selected Scratch configuration?

  2. On the same source geometries, how much does simulation-based supervision add over a geometry-only pretraining objective?

  3. What does the Scratch version of the same model achieve under the exact setup used for Table 4?

4 comments
AuthorsOpenReview ↗

(1/2) Response to Reviewer 8QXN's comments

We are grateful for the reviewer’s positive assessment and insightful suggestions. Our detailed responses to each concern are as follows:

  1. Sensitivity of Data Savings to Subsets/Seeds & Comparison Against Best Validation-Selected Scratch (W1, Q1)

  2. Simulation-Based Supervision vs. Geometry-Only Pretraining on Identical Source Geometries (W2, Q2)

  3. Scratch Version of the Same Model Under the Exact Setup of Table 4 (W3, Q3)

Please refer to the updated PDF, with all changes highlighted in blue.

1. Sensitivity of Data Savings to Subsets/Seeds & Comparison Against Best Validation-Selected Scratch (W1, Q1)

Thank you for this constructive suggestion. We conducted additional experiments to address both parts of your question:

Multi-Seed / Multi-Subset Uncertainty Quantification. Instead of a single run on one fixed subset ordering, we repeated the reduced-data experiments across 3 independent random subset permutations and seeds for each data fraction (10%, 20%, 40%, 60%, 80%) to compute the mean and standard deviation of the learning curves and interpolated label savings. Using the mean learning curve, the estimated high-fidelity label savings are 76.1% on Car-Crash, 57.8% on Aircraft, 29.5% on NASA-CRM, and 28.8% on DrivAerML. Substantial savings therefore persist after varying the training subsets and seeds. We have updated the results in Appendix E.2 / Table 12, 13 / Section 6.3 of the revised manuscript.

Comparison against validation-selected Scratch. We kept the pretrained learning curves fixed and recomputed savings against the full-data Scratch configuration with the lowest validation error. Pretraining matches these stronger references with 67.5% fewer labels on Car-Crash, 40.3% fewer on Aircraft, and 28.8% fewer on NASA-CRM. For DrivAerML, the available ranking uses test pressure error. Our pretrained L8 model reaches 3.93% error at full data, above the 3.53% achieved by the best L32 Scratch run.

AuthorsOpenReview ↗

(2/2) Response to Reviewer 8QXN's comments

2. Simulation-Based Supervision vs. Geometry-Only Pretraining on Identical Source Geometries (W2, Q2)

Thank you for raising this important point. We ran Transolver L8 and AB-UPT-24M to isolate the exact effect of the supervision targets on the identical source geometries, model architecture, training protocol, then compare:

Scratch: Random initialization without pretraining.

Geometry-only Pretraining: Pretraining the backbone on the same geometry corpus using masked surface reconstruction, similar to GMP [1]. Randomly mask 50% of surface mesh nodes and predict their normals and curvature from the global coordinates of all nodes and the mask indicators, without physical simulation data.

Low-Fidelity Physics Pretraining (Ours): Pretraining on the exact same geometries using simulated LBM fluid fields.

BackbonePretraining ObjectiveDriAverML CpC_pDriAverML ‖τwτ_w‖AircraftCar-CrashNASA-CRM
Transolver L8Scratch4.37 ± 0.047.40 ± 0.045.08 ± 0.3618.36 ± 0.119.20 ± 0.15
Transolver L8Geometry-Only Pretraining5.19 ± 0.029.09 ± 0.044.35 ± 0.1517.61 ± 0.018.41 ± 0.11
Transolver L8Low-Fidelity Physics (Ours)4.16 ± 0.027.12 ± 0.014.16 ± 0.0615.98 ± 0.057.97 ± 0.17
AB-UPT-24MScratch4.99 ± 0.107.93 ± 0.176.46 ± 0.1619.81 ± 4.618.30 ± 0.28
AB-UPT-24MGeometry-Only Pretraining4.70 ± 0.047.50 ± 0.077.43 ± 2.5216.63 ± 0.189.09 ± 0.65
AB-UPT-24MLow-Fidelity Physics (Ours)4.42 ± 0.027.05 ± 0.064.82 ± 0.0815.69 ± 0.017.76 ± 2.03

As shown in the table above, physical supervision provides substantial gains over geometry-only pretraining. Across both Transolver and AB-UPT-24M backbone, while geometry-only pretraining improves modestly over Scratch on some of the downstream applications, low-fidelity physics supervision consistently outperforms Geometry-Only pretraining and scratch training across all the benchmarks. We have added these experiments and analysis to Section 6.5 of the revised manuscript.

3. Scratch Version of the Same Model Under the Exact Setup of Table 4 (W3, Q3)

Thank you for pointing this out. We evaluated the randomly initialized Scratch version of the exact same model (Transolver L32) for DrivAerML and NASA-CRM using the same setup as the pretrained L32 model in Table 4:

MethodDrivAerML CpC_pDrivAerML ||τwτ_w||NASA-CRM CpC_pNASA-CRM CfC_f
Ours Scratch3.946.949.056.64
Ours pretrained3.485.898.144.78

Under the same experimental setup, pretraining demonstrates substantial gains over training the identical model from scratch. We have added the Ours (Scratch) row directly to Table 4 in the revised manuscript.

Summary

We sincerely thank you for your thoughtful and constructive review. We greatly appreciate your positive evaluation of our work, and we hope that our new results on multi-seed / best-Scratch data savings, Physical simulation supervision vs. Geometry-Only pretraining, and the matched Table 4 Scratch baseline fully address all of your questions.

Reference

[1] Luis Medrano-Navarro, et al. "Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers."

I am willing to maintain my initial score, though I agree with the concern raised by other reviewers regarding the novelty: if this is a ML problem or a learning pipeline. I will then respect the final decision of AC. thanks.

Thank you for following up and for maintaining your positive assessment. Before the discussion closes, could you confirm whether our responses resolved your three original questions (multi-seed/best-Scratch savings, geometry-only pretraining on identical source geometries, and the matched Scratch row in Table 4)? Each of these questions now has a dedicated experiment in the revision. An explicit confirmation would also help AC considerably.

On whether this is "an ML problem or a learning pipeline", we respectfully believe the new controlled experiments answer this directly. The question our paper asks is a learning question: which source of pretraining supervision can transfer to downstream high-fidelity physics tasks? The experiments in the rebuttal and paper further isolate it by:

  1. Same geometries, same backbone, same protocol, different supervision. Geometry-only pretraining is inconsistent and even hurts on DrivAerML, while low-fidelity physics supervision gives the lowest error on all five metrics for both Transolver and AB-UPT backbone. The gain therefore comes from the supervision signal, not from the pipeline or from geometry exposure.
  2. Supervision fidelity is not what matters most. An AB-UPT checkpoint pretrained on six high-fidelity public CFD datasets (CD-CFD) transfers worse than our low-fidelity initialization on all three held-out benchmarks, while at a much higher generation cost.
  3. The learned representation transfers across governing equations and removes the depth degradation seen in from-scratch training.

We believe such findings are of direct interest to the ML and science community. Beyond the findings themselves, we will also release the full data generation pipeline, the low-fidelity pretraining corpora, and pretrained checkpoints upon acceptance. This gives the community a reusable starting point on their own high-fidelity tasks, extend the corpora to new geometries and physics, or use them to benchmark future pretraining methods.

If any specific concern remains, we would be glad to address it in the remaining time.

Reviewer_fjdSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates low-fidelity simulation as scalable pretraining for neural surrogates on unstructured 3D geometries. It generates fluid and solid-mechanics data using inexpensive solvers and fine-tunes standard architectures on high-fidelity tasks. Experiments compare the proposed approach across tasks like fluid fields and deformable-body responses. Results demonstrate improved accuracy and reduced high-fidelity supervision requirements. Cross-physics transfer, model scaling and data efficiency are also evaluated.

Critical strengths
  1. The paper proposes a pipeline for training neural surrogates for high-fidelity physics simulation. The major methodology lies in utilizing low-fidelity pre-training to improve downstream accuracy. More importantly, Pre-training substantially reduces high-fidelity data requirements. In the tasks introduced in the experiments, these benefits are adaquately justified.
Critical weaknesses
  1. The paper is more about proposing a learning-based pipeline for physical simulation than proposing a novel learning algorithm. Since the methodological novelty is somehow limited, for the interest of machine learning community, it would be important to justify the approach in real downstream applications. The current paper introduces experiments several high-fidelity simulation benchmark with real data, while a significant gap still remains towards a solid real application. Furthermore, in my view, a major technical question raised in the paper is that why low-fidelity pre-training can take the place of sufficient high-fidelity data? The paper focuses on “how” to build the pipeline, while does not sufficiently explore this “why” question.

  2. The current experiments do not sufficiently distinguish between the contributions of “seeing more geometric examples” and “seeing more physical simulation examples”. For example, in CFD tasks, the proposed pre-training also exposes the network to a large collection of 3D geometries. The improvements might therefore arise partly from learning geometric representations rather than physically meaningful responses. The justification on this is important since the major target of the paper is to justify the significance of physical-process-based pre-training, not geometric information.

Questions for authors
  1. Consider Weakness 1, please either provide sound evidence that the current empircal results justify the usefulness in important real applications, or provide further methodological analysis on “why low-fidelity pre-training can the place of sufficient high-fidelity data”.

  2. Consider Weakness 2, please conduct further experiments to distinguish between the contributions of “seeing more geometric examples” and “seeing more physical simulation examples”.

5 comments
AuthorsOpenReview ↗

(1/2) Response to Reviewer fjdS's comments

We are grateful for the reviewer’s valuable comments and insightful suggestions. Our detailed responses to each concern are as follows:

  1. Why Low-Fidelity Pretraining Can Replace High-Fidelity Data & Justification in Real Downstream Applications (W1, Q1)
  2. Distinguishing "Seeing More Geometric Examples" from "Seeing More Physical Simulation Examples" (W2, Q2)

Please refer to the updated PDF, with all changes highlighted in blue.

1. Why Low-Fidelity Pretraining Can Replace High-Fidelity Data & Justification in Real Downstream Applications (W1, Q1)

Thank you for raising this fundamental question. Below we provide both (A) a deeper methodological analysis of why low-fidelity pretraining can substitute for a large fraction of high-fidelity data, and (B) concrete evidence of its practical impact in real industrial engineering applications.

A. Methodological Analysis: Why Low-Fidelity Pretraining Substitutes for Supervision by Sufficient High–Fidelity Data. Building an accurate physics surrogate requires learning both (1) how geometry and boundary conditions shape the response fields, including local geometric effects and non-local interactions, and (2) response characteristics specific to the target high-fidelity physical regime.

Low-fidelity simulation can already provide useful supervision for geometry-physics interactions, which is (1) above, despite approximations in their governing models, discretization, and numerical accuracy. High-fidelity simulations remain necessary to learn the target-specific responses and correct discrepancies in the approximate relationships (2). Training from scratch must learn both aspects jointly from high-fidelity labels. Low-fidelity pretraining instead allows the model to acquire useful information about geometry-conditioned response fields. Consequently, fewer high-fidelity examples may be needed to achieve the same predictive accuracy. This helps explain our observed data savings in Section 6. Our same-geometry comparison between geometric and physical supervision further tests whether simulated response fields contribute useful transfer information beyond geometry exposure alone, which is described below.

B. Justification in Real Industrial Applications. We emphasize that the downstream benchmarks evaluated in our paper are real-world industrial engineering tasks with high computational bottlenecks, including Automotive External Aerodynamics (DrivAerML), Aircraft Transonic Wing-Body Design (NASA-CRM), Vehicle Crashworthiness (Car-Crash), Mechanical Structural Engineering (SimJEB). For instance, a typical DrivAerML case requires about 40 hours on 1,536 CPU cores (Section 3.4). The practical value of our approach is that the source pretraining corpus and resulting initialization are reusable across downstream tasks. Each target task can then be adapted using a smaller set of high-fidelity simulations. As reported in Section 6, low-fidelity pretraining reduces the high-fidelity training data required to match the corresponding full-data Scratch accuracy by up to 76% in the evaluated settings. Thus, rather than generating a large high-fidelity training set independently for every new task, reusable low-fidelity pretraining can substantially reduce the cost for generating additional high-fidelity supervision data.

AuthorsOpenReview ↗

(2/2) Response to Reviewer fjdS's comments

2. Distinguishing "Seeing More Geometric Examples" from "Seeing More Physical Simulation Examples" (W2, Q2)

Thank you for highlighting this critical distinction. To clearly separate the contribution of seeing more 3D geometries from seeing more physical simulation examples, we conducted new controlled experiments. We pretrained both Transolver L8 and AB-UPT 24M on the exact same geometry corpus under the same training protocol, comparing:

Scratch: Random initialization without pretraining.

Geometry-only Pretraining: Pretraining the backbone on the same geometry corpus using masked surface reconstruction, similar to GMP [1]. We randomly mask 50% of surface mesh nodes and predict their normals and curvature from the global coordinates of all nodes and the mask indicators, without physical simulation data.

Low-Fidelity Physics Pretraining (Ours): The network sees the exact same 3D geometries, supervised by simulated LBM physical fields.

BackbonePretraining ObjectiveDriAverML CpC_pDriAverML ‖τwτ_w‖AircraftCar-CrashNASA-CRM
Transolver L8Scratch4.37 ± 0.047.40 ± 0.045.08 ± 0.3618.36 ± 0.119.20 ± 0.15
Transolver L8Geometry-Only Pretraining5.19 ± 0.029.09 ± 0.044.35 ± 0.1517.61 ± 0.018.41 ± 0.11
Transolver L8Low-Fidelity Physics (Ours)4.16 ± 0.027.12 ± 0.014.16 ± 0.0615.98 ± 0.057.97 ± 0.17
AB-UPT-24MScratch4.99 ± 0.107.93 ± 0.176.46 ± 0.1619.81 ± 4.618.30 ± 0.28
AB-UPT-24MGeometry-Only Pretraining4.70 ± 0.047.50 ± 0.077.43 ± 2.5216.63 ± 0.189.09 ± 0.65
AB-UPT-24MLow-Fidelity Physics (Ours)4.42 ± 0.027.05 ± 0.064.82 ± 0.0815.69 ± 0.017.76 ± 2.03

As shown in the table above, physical supervision provides substantial gains over geometry-only pretraining. Across both Transolver and AB-UPT-24M backbone, while geometry-only pretraining improves modestly over Scratch on some of the downstream applications, low-fidelity physics supervision consistently outperforms Geometry-Only pretraining and scratch training across all the benchmarks. We have incorporated this discussion into Section 6.5 of the revised paper.

Summary

We sincerely thank you for your constructive review and insightful questions. We hope our methodological analysis on why low-fidelity physics pretraining reduces high-fidelity sample complexity, together with our new controlled experiments decoupling geometry exposure from physical simulation supervision fully address your concerns.

Reference

[1] Luis Medrano-Navarro, et al. "Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers."

Dear Reviewer,

As the discussion period is closing soon, we would like to kindly follow up.

Your Q2 asked us to separate "seeing more geometric examples" from "seeing more physical simulation examples". We ran exactly this control on the same geometry corpus, backbone and protocol, with only the supervision changed. Geometry-only pretraining is inconsistent and sometimes worse than Scratch, while physics supervision is best on all benchmarks for both backbones (Section 6.5).

For Q1, we provided both the methodological analysis and the industrial cost context you requested in the responses above.

Could you let us know whether these address your concerns, and if so, whether you would consider updating your rating? We are happy to clarify anything further.

Reviewer_a7bMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors explore pretraining surrogate models for high fidelity physics simulations on cheap, low-fidelity simulations and fine-tuning them on smaller high-fidelity datasets. The claim is that pre-training can be done on dufferent geometries or even governing equations than the downstream task.

Using Transolver as their example surrogate model, the authors show that indeed pre-training on low fidelity data and fine-tuning on high fidelity data leads to a performance increase on various physics datasets such as CFD or crash modelling, when compared to models trained from scract or pretrained with GeoP,. while also showing that pretrained models can match models trained from scratch using substantially fewer high fidelity examples. Moreover, they also show that improvements can be obtained by pre-training on completely different physics operators.

Critical strengths
  1. The paper tackles the problem of scarcity of high fidelity physical simulation data through pre-training on cheap, low fidelity data, It shows that using this low fidelity data can lead to increases in performance after fine-tuning on high fidelity datasets, and more interestingly that it allows to match previous models trained from scratch by using far fewer high fidelity datapoints (by up to 77%). Since high fidelity data can be very expensive to produce, this opens the door to the potential training of performant foundation models for physics. The results show that the pretraining is even transferable to different equations.

  2. The authors perform a vast array of experiments and across an array of datasets spanning CFD, structural mechanics, acoustics, electrostatics and radiosity, and compare against GeoPT which is a popular pretraining baseline.

Critical weaknesses
  1. The approach doesn't introduce any methodological novelty and only showcases a the effect of pre-training on low fidelity data followed by fine-tuning, which isn't a novel idea.

  2. The paper lacks an analysis regarding what is actually responsible for the increased performance when pre-training. Since an increase in performance is observed even for different governing equations, a plausible candidate would be that the model is pre-trained to better understand geometry representations, but no deeper analysis is provided.

  3. Only one model is pre-trained per case; it would be interesting to show that gains in performance are valid across different models.

Questions for authors
  1. Could the authors explore whether pre-training is actually teaching the model useful physics or simply better 3D geometry representations ? The authors could try to compare the current low-fidelity physics pre-training objectives and geometry based pre-training objectives (like SDF, curvature, etc..).

  2. Could the authors show how the framework would improve pre-training other models ? For the CFD cases for instance, the authors could try to pretrain AB-UPT and see if the improvements seen in Transolver are also valid for this other architecture.

  3. For pertinence, it would be interesting to add a comparison of the performance of the fine-tuned surrogate in comparison to a low-fidelity solver. Does the fine-tuned surrogate provide an increased accuracy ? How much faster is inference time with the surrogate ?

7 comments
AuthorsOpenReview ↗

(1/3) Response to Reviewer a7bM's comments

We are grateful for the reviewer’s valuable comments and insightful suggestions. Our detailed responses to each concern are as follows:

  1. Clarification on Novelty (W1)

  2. Physical Supervision vs. 3D Geometry Representations (W2, Q1)

  3. Generalization Across Other Architectures: Pretraining AB-UPT (W3, Q2)

  4. Comparison of Fine-Tuned Surrogate vs. Low-Fidelity Solver (Q3)

Please refer to the updated PDF, with all changes highlighted in blue.

1. Clarification on Novelty (W1)

Thank you for raising this question. We would like to clarify that the contribution of our work extends beyond simply applying standard pretraining and fine-tuning. It lies in establishing, validating, and scaling solver-driven low-fidelity physical supervision on arbitrary 3D geometries. Its value is to address fundamental bottlenecks that previously prevented supervised pretraining at scale in 3D computer-aided engineering (CAE):

Overcoming the Geometry-Pairing Bottleneck in Multi-Fidelity Physics Learning: Traditional multi-fidelity surrogate modeling typically starts by constructing low- and high-fidelity simulation datasets for the same target geometries or parametric CAD family. Prior 3D pretraining work (such as GeoPT) resorted instead to hand-crafted synthetic kinematics. To the best of our knowledge, we demonstrate for the first time that non-paired, approximate physical responses generated on out-of-domain 3D geometry repositories provide a reusable pretraining initialization that transfers across disjoint downstream geometries, mesh resolutions, and boundary-value operators. Furthermore, we showcase that this paradigm can lead to better generalization and accuracy compared to pretraining based on synthetic kinematics.

Tailored Mesh-aligned Low-Fidelity Data Generation Pipelines at Scale: Directly applying an off-the-shelf solver to raw 3D assets is insufficient to produce reliable physical supervision. We introduce mesh-normal-based reconstruction of surface responses from coarse LBM simulations, and procedural generation of solid geometries and loading conditions for elastostatic simulation.

Systematic Study of Cross-Operator Transfer and Scaling Behavior in 3D Surrogates: Beyond proposing the pipeline, we uncover non-trivial scientific findings for the ML-for-physics community: (i) low-fidelity pretraining transfers to unseen governing equations (radiosity, acoustics, electrostatics) without modifying the pretraining objective; (ii) pretraining unlocks depth scaling (preventing model layer depth degradation seen in from-scratch training); and (iii) task-dependent benefits on increasing geometric diversity and physical-condition diversity.

Together, our work is not merely an application of fine-tuning, but a holistic framework and systematic empirical study showing how inexpensive physical solvers can unlock scalable, reusable supervision for high-fidelity 3D engineering surrogates.

AuthorsOpenReview ↗

(2/3) Response to Reviewer a7bM's comments

2. Physical Supervision vs. 3D Geometry Representations (W2, Q1)

Thank you for this insightful suggestion. To strictly determine whether our pretraining teaches useful physical priors or merely better 3D geometry representations, we conducted controlled experiments suggested by the reviewer. Specifically, we hold the source 3D geometry corpus, model architecture, and training protocol strictly fixed, and compare:

Scratch: Random initialization without pretraining.

Geometry-only Pretraining: Pretraining the backbone on the same geometry corpus using masked surface reconstruction, similar to GMP [1]. We randomly mask 50% of surface mesh nodes and predict their normals and curvature from the global coordinates of all nodes and the mask indicators, without physical simulation data.

Low-Fidelity Physics Pretraining (Ours): Pretraining on the exact same geometries using simulated LBM fluid fields.

BackbonePretraining ObjectiveDriAverML CpC_pDriAverML ‖τwτ_w‖AircraftCar-CrashNASA-CRM
Transolver L8Scratch4.37 ± 0.047.40 ± 0.045.08 ± 0.3618.36 ± 0.119.20 ± 0.15
Transolver L8Geometry-Only Pretraining5.19 ± 0.029.09 ± 0.044.35 ± 0.1517.61 ± 0.018.41 ± 0.11
Transolver L8Low-Fidelity Physics (Ours)4.16 ± 0.027.12 ± 0.014.16 ± 0.0615.98 ± 0.057.97 ± 0.17
AB-UPT-24MScratch4.99 ± 0.107.93 ± 0.176.46 ± 0.1619.81 ± 4.618.30 ± 0.28
AB-UPT-24MGeometry-Only Pretraining4.70 ± 0.047.50 ± 0.077.43 ± 2.5216.63 ± 0.189.09 ± 0.65
AB-UPT-24MLow-Fidelity Physics (Ours)4.42 ± 0.027.05 ± 0.064.82 ± 0.0815.69 ± 0.017.76 ± 2.03

As shown in the table above, physical supervision provides substantial gains over geometry-only pretraining. Across both Transolver and AB-UPT-24M backbone, while geometry-only pretraining improves modestly over Scratch on some of the downstream applications, low-fidelity physics supervision consistently outperforms Geometry-Only pretraining and scratch training across all the benchmarks. We have added these experiments and analysis to Section 6.5 of the revised manuscript.

3. Generalization Across Other Architectures (W3, Q2)

Thank you for this constructive suggestion. First, we note that in our original submission, we already validated our framework on a second, distinct architecture family for deformable solid mechanics. We utilized Adaptive Spatial Tokenization (AST, 4M and 40M) (Table 5) as the backbone model, and our low-fidelity solid pretraining reduces downstream error by up to 68.2% on DeformingPlate and 37.2% on SimJEB.

Second, following your specific suggestion for CFD, we pretrained AB-UPT on our low-fidelity CFD corpus, and evaluated downstream fine-tuning against from-scratch AB-UPT across all four engineering benchmarks. As shown in the table above, AB-UPT-24M exhibits consistent downstream accuracy improvements when initialized with our low-fidelity CFD checkpoints, confirming that our pretraining framework generalizes effectively across different surrogate architectures (Transolver, AB-UPT, and AST).

We have added these experiments and analysis to Table 2 / Section 5 of the revised manuscript.

4. Comparison of Fine-Tuned Surrogate vs. Low-Fidelity Solver (Q3)

Thank you for pointing this out. To directly compare the accuracy and speed of our fine-tuned surrogate against the low-fidelity solver itself, we ran our low-fidelity LBM solver (XLB, 300×120×90 grid, 16,000 steps) directly on the downstream test geometries of DrivAerML and compared its predictions accuracy (Relative L2) and runtime against our fine-tuned surrogates:

MethodCpC_p (%)||τwτ_w|| (%)Runtime per Case
Low-Fidelity Solver65.6%63.7%542 s (1 A100 GPU)
Fine-Tuned Surrogate (Ours)3.48%5.89%84.3 ms (1 A100 GPU)

As shown in the table above, our fine-tuned surrogate produces substantially higher accuracy than the low-fidelity solver with orders-of-magnitude faster inference. We have added this comparison in Appendix C.2 of the revised manuscript.

AuthorsOpenReview ↗

(3/3) Response to Reviewer a7bM's comments

Summary

We sincerely thank you for your thoughtful and constructive review. We greatly appreciate the time and effort you dedicated to evaluating our work, and we hope that our new experiments on Geometry-Only pretraining, cross-backbone pretraining, and direct low-fidelity solver comparison fully address your concerns.

Reference

[1] Luis Medrano-Navarro, et al. "Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers."

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for your time and constructive feedback. We are encouraged that the reviewers note that low-fidelity pretraining "opens the door to the potential training of performant foundation models for physics" (a7bM) and that "the curated datasets and associated pipeline could be useful to the community" (x3c4). We also appreciate the recognition of our "vast array of experiments" across diverse physical domains (a7bM), showing that pretraining "substantially reduces high-fidelity data requirements" (fjdS), transfers across different governing equations (a7bM, 8QXN).

We have addressed all of the reviewers' concerns in the individual responses, and we have updated the submission PDF and its appendix (with all changes highlighted in blue) to incorporate these changes. The main updates are summarized below.

New experiments

  • Physical supervision vs. 3D geometry-only pretraining (Section 6.5): To isolate the contribution of simulated physical fields from seeing more 3D geometries, we added a controlled comparison against geometr-only pretraining across 2 backbones (a7bM, fjdS, 8QXN).
  • Generalization to an additional CFD architecture (Table 2, Section 5): Beyond Transolver and AST, we pretrained and fine-tuned AB-UPT 24M on our low-fidelity CFD corpus across all four engineering benchmarks (a7bM).
  • Low-fidelity pretraining vs. high-fidelity CFD corpora (Section 6.5): We compared our low-fidelity initialization against CD-CFD, demonstrating stronger downstream adaptation with much less generation cost (x3c4).
  • Multi-seed data-efficiency uncertainty and best-Scratch reference (Section 6.3, Appendix E.2, Tables 12–13): We repeated the reduced-data experiments across 3 random subset permutations and recomputed label savings against the best validation-selected full-data Scratch configuration across depths (8QXN).
  • Matched from-scratch baseline in full-mesh evaluation (Table 4, Section 5): We added the randomly initialized counterpart of our identical L32 model (Ours, scratch) under the exact setup of Table 4(8QXN).
  • Fine-tuned surrogate vs. direct low-fidelity solver (Appendix C.2): We evaluated our low-fidelity LBM solver directly on DrivAerML test geometries (a7bM).

Clarifications

  • Novelty and technical contributions: Clarified how our framework overcomes the geometry-pairing bottleneck in traditional multi-fidelity learning, introduces mesh-aligned low-fidelity data generation pipelines at scale, and establishes cross-operator transfer and depth-scaling behavior in 3D surrogates (a7bM).
  • Methodological basis and industrial relevance: Clarified why low-fidelity pretraining reduces the requirement for high-fidelity data, and highlighted its practical value in reducing simulation bottlenecks on industrial CAE applications (fjdS).

We hope that these additions strengthen the manuscript and fully address the reviewers' comments, and we gladly welcome any follow-up discussion.

AuthorsOpenReview ↗

Summary of the discussion.

We thank all reviewers and the AC for their time and engagement during the discussion period. We summarize the outcome below.

Updated assessments

  • Reviewer a7bM indicated that they will raise their rating to weak accept, after our new experiments on additional backbone, geometry-only pretraining, and the comparison with the low-fidelity solver.
  • Reviewer x3c4 raised their rating to weak accept, after we added the comparison with CD-CFD pretraining.
  • Reviewer 8QXN maintains a weak accept.
  • Reviewer fjdS confirmed that their Weakness 2 is largely addressed. For Weakness 1, we provided a further theoretical and experimental analysis (Section 6.5 and Appendix G).

Concerns addressed during the discussion

  1. Physics supervision vs. geometry exposure (a7bM, fjdS, 8QXN). On identical geometries, backbones, and pretaining protocol, geometry-only pretraining does not reproduce the gains compare to from-scratch training, while low-fidelity physics supervision gives the lowest error on every benchmark for both Transolver and AB-UPT (Table 7).
  2. Generality across architectures (a7bM). AB-UPT pretrained on our corpus improves on all four engineering benchmarks (Table 2), in addition to Transolver and AST.
  3. Low-fidelity pretraining vs. high-fidelity public data pretraining baselines (x3c4). A checkpoint pretrained on six high-fidelity CFD datasets (CD-CFD) transfers worse than our low-fidelity corpus on all three held-out benchmarks, at a much higher generation cost (Table 7).
  4. Robustness of the data savings (8QXN). With three random subsets per data fraction, pretraining matches full-data Scratch training accuracy with 29–76% fewer high-fidelity labels, and remains effective against the best validation-selected Scratch configuration (Tables 12–13).
  5. Attribution of the full-data results (8QXN). The randomly initialized version of the same model under the setup of Table 4 further confirms that the gains come from pretraining.
  6. Why low-fidelity pretraining saves high-fidelity data (fjdS). A sample-complexity argument, modelled on representation-transfer theory, explains the savings and predicts when they shrink. Our experiment results also support such argument (Appendix G).
  7. Surrogate vs. low-fidelity solver (a7bM). The fine-tuned surrogate reaches 3.48% error against 65.6% for the low-fidelity solver, at 84 ms instead of 542 s per case (Appendix C.2).

We'd also like to resummarize the main contribution and novelty of our work:

  1. Low-fidelty physics output from approximate solver on geometries unrelated to the downstream tasks, with no paired simulations, provides reusable pretraining supervision for 3D physics surrogates, transferring across geometries, backbones, and governing equations.
  2. Controlled evidence that the benefit comes from physical supervision and coverge of pretraining corpus, instead of geometry exposure or label fidelity.
  3. A systematic study of cross-operator transfer, data efficiency, depth scaling, and pretraining corpus design.

High-fidelity simulation remains the main bottleneck for building neural surrogates in engineering design, where a single case can take tens of hours on thousands of CPU cores. Our results show that this bottleneck can be eased with inexpensive, reusable supervision from low-fidelity physics label generated by approximate solver. We hope that our corpora, pretrained checkpoints and the thorough pipeline will make contributions for the ML-for-physics community.

Again, we are sincerely grateful to all reviewers for your constructive feedback, which has substantially strengthened the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.