acceptodds
Under review as a conference paper at ICLR 2027

Beyond Source Simulation: Pretraining with Proxy Governing Equations

Abstract

Physical simulation is fundamental to aerospace, marine, and automotive engineering design, but obtaining high-fidelity solutions for diverse geometries and operating conditions remains computationally expensive. Although neural surrogates accelerate repeated prediction, their training still relies heavily on costly simulated fields for supervision. To overcome this limitation, we propose proxy governing equation pretraining, a new paradigm that learns transferable physical structure directly from unlabeled 3D geometries and governing laws, without precomputed source simulation fields. Specifically, we pair each geometry with randomly sampled physical conditions and pretrain a shared predictor using proxy equation residuals and boundary constraints. We adopt the classical Oseen system as a physically rich proxy that retains transport, pressure coupling, and viscous diffusion, but its second-order derivatives are difficult to estimate accurately on irregular 3D point sets. To address this, we introduce auxiliary momentum fluxes and reformulate the system into a provably equivalent first-order mixed form that preserves the proxy dynamics while requiring only first-order derivatives. Combined with boundary-aware operators and volume–surface pressure consistency, this formulation enables effective physical supervision on irregular 3D geometries. The pretrained model is then fine-tuned with high-fidelity target labels for diverse downstream tasks. Across five benchmarks spanning aerodynamic, hydrodynamic, and structural prediction, it transfers to substantially different target physics, including fluid-to-structure transfer. Our method matches full-data scratch accuracy with up to 60% fewer labeled simulations and further improves prediction accuracy on the data-rich DrivAerNet++ benchmark. These results show that proxy governing equations provide a principled way to lift geometry into transferable physical structure, extending physical-field pretraining beyond fixed collections of simulated solutions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
physical pretraining, proxy governing equations, unlabeled geometry, cross-physics transfer

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_A1GfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a pretraining method for fluid dynamics prediction using unlabeled 3D geometries. The method employs the Oseen equations as proxy dynamics and uses a physics-informed loss for pretraining, without requiring precomputed source simulation fields. Experiments show that the pretrained model improves prediction accuracy across multiple fluid dynamics benchmarks.

Critical strengths
  1. The paper presents a coherent geometry-based pretraining framework that incorporates simplified fluid physics and improves downstream prediction performance.
  2. The ablation study supports the benefit of the coupled constitutive and momentum constraints: the full objective outperforms a variant retaining only incompressibility and pressure-trace consistency.
Critical weaknesses

My main concerns lie in the experimental evaluation.

  1. The improvements over the strongest competing methods, typically GeoPT, are relatively modest, and the test sets of the four smaller benchmarks contain only 10–44 cases. Without results across multiple random seeds or uncertainty estimates, it is difficult to determine whether these gains are robust. Reporting repeated runs, particularly for the proposed method and GeoPT, would make the comparison more convincing.
  2. The experimental configurations of the baselines are insufficiently documented, making it difficult to assess the fairness of the comparisons. In particular, the GeoPT model variant, pretraining and fine-tuning settings, and whether its results were reproduced under the same evaluation protocol should be clarified. The introduction emphasizes GeoPT’s lack of explicit fluid coupling, whereas Figure 1(b) emphasizes its offline data-generation cost. These are distinct motivations: the paper should clarify their respective contributions and provide a comparable cost breakdown to substantiate the claimed efficiency advantage.
Questions for authors
  1. The paper motivates the Oseen equations as a simplified proxy retaining key fluid interactions. What makes this choice particularly suitable for transferable pretraining compared with alternatives, such as Stokes or the full Navier–Stokes equations? Is there analytical or empirical evidence supporting this trade-off between physical expressiveness and training difficulty?

  2. What justifies characterizing DrivAerNet++ as “data-rich” beyond its larger dataset size? To support the claim that the benefits extend beyond label scarcity, could the authors provide learning curves comparing training from scratch and pretrained initialization across different training-set sizes on this benchmark?

9 comments

We thank the reviewer for the detailed concerns about experimental reliability and the choice of proxy equation. We have added multi-seed and Stokes comparison experiments and clarified the methodological rationale, computational costs, and scope of the large-scale benchmark results. Our responses follow.

Critical Weakness 1: The improvements over the strongest competing methods, typically GeoPT, are relatively modest, and the test sets of the four smaller benchmarks contain only 10–44 cases. Without results across multiple random seeds or uncertainty estimates, it is difficult to determine whether these gains are robust. Reporting repeated runs, particularly for the proposed method and GeoPT, would make the comparison more convincing.

To assess training variability, we performed three independent downstream fine-tuning runs for each of our method and GeoPT on all four fully labeled tasks, holding the pretrained checkpoints fixed and varying the fine-tuning seeds. The table reports mean ± standard deviation of relative L2 error.

DatasetOurs ↓GeoPT ↓
Aircraft0.04094 ± 0.000380.04136 ± 0.00042
DTCHull0.14152 ± 0.005050.14583 ± 0.00509
NASA-CRM0.08870 ± 0.003370.09289 ± 0.00164
Car-Crash0.17575 ± 0.000800.22173 ± 0.00636

Our method yields lower mean errors on all four tasks under the fixed benchmark splits, with the largest improvement on Car-Crash. These repeated runs quantify fine-tuning variability, not uncertainty arising from the limited number of test geometries; our conclusions are therefore confined to the evaluated splits.

Q1: The paper motivates the Oseen equations as a simplified proxy retaining key fluid interactions. What makes this choice particularly suitable for transferable pretraining compared with alternatives, such as Stokes or the full Navier–Stokes equations? Is there analytical or empirical evidence supporting this trade-off between physical expressiveness and training difficulty?

Our goal is not to solve high-fidelity Navier–Stokes equations during source pretraining, but to construct condition-dependent proxy tasks that retain essential physical interactions with a tractable residual structure. This motivates our choice of Oseen.

First, unlike Stokes, Oseen preserves the explicit dependence of the volumetric velocity equation on Reynolds number and background-flow direction. With π=Re,p\pi=Re\\,p, the equations become

Stokes:∇π−Δv=0.\text{Stokes:}\quad \nabla\pi-\Delta v=0.

Oseen:Re(q^⋅∇)v+∇π−Δv=0.\text{Oseen:}\quad Re(\hat q\cdot\nabla)v+\nabla\pi-\Delta v=0.

Under our velocity boundary conditions and zero-pressure reference, rescaling pressure removes the explicit Reynolds-number dependence from the continuous Stokes velocity equation. By contrast, Oseen retains the interplay of background advection and viscous diffusion. This makes Oseen better suited to generating condition-dependent supervision for the same geometry across different flow regimes.

Second, relative to full Navier–Stokes, Oseen retains essential transport and coupling terms while eliminating the nonlinear disturbance self-advection term. The momentum residuals satisfy

rNS=rOseen+(v⋅∇)v.r_{\mathrm{NS}}=r_{\mathrm{Oseen}}+(v\cdot\nabla)v.

Oseen retains background advection, pressure coupling, and viscous diffusion, thereby preserving condition-dependent transport structure. Full Navier–Stokes adds the disturbance self-advection term (v⋅∇)v(v\cdot\nabla)v. With fixed geometry, physical conditions, and discrete operators, the Oseen residual is linear in the predicted fields and yields a quadratic least-squares objective, whereas the nonlinear self-advection term introduces higher-order field couplings into the Navier–Stokes loss. This structural simplification motivates Oseen as a proxy for large-scale pretraining without claiming a guarantee of faster neural-network optimization.

The additional Stokes experiments provide empirical support:

DatasetScratchStokesOseen (Ours)
Aircraft0.09220.05870.0405
DTCHull0.18760.15200.1362
NASA-CRM0.10910.09380.0866
Car-Crash0.19710.18330.1762

Stokes pretraining improves upon Scratch on all four tasks, confirming that even simplified proxy physics can provide useful supervision. Oseen reduces the error further by approximately 31.01%, 10.39%, 7.68%, and 3.87%, respectively, supporting the value of background transport for downstream transfer.

Together, the theoretical comparison and Stokes experiments support Oseen as a principled compromise between condition-dependent physical expressiveness and residual complexity. The Stokes comparison tests the value of background transport empirically; the Navier–Stokes comparison establishes the structural effect of omitting nonlinear disturbance advection.

Q2: What justifies characterizing DrivAerNet++ as “data-rich” beyond its larger dataset size? To support the claim that the benefits extend beyond label scarcity, could the authors provide learning curves comparing training from scratch and pretrained initialization across different training-set sizes on this benchmark?

DrivAerNet++ provides a substantially richer supervised setting than our smaller benchmarks through both geometric diversity and dense spatial field annotations. Its 5,817 labeled training geometries span different vehicle body types, wheel configurations, and underbody designs, with pressure and wall shear stress (WSS) fields evaluated at 32,768 surface points per geometry. We therefore use “data-rich” to describe the scale and diversity of the available high-fidelity supervision, not to imply that performance has saturated.

With the complete labeled training set, our method achieves relative L2 errors of 17.91% for pressure and 18.51% for WSS. The corresponding literature-reported scratch-trained Transolver errors are 23.87% and 22.49%, giving relative error reductions of 24.97% and 17.70%, respectively. These results demonstrate competitive performance on a substantially larger benchmark and broaden the empirical evidence beyond small-label tasks.

To characterize how pretraining benefits change with increasing target supervision, we are conducting additional learning-curve experiments across different label budgets on DrivAerNet++. Given the computational cost of repeated training on this benchmark, these experiments remain in progress, and we will report the results once available.

Critical Weakness 2(a): The experimental configurations of the baselines are insufficiently documented, making it difficult to assess the fairness of the comparisons. In particular, the GeoPT model variant, pretraining and fine-tuning settings, and whether its results were reproduced under the same evaluation protocol should be clarified.

We use GeoPT's officially released eight-layer pretrained checkpoint (GeoPT_8layers.pt), with a hidden dimension of 256, eight attention heads, and 32 slices. During downstream adaptation, we reinitialize the task-specific output layers while retaining the remaining pretrained parameters. Rather than copying results from the original paper, we independently fine-tuned and evaluated GeoPT on all four downstream benchmarks.

To ensure a fair comparison, GeoPT and our method use identical dataset splits, labeled subsets, input preprocessing, prediction targets, and evaluation metrics. Both are fine-tuned for 200 epochs with the same optimizer and checkpoint selection protocol. Relative L2 errors are evaluated per test sample after restoring predictions to the original scale. Thus, the comparisons are conducted under a unified downstream protocol.

Critical Weakness 2(b): The introduction emphasizes GeoPT’s lack of explicit fluid coupling, whereas Figure 1(b) emphasizes its offline data-generation cost. These are distinct motivations: the paper should clarify their respective contributions and provide a comparable cost breakdown to substantiate the claimed efficiency advantage.

We agree that these motivations should be distinguished. Our primary methodological contribution is learning geometry–condition–field relationships through explicit proxy-physics supervision, whereas GeoPT learns geometry–motion representations from synthetic geometric dynamics. By directly enforcing governing-equation residuals during pretraining, our approach additionally avoids offline synthetic trajectory generation.

To quantify the computational benefits, we compare source-stage investments and downstream costs under a common accounting framework. Our measured preprocessing time is 7 h 15 min 52 s, and pretraining requires approximately 127.48 GPU-hours, including data-loading overhead. GeoPT reports approximately 80 CPU cores for 3 days to generate around 5 TB of synthetic trajectories, followed by 144 GPU-hours for Base pretraining.

Using illustrative rates of USD 0.02 per CPU-core-hour and USD 2 per GPU-hour, we assume 200 CPU-core-hours for our preprocessing. For a downstream task at matched accuracy, we assume 100 labels and 6 GPU-hours for Scratch, versus 60 labels and 3 GPU-hours for both pretrained methods, with each high-fidelity label costing USD 2.56.

Cost component (USD)ScratchGeoPTOurs
Source preprocessing and supervision generation0115.204.00
Source pretraining0288.00256.00
One-time investment0403.20260.00
Downstream geometry processing per task5.005.005.00
High-fidelity labels per task256.00153.60153.60
Downstream training per task12.006.006.00
Total per downstream task273.00164.60164.60
Break-even task count—43

Under these assumptions, our method requires 35.5% less upfront investment than GeoPT and reduces recurring downstream costs by 39.7% relative to Scratch. Consequently, our pretraining investment is recovered after 3 downstream tasks, compared with 4 for GeoPT.

Although these monetary estimates rely on illustrative assumptions, they demonstrate the potential benefits of eliminating offline supervision generation and amortizing physics-aware pretraining across downstream tasks.

Reviewer_sGUkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores whether governing-equation constraints can replace source simulation labels for neural operator pretraining. The authors construct Oseen proxy tasks on unlabeled 3D geometries, train a Transolver using equation residuals and boundary conditions, and then fine-tune it on labeled downstream tasks. To handle irregular point clouds, they use a first-order mixed formulation, boundary-aware differential operators, and a volume–surface pressure consistency constraint. As I understand it, the central question is whether a simplified equation can provide useful pretraining signals even when it does not match the target physics. The results across five downstream tasks support this idea, although they leave open which aspects of the proxy physics matter most.

Critical strengths

What I find most compelling is the reduction in target-label requirements. In Table 9, Aircraft reaches 0.0598 with 40% of the labels, outperforming the full-label scratch model at 0.0922. When simulation labels are expensive, this is more consequential than a small improvement in final accuracy. With all target labels available, the method also outperforms the same backbone trained from scratch on all four tasks, so the benefit is not confined to one label budget.

The Car-Crash result is also worth highlighting. The model is pretrained with a fluid proxy but fine-tuned to predict peak von Mises stress, reducing the error from 0.1971 to 0.1762. This does not establish a universal physical representation, but it does show that a mismatch between proxy and target physics need not prevent useful transfer. The DrivAerNet++ results in Table 3 further broaden the empirical evidence.

Critical weaknesses
  1. I am persuaded that the pretraining helps, but less clear on which gains require the particular design proposed here. Equation-based learning without solution labels has precedents in PINO and PFEM; GeoPT studies geometry- and dynamics-based pretraining with transfer across physical tasks; and first-order reformulation has precedent in FO-PINN [1–4]. The authors discuss these connections, so my concern is not missing citations. It is which of the choices made here are responsible for the additional value. Table 2 provides useful evidence that the full objective outperforms the partial objective without the coupled flux–momentum constraints. However, the partial objective no longer includes the full momentum relation. This does not tell us whether a simpler, well-defined proxy equation could provide similar benefits. The first-order formulation, boundary-aware operators, and pressure trace are also important parts of the method, but their individual effects are not directly evaluated. The equivalence argument in the appendix establishes the relationship between the continuous equations; it does not establish an improvement in discrete training accuracy or stability.

  2. The label-efficiency results are encouraging, but the comparisons and cost accounting remain incomplete. Figure 4 establishes label savings relative to training from scratch. I would like to see GeoPT under the same label budgets, since this would directly affect my assessment of the added value. With full labels, some improvements over GeoPT are modest: 0.0412 to 0.0405 on Aircraft and 18.28% to 17.91% for DrivAerNet++ pressure. Without run-to-run variability, their stability is difficult to assess. The Car-Crash result with 80% of the labels, 0.1965, is also very close to the full-label scratch result of 0.1971, so the claimed 20% label saving needs a more cautious interpretation. I would also distinguish avoiding source simulation labels from reducing total cost. Pretraining takes 103.94 GPU-hours, in addition to geometry processing and discrete-operator construction. This investment may well be worthwhile, particularly when reused across tasks, but the paper does not provide a complete accounting. The statement “No extra data processing” in Figure 1 is also inconsistent with the preprocessing described in the appendix and should be revised.

References

[1] Li et al. Physics-Informed Neural Operator for Learning Partial Differential Equations. arXiv:2111.03794.

[2] Wang et al. Pretrain Finite Element Method: A Pretraining and Warm-start Framework for PDEs via Physics-Informed Neural Operators. arXiv:2601.03086.

[3] Wu et al. GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Training. arXiv:2602.20399.

[4] Gladstone et al. FO-PINNs: A First-Order Formulation for Physics Informed Neural Networks. arXiv:2210.14320.

Questions for authors
  1. If one comparison could be added, I would most like to see GeoPT under the same label budgets as Figure 4. Are these results available, together with variability across repeated runs?

  2. What evidence distinguishes the benefit of the Oseen proxy from that of other reasonable pretraining objectives? Have the first-order reformulation, boundary-aware operators, and pressure trace been evaluated separately?

  3. What is the total pretraining cost, including geometry preprocessing and operator construction? Under what level of downstream reuse or reduction in target simulations does this investment become cost-effective?

  4. In Figure 2, the pressure and viscous terms in the flux definition have different signs from Equation (7), and an additional “=0” appears. Please confirm whether this is a figure typo and whether the implementation follows the equations in the main text.

10 comments

We thank the reviewer for recognizing the importance of label efficiency and cross-physics transfer. In response, we have expanded the comparisons with existing pretraining methods, added stability and design ablations, and clarified the computational costs and implementation details. We address each question below.

Q1: If one comparison could be added, I would most like to see GeoPT under the same label budgets as Figure 4. Are these results available, together with variability across repeated runs?

Comparison across label budgets. We evaluated GeoPT at 20%, 40%, 60%, 80%, and 100% label budgets on all four downstream tasks, using the same labeled subsets, data splits, and evaluation protocol as our method.

The table below reports single-run results for each budget. The first row of each dataset gives the relative L2) error of Scratch trained with 100% of the labels. Boldface marks the smallest evaluated budget at which each method matches or improves upon that reference.

DatasetMethod20%40%60%80%100%
AircraftScratch 100%————0.0922
GeoPT0.11500.06090.05310.05150.0412
Ours0.13460.05980.04990.04230.0405
DTCHullScratch 100%————0.1876
GeoPT0.36380.22180.19730.16320.1444
Ours0.33310.21320.18380.16200.1362
NASA-CRMScratch 100%————0.1091
GeoPT0.30490.20680.14230.09790.0919
Ours0.29750.18910.15230.09580.0866
Car-CrashScratch 100%————0.1971
GeoPT0.52760.43860.33290.25490.2183
Ours0.53340.41670.26580.19650.1762

Our method yields lower relative L2) errors in 17 of 20 task–budget combinations, including 13 of 16 combinations at 20%–80% labels. Thus, its gains are not confined to full supervision but persist across most limited-label settings.

Crucially, our method also reduces the label budget needed to match full-label Scratch on several tasks. On DTCHull, this threshold falls from 80% for GeoPT to 60% for our method. On Car-Crash, our method reaches it at 80%, whereas GeoPT does not reach it even with 100% of the labels. Both methods reach the threshold at 40% on Aircraft and 80% on NASA-CRM, although ours achieves lower errors at most budgets. These comparisons more directly substantiate the additional value of proxy-physics supervision over geometry-based pretraining.

Repeated runs with full supervision. To assess sensitivity to training randomness, we conducted three independent downstream fine-tuning runs for both our method and GeoPT at the 100% label budget. We report mean ± standard deviation of relative L2) error below.

DatasetOurs ↓GeoPT ↓
Aircraft0.04094 ± 0.000380.04136 ± 0.00042
DTCHull0.14152 ± 0.005050.14583 ± 0.00509
NASA-CRM0.08870 ± 0.003370.09289 ± 0.00164
Car-Crash0.17575 ± 0.000800.22173 ± 0.00636

Across three runs, our method achieves lower mean error than GeoPT on all four tasks. On Car-Crash, the mean relative L2) error decreases by approximately 20.74%, a difference substantially larger than the observed run-to-run variation. For the remaining tasks, we report the standard deviations alongside the mean improvements to avoid relying on a single favorable run.

Repeated runs at key label-saving budgets. Because the Car-Crash result at 80% labels is close to full-label Scratch, we also examined variability at two representative savings thresholds: 80% for Car-Crash and 60% for DTCHull. We repeated the corresponding runs against full-label Scratch, retaining each method's training configuration and using the same test sets and evaluation protocols.

DatasetOurs label budgetScratch 100% ↓Ours ↓
Car-Crash80%0.198551 ± 0.0028690.198175 ± 0.003849
DTCHull60%0.189628 ± 0.0124550.178202 ± 0.004964

These results further characterize label efficiency under repeated training. On DTCHull, our method achieves a lower mean error with only 60% of the labels than full-label Scratch (0.1782 vs. 0.1896). On Car-Crash, 80% of the labels yields a mean error comparable to full-label Scratch (0.1982 vs. 0.1986).

Q2: What evidence distinguishes the benefit of the Oseen proxy from that of other reasonable pretraining objectives? Have the first-order reformulation, boundary-aware operators, and pressure trace been evaluated separately?

Our contribution is not the individual introduction of PDE residuals, first-order formulations, or MLS discretization. Rather, it is the use of proxy governing equations to provide transferable, condition-dependent supervision across unlabeled geometries. Instead of requiring source simulation labels or offline-generated proxy fields, our framework learns geometry–condition–field relationships from equation constraints and transfers them to downstream tasks with different governing physics. To isolate the role of the proxy design, we added a Stokes comparison and a pressure-trace ablation alongside our original coupled-constraint study.

1. Stokes proxy control: the contribution of background transport.

Our original ablation removed both the flux constitutive and momentum-balance constraints, showing the benefit of full coupled supervision over partial constraints. However, that comparison does not establish whether a simpler, physically complete proxy could provide similar transfer benefits. We therefore added Stokes-based pretraining as a direct comparison.

Stokes retains incompressibility, pressure coupling, and viscous diffusion but omits Oseen's background transport term. This comparison evaluates whether direction-dependent transport adds value beyond a simpler complete proxy. We used the same downstream fine-tuning settings and evaluation protocol. Separately, we set λtr=0\lambda_{\mathrm{tr}}=0 to isolate the contribution of pressure-trace consistency.

The table reports full-label relative L2) errors on the four downstream tasks (lower is better).

DatasetScratchStokes pretrainingOseen without pressure traceFull Oseen (Ours)
Aircraft0.09220.05870.04320.0405
DTCHull0.18760.15200.15060.1362
NASA-CRM0.10910.09380.09310.0866
Car-Crash0.19710.18330.17830.1762

Stokes pretraining outperforms Scratch on all four tasks, while full Oseen achieves the lowest error in every case. Relative to Stokes, Oseen reduces error by approximately 31.01% on Aircraft and 10.39% on DTCHull. These results show that even a simpler governing equation provides useful pretraining supervision, while retaining background transport yields further transfer gains. They also demonstrate the flexibility of our framework to accommodate alternative proxy equations.

2. Independent pressure-trace ablation: the contribution of volume–surface coupling.

To isolate pressure-trace consistency, we set λtr=0\lambda_{\mathrm{tr}}=0 while retaining Oseen's volumetric constraints and boundary treatment. This controlled ablation measures the additional benefit of explicitly coupling volumetric and surface pressure.

Removing pressure-trace consistency increases errors on all four tasks. Relative to the no-trace variant, the full model reduces error by approximately 6.25% on Aircraft, 9.56% on DTCHull, 6.98% on NASA-CRM, and 1.18% on Car-Crash. Thus, volume–surface consistency provides complementary supervision, although its benefit varies with the downstream task.

3. Rationale for the first-order mixed formulation and boundary-aware operators.

These components are designed to make equation-based supervision computationally tractable on irregular 3D point sets; they are not presented as a new high-accuracy PDE discretization solver.

Specifically, the first-order mixed formulation introduces an auxiliary momentum flux,

F=v⊗q^+pI−Re−1∇vF=v\otimes\hat q+pI-Re^{-1}\nabla v

and expresses the Oseen momentum equation as

\qquad \nabla\cdot F=0.$$ This avoids explicitly evaluating second-order spatial derivatives while preserving the underlying physical relations at the continuous level. Boundary-aware MLS incorporates known body and far-field velocities as anchors in local derivative estimation, allowing boundary information to enter the volumetric residuals directly. The first-order formulation and boundary augmentation are numerical mechanisms for implementing the same proxy-physics problem, rather than independent loss terms that can simply be switched off. Removing either changes the differential discretization or the enforcement of boundary data. A meaningful comparison would require a matched alternative implementation that preserves the equations and boundary conditions. We therefore prioritized the Stokes and pressure-trace controls, which more directly isolate the effects of physical supervision without conflating them with numerical implementation changes. Appendix Table 7 provides complementary training diagnostics: by the end of pretraining, the RMS divergence, constitutive, and momentum residuals reach 0.00708, 0.00749, and 0.00353, respectively, while 99.43% of MLS rows remain valid. These measurements support the feasibility of evaluating and minimizing proxy-physics residuals with the proposed discretization.
Reviewer_pMZyOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes to pretrain a neural operator (Transolver) on unlabeled ShapeNet geometries, without any simulation data. Each geometry is paired with randomly sampled flow conditions, and the model is trained to minimize the residuals of the Oseen equations. The model is then fine-tuned on five downstream tasks. The authors report better accuracy than training from scratch and than other baselines, and fewer labels needed to reach the scratch performance.

Critical strengths

The idea is good in essence. Pretraining without simulation data is an interesting direction, and evaluating how the performance changes with the number of samples is an important formulation of the problem, given that gathering samples is quite expensive. The improvement over the same model trained from scratch is significant on some tasks (e.g. Aircraft), and the set of benchmarks is quite diverse, including a structural task.

Critical weaknesses

The idea is good in essence. Pretraining without simulation data is an interesting direction, and evaluating how the performance changes with the number of samples is an important formulation of the problem, given that gathering samples is quite expensive. The improvement over the same model trained from scratch is significant on some tasks (e.g. Aircraft), and the set of benchmarks is quite diverse, including a structural task.

Questions for authors
  • The same-architecture scratch model only appears in the ablation and not in Table 1, and it is weaker than several baselines (0.092 on Aircraft vs. 0.052 for LinearNO): can you explain this descrepancy?

  • Could you clarify how the baseline implementations and evaluation protocols were aligned with the cited papers?

  • Training configurations are only given for the proposed method, so I cannot tell whether baselines were run under the same protocol.

3 comments

We thank the reviewer for recognizing the value of our proposed approach and for the constructive feedback. We appreciate the opportunity to clarify the same-backbone comparisons, baseline implementations, and experimental protocols.

Q1: The same-architecture scratch model only appears in the ablation and not in Table 1, and it is weaker than several baselines (0.092 on Aircraft vs. 0.052 for LinearNO): can you explain this descrepancy?

We appreciate the reviewer highlighting this presentation issue. Table 1 was intended to compare the final predictive performance of different neural operators and pretraining methods, whereas the same-backbone Scratch baseline was reported in Table 2 and Figure 4 to directly quantify the contribution of pretraining. We agree that including Scratch in Table 1 would make this comparison more complete and will add the corresponding results in the revision.

Importantly, Scratch and Ours use the same Transolver backbone and downstream training configuration, differing in the initialization of the transferable model parameters. Both are trained or fine-tuned for 200 epochs under identical optimization settings and checkpoint-selection procedures. This controlled comparison allows us to assess the effect of proxy-physics pretraining independently of architectural differences.

Regarding the discrepancy with LinearNO, different neural operator architectures incorporate distinct inductive biases and representation mechanisms, which can lead to different performance when trained from scratch, even under the same optimization protocol. Therefore, the fact that LinearNO outperforms scratch-trained Transolver does not contradict the benefits of pretraining. The same-backbone comparison measures the contribution of our pretraining strategy, while the cross-architecture comparison assesses the competitiveness of the resulting pretrained model.

The benefits of pretraining are consistent across all four downstream benchmarks. Under full supervision, our method reduces the same-backbone relative L2 errors from 0.0922 to 0.0405 on Aircraft, 0.1876 to 0.1362 on DTCHull, 0.1091 to 0.0866 on NASA-CRM, and 0.1971 to 0.1762 on Car-Crash, corresponding to relative error reductions of 56.07%, 27.40%, 20.62%, and 10.60%, respectively. Moreover, our pretrained model also outperforms LinearNO on Aircraft (0.0405 vs. 0.0517), demonstrating competitive performance beyond the same-backbone comparison.

Our contribution is therefore not to introduce a stronger neural operator architecture, but to improve the predictive capability of an existing backbone through pretraining with proxy governing equations, without requiring source simulation labels.

Q2: Could you clarify how the baseline implementations and evaluation protocols were aligned with the cited papers?

We clarify that all baseline results for the four cross-physics benchmarks in Table 1 were obtained through our own experiments using publicly available implementations, rather than taken directly from the original publications. These include GNOT, UPT, Transolver++, LinearNO, HNO, and GeoPT.

Specifically, we trained the neural operator baselines on the corresponding downstream benchmarks. For GeoPT, we initialized the model from the authors' publicly released pretrained checkpoint and independently performed downstream fine-tuning. All methods were evaluated using the same dataset partitions, prediction targets, and metric computation procedures, with unified downstream training configurations. Thus, the results in Table 1 were obtained within a consistent experimental framework, rather than assembled from separately reported results with potentially different evaluation protocols.

For DrivAerNet++, the other baseline results were taken from the respective publications. Importantly, these methods follow the established DrivAerNet++ benchmark protocol, including the standard training, validation, and test splits, 32,768 sampled surface points per geometry, pressure and wall shear stress (WSS) prediction targets, and consistent evaluation metrics. Although these results are literature-reported, they are based on the same benchmark and evaluation protocol, providing a common basis for comparison.

To further strengthen the evidence, we additionally reproduced the same-backbone Scratch baseline on DrivAerNet++ and compared it with our pretrained model at 20%, 60%, and 100% label budgets under identical downstream training and evaluation settings. Under full supervision, our method achieves relative L2 errors of 17.91% for pressure and 18.51% for WSS, compared with 18.72% and 19.28% for the reproduced Scratch baseline. These additional experiments provide a direct controlled comparison that complements the literature-reported benchmark results.

Dear Reviewer pMZy,

As the discussion period is approaching its deadline, we would like to kindly follow up and ask whether you have had an opportunity to review our response, and whether there are any remaining questions or concerns that we could further address.

Your feedback, including both constructive suggestions and points of concern, has been invaluable in helping us improve our work. We have made every effort to carefully address each of your comments and clarify any potential misunderstandings. We sincerely value the opportunity for further discussion and would greatly appreciate any additional feedback you may have.

Thank you again for your time and thoughtful consideration.

Best regards,

Authors of Submission 53039

Q3: Training configurations are only given for the proposed method, so I cannot tell whether baselines were run under the same protocol.

We thank the reviewer for raising this important point. We clarify that all methods in Table 1, including GNOT, UPT, Transolver++, LinearNO, HNO, GeoPT, Scratch, and Ours, were trained or fine-tuned using the same downstream optimization configuration. Specifically, all methods employed AdamW with an initial learning rate of 1.2 × 10⁻³, weight decay of 1 × 10⁻⁵, cosine learning-rate decay, a batch size of 1, and 200 training epochs.

Beyond these optimization settings, we adopted a unified downstream experimental protocol, including identical dataset partitions, labeled training subsets, task-level preprocessing, point-sampling procedures, prediction targets, and evaluation metrics. Architecture-specific input representations followed the corresponding implementations, while the downstream optimization settings and training budgets remained consistent across methods.

These shared settings ensure that the comparisons are not affected by differences in downstream training budgets or evaluation procedures. In particular, Scratch and Ours additionally share the same backbone and downstream adaptation configuration, providing a direct comparison that isolates the contribution of pretraining initialization.

For DrivAerNet++, we distinguish the literature-reported baselines from our additional controlled Scratch-versus-pretrained experiments. The latter use identical downstream training and evaluation configurations across all label budgets.

We will explicitly document the shared optimization settings and baseline configurations in Appendix B to improve the transparency and reproducibility of our experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.