acceptodds
Under review as a conference paper at ICLR 2027

uPDEt: Efficient Transformer for Unstructured Physics Simulations

Abstract

Deep learning surrogate models have emerged as powerful tools for accelerating Partial Differential Equation (PDE) simulations on complex, unstructured domains and arbitrary geometries. However, existing state-of-the-art architectures often exhibit excessive memory consumption, steep computational overhead, and limited capacity to perform autoregressive dynamics. In this work, we introduce uPDEt, a highly efficient and scalable Transformer-based neural operator designed for unstructured physical systems. uPDEt yields highly competitive accuracy while, at the same time, significantly reducing memory footprint and runtime. This results in substantially reduced wall-clock training and inference times compared to top-performing models. With this pairing of high accuracy and high efficiency, uPDEt outperforms existing unstructured neural operators in terms of Pareto optimality across a range of complex 3D benchmarks. Furthermore, while many unstructured models are primarily restricted to steady-state predictions, uPDEt extends to autoregressive rollouts for transient dynamics. Our results highlight uPDEt as a versatile, high-performance foundation for large-scale physical surrogate modeling.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operators, Fluid Mechanics, Physics simulations, Efficient Transformer, Geometric Deep Learning, Surrogate Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_G74tOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes uPDEt, a hierarchical Transformer neural operator with a U-Net-like structure, designed for large-scale physics simulations on unstructured meshes. By using local attention and hierarchical processing, the computational complexity scales linearly with the number of input points N.

The authors evaluate the method on three steady-state benchmarks, including DrivAerML, SHIFT-Wing, and SHIFT-Crash, using models with around 10–15M parameters. They also consider autoregressive prediction on 2D KS with deformed points, 2D Ellipse flow, and SHIFT-Crash.

In terms of accuracy, uPDEt performs slightly worse than SMART with similar parameter counts, achieves comparable results to GeoTransolver, and outperforms AB-UPT, Transolver++, Erwin, and PTV3 in the reported comparisons. The main advantage is computational efficiency. Compared with SMART, uPDEt is about 3.7 times faster, uses around half the memory, and requires 7.8 times fewer FLOPs. The method can also process up to 16.8M points on a single H100 GPU.

Critical strengths
  • The main strength of this paper is its computational efficiency while maintaining competitive accuracy. According to Table 30, uPDEt achieves similar accuracy to GeoTransolver on DrivAerML, but with much higher throughput (3.92 vs 0.51 samples/s). Compared with SMART, it is around 3.7 times faster and requires only about half the GPU memory. Table 31 also shows approximately linear scaling of computation time and memory consumption, and the model can handle 16.8M points on a single GPU, while SMART and Transolver++ run out of memory. I think this is an important practical advantage for large-scale CFD applications. The efficiency analysis is also relatively detailed.

  • Another strength is that the same architecture can be applied to different types of problems, including steady-state prediction, surface and volume queries, and autoregressive rollouts. On SHIFT-Crash autoregressive prediction, uPDEt achieves around 2 times lower NRMSE than Transolver++ across different prediction horizons (Table 7). I also appreciate that the paper clearly acknowledges that uPDEt does not outperform SMART in terms of pure prediction accuracy.

Critical weaknesses
  1. My main concern is about the fairness of the accuracy comparison, especially the differences in input features and computational budgets.
  • Input features: According to Sec. 4 and Appendix E.9, uPDEt uses additional geometric information, including surface normals and cell areas, while SMART and AB-UPT only use point coordinates. These additional features can be helpful for learning physical quantities, especially on complex geometries. The paper's own ablation in Table 27 shows that coordinates-only uPDEt has a channel-averaged error of 13.47% on DrivAerML. Therefore, it is difficult to determine how much of the accuracy improvement comes from the proposed architecture rather than the additional input information.

  • Computational budget: Although the paper compares models with similar parameter counts, I think this is not sufficient for a fair comparison. Since uPDEt and SMART have very different computational costs, a comparison with matched training FLOPs or training time would be more informative. The current comparison with a smaller SMART model does not fully show the trade-off between accuracy and computational resources.

  1. I also have some concerns about the methodological novelty. The proposed architecture seems to mainly combine several existing techniques, including local window attention from PDE-Transformer, ball-tree partitioning and rotation from Erwin, RoPE, FiLM, and cross-attention decoding similar to AB-UPT and SMART. These are all reasonable design choices, and their combination can be practically useful. However, it is not very clear which component represents the main methodological contribution compared with previous work. I think the paper should better distinguish the proposed method from these existing architectures and explain which design is responsible for the improvement.

  2. Some claims in the paper seem stronger than what the experimental results actually show.

  • Speedup: In Sec. 4.1, the paper claims an "order-of-magnitude speedup relative to SMART". However, Table 30 reports a speedup of around 3.7 times, while the reduction in FLOPs is about 7.8 times. Table 31 also reports speedups of around 3–5 times. Therefore, I think the statement about an order-of-magnitude speedup is not accurate for the actual runtime results.

  • Autoregressive prediction: The conclusion claims "superior autoregressive rollout accuracy". However, on the Ellipse benchmark, Transolver++ achieves equal or slightly better results at every prediction horizon. Therefore, this claim should be qualified rather than presented as a general conclusion.

  1. The statistical reliability of the results is also unclear. The standard deviations reported in Appendix I are computed across test samples rather than different training seeds. Therefore, they do not reflect the randomness of model training. Some performance differences are also quite small. For example, on DrivAerML surface pressure, uPDEt achieves 5.72 ± 0.71 compared with 5.76 ± 0.63 for GeoTransolver. Without multiple training runs, it is difficult to know whether such a small difference is meaningful. I think reporting results from multiple random seeds would make the conclusions more convincing.
Questions for authors
  1. Could the authors provide comparisons where all models use the same input features? In particular, it would be helpful to evaluate SMART and AB-UPT with surface normals and cell areas, or evaluate uPDEt with coordinates only, across all three steady-state benchmarks. Also, could the authors clarify which input features were used for Transolver++, GeoTransolver, Erwin, and PTV3?

  2. How was SMART-S trained compared with uPDEt? Did the two models use the same number of epochs, training time, or total training FLOPs? I think it would be useful to provide accuracy-versus-compute curves for both uPDEt and SMART, using several model sizes for each. This would give a more complete comparison than using only one smaller SMART model.

  3. I have a question about the receptive field at different resolutions. For 32K input points, the coarsest level contains a single ball with 8192 tokens, so the model can perform global attention at this level. However, when the number of input points increases, the coarsest level will contain multiple balls if the number of levels and ball size remain unchanged. In this case, the model may no longer have a global receptive field. Could the authors clarify which hierarchical configurations were used for the different resolutions in Fig. 4, Table 6, and Table 31? In particular, is the reported linear scaling achieved while maintaining global attention at the bottleneck?

  4. On SHIFT-Wing, uPDEt and all other models except SMART have quite large errors on volume velocity, around 28–33%, compared with approximately 12% for SMART. This is a relatively large performance gap. Could the authors explain why this happens? Is it related to the limited global information exchange caused by local window attention, especially for transonic flow where long-range interactions may be important?

4 comments

We thank the reviewer for the detailed, thoughtful assessment and constructive feedback. We appreciate the positive evaluation of our work's strengths and address each weakness (W) and question (Q) below.

  • W1.1: We appreciate the opportunity to clarify the input configurations across models. Four of the baseline architectures evaluated (Transolver++, GeoTransolver, Erwin, and PTV3) use the exact same input features as uPDEt (spatial coordinates, surface normals, and cell areas). In contrast, SMART and AB-UPT were designed exclusively for spatial coordinate inputs; their published architectures do not natively ingest auxiliary geometric channels without structural alterations. To ensure a fair benchmark, all models were evaluated under their standard, intended configurations at their maximum capability.

    The capacity to naturally incorporate rich boundary geometric features (such as normals and area weighting) is an inherent architectural advantage of flexible modern operators like uPDEt. Therefore, we emphasize that omitting normals and cell areas for AB-UPT and SMART was not an arbitrary choice, but a consequence of their architectural designs, which do not natively support these auxiliary inputs. Thus, all models are compared fairly based on their intended designs at their maximum capabilities. Furthermore, our baseline selection reflects the standard in the field, with AB-UPT and Transolver being widely adopted benchmarks and SMART representing the strongest recent baseline. We have revised the discussion around line 297 to explain these architectural distinctions and baseline setups more clearly.

    Specifically, Appendix E.9 provides an ablation isolating the effect of the input channels. Indeed, we have quantified the isolated contribution the input channels: for DrivAerML, if we ignore normals and cell areas in uPDEt, the relative L2L_2 grows from 8.81 to 13.47. Together with the rest of Appendix E and Table 5 (now table 6), we systematically analyzed which design and architectural choices yield the greatest performance gains.

  • W1.2: Regarding the computational budget, our main results provide comparisons under parameter-matched configurations, as is standard in prior literature. However, we agree with the reviewer that resource-matched comparisons provide an interesting, complementary perspective. Therefore, we included the efficiency-matched comparison against SMART in Table 4. We focused on SMART for this comparison because it represents the strongest competing baseline; for the remaining baselines, the Pareto trade-off is already evident (i.e., uPDEt matches or exceeds their accuracy while consuming substantially fewer resources). In Table 4, we evaluated a smaller configuration of SMART (SMART-S) with a peak memory footprint comparable to our standard uPDEt, trained for the exact same duration. This establishes an equitable comparison where both models share identical memory budgets and training time.

    Additionally, we just performed the opposite resource-matched comparison, scaling uPDEt to match the computational budget of full SMART, in addition to the opposite that was already provided in the manuscript. The relative L2 errors on DrivAerML are:

Modelpsp_sτs\tau_spvp_vuvu_v
SMART4.419.113.968.02
uPDEt-matched3.938.293.487.46

Under matched computational budgets, uPDEt consistently outperforms SMART across all surface (ps,τsp_s, \tau_s) and volumetric (pv,uvp_v, u_v) fields for DrivAerML. We are currently finalizing the corresponding resource-matched runs for SHIFT-Wing and SHIFT-Crash, and will incorporate them into the final manuscript (or in an updated rebuttal response as soon as they complete).

  • W2: We agree with the reviewer that understanding the individual contributions of each architectural component is crucial. To this end, Appendix E provides a comprehensive ablation across all design choices. Table 5 (now Table 6) summarizes the primary ablation of our contributions. While all of the modifications contribute with some improvement, RoPE represents clearly the biggest one. We'd be happy to expand on any specific component if further clarification is helpful.

  • W3: We agree with the reviewer that this was an overstatement. We have carefully revised these claims in the manuscript. While the statement holds for the SHIFT-Crash visualization in Figure 3d, we have tempered the phrasing to reflect that it does not apply across all regimes. Likewise, we revised "superior autoregressive rollout accuracy" to accurately reflect that Transolver++ achieves the best accuracy on the Ellipse benchmark.

  • W4: Regarding variance across runs: training runs on these datasets exhibit high stability and very low seed-to-seed variance. In the submission, the tables in the appendix reported standard deviations across test samples to reflect sample-level prediction spread, which is much higher than training seed spread. However, we fully agree with the reviewer that reporting run-to-run variance across different random seeds is important for completeness. Following the reviewer's recommendation, we trained both uPDEt and SMART across 5 different random seeds on DrivAerML to show that the variance remains very low, obtaining a standard deviation of 0.00058 and 0.00094 on the relative L2 error, respectively. This represents only around 0.62% and 1.20% of the mean magnitude. These results have been added to the appendix. We will include a similar analysis for all the experiments for future revisions of our paper.

  • Q1: As noted in W1.1, SMART and AB-UPT were designed for coordinate inputs and do not natively support additional input channels. In contrast, the remaining baselines (Transolver++, GeoTransolver, Erwin, and PTV3) utilize the exact same inputs as uPDEt: spatial coordinates, surface normals, and cell areas.

  • Q2: SMART-S uses the same memory and training time as our standard uPDEt configuration. We agree that it is valuable to provide a broader model scaling analysis across architectures. While our initial manuscript already reported model scaling on the transient benchmarks, we agree that including scaling behavior on steady-state benchmarks is likewise interesting. Repeating the scaling analysis of Figure 5 (now Figure 6) on DrivAerML yields the following table, showing the relative rel L2L_2 averaged across all the channels and normalized with the value achieved of the S version. Here, S is our base configurations from Table 3 (around 10-15M) and goes up to L with around 200M parameters.

Model SizeSSMMMLL
uPDEt (ours)1.000.860.700.630.56
SMART1.000.850.660.580.51
Transolver++1.000.810.790.750.71

Thus, both architectures demonstrate consistent and comparable power-law scaling across model capacities. Still, in this scaling analysis, uPDEt presents other clear advantages. For bigger models, efficiency is even more critical. While bigger versions of SMART need big GPUs and many days of training, uPDEt is much faster to train. We are currently finalizing the corresponding runs for SHIFT-Wing and SHIFT-Crash, and will incorporate them into the final manuscript (or in an updated rebuttal response as soon as they complete).

  • Q3: We thank the reviewer for this very good question. In our reported experiments, the ball radius scales with point density to preserve a consistent neighborhood size (physically) and ensure effective global communication at the coarsest level. In our benchmarks, we did not observe substantial differences compared to a fixed-radius setup; however, guided by the ablation results in Table 23, fixing the radius across resolutions can degrade predictive accuracy.

  • Q4: We thank the reviewer for highlighting this interesting observation. We also noticed this discrepancy and verified its reproducibility through repeated runs. We are investigating possible factors behind this behavior and greatly appreciate the hypothesis suggested by the reviewer. In addition, we want to highlight that in many industrial aerodynamic workflows, surface quantities remain the primary engineering metrics since they are the basis for computing aerodynamic forces; nonetheless, fully understanding volumetric dynamics is important. We do not have a clear answer yet, but we believe that our global communication step at the coarsest level should account for long range interactions in transonic flows. Since this only happens in our SHIFT-Wing experiments, it could be more related to the dataset than the models.

We thank the reviewer once again for their constructive suggestions and deep engagement with our work. We look forward to continuing this discussion during the rebuttal phase.

Thanks for addressing my concerns. A follow up question: is it possible for coordinate-only uPDEt next to SMART and AB-UPT on all three benchmarks?

Also, I recommend the authors highlight the novelty of the method in the abstract. As for now, the abstract only talks about the performance improvement.

Thank you for positively acknowledging our previous answers. We added a general official comment about the input representations, where we ran all the coordinate-only uPDEt benchmarks. In addition, we have modified the abstract as suggested.

As a final note, in our answer to W1.2 and Q2, we provided the first results of our additional scaling experiments. The remaining additional experiments are now finished and are included in the revised manuscript. In the final resource-matching experiments (Table 4), uPDEt outperforms SMART in 90% of the cases. Regarding the general model size scaling (Figures 5, 6 and 12), uPDEt shows a better progression than Transolver++. For steady-state experiments, uPDEt closely matches SMART, although the final order depends on the particular dataset used.

Reviewer_CPocOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes uPDEt, a hierarchical Transformer neural operator for PDE surrogate modeling on unstructured domains. It combines PDE-Transformer’s multiscale windowed attention with Erwin’s ball-tree partitioning, and adds four components: cross-attention graph pooling and unpooling, a D-dimensional rotary position embedding, a queryable cross-attention decoder for joint surface and volume prediction, and parameter-modulated MLPs. The authors derive linear complexity in the number of points. They evaluate on three steady 3D benchmarks and three transient benchmarks against six baselines at matched parameter count. The results highlight uPDEt as a versatile, high-performance foundation for large-scale physical surrogate modeling.

Critical strengths
  1. The model shows efficiency at competitive accuracy, with a practical comparison. Section 4.1 and Figure 3c report that, at about 10M parameters, uPDEt uses less than half of SMART’s peak memory and runs much faster. In Table 3 it is second only to SMART on most metrics and beats GeoTransolver on DrivAerML volume fields. In Figure 4a, uPDEt is the only model that is both accurate and fast with low memory.

  2. The model scales to large meshes and handles both steady and time-stepping tasks. Figure 3a shows throughput growing linearly up to about 8M points (2²³), while the baselines run out of memory. Overall, uPDEt is a general base model for large-scale surrogates.

Critical weaknesses
  1. Section 4 says SMART and AB-UPT see only spatial coordinates, while uPDEt also gets face normals and cell areas. Section 4.2 then presents uPDEt’s win over SMART-S (Table 4) and over AB-UPT (Table 3) as architectural gains. The main text never shows how much of the gap comes from the extra inputs.
  2. Some claims also outrun the numbers:
  • Section 4.2 says uPDEt has “superior accuracy” under matched resources. Yet SMART-S is clearly better on SHIFT-Wing volume velocity (17.05 vs 27.92).
  • Section 4.1 says uPDEt “systematically outperforms other baselines in throughput and memory footprint.” But Figures 3c and 3d compare it only with SMART and Transolver++. The paper’s own discussion of Figure 4a says Erwin and PTV3 achieve higher throughput.
  • GeoTransolver, which is not part of the efficiency plots, matches or beats uPDEt on several Table 3 metrics: SHIFT-Wing volume pressure (0.42 vs 0.83) and both SHIFT-Crash fields. The comparison needs equal inputs and claims.
  1. The ablation (Table 5) shows that almost all of the gain over the Erwin/PDE-Transformer base comes from adding RoPE (24.29 → 9.56). Attention pooling helps only a little (26.12 → 24.29). RoPE could be added to existing models directly, so it is unclear how much the new architecture itself contributes. That ablation also covers only one dataset, and no main-text table reports variance across runs.
Questions for authors

See weaknesses above.

2 comments

We thank the reviewer for his commitment and feedback. We appreciate his positive assessment and would like to address the concerns raised below.

  • W1: We appreciate the opportunity to clarify the input configurations across models. Four of the baseline architectures evaluated (Transolver++, GeoTransolver, Erwin, and PTV3) use the exact same input features as uPDEt (spatial coordinates, surface normals, and cell areas). In contrast, SMART and AB-UPT were designed exclusively for spatial coordinate inputs; their published architectures do not natively ingest auxiliary geometric channels without structural alterations. To ensure a fair benchmark, all models were evaluated under their standard, intended configurations at their maximum capability.

    The capacity to naturally incorporate rich boundary geometric features (such as normals and area weighting) is an inherent architectural advantage of flexible modern operators like uPDEt. Therefore, we emphasize that omitting normals and cell areas for AB-UPT and SMART was not an arbitrary choice, but a consequence of their architectural designs, which do not natively support these auxiliary inputs. Thus, all models are compared fairly based on their intended designs at their maximum capabilities. Furthermore, our baseline selection reflects the standard in the field, with AB-UPT and Transolver being widely adopted benchmarks and SMART representing the strongest recent baseline. We have revised the discussion around line 297 to explain these architectural distinctions and baseline setups more clearly.

    Regarding the concern that "the main text never shows how much of the gap comes from the extra inputs": we have quantified the isolated contribution the input channels: for DrivAerML, if we ignore normals and cell areas in uPDEt, the relative L2L_2 grows from 8.81 to 13.47. This ablation was presented in Appendix E.9 of our submission. Together with the rest of Appendix E and Table 5 (now Table 6), we systematically analyzed which design and architectural choices yield the greatest performance gains.

  • W2.1: We thank the reviewer for noting the imprecision in our initial wording. While uPDEt outperforms SMART-S across most metrics, we have revised the manuscript to explicitly state that SMART-S attains better accuracy on volumetric velocity for SHIFT-Wing. Nonetheless, we believe the overall advantage of uPDEt across these benchmarks remains clear. Furthermore, in many downstream aerodynamic engineering applications, surface quantities are of primary importance, as they directly determine aerodynamic forces such as drag and lift.

  • W2.2: We thank the reviewer for pointing out this overstatement. A better wording here is that uPDEt achieves superior efficiency compared to baselines in a comparable accuracy regime. While Erwin and PTV3 exhibit lower runtime, their predictive error is substantially higher than the other baselines. We have rephrased this statement in the revised manuscript to be more precise.

  • W2.3: While GeoTransolver matches or slightly surpasses uPDEt in certain individual metrics, it does so at a substantially lower computational efficiency, as detailed in Appendix F: it is around 7x slower, and uses at least 15% more memory. Consequently, uPDEt demonstrates a significantly more favorable accuracy-efficiency Pareto trade-off. Moreover, GeoTransolver uses the exact same input features as uPDEt (including normals and cell areas). We would be glad to provide further clarification here if desired.

  • W3: We agree that the 3D adaptation of RoPE provides substantial performance gains, and we consider this one of our core architectural improvements. This position encoding is widely used in LLMs, but not much in physics surrogate models. While exploring 3D RoPE within other architectures is an interesting direction, our evaluation strictly adheres to the canonical, officially published implementations of each baseline to ensure faithful, reproducible comparisons. Given the strong empirical benefits demonstrated by 3D RoPE in our experiments, we believe this mechanism represents a valuable insight for the broader community and we believe that enhancing other geometric architectures based on this contribution is promising future work. This suggests that the accuracy gap of local window-attention models such as Erwin and PTV3 stems mainly from how geometry is encoded, rather than from their restricted receptive field. However, for formal benchmarking, we believe that comparing against unaltered official implementations of the benchmarks is best option.

    Regarding variance across runs: training runs on these datasets exhibit high stability and very low seed-to-seed variance. In the submission, the tables in the appendix reported standard deviations across test samples to reflect sample-level prediction spread, which is much higher than training seed spread. However, we fully agree with the reviewer that reporting run-to-run variance across different random seeds is important for completeness. Following the reviewer's recommendation, we trained both uPDEt and SMART across 5 different random seeds on DrivAerML to show that the variance remains very low, obtaining a standard deviation of 0.00058 and 0.00094 on the relative L2 error, respectively. This represents only around 0.62% and 1.20% of the mean magnitude. These results have been added to the appendix.

We thank the reviewer once again for their valuable time and constructive feedback. We look forward to engaging in further fruitful discussions during the rebuttal phase.

Reviewer_3nsFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces uPDEt, a Transformer-based neural operator for predicting physical fields on unstructured geometries. The architecture combines a multiscale ball-tree hierarchy with local self-attention, spatial rotary position embeddings, and attention-based pooling and unpooling. It also incorporates parameter-modulated MLPs and a queryable decoder to handle global physical parameters and predictions at arbitrary spatial locations. The model is applied to both direct steady-state prediction and autoregressive modeling of transient dynamics. Experiments cover several CFD and structural mechanics benchmarks, evaluating prediction accuracy, computational efficiency, resolution scaling, and the effects of individual architectural components.

Critical strengths

Strong practical relevance. The paper addresses the important challenge of efficient neural surrogate modeling on large-scale, unstructured geometries, with applications to computational fluid dynamics and structural mechanics.

Effective architectural integration. The combination of multiscale local attention, ball-tree partitioning, attention-based pooling and unpooling, and queryable decoding provides a practical architecture for processing unstructured physical domains.

Convincing computational efficiency. The experiments demonstrate substantial reductions in inference time, FLOPs, and memory consumption compared with high-accuracy baselines such as SMART, while retaining competitive predictive accuracy at the evaluated configurations.

Promising empirical scalability. The resolution-scaling experiments demonstrate favorable runtime and memory scaling on large point clouds. The cross-resolution experiments also provide encouraging evidence of practical generalization to different spatial resolutions.

Broad empirical evaluation. The paper evaluates steady-state and transient problems across several physical systems, includes comparisons under different resource-matching protocols, and provides extensive architectural ablations. The short-horizon autoregressive results and aerodynamic coefficient evaluations offer additional evidence of practical utility.

Critical weaknesses

Limited methodological clarity and formalization. Although the main architectural components are described, important implementation details remain unclear. Such examples are the ball-tree construction and updates, query-to-support neighborhood assignments, and multiscale skip connections. The mathematical formulation is also somewhat informal, with insufficient distinction between established properties and architectural intuitions.

Limited theoretical justification. The paper provides little theoretical analysis beyond the computational complexity estimate. In particular, the connections to multigrid methods and multiscale physical phenomena remain largely heuristic. Moreover, the linear-complexity analysis does not explicitly account for all architectural components, including tree construction, pooling/unpooling, and query processing.

Incomplete accuracy–efficiency trade-off evaluation. Although the reported results demonstrate favorable trade-offs at selected configurations, the paper does not systematically compare accuracy–throughput or accuracy–compute curves across varying model capacities. In particular, the resource-matched comparison downsizes SMART without providing the complementary experiment of scaling uPDEt toward the resource budget of full SMART, limiting the evidence for general Pareto superiority.

Limited autoregressive rollout evaluation. The transient experiments consider only 1, 5, and 10 rollout steps without clearly reporting the physical time intervals between snapshots. Consequently, the significance of these horizons is difficult to assess. Longer rollouts, especially for the KS and Ellipse datasets, would provide stronger evidence of long-term predictive accuracy and stability.

Potential confounding factors and insufficient validation of physical claims. The baseline comparisons do not always use equivalent input information, as uPDEt benefits from auxiliary geometric attributes unavailable to some competing models. This leaves the choice of baselines somewhat questionable.

Questions for authors

Please interpret the weakness points as my questions.

7 comments

We thank the reviewer for the valuable feedback. We appreciate the positive remarks on our work's strengths and address the concerns raised below.

  • W1: We agree that additional implementation details could be included in the main text, in addition to the details provided in the appendix. The ball tree construction is detailed in Appendix E.2, where two implementations are analyzed and the more efficient variant is selected. In our pipeline, the hierarchical ball tree is constructed only once at the beginning across all levels. The model then queries the precomputed neighbor lists at the required resolution during each forward pass.

    Regarding query-to-support neighborhood assignments and multiscale skip connections: query points cross-attend to all support points, although this can be easily modified in our implementation, and our skip connections follow the standard formulation used in classical UNets and PDE-Transformer. We have improved the algorithm in the manuscript for more clarity, and we'd be happy to hear whether these explanations address all previously unclear aspects.

    The distinction between established properties and architectural intuition is a very good point: we aimed to clearly and broadly categorize these in Table 5 (now Table 6) and Appendix E. To summarize, all our new contributions add extra accuracy, but with different magnitude of improvement, with 3D RoPE being the biggest one. This contributions were designed based on our intuition, and these experiments help to verify our hypothesis.

  • W2: We appreciate this constructive perspective and agree that strong conceptual and theoretical grounding is essential. While our primary contribution is architectural and empirical, our design is directly grounded in established principles from computational physics and neural operators, such as the connection between multiscale hierarchies and multigrid solvers. Rather than introducing complex formalisms, our goal is to maintain a rigorous connection between these physical principles and their practical realization. The connection between multiscale phenomena, UNet hierarchies, and multigrid solvers has been widely established in computational physics and neural operator literature, and our design directly builds upon these principles. We have included some references to previous work to justify this connection around line 169.

    Regarding computational complexity, our initial derivation in the submission focused on the support and geometry attention mechanism. In response to the reviewer's feedback, we have expanded this analysis in the revised manuscript to include all remaining operational components. The total computational complexity remains quasi-linear, which is fully corroborated by our empirical runtime and scaling benchmarks.

  • W3: Our initial benchmarks contain a parameter-matched comparison between models with parameter counts in the common ranges in this field, to make it compatible with previous work. While our initial manuscript reported model scaling on the transient benchmarks, we agree that including the same scaling behavior on steady-state benchmarks is valuable. Repeating the scaling analysis of Figure 5 (now Figure 6) on DrivAerML yields the results in the following table, showing the relative rel L2L_2 averaged across all the channels and normalized with the value achieved of the S version. Here, S is our base configurations from Table 3 (around 10-15M) and goes up to L with around 200M parameters.

Model SizeSSMMMLL
uPDEt (ours)1.000.860.700.630.56
SMART1.000.850.660.580.51
Transolver++1.000.810.790.750.71

Both architectures demonstrate consistent and comparable power-law scaling across model capacities. Additionally, we performed the suggested resource-matched comparison, scaling uPDEt to match the computational budget of SMART. The relative L2L_2 errors on DrivAerML are:

Modelpsp_sτs\tau_spvp_vuvu_v
SMART4.419.113.968.02
uPDEt-matched3.938.293.487.46

Thus, under matched computational budgets, uPDEt consistently outperforms SMART across all surface (ps,τsp_s, \tau_s) and volumetric (pv,uvp_v, u_v) fields for DrivAerML. We are currently finalizing the corresponding resource-matched runs for SHIFT-Wing and SHIFT-Crash, and will incorporate them into the final manuscript (or in an updated rebuttal response as soon as they complete).

  • W4: In unstructured 3D surrogate modeling, existing literature typically reports only very short rollout horizons (e.g., 5–10 steps), as several baseline methods quickly accumulate errors approaching a relative L2L_2 error of ∼1.0\sim 1.0, beyond which predictions no longer retain any meaningful physical fidelity. The physical time step sizes for our tests are 0.1s0.1\text{s} for KS, 0.05s0.05\text{s} for Ellipse, and 0.01s0.01\text{s} for SHIFT-Crash (the latter having a physical simulation duration capped at 12 steps).

    Following the reviewer's suggestion, we evaluated long-horizon rollouts up to the maximum available ground truth timesteps for both KS and Ellipse (SHIFT-Crash already reaches almost the maximum in the original manuscript):

Model (KS)15 Steps20 Steps30 Steps
PTV31.1131.1271.219
Erwin1.3651.4671.578
Transolver++1.1481.1751.560
uPDEt (ours)0.7240.9931.271
Model (Ellipse)20 Steps50 Steps100 Steps
PTV30.7691.8571.957
Erwin0.1080.4997.686
Transolver++0.0200.0270.046
uPDEt (ours)0.0220.0300.058

Even over extended horizons, uPDEt remains competitive, exhibiting significantly more stable error growth than PTV3 and Erwin. Only Transolver++, and only in Ellipse, shows slightly better results.

  • W5: We appreciate the opportunity to clarify the input configurations across models. Four of the baseline architectures evaluated (Transolver++, GeoTransolver, Erwin, and PTV3) use the exact same input features as uPDEt (spatial coordinates, surface normals, and cell areas). In contrast, SMART and AB-UPT were designed exclusively for spatial coordinate inputs: their published architectures do not natively ingest auxiliary geometric channels without structural alterations. To ensure a fair benchmark, all models were evaluated under their standard, intended configurations at their maximum capability.

    The capacity to naturally incorporate rich boundary geometric features (such as normals and area weighting) is an inherent architectural advantage of flexible operators like uPDEt. Therefore, we emphasize that omitting normals and cell areas for AB-UPT and SMART was not an arbitrary choice, but a consequence of their architectural designs. Thus, all models are compared fairly, based on their intended designs at their maximum capabilities. Furthermore, our baseline selection reflects the standard in the field, with AB-UPT and Transolver being widely adopted benchmarks and SMART representing the strongest recent baseline. We have revised the discussion around line 297 to explain these architectural distinctions and baseline setups more clearly.

We hope these clarifications and additional experiments effectively address the reviewer's concerns. We would be happy to answer any additional questions.

I appreciate the authors' in-depth response. The revised manuscript improves several aspects of the presentation and empirical evaluation. In particular, the explicit skip connections, enlarged-uPDEt comparison, and extended rollouts address parts of my original concerns. However, the query-decoder complexity remains insufficiently justified, and the resource requirements of the new matched-budget configuration are not reported clearly enough to assess the comparison. Also, although longer rollouts have been added, the time interval between consecutive prediction steps remains unclear in the revised manuscript, making it difficult to assess the duration and dynamical significance of the evaluated horizons. More importantly, the work appears primarily to integrate architectural ideas widely adopted in the respective fields. While the resulting model has some practical advantage, the broader insight, significance, and its indication of this integration remain largely limited.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Weakness asked by all reviewers: input representations

All reviewers questioned how fair the baseline comparisons with SMART and AB-UPT are, given that these models do not ingest normals and areas as input, only coordinates, while the other models do. In the individual rebuttals, we already explained that these two models work only with coordinates by design, and that they were benchmarked to the best of their capabilities to make the comparison fair. Still, we tried to run SMART with the additional attributes concatenated to the input. By default, this yields runtime errors because the model is not designed for this. We then modified the source code to ingest them correctly and to keep the coordinate embedding separate for a correct positional encoding. Our modifications guarantee that the model can now run with additional inputs, but we acknowledge that this is a modification of the model and no longer the official implementation. After our modifications, we verified that the model reproduces the previous results when only the coordinates are passed. Then, in a second run, we passed the normals and areas as input. Our most important finding is that this makes the DrivAerML results only around 1% better, which might be pure noise (final relative L2 error of 7.47 vs. 7.57). This helps to support our training setup and baseline comparisons. The comparison of uPDEt against Erwin, PTV3, Transolver++ and GeoTransolver was always fair. Against SMART and AB-UPT, we believe it is fair as well, particularly given these findings. This supports the fact that SMART is not designed to ingest anything but the coordinates: even after we enabled it to do so by modifying the official implementation, the results did not improve.

As suggested by reviewer G74t, we now have the results for uPDEt coordinate-only for SHIFT-Wing and SHIFT-Crash. For SHIFT-Wing, the additional attributes improve the error a 6%. For SHIFT-Crash, we must note that our original dataset did not contain the normals of the mesh, we only reported the coordinates-only results, also for uPDEt, so that comparison was completely fair from the begining. Now, we added the normals to the input of uPDEt and there is a 27% improvement, now outperforming SMART. Because of previous discussion, we believe that our initial setup was still the most fair one.

uPDEtCoordinates onlyWith attributes
SHIFT-Wing15.24 (new)14.38
SHIFT-Crash23.2618.58 (new)

We have further improved this discussion in Appendix E.9.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.