- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes uPDEt, a hierarchical Transformer neural operator with a U-Net-like structure, designed for large-scale physics simulations on unstructured meshes. By using local attention and hierarchical processing, the computational complexity scales linearly with the number of input points N.
The authors evaluate the method on three steady-state benchmarks, including DrivAerML, SHIFT-Wing, and SHIFT-Crash, using models with around 10–15M parameters. They also consider autoregressive prediction on 2D KS with deformed points, 2D Ellipse flow, and SHIFT-Crash.
In terms of accuracy, uPDEt performs slightly worse than SMART with similar parameter counts, achieves comparable results to GeoTransolver, and outperforms AB-UPT, Transolver++, Erwin, and PTV3 in the reported comparisons. The main advantage is computational efficiency. Compared with SMART, uPDEt is about 3.7 times faster, uses around half the memory, and requires 7.8 times fewer FLOPs. The method can also process up to 16.8M points on a single H100 GPU.
Critical strengths
-
The main strength of this paper is its computational efficiency while maintaining competitive accuracy. According to Table 30, uPDEt achieves similar accuracy to GeoTransolver on DrivAerML, but with much higher throughput (3.92 vs 0.51 samples/s). Compared with SMART, it is around 3.7 times faster and requires only about half the GPU memory. Table 31 also shows approximately linear scaling of computation time and memory consumption, and the model can handle 16.8M points on a single GPU, while SMART and Transolver++ run out of memory. I think this is an important practical advantage for large-scale CFD applications. The efficiency analysis is also relatively detailed.
-
Another strength is that the same architecture can be applied to different types of problems, including steady-state prediction, surface and volume queries, and autoregressive rollouts. On SHIFT-Crash autoregressive prediction, uPDEt achieves around 2 times lower NRMSE than Transolver++ across different prediction horizons (Table 7). I also appreciate that the paper clearly acknowledges that uPDEt does not outperform SMART in terms of pure prediction accuracy.
Critical weaknesses
- My main concern is about the fairness of the accuracy comparison, especially the differences in input features and computational budgets.
-
Input features: According to Sec. 4 and Appendix E.9, uPDEt uses additional geometric information, including surface normals and cell areas, while SMART and AB-UPT only use point coordinates. These additional features can be helpful for learning physical quantities, especially on complex geometries. The paper's own ablation in Table 27 shows that coordinates-only uPDEt has a channel-averaged error of 13.47% on DrivAerML. Therefore, it is difficult to determine how much of the accuracy improvement comes from the proposed architecture rather than the additional input information.
-
Computational budget: Although the paper compares models with similar parameter counts, I think this is not sufficient for a fair comparison. Since uPDEt and SMART have very different computational costs, a comparison with matched training FLOPs or training time would be more informative. The current comparison with a smaller SMART model does not fully show the trade-off between accuracy and computational resources.
-
I also have some concerns about the methodological novelty. The proposed architecture seems to mainly combine several existing techniques, including local window attention from PDE-Transformer, ball-tree partitioning and rotation from Erwin, RoPE, FiLM, and cross-attention decoding similar to AB-UPT and SMART. These are all reasonable design choices, and their combination can be practically useful. However, it is not very clear which component represents the main methodological contribution compared with previous work. I think the paper should better distinguish the proposed method from these existing architectures and explain which design is responsible for the improvement.
-
Some claims in the paper seem stronger than what the experimental results actually show.
-
Speedup: In Sec. 4.1, the paper claims an "order-of-magnitude speedup relative to SMART". However, Table 30 reports a speedup of around 3.7 times, while the reduction in FLOPs is about 7.8 times. Table 31 also reports speedups of around 3–5 times. Therefore, I think the statement about an order-of-magnitude speedup is not accurate for the actual runtime results.
-
Autoregressive prediction: The conclusion claims "superior autoregressive rollout accuracy". However, on the Ellipse benchmark, Transolver++ achieves equal or slightly better results at every prediction horizon. Therefore, this claim should be qualified rather than presented as a general conclusion.
- The statistical reliability of the results is also unclear. The standard deviations reported in Appendix I are computed across test samples rather than different training seeds. Therefore, they do not reflect the randomness of model training. Some performance differences are also quite small. For example, on DrivAerML surface pressure, uPDEt achieves 5.72 ± 0.71 compared with 5.76 ± 0.63 for GeoTransolver. Without multiple training runs, it is difficult to know whether such a small difference is meaningful. I think reporting results from multiple random seeds would make the conclusions more convincing.
Questions for authors
-
Could the authors provide comparisons where all models use the same input features? In particular, it would be helpful to evaluate SMART and AB-UPT with surface normals and cell areas, or evaluate uPDEt with coordinates only, across all three steady-state benchmarks. Also, could the authors clarify which input features were used for Transolver++, GeoTransolver, Erwin, and PTV3?
-
How was SMART-S trained compared with uPDEt? Did the two models use the same number of epochs, training time, or total training FLOPs? I think it would be useful to provide accuracy-versus-compute curves for both uPDEt and SMART, using several model sizes for each. This would give a more complete comparison than using only one smaller SMART model.
-
I have a question about the receptive field at different resolutions. For 32K input points, the coarsest level contains a single ball with 8192 tokens, so the model can perform global attention at this level. However, when the number of input points increases, the coarsest level will contain multiple balls if the number of levels and ball size remain unchanged. In this case, the model may no longer have a global receptive field. Could the authors clarify which hierarchical configurations were used for the different resolutions in Fig. 4, Table 6, and Table 31? In particular, is the reported linear scaling achieved while maintaining global attention at the bottleneck?
-
On SHIFT-Wing, uPDEt and all other models except SMART have quite large errors on volume velocity, around 28–33%, compared with approximately 12% for SMART. This is a relatively large performance gap. Could the authors explain why this happens? Is it related to the limited global information exchange caused by local window attention, especially for transonic flow where long-range interactions may be important?
4 comments
-
W4: Regarding variance across runs: training runs on these datasets exhibit high stability and very low seed-to-seed variance. In the submission, the tables in the appendix reported standard deviations across test samples to reflect sample-level prediction spread, which is much higher than training seed spread. However, we fully agree with the reviewer that reporting run-to-run variance across different random seeds is important for completeness. Following the reviewer's recommendation, we trained both uPDEt and SMART across 5 different random seeds on DrivAerML to show that the variance remains very low, obtaining a standard deviation of 0.00058 and 0.00094 on the relative L2 error, respectively. This represents only around 0.62% and 1.20% of the mean magnitude. These results have been added to the appendix. We will include a similar analysis for all the experiments for future revisions of our paper.
-
Q1: As noted in W1.1, SMART and AB-UPT were designed for coordinate inputs and do not natively support additional input channels. In contrast, the remaining baselines (Transolver++, GeoTransolver, Erwin, and PTV3) utilize the exact same inputs as uPDEt: spatial coordinates, surface normals, and cell areas.
-
Q2: SMART-S uses the same memory and training time as our standard uPDEt configuration. We agree that it is valuable to provide a broader model scaling analysis across architectures. While our initial manuscript already reported model scaling on the transient benchmarks, we agree that including scaling behavior on steady-state benchmarks is likewise interesting. Repeating the scaling analysis of Figure 5 (now Figure 6) on DrivAerML yields the following table, showing the relative rel averaged across all the channels and normalized with the value achieved of the S version. Here, S is our base configurations from Table 3 (around 10-15M) and goes up to L with around 200M parameters.
| Model Size | S | SM | M | ML | L |
|---|---|---|---|---|---|
| uPDEt (ours) | 1.00 | 0.86 | 0.70 | 0.63 | 0.56 |
| SMART | 1.00 | 0.85 | 0.66 | 0.58 | 0.51 |
| Transolver++ | 1.00 | 0.81 | 0.79 | 0.75 | 0.71 |
Thus, both architectures demonstrate consistent and comparable power-law scaling across model capacities. Still, in this scaling analysis, uPDEt presents other clear advantages. For bigger models, efficiency is even more critical. While bigger versions of SMART need big GPUs and many days of training, uPDEt is much faster to train. We are currently finalizing the corresponding runs for SHIFT-Wing and SHIFT-Crash, and will incorporate them into the final manuscript (or in an updated rebuttal response as soon as they complete).
-
Q3: We thank the reviewer for this very good question. In our reported experiments, the ball radius scales with point density to preserve a consistent neighborhood size (physically) and ensure effective global communication at the coarsest level. In our benchmarks, we did not observe substantial differences compared to a fixed-radius setup; however, guided by the ablation results in Table 23, fixing the radius across resolutions can degrade predictive accuracy.
-
Q4: We thank the reviewer for highlighting this interesting observation. We also noticed this discrepancy and verified its reproducibility through repeated runs. We are investigating possible factors behind this behavior and greatly appreciate the hypothesis suggested by the reviewer. In addition, we want to highlight that in many industrial aerodynamic workflows, surface quantities remain the primary engineering metrics since they are the basis for computing aerodynamic forces; nonetheless, fully understanding volumetric dynamics is important. We do not have a clear answer yet, but we believe that our global communication step at the coarsest level should account for long range interactions in transonic flows. Since this only happens in our SHIFT-Wing experiments, it could be more related to the dataset than the models.
We thank the reviewer once again for their constructive suggestions and deep engagement with our work. We look forward to continuing this discussion during the rebuttal phase.
Thanks for addressing my concerns. A follow up question: is it possible for coordinate-only uPDEt next to SMART and AB-UPT on all three benchmarks?
Also, I recommend the authors highlight the novelty of the method in the abstract. As for now, the abstract only talks about the performance improvement.
Thank you for positively acknowledging our previous answers. We added a general official comment about the input representations, where we ran all the coordinate-only uPDEt benchmarks. In addition, we have modified the abstract as suggested.
As a final note, in our answer to W1.2 and Q2, we provided the first results of our additional scaling experiments. The remaining additional experiments are now finished and are included in the revised manuscript. In the final resource-matching experiments (Table 4), uPDEt outperforms SMART in 90% of the cases. Regarding the general model size scaling (Figures 5, 6 and 12), uPDEt shows a better progression than Transolver++. For steady-state experiments, uPDEt closely matches SMART, although the final order depends on the particular dataset used.
We thank the reviewer for the detailed, thoughtful assessment and constructive feedback. We appreciate the positive evaluation of our work's strengths and address each weakness (W) and question (Q) below.
W1.1: We appreciate the opportunity to clarify the input configurations across models. Four of the baseline architectures evaluated (Transolver++, GeoTransolver, Erwin, and PTV3) use the exact same input features as uPDEt (spatial coordinates, surface normals, and cell areas). In contrast, SMART and AB-UPT were designed exclusively for spatial coordinate inputs; their published architectures do not natively ingest auxiliary geometric channels without structural alterations. To ensure a fair benchmark, all models were evaluated under their standard, intended configurations at their maximum capability.
The capacity to naturally incorporate rich boundary geometric features (such as normals and area weighting) is an inherent architectural advantage of flexible modern operators like uPDEt. Therefore, we emphasize that omitting normals and cell areas for AB-UPT and SMART was not an arbitrary choice, but a consequence of their architectural designs, which do not natively support these auxiliary inputs. Thus, all models are compared fairly based on their intended designs at their maximum capabilities. Furthermore, our baseline selection reflects the standard in the field, with AB-UPT and Transolver being widely adopted benchmarks and SMART representing the strongest recent baseline. We have revised the discussion around line 297 to explain these architectural distinctions and baseline setups more clearly.
Specifically, Appendix E.9 provides an ablation isolating the effect of the input channels. Indeed, we have quantified the isolated contribution the input channels: for DrivAerML, if we ignore normals and cell areas in uPDEt, the relative L2 grows from 8.81 to 13.47. Together with the rest of Appendix E and Table 5 (now table 6), we systematically analyzed which design and architectural choices yield the greatest performance gains.
W1.2: Regarding the computational budget, our main results provide comparisons under parameter-matched configurations, as is standard in prior literature. However, we agree with the reviewer that resource-matched comparisons provide an interesting, complementary perspective. Therefore, we included the efficiency-matched comparison against SMART in Table 4. We focused on SMART for this comparison because it represents the strongest competing baseline; for the remaining baselines, the Pareto trade-off is already evident (i.e., uPDEt matches or exceeds their accuracy while consuming substantially fewer resources). In Table 4, we evaluated a smaller configuration of SMART (SMART-S) with a peak memory footprint comparable to our standard uPDEt, trained for the exact same duration. This establishes an equitable comparison where both models share identical memory budgets and training time.
Additionally, we just performed the opposite resource-matched comparison, scaling uPDEt to match the computational budget of full SMART, in addition to the opposite that was already provided in the manuscript. The relative L2 errors on DrivAerML are:
Under matched computational budgets, uPDEt consistently outperforms SMART across all surface (ps,τs) and volumetric (pv,uv) fields for DrivAerML. We are currently finalizing the corresponding resource-matched runs for SHIFT-Wing and SHIFT-Crash, and will incorporate them into the final manuscript (or in an updated rebuttal response as soon as they complete).
W2: We agree with the reviewer that understanding the individual contributions of each architectural component is crucial. To this end, Appendix E provides a comprehensive ablation across all design choices. Table 5 (now Table 6) summarizes the primary ablation of our contributions. While all of the modifications contribute with some improvement, RoPE represents clearly the biggest one. We'd be happy to expand on any specific component if further clarification is helpful.
W3: We agree with the reviewer that this was an overstatement. We have carefully revised these claims in the manuscript. While the statement holds for the SHIFT-Crash visualization in Figure 3d, we have tempered the phrasing to reflect that it does not apply across all regimes. Likewise, we revised "superior autoregressive rollout accuracy" to accurately reflect that Transolver++ achieves the best accuracy on the Ellipse benchmark.