acceptodds
Under review as a conference paper at ICLR 2027

Seeing Near and Far: A Neural Operator That Efficiently Tokenizes and Decodes on Industrial-Scale CFD Meshes

Abstract

Industrial-scale computational fluid dynamics (CFD) simulations typically require more than a million mesh points to fully capture complex nonlinear physics. Consequently, recent efforts have focused on designing neural operators that can handle such large, complex meshes without sacrificing speed or accuracy. Approaches that aggregate only local information during tokenization and decoding are highly compute-efficient, especially as meshes grow larger. However, on 3D industrial-scale benchmarks, they are currently outperformed by models that retain global information through pointwise computation across the entire domain. We hypothesize that this performance gap is partly attributable to a mismatch between latent-point allocation and spatially varying reconstruction demands, and propose the use of a KD-tree algorithm for more efficient allocation. To further generalize this idea, we propose the Importance-Balanced Operator (IBO), which integrates an importance measure into the KD-tree algorithm to distribute latent points more effectively, substantially closing this performance gap while retaining computational efficiency. In our experiments, IBO achieves the best performance across multiple metrics on 2D single- and multi-airfoil aerodynamic problems. Crucially, on large-scale 3D datasets involving automotive aerodynamics (DrivAerML) and urban aerodynamics (the newly introduced UrbanWind dataset), IBO matches or outperforms state-of-the-art models across nearly all field metrics while requiring 5–40× fewer FLOPs per case, thereby establishing a new accuracy–compute Pareto frontier for learning industrial-scale CFD.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operators, computational fluid dynamics, scientific machine learning, operator learning, point clouds, industrial-scale CFD

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4wGKOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

To address performance gaps on large-scale CFD meshes while maintaining low computational cost, the authors propose the Importance-Balanced Operator (IBO), which uses an importance-weighted KD-tree to allocate latent tokens in proportion to spatial physical complexity.

Critical strengths
  1. The paper mathematically proves that allocating latent tokens in proportion to spatial physical complexity (λ∗(x)=Lρ(x)\lambda^*(x) = L \rho(x)) minimizes the decoder resolution error bound, providing a solid theoretical foundation for physics-informed tokenization.

  2. On industrial-scale 3D benchmarks (e.g., DrivAerML and UrbanWind with millions of mesh points), IBO matches or exceeds the predictive accuracy over state-of-art methods

Critical weaknesses
  1. The paper emphasizes scaling to industrial-scale CFD meshes; however, key industrial engineering metrics (e.g., pressure distributions, skin friction, and surface drag/lift forces) rely predominantly on accurate surface-field predictions. On datasets such as DriverNetML, IBO performs worse on surface pressure fields. I feel like that a primary reason for this limitation is that IBO’s weighting mechanism (wi=1w_i = 1) on locally refined meshes relies entirely on the node distribution of the input CFD mesh, effectively outsourcing the estimation of physical difficulty back to the human mesh generator. Consequently, the paper’s claim of bridging the performance gap between spatially localized and globally-resolved models remains largely unsolved for surface data. So it seems that this method appears restricted to fluid flow problems rather than general operator learning.

  2. The authors use an axis-aligned, weighted KD-tree to partition 3D spaces. However, orthogonal bounding-box splits cut arbitrarily across complex 3D curved surfaces, thin boundary layers, and non-axis-aligned geometries. Considering that, the paper lacks a literature review on domain decomposition and spatial partitioning in scientific machine learning, raising concerns about whether alternative geometric decomposition schemes were adequately evaluated. Furthermore, given that local mesh resolution in adaptive CFD grids directly reflects regional physical difficulty, can local resolution itself be used directly to infer physical difficulty without relying on hand-crafted distance heuristics? With that, I think at least 2-3 baseline is missing: 1) Use mesh density directly to partition; 2) existing method for complex geometry such as PGD-NO (https://openreview.net/forum?id=h5vFCxA37Y). Also, the literature review of domain partition / reducing memory requirement of a neural operator (for instance the PGD-NO [1], SMART[2]) is also required to be added.

[1] Zhong, W., Bi, J., Oancea, V., & Meidani, H. (2026). PGD-NO: A Neural Operator with Precomputed Geometry Decomposition for 3D Million-scale Physics Simulations. Proceedings of the 43rd International Conference on Machine Learning

[2] Hagnberger, J., & Niepert, M. (2026, January). SMART: Scalable Mesh‑free Aerodynamic Simulations from Raw Geometries using a Transformer‑based Surrogate Model. In Forty-third International Conference on Machine Learning.

  1. Taking Points 1 and 2 together, the primary empirical advantage of IBO appears confined mainly to fluid volume field predictions, which limits the overall novelty and practical impact of the work for real-world design decisions. Could the authors clarify how this framework extends beyond steady-state external fluid flows? Specifically, in many engineering domains (e.g., structural mechanics, dynamic shock waves, coupled multiphysics, or dynamic time-dependent flows), defining a scalar "importance score" ρ(x)\rho(x) requires deep domain-specific knowledge and can be highly non-trivial or time-varying.
Questions for authors

Please see my Critical Weaknesses for my questions.

7 comments
AuthorsOpenReview ↗

Response to Reviewer 4wGK

We thank the reviewer for the careful assessment and provide our responses below.

1. The paper emphasizes scaling to industrial-scale CFD meshes; however, key industrial engineering metrics (e.g., pressure distributions, skin friction, and surface drag/lift forces) rely predominantly on accurate surface-field predictions. ... I feel like that a primary reason for this limitation is that IBO’s weighting mechanism on locally refined meshes relies entirely on the node distribution of the input CFD mesh, effectively outsourcing the estimation of physical difficulty back to the human mesh generator.

We acknowledge that surface fields and integrated forces can be important for design, and that IBO is not the best model on them across all our benchmark problems. Interestingly, on the 2D benchmarks, IBO is best on the surface: 0.0006 vs 0.0011 (AB-UPT) on AirfRANS, and 9–40% below the best alternate baseline model on all five TandemFoilSet subsets. However, this is not true for the 3D benchmarks.

While re-examining DrivAerML, we found that Table 3 of the submission was not computed under a single protocol as the baseline rows taken from Transolver-3's table mix magnitude errors (∣u∣|\mathbf u|, ∣τ∣|\boldsymbol\tau|) with vector errors (u\mathbf u, τ\boldsymbol\tau). We provide the corrected comparison under both Transolver-3 and AB-UPT evaluation protocols here, and will update the manuscript accordingly:

Modelpsp_s∣τ∣\lvert\tau\rvertCptC_{p_t}∣u∣\lvert u\rvertR2R^2 CDC_DR2R^2 CLC_LPFLOP
Transolver-33.715.855.724.140.9720.98525.3
IBO3.86±0.036.11±0.015.68±0.024.13±0.020.961±0.0070.975±0.0042.6

Under AB-UPT's protocol, IBO is almost level with AB-UPT on the surface fields (psp_s 3.87 vs 3.82, τ\tau 7.23 vs 7.29). We note that under the same evaluation protocol used by Transolver-3, we no longer see any large performance difference, being only 4% behind Transolver-3 on the surface fields. It is also competitive with Transolver-3 on the UrbanWind problem, showing that IBO does not have a systematic weakness for surface prediction relative to other state-of-the-art models currently. Nonetheless, we do acknowledge that IBO's surface-pressure error on UrbanWind is twice AB-UPT's (0.0023 vs 0.0012).

We believe the 3D surface gap comes from how we allocated the budget, not from inherent limitations in the model:

  • UrbanWind. AB-UPT places 16,384 anchors on the surface (22% of all surface points of a mean test case) plus 16,384 in the volume. IBO's 16,384 tokens all lie in the volume, and its importance measure ignores the surface: surface points are only attached to volume leaves (App. D.3).
  • DrivAerML. Transolver-3 trains a separate 24-layer surface model, whereas IBO and AB-UPT predict surface and volume with one 12-block model.

Given the larger discrepancy in surface-pressure error relative to AB-UPT, we are training IBO on UrbanWind with 32,768 tokens (AB-UPT's anchor count) as a fairer comparison. Related to your helpful suggestion regarding the current weighting mechanism neglecting surface data, we have modified the model such that surface points are now included in the importance measure with a higher weight, the wall-distance cap is lowered from 2 m to 0.2 m, and half of the supervised points lie on the surface, as in AB-UPT. We will post the results subsequently, to provide insight into whether this was indeed the reason for the discrepancy in performance, and will update our manuscript accordingly.

AuthorsOpenReview ↗

Response to Reviewer 4wGK

1-2. Outsourcing difficulty to the mesh generator. We do agree with the reviewer's insight that the definiton of the importance measure may still depend on the human mesh generator. In particular, we highlight that where a refined mesh exists with insights from a domain expert, its density is not arbitrary and should indeed be utilized as prior. Meshes are built or adapted to equidistribute an error indicator, through wall-spacing targets, wake and gap refinement or a-posteriori adaptation (Huang & Russell, 2011). The mesh density is thus the best available difficulty prior, and it is free at inference, although it may seem that this appears to outsource the estimation of field complexity back to the human expert. Nonetheless, our work is focused on adddressing how the latent layout should be designed for such models. Other methods can be designed to predict such physical field complexity, and we agree is an important future direction.

Crucially, Eq. 11 also shows that an approximately correct prior costs only a second-order penalty, meaning that simple priors can also be effective. IBO does not necessarily require a perfectly designed mesh from a human mesh generator. For example, UrbanWind's mesh is nearly regular (Cartesian with wall snapping), a fairly typical meshing strategy when the human mesh generator has little insight to offer, and the mesh carries correspondingly little prior. Hence, the importance comes from the geometry alone, through the wall-distance weight of Eq. 23. Unit weights already beat the uniform grid on all six channels, and the geometric weight improves all six further.

We will further include this in the Discussion as an important future direction.

2. The authors use an axis-aligned, weighted KD-tree to partition 3D spaces. However, orthogonal bounding-box splits cut arbitrarily across complex 3D curved surfaces, thin boundary layers, and non-axis-aligned geometries. Considering that, the paper lacks a literature review on domain decomposition and spatial partitioning in scientific machine learning, raising concerns about whether alternative geometric decomposition schemes were adequately evaluated.

Unlike the typical use of KD-tree for spatial partitioning, we note that the KD partition here is not meant to fit or delineate actual physical boundaries. It decides (i) where the latent points lie and (ii) which points each token summarizes. Leaves are small where the importance is high: on DrivAerML, the near-wall read-out distance is 2.3 cm, against 86 cm for a uniform grid of the same budget (Table 16). Boundary geometry enters through the features rather than the cell shapes, since every member and query carries its signed wall distance and wall direction (App. C.2, D.3). Reconstruction also ignores the leaf boundaries: each query reads its kk nearest tokens across the cuts, with distances normalized by the local spacing (Eq. 22), so a cut through a boundary layer does not produce a discontinuity in the prediction. Our current implementation therefore does not suffer from the typical issues such split-cuts may encounter where no additional featurization (in the latent tokens) is provided.

Literature. Hence, there are specific differences to what is typically used from spatial partitioning, but we do agree that a more detailed literature review is lacking here. We will include additional Related Work on domain decomposition and spatial partitioning:

  • quadtrees/octrees and AMR-Transformer: fixed geometric splits, refinement from the solution on structured grids, a variable number of tokens;
  • NUNO: a KD-tree into a few subdomains for grid-based operators;
  • Erwin: a balanced ball tree that keeps every point as a token;
  • PGD-NO: a sharp-edge decomposition of the surface mesh into geometry tokens;
  • SMART: a mesh-free model that encodes the raw geometry;

In particular, we think it worth highlighting that IBO differs by partitioning the mesh points by importance mass. This leaves no empty cells on non-convex multi-body domains, gives a fixed number of tokens, and realizes the density of Prop. 4.2.

AuthorsOpenReview ↗

Response to Reviewer 4wGK

3. Furthermore, given that local mesh resolution in adaptive CFD grids directly reflects regional physical difficulty, can local resolution itself be used directly to infer physical difficulty without relying on hand-crafted distance heuristics? With that, I think at least 2-3 baseline is missing: 1) Use mesh density directly to partition; 2) existing method for complex geometry such as PGD-NO (https://openreview.net/forum?id=h5vFCxA37Y).

Yes, and this is IBO's default: on AirfRANS, TandemFoilSet and DrivAerML every weight is 1, so every leaf holds the same number of mesh points and the partition follows the mesh density, with no distance heuristic. The distance weight is used only on UrbanWind, whose mesh carries no refinement prior; Table 6 compares the two weightings there (discussed in Point 1 above).

3-1. Mesh density used directly, without the KD partition.

Drawing the latent points at random from the mesh cells follows the mesh density in expectation, without any partition. By using a single, consistent IBO model, we investigate the impact of changes to the choice of latent layout. The 32,768 latent points are drawn uniformly from the volume mesh cells (akin to AB-UPT's scheme):

Latent layoutTokenspsp_s∣τ∣\lvert\tau\rvertCptC_{p_t}∣u∣\lvert u\rvertR2R^2 CDC_DR2R^2 CLC_L
Uniform grid32,7688.8610.989.026.580.8270.730
Mesh-uniform sampling32,7684.42±0.026.87±0.056.24±0.084.49±0.060.959±0.0030.976±0.004
IBO, KD-tree32,7683.86±0.036.11±0.015.68±0.024.13±0.020.961±0.0070.975±0.004

Read from top to bottom:

  • Uniform grid → mesh-uniform sampling: the errors fall by 31–50%. In expectation, sampling inherits the mesh density, which is the presumed optimal density of Prop. 4.2.
  • Sampling → KD-tree: at the same expected density, sampling remains 9–15% above the KD-tree on every field error, although the impact is within seed variation for R2R^2. The difference is much smaller than the performance gains relative to the use of a uniform grid as expected. We further discuss this below.

Why the KD-tree is better at the same density. All 3 methods being compared use the same number of tokens. The uniform grid ignores the importance measure, whereas mesh-uniform sampling and the KD-tree both inherit the prior from the mesh construction. However, mesh-uniform sampling has the optimal density λ⋆=Lρ\lambda^\star = L\rho of Prop. 4.2 in expectation, whereas the KD-tree realizes it at leaf scale (Eq. 12). An additional difference lies in the distances. This is consistent with Section 4.1, which bounds the error through the spatial extent ∼λ−1/d\sim\lambda^{-1/d} that each local representation must cover (Eq. 6; Appendix B for the encoder): the farther a point lies from the latent points that encode and reconstruct it, the larger its error. Under a second-order criterion, the mean squared distance between the points and the latent points that serve them, the optimal placement of a fixed number of latent points is the k-means solution, in which every latent point sits at the centroid of the points it serves (Lloyd, 1982; Gersho, 1979, Sec. IV). A sampled latent point is a single mesh point, not the centroid of a group of points. Every one of the 32,768 sampled points is therefore offset from the centroid of its neighboring points, such that there is systematic error. The KD-tree instead places every latent point at the weighted centroid of its leaf (Section 5), deterministically and with median splits alone, so it keeps this property of the k-means solution without the k-means iterations that are prohibitive on 1.4×1081.4\times10^8 points.

Read-out distance by wall distance. As in Figure 4a, we split the test volume points into three equal-count bins by wall distance (which we will include in Appendix D.6 of the revision). The read-out distance RkR_k, the mean distance from a query to the 16 latent points it reads, is 3% shorter with the KD-tree than with mesh-uniform sampling in the near and middle bins, and the error of IBO is 7–11% lower in every bin.

Reviewer_eG2BOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a method called Importance-Balanced Operator (IBO). Their main innovation is to choose latent points with a deterministic KD-tree procedure. As the field complexity is highly correlated with mesh resolution, this is a useful proxy that has been also observed in prior art [1].

Additionally, the paper introduces a new benchmark called UrbanWind. They show that IBO is the strongest model on Urbanwind and competitive on DrivAerML, an existing benchmark.

[1]: Alkin, Benedikt, et al. "AB-UPT: Scaling neural CFD surrogates for high-fidelity automotive aerodynamics simulations via anchored-branched universal physics transformers." arXiv preprint arXiv:2502.09692 (2025).

Critical strengths
  • the paper mathematically formalizes an empirically known observation, namely that spatial sampling must adapt to the difficulty of predicting the field.
  • the paper introduces a new dataset.
Critical weaknesses

The contributions are limited. The paper aims to formalize the notion of "spatially varying difficulty" which has been emphasized in prior work (e.g. AB-UPT [1], Table 5).

If the central claim is the importance, there should be more focus on isolating its effect, instead of introducing a new method that additionally changes other factors (e.g. the encoding and decoding). Taking a step, I think IBO is very similar to AB-UPT, as it spatially "subsamples" the points but with a deterministic KD-Tree procedure. I suggest adding experiments that isolate this subsampling effect on the AB-UPT alone. Then from there you could add additional experiments that gradually add the changes in the encoding/decoding. Such a chain of experiments would also give insights on the performance of this modules and how to improve the whole family of architectures with local subsampling -> global processsing -> local decoding.

The ablation is not clear as it just ablates with a uniform grid, but not "uniform according to the mesh" which was used in AB-UPT. I hand-checked some of the results of the uniform-grid ablation and it seems to perform worse than AB-UPT. A proper experiment with equal sampling (maybe even sampling the same points) would isolate this effect better.

Minor, but still worth mentioning: I find the title extremely weak. The paper is about imporance-balanced spatial subsampling. The title does not reflect this at all.

[1]: Alkin, Benedikt, et al. "AB-UPT: Scaling neural CFD surrogates for high-fidelity automotive aerodynamics simulations via anchored-branched universal physics transformers." arXiv preprint arXiv:2502.09692 (2025).

Questions for authors
  • It is unclear why in 4.1 the error is split into encoder, decoder and learned error and then in Section 4.2 only the decoder error is analyzed. As the network is trained end to end, these are related. E.g. in regions where the decoder makes a large error, the encoder or "learner" could try to compensate.

minor:

  • what exactly is meant with long skip connections?
5 comments
AuthorsOpenReview ↗

Response to Reviewer eG2B

We thank the reviewer for the detailed review, and in particular for suggesting the mesh-uniform sampling control and the isolation on AB-UPT. We have added both these experiments below. While doing so, we found an evaluation inconsistency in our DrivAerML table, which we detail below.

Correction of the DrivAerML results

Table 3 of the original submission was not computed under a single protocol as we found that the two papers we referenced (Transolver-3 vs AB-UPT) each had a slightly different evaluation protocol. Hence, the baseline rows taken from Transolver-3's table mix magnitude errors (∣u∣|\mathbf u|, ∣τ∣|\boldsymbol\tau|, Transolver-3's protocol) with vector errors (u\mathbf u, τ\boldsymbol\tau, AB-UPT's protocol). We now report the errors for IBO relative to each baseline's own protocol, against their corresponding published baseline values:

Transolver-3's protocol (magnitude errors)

Modelpsp_s∣τ∣\lvert\tau\rvertCptC_{p_t}∣u∣\lvert u\rvertR2R^2 CDC_DR2R^2 CLC_LPFLOP
Transolver-33.715.855.724.140.9720.98525.3
IBO3.86±0.036.11±0.015.68±0.024.13±0.020.961±0.0070.975±0.0042.6

AB-UPT's protocol (vector errors)

Modelpsp_sτ\tauCptC_{p_t}uuR2R^2 CDC_DR2R^2 CLC_LPFLOP
AB-UPT3.827.296.085.930.9630.97522.2
IBO3.87±0.037.23±0.005.68±0.025.20±0.030.961±0.0070.975±0.0042.6

IBO matches Transolver-3 on the volume fields, is 4% behind it on the surface fields. It is however, level with AB-UPT on the surface fields and forces, and 7–12% better on the volume fields, at ~9× fewer FLOPs than either. We will revise the section in our manuscript accordingly with these results. All DrivAerML ablations below were rerun under Transolver-3's protocol.

1. I think IBO is very similar to AB-UPT, as it spatially "subsamples" the points but with a deterministic KD-Tree procedure. I suggest adding experiments that isolate this subsampling effect on the AB-UPT alone. Then from there you could add additional experiments that gradually add the changes in the encoding/decoding. Such a chain of experiments would also give insights on the performance of this modules and how to improve the whole family of architectures with local subsampling -> global processsing -> local decoding.

We agree that AB-UPT observed that sampling anchors from the CFD mesh implicitly assigns more resources to refined regions (also mentioned in our Sec. 2). However, while this provides some initial motivation for our work, we want to highlight that our method focuses on spatially localized neural operators, and not neural operators like AB-UPT which utilize a set of global points for decoding. Crucially, we believe the latent layout to be one key reason why spatially localized models have tended to underperform models like Transolver-3 and AB-UPT, and seek to demonstrate that with our work. This is also a key distinction between IBO and AB-UPT.

In relation to your suggestion, we reproduced AB-UPT on DrivAerML and retrained it with one change. Its 16,384 volume anchors are chosen by farthest point sampling (space-filling) instead of uniformly from the mesh cells (mesh-following). This also better resembles a uniform grid sampling procedure, such that field complexity is not taken into account in modelling. Both runs are supervised on the same random mesh query points (AB-UPT's query-training mode, its App. A.3), so only the anchor positions differ. On DrivAerML, two thirds of the volume cells lie within 12.6 mm of the car, so the two layouts are very different.

AB-UPT volume anchorspsp_s∣τ∣\lvert\tau\rvertCptC_{p_t}∣u∣\lvert u\rvertR2R^2 CDC_DR2R^2 CLC_L
Random mesh cells (as published)3.776.025.754.240.9470.980
Farthest point sampling (space-filling)3.826.116.224.510.9560.980

Space-filling anchors raise AB-UPT's errors by only 1–8%. In IBO, the corresponding step (uniform grid → mesh-following sampling, point 3) improves errors by 31–50%. This further highlights our distinction, in that the layout matters far less when every query attends to all anchors than when each query is reconstructed from its kk nearest tokens, which is the class of models that Section 4 analyzes. We see this as the clearest statement of our contribution: importance-balanced allocation is what lets a spatially localized model reach the accuracy of global read-out at ~9× lower inference cost.

Hence, instead of performing the recommended chain of experiments with AB-UPT, we sought to address the same question by comparing the performance of IBO with a series of changes to the layout allocation protocol (detailed below in Point 2).

AuthorsOpenReview ↗

Response to Reviewer eG2B

2. The ablation is not clear as it just ablates with a uniform grid, but not "uniform according to the mesh" which was used in AB-UPT. I hand-checked some of the results of the uniform-grid ablation and it seems to perform worse than AB-UPT. A proper experiment with equal sampling (maybe even sampling the same points) would isolate this effect better.

By using a single, consistent IBO model, we investigate the impact of changes to the choice of latent layout. The 32,768 latent points are drawn uniformly from the volume mesh cells (akin to AB-UPT's scheme), with three seeds, each with its own draw:

Latent layoutTokenspsp_s∣τ∣\lvert\tau\rvertCptC_{p_t}∣u∣\lvert u\rvertR2R^2 CDC_DR2R^2 CLC_L
Uniform grid32,7688.8610.989.026.580.8270.730
Mesh-uniform sampling32,7684.42±0.026.87±0.056.24±0.084.49±0.060.959±0.0030.976±0.004
IBO, KD-tree32,7683.86±0.036.11±0.015.68±0.024.13±0.020.961±0.0070.975±0.004
IBO, KD-tree16,3844.046.325.824.220.9620.977

Read from top to bottom:

  • Uniform grid → mesh-uniform sampling: the errors fall by 31–50%. In expectation, sampling inherits the mesh density, which is the presumed optimal density of Prop. 4.2.
  • Sampling → KD-tree: at the same expected density, sampling remains 9–15% worse than KD-tree on every field error, although the impact is within seed variation for R2R^2. The difference is much smaller than the performance gains relative to the use of a uniform grid as expected. We further discuss this below.
  • Half the tokens: For completeness, and consistency with the manuscript, we include a comparison with a reduced number of tokens, showing that the errors rise by only 2–5%, and IBO with 16,384 tokens remains more accurate on every error than mesh-uniform sampling with all 32,768. With an importance-balanced layout, IBO can thus drop half its tokens, which cuts the cost of its self-attention to about a quarter.

Why the KD-tree is better at the same density. All 3 methods being compared use the same number of tokens. The uniform grid ignores the importance measure, whereas mesh-uniform sampling and the KD-tree both inherit the prior from the mesh construction. However, mesh-uniform sampling has the optimal density λ⋆=Lρ\lambda^\star = L\rho of Prop. 4.2 in expectation, whereas the KD-tree realizes it at leaf scale (Eq. 12). An additional difference lies in the distances. This is consistent with Section 4.1, which bounds the error through the spatial extent ∼λ−1/d\sim\lambda^{-1/d} that each local representation must cover (Eq. 6; Appendix B for the encoder): the farther a point lies from the latent points that encode and reconstruct it, the larger its error. Under a second-order criterion, the mean squared distance between the points and the latent points that serve them, the optimal placement of a fixed number of latent points is the k-means solution, in which every latent point sits at the centroid of the points it serves (Lloyd, 1982; Gersho, 1979, Sec. IV). A sampled latent point is a single mesh point, not the centroid of a group of points. Every one of the 32,768 sampled points is therefore offset from the centroid of its neighboring points, such that there is systematic error. The KD-tree instead places every latent point at the weighted centroid of its leaf (Section 5), deterministically and with median splits alone, so it keeps this property of the k-means solution without the k-means iterations that are prohibitive on 1.4×1081.4\times10^8 points.

Read-out distance by wall distance. As in Figure 4a, we split the test volume points into three equal-count bins by wall distance (which we will include in Appendix D.6 of the revision). The read-out distance RkR_k, the mean distance from a query to the 16 latent points it reads, is 3% shorter with the KD-tree than with mesh-uniform sampling in the near and middle bins, and the error of IBO is 7–11% lower in every bin.

Hence, we believe this additional ablation experiment fulfills a similar purpose from IBO's end such that the layout is systematically varied (grid → sampling → KD-tree, table above; grid → KD-tree also provided on all five TandemFoilSet subsets, Table 17). We do not build up from GINO or GAOT as both assume a regular latent grid (an FNO processor, or grid patches with fixed-radius neighbourhoods). An importance-balanced spacing necessitates a choice of model architecture that accommodates corresponding spatial variation in the layout.

AuthorsOpenReview ↗

Response to Reviewer eG2B

3. It is unclear why in 4.1 the error is split into encoder, decoder and learned error and then in Section 4.2 only the decoder error is analyzed. As the network is trained end to end, these are related. E.g. in regions where the decoder makes a large error, the encoder or "learner" could try to compensate.

Appendix B bounds the encoder term in the same form (Eq. 13) and shows that λ⋆=Lρ\lambda^\star = L\rho minimizes the encoder and decoder bounds jointly (Eqs. 14–16). For space, the main text showed only the decoder term; in our final revision (to be uploaded), we will state the joint result after Prop. 4.2. On compensation: εenc\varepsilon_{\rm enc} and εdec\varepsilon_{\rm dec} are defined through reference (oracle) encoding and reconstruction maps for a given layout and token capacity (Eq. 4), so they do not depend on the trained weights. Any compensation between the parts of the trained network is contained in εlearn\varepsilon_{\rm learn}. The analysis therefore concerns the best resolution error a layout permits.

4. What exactly is meant with long skip connections?

These are the long skip connections of U-ViT (Bao et al., 2023). Each early processor block is connected to its mirror late block through an RMS normalization, a learned linear projection and a scalar gate initialized to 0.01 (App. C.3). We will include this in the revision for clarity.

5. Title

We agree. We will retitle the paper: Seeing Near and Far: Importance-Balanced Spatial Latent Layouts for Globally Coupled, Locally Sharp Flows.

We are grateful for the suggestions, which have made the central claim considerably cleaner, and we would be glad to discuss further.

Reviewer_TjV7OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces importance balanced operator for large-scale CFD problems. To bridge the gap between localized operators and global methods, the method proposes KD-Trees for an importance measure based tokenization-like scheme. The authors prove that allocating latent points proportionally to this physical complexity minimizes the decoder's resolution error bound. The architecture processes this partitioned data using an off-the-shelf PointNet encoder, a standard pre-norm Transformer, and a local cross-attention decoder. Evaluated on massive 3D benchmarks, including DrivAerML (∼1.4×108\sim1.4 \times 10^8 points) and a newly introduced UrbanWind dataset, IBO achieves competitive accuracy while requiring significantly fewer inference FLOPs.

Critical strengths

The method exhibits strong empirical scaling, while slashing computational costs.

The authors extend the evaluation scenario to complex multi-body settings.

The authors provide clear theoretical motivation for their work, along with a theoretical justification section within the appendix.

Critical weaknesses

Lack of Novelty: The core contribution of the paper is essentially a static data preprocessing step. Using a KD-tree to partition spatial coordinates based on an importance measure is a classical algorithmic technique.

Overlap with existing approaches: The conceptual leap of using tree structures (e.g., quadtrees, octrees) and Adaptive Mesh Refinement (AMR) principles to dynamically or statically aggregate spatial data into a reduced latent grid is already established in recent neural fluid simulation literature (e.g., AMR-Transformer, CVPR 2025). The paper does not adequately differentiate its KD-tree tokenization from these existing hierarchical tokenization paradigms.

The method relies heavily on static, pre-computed importance measures and the current framing largely reads like a new data preprocessing step. Despite the theoretical grounding, this does not feel like the authors are proposing a new way of modeling the problem. What I mean is, perhaps the authors can refer to AMR-Transformer paper to provide an intuition of how their proposed approach changes how the problem is modeled compared to existing approaches that simply sample points. Are there some PDE structures that are fundamentally not captured with local settings? Perhaps the authors can start with such a setting and then expand on how a new formulation can help.

Questions for authors

How does the proposed KD-tree tokenization pipeline fundamentally differ in capability or design from existing hierarchical spatial tokenizers (e.g., quadtree/octree tokenization or AMR-Transformers)?

8 comments
AuthorsOpenReview ↗

Response to Reviewer TjV7

We thank the reviewer for the careful reading and for pointing us to AMR-Transformer. Below we address the reviewer's 3 key concerns highlighted in the review: (1) what is new, (2) how IBO's partition differs from quadtree/octree/AMR tokenizers, and (3) which PDE structure motivates the design.

1. Lack of Novelty: The core contribution of the paper is essentially a static data preprocessing step. Using a KD-tree to partition spatial coordinates based on an importance measure is a classical algorithmic technique.

We agree that the KD-tree is a classical algorithm and do not claim the sole use of a KD-tree algorithm for spatial partitioning as our contribution. Rather, in a spatially localized operator (local encoding → global processing → local decoding), the latent points require a choice of spatial allocation, similar to the concept of a mesh in traditional numerical methods. Choosing them is part of the model, just as the mesh is part of a numerical implementation. In our work, the key contribution is to better understand what sort of allocation principle (analogous to numerical discretization) given a fixed budget, minimizes the error of such neural operator architectures.

Though mentioned in the manuscript, we believe it worth highlighting again that localized models (e.g. GINO, GAOT) have trailed globally-resolved models (AB-UPT, Transolver-3) by about an order of magnitude at industrial scale. With architecture, budget and training fixed, replacing a uniform grid by the importance-balanced layout lowers errors substantially. With this layout, a localized model now can approach or even better the accuracy of the best-performing globally-resolved models at ~9× fewer inference FLOPs on DrivAerML and 6–40× fewer on UrbanWind. We believe this insight to be crucial for better design of such spatially localized models.

This is encapsulated by the accompanying theoretical analysis: The encoder and decoder resolution bounds are jointly minimized by the latent density λ⋆=Lρ\lambda^\star = L\rho (Prop. 4.2, App. B.2). Based on this insight, we propose the use of a classical algorithm, KD-tree partitioning, even as we show that a uniform layout is suboptimal by the factor in Eq. 10, while an approximately correct prior degrades performance by O(η2)O(\eta^2) (Eq. 11). The core contribution here is the insight, which we believe crucial to performance of spatially localized operators, and not the static preprocessing step via an importance-weighted KD-tree per se, which is just a very effective algorithmic implementation for achieving this outcome.

In fact, we add further experiments below for IBO, where we show that errors can also be substantially reduced by the use of a mesh-following sampling instead of the use of a uniform spatial layout (or uniform grid), as opposed to a KD-tree. This also presupposes that the mesh was generated with some domain knowledge, similar to what was discussed in the manuscript (and providing further evidence for the above analysis for Eq. 10 and Eq. 11). Nonetheless, at the same density, errors from the use of a mesh-following sampling still remains 9–15% above the KD-tree, precisely because KD-tree provides more effective implementation for spatial partitioning of points (coordinates).

ModelLayoutpsp_s∣τ∣\lvert\tau\rvertCptC_{p_t}∣u∣\lvert u\rvert
IBO (local read-out)Space-filling (uniform grid)8.8610.989.026.58
Mesh-following (random mesh cells, 3 seeds)4.426.876.244.49
Importance-balanced KD-tree (3 seeds)3.866.115.684.13
AB-UPT (our reproduction, global read-out)Space-filling (farthest point sampling)3.826.116.224.51
Mesh-following (random mesh cells)3.776.025.754.24

In addition, we reproduce AB-UPT and conduct a similar experiment, showing that the change from a uniform grid (space-filling set of points) to a mesh-following layout lowers AB-UPT's errors by only 1–8%, but IBO's by 31–50%. When every query attends to all anchors (as in the case of AB-UPT), resolution is pooled globally. When each query reads only its kk nearest tokens (i.e. in spatially localized models like IBO), the layout dictates the local resolution. This further highlights why this is an important design principle for localized neural operators but globally-resolved models like AB-UPT never had to vigorously grapple with the choice of anchor points.

We will further incorporate the layout experiments mentioned here in our revised manuscript (to be uploaded once all experiments have been thoroughly checked).

AuthorsOpenReview ↗

Response to Reviewer TjV7

2. How does the proposed KD-tree tokenization pipeline fundamentally differ in capability or design from existing hierarchical spatial tokenizers (e.g., quadtree/octree tokenization or AMR-Transformers)?

On a very high level, we do agree that these are all potential spatial partitioning algorithms. However, there are still key nuances to highlight which make the choice of KD-tree more appropriate in our work relative to other previously published work, specifically the difference in what the tree splits and what drives the split. Quadtree/octree tokenizers subdivide space into fixed sub-cells by a threshold rule, with AMR-Transformer more precisely choosing to refine based on where indicators computed from the input velocity field are large. More specific to AMR-Transformer, the indicator needs the flow as input, which is not possible for steady-state problems like the benchmark problems in this work. The number of tokens follows from the chosen thresholds and indicators. On the other hand, IBO uses a KD-tree tokenization pipeline to split the set of mesh points at the median of an importance mass available. This yields exactly LL leaves of equal mass on a domain of any shape, with no empty cells between bodies, and this equal-mass partition realizes the optimal density λ⋆=Lρ\lambda^\star = L\rho of Prop. 4.2 at leaf scale (Eq. 12).

Quadtree/octree tokenizers, AMR-TransformerIBO
Refinement signal(AMR-Transformer) Indicator computed from the current velocity field (available during autoregressive rollout)Prior available before solving: mesh density or geometry. A steady geometry→field surrogate has no solution at input.
Split ruleFixed geometric subdivision + thresholdsMedian of importance mass
Token countData-dependent (thresholds)Exactly LL; every leaf carries mass 1/L1/L
Empty cellsMidpoint cells in the non-convex region between bodies may contain no pointsNone: splits act on the points themselves
Target densityHeuristicOptimal λ⋆=Lρ\lambda^\star = L\rho (Prop. 4.2), realized at leaf scale (Eq. 12)

In particular, we note that the fixed LL also matters in practice. Cost and memory are known in advance for every case, which allows batched training on 10810^8-point cases and on urban scenes with 1 to 124 buildings.

We thank the reviewer for this suggestion, and will add a Related Work paragraph on hierarchical tokenizers and domain decomposition methods (e.g. quadtrees/octrees, AMR-Transformer, PGD-NO, SMART), in the revised manuscript, including a concise version of the comparison above.

AuthorsOpenReview ↗

Response to Reviewer TjV7

3. The method relies heavily on static, pre-computed importance measures and the current framing largely reads like a new data preprocessing step. Despite the theoretical grounding, this does not feel like the authors are proposing a new way of modeling the problem. What I mean is, perhaps the authors can refer to AMR-Transformer paper to provide an intuition of how their proposed approach changes how the problem is modeled compared to existing approaches that simply sample points. Are there some PDE structures that are fundamentally not captured with local settings? Perhaps the authors can start with such a setting and then expand on how a new formulation can help.

We thank the reviewer for this question. This helps provide useful advice for sharpening additional contributions of the paper as well.

We know that the real-world physics to be modelled usually has both a global and a local component, especially for fluid flow. In particular, the pressure satisfies an elliptic Poisson equation where the pressure at any point depends on the whole domain and on every body. At the same time, the sharpest gradients, and much of the source of this coupling, originate from boundary layers, shear layers and wakes that are orders of magnitude thinner than the domain. Multi-body flows are the clearest instance of how this is important, although we note that many current benchmarks, including for steady incompressible fluid flow, still focus on single body problems (e.g. DrivAerML). The bodies interact through wakes, blockage and the pressure field, and the flow behavior changes with spacing, from isolated bodies to interfering wakes. The learnt neural operator therefore needs to be capable of modelling both the globally coupled behavior while resolving the important locally sharp features and gradients. An improperly designed local model may not capture the global effects from this elliptic coupling. while a global model that utilizes uniform resolution may spend most of its token budget where the field is smooth, thereby necessitating larger number of points to be retained and the corresponding pointwise global computation on the large set of points to be expensive (22–25 PFLOP per DrivAerML case for AB-UPT and Transolver-3, vs. 2.6 for IBO).

IBO attempts to resolve this conundrum by accounting for both needs. Global attention among LL tokens helps with the global coupling, and the importance-balanced layout places local resolution where the key features are. This is particularly crucial when the regions that decide the interaction are thin and lie next to each body. This is also corroborated by our experiments where we observe that the latent layout matters most where this multibody physics is particularly significant: a 9× volume-error reduction over the uniform grid on TandemFoilSet Cruise Random, with Transolver-3, despite being a state-of-the-art globally-resolved model, remaining an order of magnitude behind on the volume fields of four TandemFoilSet subsets.

Following the reviewer's suggestion, and AMR-Transformer's exposition, we will further motivate the Introduction with this "globally coupled, locally important" structure and emphasize its importance for multi-body flows, before introducing the need for IBO.

Comments on the paper

1 comment

Thank you for your very constructive feedback, and agreeing to raise the score by 1. We will incorporate the discussion and results , and better refine the contribution as suggested in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.