acceptodds
Under review as a conference paper at ICLR 2027

AMS-GNO: An Aggregation-based MultiScale Graph Neural Operator for Large Scale PDEs

Abstract

Neural operators provide efficient surrogates for repeated numerical simulation by learning mappings from PDE parameters, geometries, and boundary conditions to solution fields. Scaling neural operators to large unstructured meshes and point clouds, however, remains challenging: purely local operators require many hops to capture long-range dependencies, whereas global attention or dense soft-assignment mechanisms introduce non-negligible computational and memory overhead by coupling every node to every latent token.Motivated by the inherently local structure of PDEs, we propose AMS-GNO, an Aggregation-based Multiscale Graph Neural Operator that performs long-range interaction on a coarsened graph while preserving sparse, topology-aware communication across scales. AMS-GNO first partitions the original mesh or point cloud into balanced local aggregates using geometric and topological priors, preserving physical connectivity while controlling aggregation size. Local smoothing then expands these hard assignments into a sparse overlapping cross-scale graph, allowing nodes near aggregate boundaries to interact with multiple neighboring coarse nodes. Restricted to this sparse support, attention-based operators separately learn downsampling and upsampling, adaptively determining which fine-scale information should be retained at coarse scales and how coarse representations should be reconstructed. The resulting architecture integrates fine-scale local processing, overlapping cross-scale transfer, and coarse-scale long-range interaction within a unified multiscale framework.Across multiple PDE benchmarks involving complex geometries, AMS-GNO achieves competitive or superior performance relative to state-of-the-art baselines while retaining favorable computational efficiency. Scalability experiments further show approximately linear growth in training time and memory consumption with the number of nodes, and demonstrate applicability to industrial-scale datasets containing millions of nodes. These results suggest that AMS-GNO provides an effective route toward scalable neural operators for high-resolution simulation on complex unstructured domains.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operators, Graph Neural Network, Partial Differential Equations, Multiscale Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Hyz3OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces AMS-GNO, a method for solving PDEs on large, unstructured grids. The operator keeps local structure at fine scales, captures long-range dependencies at coarse scales, and uses sparse attention across scales. This approach lowers the computational cost of graph networks, avoids information loss, and balances prediction accuracy with efficiency.

Critical strengths
  1. This paper introduces a multi-scale graph construction method that uses Balanced-Lloyd clustering and Jacobi smoothing. It also applies independently parameterized bidirectional cross-scale attention to help with feature aggregation and reconstruction. At the coarsest scale, Performer linear attention is used to lower the global interaction overhead on large grids. The technical motivations for this design are clear, and ablation experiments show that cross-scale attention and graph aggregation strategies improve prediction accuracy.
Critical weaknesses
  1. The paper does not provide enough direct comparisons with existing multi-scale graph neural networks, which makes it hard to show the method’s advantages clearly. Most of the comparisons are with operator-based methods like MeshGraphNet, GINO, Geo-FNO, and Transolver. GAOT is the only multi-scale-related method included, but there are no direct quantitative comparisons with other multi-scale graph networks such as MPG [1], BSMS-GNN [2], and MultiScale MeshGraphNets [3]. As a result, the current experiments do not clearly show that AMS-GNO’s performance improvement comes from its unique multi-scale graph construction and cross-scale transfer mechanism.
  2. The paper still needs to better show the independent innovation of the method. The design mainly combines and modifies existing technologies. Encoder–Processor–Decoder, V-cycle multi-scale structure, graph coarsening, attention aggregation, and linear global attention all have established research backgrounds [4-8]. The main difference in this work is the integration of balanced aggregation, sparse overlapping connections, bidirectional learnable cross-scale attention, and coarse-scale global interaction. However, the current ablation experiments mostly test the effects of cross-scale operators and aggregation strategies, which does not fully separate the contributions of each part. They also do not clearly show the unique advantages of this combination compared to other multi-scale graph architectures.

References:

[1]Wu, Q., Liu, P., Ren, X., & Ni, D. (2026). “MPG: An Efficient Multi-scale Point-Based GNN for Non-uniform Meshes.” Machine Learning and Knowledge Discovery in Databases.

[2]Cao, Y., Chai, M., Li, M., & Jiang, C. (2023). “Efficient Learning of Mesh-Based Physical Simulation with Bi-Stride Multi-Scale Graph Neural Network.” Proceedings of the 40th International Conference on Machine Learning (ICML 2023).

[3]Fortunato, M., Pfaff, T., Wirnsberger, P., Pritzel, A., & Battaglia, P. (2022). “MultiScale MeshGraphNets.” arXiv:2210.00612.

[4]Pfaff, T., Fortunato, M., Sánchez-González, A., & Battaglia, P. W. (2021). “Learning Mesh-Based Simulation with Graph Networks.” International Conference on Learning Representations (ICLR 2021).

[5] Vaněk, P., Mandel, J., & Brezina, M. (1996). “Algebraic Multigrid by Smoothed Aggregation for Second and Fourth Order Elliptic Problems.” Computing.

[6] Zaman, T., Nytko, N., Taghibakhshi, A., MacLachlan, S., Olson, L., & West, M. (2024). “Generalizing Lloyd’s Algorithm for Graph Clustering.” SIAM Journal on Scientific Computing.

[7] Wen, S., Kumbhat, A., Lingsch, L., Mousavi, S., Zhao, Y., Chandrashekar, P., & Mishra, S. (2025). “Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domains.” Advances in Neural Information Processing Systems (NeurIPS 2025).

[8] Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., et al. (2021). “Rethinking Attention with Performers.” International Conference on Learning Representations (ICLR 2021).

Questions for authors
  1. The paper lacks a direct comparison with existing multi-scale graph neural networks like MPG, BSMS-GNN, and MultiScale MeshGraphNets. Could you provide a fair quantitative comparison of these methods on the same dataset to show the real advantages of AMS-GNO over existing approaches?
  2. AMS-GNO uses Balanced-Lloyd aggregation, sparse overlapping connections from Jacobi smoothing, bidirectional learnable cross-scale attention, and coarse-scale Performer global attention. The current ablation experiments do not clearly show the separate effects of each part. Could you add more detailed ablation experiments to explain how each component affects prediction accuracy, long-distance dependency modeling, and computational efficiency? This would help show why each design choice is necessary.
  3. The paper’s use of multi-scale hierarchy, attention aggregation, and coarse-scale global interaction builds on existing research. It is not clear what the main innovation of AMS-GNO is compared to other methods. Could you explain more clearly how AMS-GNO differs from GAOT and other multi-scale graph networks in terms of architecture and information transmission? Also, please show through experiments whether the performance gains come from your specific design, rather than just combining existing modules or changing model settings.
4 comments

Thank you for your comments. We will address your concerns point by point below.

1. [W1] there are no direct quantitative comparisons with other multi-scale graph networks such as MPG [1], BSMS-GNN [2], and MultiScale MeshGraphNets [3]. [Q1] Could you provide a fair quantitative comparison of these methods on the same dataset to show the real advantages of AMS-GNO over existing approaches?

The experimental results show that AMS-GNO achieves higher predictive accuracy than MPG, BSMS-GNN, and MS-MGN across all benchmarks in our comparison. Therefore, the improvement in predictive accuracy cannot be attributed solely to multiscale processing itself. We have included BSMS-GNN, MS-MGN, RIGNO, and MPG in the experiment. The results are summarized below:

ModelShapeNet-CarMotor SourceMotor TargetNACA0012Poisson-C-SinesDeepJEBAhmed BodyAirfRANS
AMS-GNO0.05330.02450.00870.01400.02250.25290.05640.0109
BSMS-GNNOOT0.05360.02850.01810.2185OOT0.1403OOT
MS-MGN0.05450.05050.02400.02230.44430.30410.13240.2573
MPG0.08480.08430.04830.02410.62860.42170.25240.2330

OOT denotes runs stopped due to the time budget.

2. [W2] The paper still needs to better show the independent innovation of the method. [Q3] It is not clear what the main innovation of AMS-GNO is compared to other methods.

We have clarified our contributions in the general comment. Our main contributions are as follows:

  • Partial differential equations are inherently local: interactions between physically nearby nodes are more important than those between distant nodes. We therefore introduce a sparse aggregate partitioning strategy that preserves topology and respects spatial locality.

  • We argue that effective information compression and reconstruction are critical to multiscale neural networks. We therefore use Transformers across levels to learn physical similarities from node features and determine the corresponding compression and reconstruction weights.

  • We attribute the limited cross-resolution stability of graph neural networks to the strong dependence of mesh-based message passing on specific edge connections. We therefore omit mesh-based message passing at the finest level and instead decode directly from coarser levels.

  • By sampling aggregates to indirectly sample nodes, we enable mesh-based graph neural networks to process fewer nodes per input while achieving predictive accuracy comparable to full-graph processing. We further introduce streaming computation to scale the model to point sets containing millions of points.

  • We evaluate AMS-GNO on multiple PDE benchmarks, where it achieves highly competitive performance. We further demonstrate strong results in cross-resolution generalization and computational scalability.

We note that the contributions above are supported by extensive comparative experiments and ablation studies.

3. [Q3] Could you explain more clearly how AMS-GNO differs from GAOT and other multi-scale graph networks in terms of architecture and information transmission?

We here provide a detailed comparison of GAOT and AMS-GNO. The two methods differ in graph construction, architecture, and internal modules:

ii Architecturally, GAOT maps inputs from the physical domain to a latent representation through neighborhood-based mappings, whereas AMS-GNO adopts an explicit U-Net-style multilevel hierarchy.

iiii For graph construction, GAOT constructs connections based on spatial distances, whereas AMS-GNO builds its hierarchy from the connectivity of the original mesh.

iiiiii For information compression and reconstruction, GAOT incorporates attention-weighted kernel aggregation, but its attention weights depend only on node coordinates. In contrast, AMS-GNO incorporates both node features and edge features into the computation of physical similarity, enabling more effective information compression and reconstruction.

We here provide a detailed comparison of BSMS-GNN, MS-MGN, MPG, and AMS-GNO. AMS-GNO differs from these approaches in three key aspects.

(i)These methods all perform mesh-based message passing at the finest level, making their predictions overly sensitive to finest level graph connectivity and limiting their ability to generalize across resolutions. AMS-GNO omits mesh-based message passing at the finest level, enabling cross-resolution generalization.

iiii Regarding graph construction, MS-MGN uses COMSOL to regenerate meshes at coarse levels and constructs interlevel connections based on mesh elements. MPG applies different sampling rates in sparse and dense regions, then reconstructs connectivity through constrained Delaunay triangulation. BSMS-GNN selects breadth-first search (BFS) frontiers at each level and constructs connections between coarse and fine levels based on topology. This approach cannot control the number of edges at coarse levels and may instead increase it, resulting in substantial computational overhead. In contrast, AMS-GNO constructs its graph hierarchy solely from the input graph, requiring no additional mesh generation. More importantly, its aggregation scheme explicitly controls the number of fine-level nodes represented by each coarse node. It also naturally extends to overlapping aggregates, enabling overlapping interlevel connections. Together, these properties improve predictive accuracy.

iiiiii BSMS-GNN transfers information across levels using fixed degree-normalized transfer weights, MPG combines distance-based softmax weights with learnable parameters for neighborhood aggregation and uses fixed inverse-distance k-NN interpolation in the decoder, while MS-MGN uses conventional message passing. In contrast, AMS-GNO employs Transformers to assess node contributions based on learned physical similarities.

4. [W2] However, the current ablation experiments mostly test the effects of cross-scale operators and aggregation strategies, which does not fully separate the contributions of each part. [Q2] The current ablation experiments do not clearly show the separate effects of each part.

We have added these experiments on Poisson-C-Sines. The results are shown below:

AblationConfigurationRelative L2
OverlapNon-overlap0.6610
OverlapOverlap0.0225
Pool / UnpoolTransformer / Transformer0.0225
Pool / UnpoolMLP / Transformer0.0683
Pool / UnpoolMLP / MLP0.1781
Pool / UnpoolTransformer / MLP0.3165
Pool / UnpoolFixed / Fixed0.1706
Coarsest ProcessorGlobal Attention0.0225
Coarsest ProcessorLocal Message Passing0.0258

Regarding the independent contribution of Balanced-Lloyd, the original manuscript already included an ablation study comparing different graph construction methods.

Aggregation method (AirfRANS)Volume errorSurface error
Balanced-Lloyd0.0081560.002725
Lloyd0.0095530.003758
Random0.0124300.003344

5. [Q2] Could you add more detailed ablation experiments to explain how each component affects prediction accuracy, long-distance dependency modeling, and computational efficiency?

Our ablation studies have shown how each component affects predictive accuracy. Each component contributes to the final predictive accuracy. Specifically, overlapping aggregation connects each node to more relevant nodes during cross-level message passing. Together with Transformer-based cross-level transfer, this improves information compression and reconstruction between levels. Global attention at the coarsest level enables global information exchange, allowing AMS-GNO to better combine local and global information and thereby improve predictive accuracy. Balanced-Lloyd graph construction provides a stronger physical basis for information transfer across levels. Its approximately equal aggregate sizes also help the network attend to information from nodes in sparsely populated regions.

Regarding how each component affects long-distance dependency modeling, the multiscale graph itself enables long-range information exchange, while global attention at the coarsest level provides direct access to global information. Our ablation study of the latter also shows that global information improves the predictive accuracy of AMS-GNO.

Finally, In the ablation studies of the main components, we kept the total model parameter counts approximately equal. Thus, individual components have little impact on overall computational efficiency. The main efficiency improvement comes from omitting GNO at the finest level, since applying GNO at this level requires substantial computation, especially when the number of points per sample reaches hundreds of thousands or millions.

Thank you for your time. We would be happy to discuss any further questions.

Reviewer_Hyz3OpenReview ↗

Official Comment by Reviewer Hyz3

Thank you for the detailed rebuttal and additional experiments. The new comparisons with MPG, BSMS-GNN, and MS-MGN directly address my main concern and provide stronger evidence for AMS-GNO's effectiveness. The additional ablations also better clarify the contributions of individual components. These improvements substantially strengthen the paper, so I raise my rating from 1 to 2. However, concerns remain regarding independent methodological novelty, the fairness of baseline comparisons, and the limited direct evidence for computational efficiency and long-range modeling benefits. Therefore, I do not consider all concerns fully resolved.

Reviewer_9fCGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes AMS-GNO, a multiscale graph neural operator designed for efficient PDE surrogate modeling on large unstructured meshes and point clouds. The method constructs topology-aware graph hierarchies through balanced aggregation and sparse overlapping connections, and employs learnable attention-based pooling and unpooling to transfer information across scales. Global interactions are performed on compressed coarse representations, while memory-efficient execution strategies support training on million-scale meshes. Experiments across diverse PDE benchmarks demonstrate competitive or improved prediction accuracy, cross-resolution applicability, and approximately linear scaling in computational time and memory. The results suggest that combining sparse multiscale graph structures with adaptive cross-scale communication provides an effective approach to large-scale neural PDE simulation.

Critical strengths
  1. AMS-GNO achieves the lowest reported errors in seven of nine benchmark settings, covering fluid dynamics, structural mechanics, and complex 3D geometries. Particularly notable are its improvements on AirfRANS and Ahmed Body over the strongest reported baselines. These results provide substantial empirical evidence for the effectiveness and broad applicability of the proposed multiscale architecture, making predictive performance a key argument in favor of acceptance.
  2. By combining sparse topology-aware aggregation, adaptive cross-scale attention, and memory-efficient execution, AMS-GNO successfully scales to approximately 8.8 million nodes on a single GPU. The experiments demonstrate nearly linear growth in training time and GPU memory consumption as mesh resolution increases. This is a significant practical strength because scalability is a central challenge in applying neural operators to industrial-scale PDE simulations, and the paper provides concrete empirical evidence beyond theoretical complexity analysis.
Critical weaknesses
  1. This paper suffers from limited methodological novelty and insufficient comparison with closely related multiscale approaches. The framework primarily integrates established techniques, including Balanced-Lloyd clustering, smoothed aggregation, attention-based cross-scale transfer, and coarse-level global attention. Although this combination is reasonable, the paper lacks direct comparisons with closely related multiscale methods, particularly RIGNO and BSMS-GNN. Consequently, it remains unclear whether the reported improvements arise from the proposed architectural design or from the general benefits of multiscale processing. This is a central concern because the paper's incremental contribution over existing methods is not convincingly established.
  2. There is insufficient evidence to support the claimed architectural and scalability advantages. The ablation studies do not isolate several key design choices, particularly sparse overlapping aggregation, learned versus fixed cross-scale transfer, and coarse-level global attention. Moreover, the million-scale DrivAerML experiments report training time and memory consumption without corresponding prediction accuracy, competitive efficiency baselines, or preprocessing costs. These omissions make it difficult to determine whether the proposed components are necessary and whether AMS-GNO offers a meaningful accuracy–efficiency advantage over existing scalable neural operators. This substantially weakens the empirical support for the paper's central contributions.
Questions for authors
  1. How does AMS-GNO compare with existing multiscale approaches such as RIGNO, BSMS-GNN, and MultiScale MeshGraphNets under comparable experimental settings? What specific advantages does the proposed architecture provide beyond those of conventional multiscale graph processing?
  2. Can the authors provide additional ablations isolating the effects of Jacobi-smoothed overlapping aggregation, learned versus fixed cross-scale transfer, and coarse-level global attention? These comparisons are important for establishing which components are responsible for the reported performance gains.
  3. Can the authors report prediction accuracy on the full-resolution DrivAerML benchmark, together with preprocessing costs and comparisons of runtime and memory against competitive scalable neural operators? Without these results, it is difficult to assess whether the demonstrated scalability translates into a meaningful practical accuracy–efficiency advantage.
6 comments

Thank you for your comments. We address your concerns point by point below.

1. [W1] This paper suffers from limited methodological novelty

We have clarified our contributions in the general comment. Our main contributions are as follows:

  • Partial differential equations are inherently local: interactions between physically nearby nodes are more important than those between distant nodes. We therefore introduce a sparse aggregate partitioning strategy that preserves topology and respects spatial locality.

  • We argue that effective information compression and reconstruction are critical to multiscale neural networks. We therefore use Transformers across levels to learn physical similarities from node features and determine the corresponding compression and reconstruction weights.

  • We attribute the limited cross-resolution stability of graph neural networks to the strong dependence of mesh-based message passing on specific edge connections. We therefore omit mesh-based message passing at the finest level and instead decode directly from coarser levels.

  • By sampling aggregates to indirectly sample nodes, we enable mesh-based graph neural networks to process fewer nodes per input while achieving predictive accuracy comparable to full-graph processing. We further introduce streaming computation to scale the model to point sets containing millions of points.

  • We evaluate AMS-GNO on multiple PDE benchmarks, where it achieves highly competitive performance. We further demonstrate strong results in cross-resolution generalization and computational scalability.

We note that the contributions above are supported by extensive comparative experiments and ablation studies.

2. [Q1] What specific advantages does the proposed architecture provide beyond those of conventional multiscale graph processing?

AMS-GNO differs from these approaches (BSMS-GNN,MS-MGN)in three key aspects. (i) Regarding cross-resolution generalization, these methods all perform mesh-based message passing at the finest level, making their predictions overly dependent on the connectivity of the original mesh and limiting their ability to generalize across resolutions. AMS-GNO omits mesh-based message passing at the finest level, enabling cross-resolution generalization.

iiii Regarding graph construction, MS-MGN uses COMSOL to regenerate meshes at coarse levels and constructs interlevel connections based on mesh elements. BSMS-GNN selects breadth-first search (BFS) frontiers at each level and constructs connections between coarse and fine levels based on topology. This approach cannot control the number of edges at coarse levels and may instead increase it, resulting in substantial computational overhead. In contrast, AMS-GNO constructs its graph hierarchy solely from the input graph, requiring no additional mesh generation. More importantly, its aggregation scheme explicitly controls the number of fine-level nodes represented by each coarse node. It also naturally extends to overlapping aggregates, enabling overlapping interlevel connections. Together, these properties improve predictive accuracy.

iiiiii BSMS-GNN transfers information across levels using fixed degree-normalized transfer weights, while MS-MGN uses conventional message passing. In contrast, AMS-GNO employs Transformers to assess node contributions based on learned physical similarities.

3. [W1] Although this combination is reasonable, the paper lacks direct comparisons with closely related multiscale methods, particularly RIGNO and BSMS-GNN. [Q1] How does AMS-GNO compare with existing multiscale approaches such as RIGNO, BSMS-GNN, and MultiScale MeshGraphNets under comparable experimental settings?

We have included BSMS-GNN, MS-MGN, and RIGNO in the experiment. The results are summarized below:

ModelShapeNet-CarMotor SourceMotor TargetNACA0012Poisson-C-SinesDeepJEBAhmed BodyAirfRANS
AMS-GNO0.05330.02450.00870.01400.02250.25290.05640.0109
BSMS-GNNOOT0.05360.02850.01810.2185OOT0.1403OOT
MS-MGN0.05450.05050.02400.02230.44430.30410.13240.2573
RIGNO0.05510.09100.07150.04830.3248OOTOOTOOT

OOT denotes runs stopped due to the time budget.

To address your concern in [W1], “Consequently, it remains unclear whether the reported improvements arise from the proposed architectural design or from the general benefits of multiscale processing,” AMS-GNO achieves higher predictive accuracy than multiscale graph neural networks such as BSMS-GNN, MS-MGN, and RIGNO on every dataset with reported results. These comparisons show that the gains go beyond the benefits of multiscale processing alone.

4. [Q2] Can the authors provide additional ablations isolating the effects of Jacobi-smoothed overlapping aggregation, learned versus fixed cross-scale transfer, and coarse-level global attention? [W2] The ablation studies do not isolate several key design choices, particularly sparse overlapping aggregation, learned versus fixed cross-scale transfer, and coarse-level global attention.

We have added these experiments on Poisson-C-Sines. The results are shown below:

AblationConfigurationRelative L2
OverlapNon-overlap0.6610
OverlapOverlap0.0225
Pool / UnpoolTransformer / Transformer0.0225
Pool / UnpoolMLP / Transformer0.0683
Pool / UnpoolMLP / MLP0.1781
Pool / UnpoolTransformer / MLP0.3165
Pool / UnpoolFixed / Fixed0.1706
Coarsest ProcessorGlobal Attention0.0225
Coarsest ProcessorLocal Message Passing0.0258

5. [W2] Moreover, the million-scale DrivAerML experiments report training time and memory consumption without corresponding prediction accuracy, competitive efficiency baselines, or preprocessing costs. [Q3] Can the authors report prediction accuracy on the full-resolution DrivAerML benchmark, together with preprocessing costs and comparisons of runtime and memory against competitive scalable neural operators?

We include these comparisons as you requested. The results are summarized below:

Model (DrivAerML)Pressure relative L2Wall Shear relative L2Time (s/epoch)Training memory (GiB)Parameters (M)
AMS-GNO0.07610.125676.487.225.487
UPT0.13320.210220.951.4510.856
GAOT0.27680.5130305.004.7711.246

We also report the preprocessing costs. Full preprocessing takes an average of 9.72 seconds per sample.

Overall, AMS-GNO has the lowest error with about half the parameters of the other two models.

Thank you for your time. We would be happy to discuss any further questions.

Dear Reviewer 9fCG, Thank you once again for your valuable comments on our submission. As the discussion phase is approaching its end, we would like to kindly confirm whether we have sufficiently addressed all of your concerns. Should there be any remaining questions requiring further clarification, please do not hesitate to let us know. If you are satisfied with our responses, we would greatly appreciate your consideration in adjusting the evaluation scores accordingly.

We sincerely look forward to your feedback.

Reviewer_CAA1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AMS-GNO is a multiscale graph neural operator that compresses a mesh into three levels, runs global attention at the coarsest level, and then rebuilds the fine solution.

Groups are formed by "mesh distance" with roughly equal sizes, and a "smoothing step" adds small overlap at group borders. Separate attention modules learn the downward and upward transfers.

Critical strengths
  • Chunked execution keeps every attention neighborhood complete

  • Broad benchmarks

Critical weaknesses
  • Is the overlap never ablated in any experiment? For example, switching it off or replacing it with others?

  • How can we know "when needed" in "Same-scale message passing at the finest level is omitted by default; when needed, it is applied separately on a local radius graph." (Lines 259–261) Table 6 turns it on for NACA0012 and ShapeNet-Car (Lines 1090 and 1093), why these two?

Questions for authors
  • Can you provide experiments that compare mesh edges with 8-NN on the same dataset?

  • What is the competitive baseline performance like Transolver++?

3 comments

Thank you very much for your comments and suggestions. We have added the requested ablation studies and the comparison with Transolver++, and provide further clarification on the finest-level GNO configuration below. We will address your comments point by point.

1. [W1] Is the overlap never ablated in any experiment?

We have added an ablation study on Poisson-C-Sines comparing configurations with and without overlap. The results are shown below:

ConfigurationRelative L2
Non-overlap0.6610
Overlap0.0225

2. [W2] How can we know “when needed” in “Same-scale message passing at the finest level is omitted by default; when needed, it is applied separately on a local radius graph.”

We will address this question from two perspectives.

First, does enabling the finest-level GNO improve predictive accuracy? We conducted experiments with and without the finest-level GNO on NACA0012, ShapeNet-Car, and Poisson-C-Sines. As shown in the table below, enabling GNO slightly improves predictive accuracy on NACA0012 and ShapeNet-Car but reduces it on Poisson-C-Sines. These results show that enabling GNO does not consistently improve predictive accuracy.

DatasetGNO OnGNO Off
NACA00120.01380.0140
ShapeNetCar0.05100.0533
Poisson-C-Sines0.14720.0225

Second, applying GNO at the finest level requires substantial computation and reduces computational efficiency, especially when the number of points per sample reaches hundreds of thousands or millions. This substantially increases training time and makes the model impractical. Therefore, we disable the finest-level GNO by default. We will update the corresponding values in the main table in the revised paper. AMS-GNO’s rankings remain unchanged.

In the original submission we enabled it on these two datasets because it gave slightly lower error there. This was not a principled rule, so we now disable it on all datasets. We will also discuss this issue in the revised manuscript.

3. [Q1] Can you provide experiments that compare mesh edges with 8-NN on the same dataset?

We have added the requested comparison between 8-NN and mesh connectivity to the ablation studies. Specifically, we compare predictive accuracy on ShapeNet-Car using either 8-NN or native mesh edges to construct the finest-level graph. The results are shown below:

Finest-level base graphRelative L2
Mesh edges0.0533
8-NN0.0572

4. [Q2] What is the competitive baseline performance like Transolver++

We have included Transolver++ in the comparison. AMS-GNO achieves better predictive accuracy in all reported settings except NACA0012. The results are shown below:

ModelShapeNet-CarMotor SourceMotor TargetNACA0012Poisson-C-SinesDeepJEBAhmed BodyAirfRANSNASA-CRM
AMS-GNO0.05330.02450.00870.01400.02250.25290.05640.01090.0932
Transolver++0.05470.04500.01270.01230.54070.30400.06500.01470.0951

Thank you again for your time and feedback. We would be happy to discuss any remaining questions.

Dear Reviewer CAA1, Thank you once again for your valuable comments on our submission. As the discussion phase is approaching its end, we would like to kindly confirm whether we have sufficiently addressed all of your concerns. Should there be any remaining questions requiring further clarification, please do not hesitate to let us know. If you are satisfied with our responses, we would greatly appreciate your consideration in adjusting the evaluation scores accordingly.

We sincerely look forward to your feedback.

Dear Reviewer CAA1,

I hope this message finds you well. As the discussion period is nearing its end with less than one days remaining, I want to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you for your time and effort again in reviewing our paper.

Comments on the paper

2 comments

We thank all reviewers for their constructive comments. We would first like to clarify the high-level idea and contributions of our work before addressing the individual concerns.

Our work addresses the following challenges faced by graph neural networks for PDE surrogate modeling:

  1. Existing graph neural networks cannot preserve local topology while maintaining cross-resolution generalization.

  2. Existing multiscale graph neural networks lose information during cross-scale compression and reconstruction when balancing global and local information.

  3. Existing mesh-based multiscale graph neural networks cannot scale to industrial datasets containing millions of points in a single GPU.

To address these challenges, we propose AMS-GNO, a multiscale graph neural operator that preserves local topology, supports cross-resolution generalization, improves cross-scale information compression and reconstruction, and scales to industrial datasets containing millions of points. Our main contributions are as follows:

  • Partial differential equations are inherently local: interactions between physically nearby nodes are more important than those between distant nodes. We therefore introduce a sparse aggregate partitioning strategy that preserves topology and respects spatial locality.

  • We argue that effective information compression and reconstruction are critical to multiscale neural networks. We therefore use Transformers across levels to learn physical similarities from node features and determine the corresponding compression and reconstruction weights.

  • We attribute the limited cross-resolution stability of graph neural networks to the strong dependence of mesh-based message passing on specific edge connections. We therefore omit mesh-based message passing at the finest level and instead decode directly from coarser levels.

  • By sampling aggregates to indirectly sample nodes, we enable mesh-based graph neural networks to process fewer nodes per input while achieving predictive accuracy comparable to full-graph processing. We further introduce streaming computation to scale the model to point sets containing millions of points.

  • We evaluate AMS-GNO on multiple PDE benchmarks, where it achieves highly competitive performance. We further demonstrate strong results in cross-resolution generalization and computational scalability.

We note that the contributions above are supported by extensive comparative experiments and ablation studies.

Regarding baselines, we will compare AMS-GNO with the baselines mentioned by the reviewers, including Transolver++, RIGNO, MPG, BSMS-GNN, and MS-MGN, in the revised manuscript. The results are shown in Table 1.

Table 1. Comparison of AMS-GNO with additional baselines.

DatasetAMS-GNOTransolver++RIGNOMPGBSMS-GNNMS-MGN
ShapeNet-Car0.05330.05470.05510.0848OOT0.0545
Motor Source0.02450.04500.09100.08430.05360.0505
Motor Target0.00870.01270.07150.04830.02850.0240
NACA00120.01400.01230.04830.02410.01810.0223
Poisson-C-Sines0.02250.54070.32480.62860.21850.4443
DeepJEB0.25290.3040OOT0.4217OOT0.3041
Ahmed Body0.05640.0650OOT0.25240.14030.1324
AirfRANS0.01090.0147OOT0.2330OOT0.2573

OOT denotes Out Of Time budget. To be specific, we set a time limit of 250 seconds per epoch. BSMS-GNN and RIGNO require excessively long training times on some datasets in Table 2, making them impractical in these settings. The training times are reported in the table. Specifically, BSMS-GNN took 2.7-7.2 times as long per epoch as models such as AMS-GNO, while RIGNO took approximately 3.8–41.1 times as long per epoch as AMS-GNO on these datasets.

Table 2. Time Comparison Per-epoch time in seconds

ModelsShapeNet-CarDeepJEBAhmedBodyAirfRANS
AMS-GNO51.98204.3532.1158.60
BSMS-GNN375.42547.78-264.15
RIGNO-776.111318.841467.86

To explain this issue, we compared how the three models construct their graphs. In BSMS-GNN, connectivity augmentation increases the number of within-level edges at deeper levels, introducing substantial computational overhead. RIGNO does not impose an upper limit on the number of connections, resulting in a large number of edges in its encoder and decoder.

For the same DeepJEB sample, we compared the numbers of within-level edges in BSMS-GNN and AMS-GNO at corresponding levels in Table 3. We also report the edge counts in RIGNO’s encoder and decoder for this sample: they contain 1,683,612 and 2,491,227 cross-level edges, respectively.

Table 3. Within-level edge counts for BSMS-GNN and AMS-GNO on the same DeepJEB sample.

ModelsLevel 0Level 1Level 2Level 3Level 4Level 5
AMS-GNO30808814264525901---
BSMS-GNN30808834503247782855596438396015750

We will also report two important sets of additional experiments.

The first one is cross-resolution generalization. We conducted this experiment on the Deformed Darcy dataset, which provides data at both coarse and fine resolutions. We trained the models at the coarse resolution and then evaluated them at both resolutions, with the results reported in Table 4. For comparison, we conducted the same experiment with BSMS-GNN. The results demonstrate that our model supports cross-resolution generalization, whereas conventional mesh-based graph neural networks do not.

Table 4. Cross-Resolution Generalization

ModelCoarse →\to CoarseCoarse →\to FineError ratio
AMS-GNO0.0172650.0367492.13×2.13\times
BSMS-GNN0.0340100.36015810.59×10.59\times

And the other is comparing scalability. In these experiments, we compared AMS-GNO with UPT and GAOT, two baselines that scale to large datasets. Models such as UPT typically train on a randomly sampled subset of finest-level points and perform inference on the full input, enabling them to scale to large datasets. However, random sampling disrupts the native connectivity required by mesh-based methods.

AMS-GNO instead randomly samples nodes at the coarsest level as central aggregates and includes aggregates that overlap with them during downsampling and upsampling as boundary aggregates. Since Balanced-Lloyd ensures that each aggregate contains approximately the same number of fine-level nodes, this strategy approximates random sampling of fine-level nodes. Moreover, the sparsity of the downsampling and upsampling graphs in our hierarchy allows us to control the total number of sampled points. The results are shown in Table 5.

Table 5. Scalability

ModelPressureWall ShearTime(s/epoch)Memory(GiB)Parameters(M)
AMS-GNO0.07610.125676.487.225.487
UPT0.13320.210220.951.4510.856
GAOT0.27680.51303054.7711.246

The main revisions to the new manuscript are as follows:

  1. In the Introduction, we have reorganized the text to present our main contributions and insights.

  2. In Related Work, we have provided a detailed comparison of AMS-GNO with closely related baselines, including MPG, BSMS-GNN, MS-MGN, RIGNO, and GAOT, across multiple aspects.

  3. In Experiments, we have added the baselines requested by the reviewers to the main results table and included the suggested ablation studies to isolate the contributions of AMS-GNO’s architectural components.

We will address the reviewers’ specific questions and experimental concerns in the individual discussions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.