- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces AMS-GNO, a method for solving PDEs on large, unstructured grids. The operator keeps local structure at fine scales, captures long-range dependencies at coarse scales, and uses sparse attention across scales. This approach lowers the computational cost of graph networks, avoids information loss, and balances prediction accuracy with efficiency.
Critical strengths
- This paper introduces a multi-scale graph construction method that uses Balanced-Lloyd clustering and Jacobi smoothing. It also applies independently parameterized bidirectional cross-scale attention to help with feature aggregation and reconstruction. At the coarsest scale, Performer linear attention is used to lower the global interaction overhead on large grids. The technical motivations for this design are clear, and ablation experiments show that cross-scale attention and graph aggregation strategies improve prediction accuracy.
Critical weaknesses
- The paper does not provide enough direct comparisons with existing multi-scale graph neural networks, which makes it hard to show the method’s advantages clearly. Most of the comparisons are with operator-based methods like MeshGraphNet, GINO, Geo-FNO, and Transolver. GAOT is the only multi-scale-related method included, but there are no direct quantitative comparisons with other multi-scale graph networks such as MPG [1], BSMS-GNN [2], and MultiScale MeshGraphNets [3]. As a result, the current experiments do not clearly show that AMS-GNO’s performance improvement comes from its unique multi-scale graph construction and cross-scale transfer mechanism.
- The paper still needs to better show the independent innovation of the method. The design mainly combines and modifies existing technologies. Encoder–Processor–Decoder, V-cycle multi-scale structure, graph coarsening, attention aggregation, and linear global attention all have established research backgrounds [4-8]. The main difference in this work is the integration of balanced aggregation, sparse overlapping connections, bidirectional learnable cross-scale attention, and coarse-scale global interaction. However, the current ablation experiments mostly test the effects of cross-scale operators and aggregation strategies, which does not fully separate the contributions of each part. They also do not clearly show the unique advantages of this combination compared to other multi-scale graph architectures.
References:
[1]Wu, Q., Liu, P., Ren, X., & Ni, D. (2026). “MPG: An Efficient Multi-scale Point-Based GNN for Non-uniform Meshes.” Machine Learning and Knowledge Discovery in Databases.
[2]Cao, Y., Chai, M., Li, M., & Jiang, C. (2023). “Efficient Learning of Mesh-Based Physical Simulation with Bi-Stride Multi-Scale Graph Neural Network.” Proceedings of the 40th International Conference on Machine Learning (ICML 2023).
[3]Fortunato, M., Pfaff, T., Wirnsberger, P., Pritzel, A., & Battaglia, P. (2022). “MultiScale MeshGraphNets.” arXiv:2210.00612.
[4]Pfaff, T., Fortunato, M., Sánchez-González, A., & Battaglia, P. W. (2021). “Learning Mesh-Based Simulation with Graph Networks.” International Conference on Learning Representations (ICLR 2021).
[5] Vaněk, P., Mandel, J., & Brezina, M. (1996). “Algebraic Multigrid by Smoothed Aggregation for Second and Fourth Order Elliptic Problems.” Computing.
[6] Zaman, T., Nytko, N., Taghibakhshi, A., MacLachlan, S., Olson, L., & West, M. (2024). “Generalizing Lloyd’s Algorithm for Graph Clustering.” SIAM Journal on Scientific Computing.
[7] Wen, S., Kumbhat, A., Lingsch, L., Mousavi, S., Zhao, Y., Chandrashekar, P., & Mishra, S. (2025). “Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domains.” Advances in Neural Information Processing Systems (NeurIPS 2025).
[8] Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., et al. (2021). “Rethinking Attention with Performers.” International Conference on Learning Representations (ICLR 2021).
Questions for authors
- The paper lacks a direct comparison with existing multi-scale graph neural networks like MPG, BSMS-GNN, and MultiScale MeshGraphNets. Could you provide a fair quantitative comparison of these methods on the same dataset to show the real advantages of AMS-GNO over existing approaches?
- AMS-GNO uses Balanced-Lloyd aggregation, sparse overlapping connections from Jacobi smoothing, bidirectional learnable cross-scale attention, and coarse-scale Performer global attention. The current ablation experiments do not clearly show the separate effects of each part. Could you add more detailed ablation experiments to explain how each component affects prediction accuracy, long-distance dependency modeling, and computational efficiency? This would help show why each design choice is necessary.
- The paper’s use of multi-scale hierarchy, attention aggregation, and coarse-scale global interaction builds on existing research. It is not clear what the main innovation of AMS-GNO is compared to other methods. Could you explain more clearly how AMS-GNO differs from GAOT and other multi-scale graph networks in terms of architecture and information transmission? Also, please show through experiments whether the performance gains come from your specific design, rather than just combining existing modules or changing model settings.
4 comments
3. [Q3] Could you explain more clearly how AMS-GNO differs from GAOT and other multi-scale graph networks in terms of architecture and information transmission?
We here provide a detailed comparison of GAOT and AMS-GNO. The two methods differ in graph construction, architecture, and internal modules:
Architecturally, GAOT maps inputs from the physical domain to a latent representation through neighborhood-based mappings, whereas AMS-GNO adopts an explicit U-Net-style multilevel hierarchy.
For graph construction, GAOT constructs connections based on spatial distances, whereas AMS-GNO builds its hierarchy from the connectivity of the original mesh.
For information compression and reconstruction, GAOT incorporates attention-weighted kernel aggregation, but its attention weights depend only on node coordinates. In contrast, AMS-GNO incorporates both node features and edge features into the computation of physical similarity, enabling more effective information compression and reconstruction.
We here provide a detailed comparison of BSMS-GNN, MS-MGN, MPG, and AMS-GNO. AMS-GNO differs from these approaches in three key aspects.
(i)These methods all perform mesh-based message passing at the finest level, making their predictions overly sensitive to finest level graph connectivity and limiting their ability to generalize across resolutions. AMS-GNO omits mesh-based message passing at the finest level, enabling cross-resolution generalization.
Regarding graph construction, MS-MGN uses COMSOL to regenerate meshes at coarse levels and constructs interlevel connections based on mesh elements. MPG applies different sampling rates in sparse and dense regions, then reconstructs connectivity through constrained Delaunay triangulation. BSMS-GNN selects breadth-first search (BFS) frontiers at each level and constructs connections between coarse and fine levels based on topology. This approach cannot control the number of edges at coarse levels and may instead increase it, resulting in substantial computational overhead. In contrast, AMS-GNO constructs its graph hierarchy solely from the input graph, requiring no additional mesh generation. More importantly, its aggregation scheme explicitly controls the number of fine-level nodes represented by each coarse node. It also naturally extends to overlapping aggregates, enabling overlapping interlevel connections. Together, these properties improve predictive accuracy.
BSMS-GNN transfers information across levels using fixed degree-normalized transfer weights, MPG combines distance-based softmax weights with learnable parameters for neighborhood aggregation and uses fixed inverse-distance k-NN interpolation in the decoder, while MS-MGN uses conventional message passing. In contrast, AMS-GNO employs Transformers to assess node contributions based on learned physical similarities.
4. [W2] However, the current ablation experiments mostly test the effects of cross-scale operators and aggregation strategies, which does not fully separate the contributions of each part. [Q2] The current ablation experiments do not clearly show the separate effects of each part.
We have added these experiments on Poisson-C-Sines. The results are shown below:
| Ablation | Configuration | Relative L2 |
|---|---|---|
| Overlap | Non-overlap | 0.6610 |
| Overlap | Overlap | 0.0225 |
| Pool / Unpool | Transformer / Transformer | 0.0225 |
| Pool / Unpool | MLP / Transformer | 0.0683 |
| Pool / Unpool | MLP / MLP | 0.1781 |
| Pool / Unpool | Transformer / MLP | 0.3165 |
| Pool / Unpool | Fixed / Fixed | 0.1706 |
| Coarsest Processor | Global Attention | 0.0225 |
| Coarsest Processor | Local Message Passing | 0.0258 |
Regarding the independent contribution of Balanced-Lloyd, the original manuscript already included an ablation study comparing different graph construction methods.
| Aggregation method (AirfRANS) | Volume error | Surface error |
|---|---|---|
| Balanced-Lloyd | 0.008156 | 0.002725 |
| Lloyd | 0.009553 | 0.003758 |
| Random | 0.012430 | 0.003344 |
5. [Q2] Could you add more detailed ablation experiments to explain how each component affects prediction accuracy, long-distance dependency modeling, and computational efficiency?
Our ablation studies have shown how each component affects predictive accuracy. Each component contributes to the final predictive accuracy. Specifically, overlapping aggregation connects each node to more relevant nodes during cross-level message passing. Together with Transformer-based cross-level transfer, this improves information compression and reconstruction between levels. Global attention at the coarsest level enables global information exchange, allowing AMS-GNO to better combine local and global information and thereby improve predictive accuracy. Balanced-Lloyd graph construction provides a stronger physical basis for information transfer across levels. Its approximately equal aggregate sizes also help the network attend to information from nodes in sparsely populated regions.
Regarding how each component affects long-distance dependency modeling, the multiscale graph itself enables long-range information exchange, while global attention at the coarsest level provides direct access to global information. Our ablation study of the latter also shows that global information improves the predictive accuracy of AMS-GNO.
Finally, In the ablation studies of the main components, we kept the total model parameter counts approximately equal. Thus, individual components have little impact on overall computational efficiency. The main efficiency improvement comes from omitting GNO at the finest level, since applying GNO at this level requires substantial computation, especially when the number of points per sample reaches hundreds of thousands or millions.
Thank you for your time. We would be happy to discuss any further questions.
Official Comment by Reviewer Hyz3
Thank you for the detailed rebuttal and additional experiments. The new comparisons with MPG, BSMS-GNN, and MS-MGN directly address my main concern and provide stronger evidence for AMS-GNO's effectiveness. The additional ablations also better clarify the contributions of individual components. These improvements substantially strengthen the paper, so I raise my rating from 1 to 2. However, concerns remain regarding independent methodological novelty, the fairness of baseline comparisons, and the limited direct evidence for computational efficiency and long-range modeling benefits. Therefore, I do not consider all concerns fully resolved.
Thank you for your comments. We will address your concerns point by point below.
1. [W1] there are no direct quantitative comparisons with other multi-scale graph networks such as MPG [1], BSMS-GNN [2], and MultiScale MeshGraphNets [3]. [Q1] Could you provide a fair quantitative comparison of these methods on the same dataset to show the real advantages of AMS-GNO over existing approaches?
The experimental results show that AMS-GNO achieves higher predictive accuracy than MPG, BSMS-GNN, and MS-MGN across all benchmarks in our comparison. Therefore, the improvement in predictive accuracy cannot be attributed solely to multiscale processing itself. We have included BSMS-GNN, MS-MGN, RIGNO, and MPG in the experiment. The results are summarized below:
OOT denotes runs stopped due to the time budget.
2. [W2] The paper still needs to better show the independent innovation of the method. [Q3] It is not clear what the main innovation of AMS-GNO is compared to other methods.
We have clarified our contributions in the general comment. Our main contributions are as follows:
Partial differential equations are inherently local: interactions between physically nearby nodes are more important than those between distant nodes. We therefore introduce a sparse aggregate partitioning strategy that preserves topology and respects spatial locality.
We argue that effective information compression and reconstruction are critical to multiscale neural networks. We therefore use Transformers across levels to learn physical similarities from node features and determine the corresponding compression and reconstruction weights.
We attribute the limited cross-resolution stability of graph neural networks to the strong dependence of mesh-based message passing on specific edge connections. We therefore omit mesh-based message passing at the finest level and instead decode directly from coarser levels.
By sampling aggregates to indirectly sample nodes, we enable mesh-based graph neural networks to process fewer nodes per input while achieving predictive accuracy comparable to full-graph processing. We further introduce streaming computation to scale the model to point sets containing millions of points.
We evaluate AMS-GNO on multiple PDE benchmarks, where it achieves highly competitive performance. We further demonstrate strong results in cross-resolution generalization and computational scalability.
We note that the contributions above are supported by extensive comparative experiments and ablation studies.