- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes ShellGatedGNN for predicting electronic properties from 2D molecular graphs. The model partitions nodes into shells according to shortest-path distance and constructs receiver-relative sender environments using parent-edge features, same-shell edge features, and accumulated parent states. These environments enter both the message and gate computations, and the shell outputs are concatenated to update node representations. The paper reports improvements across six properties on QM9 and QM40, a validation MAE of 0.0957 eV on PCQM4Mv2, and depth-sweep and component-ablation experiments.
Critical strengths
The reported improvements on QM9 and QM40 cover multiple targets. SGGNN achieves the lowest reported MAE on all six properties in both datasets. Relative to the strongest reported baseline on each dataset, std-MIX improves by approximately 12.4% and 3.9%, respectively. These results provide meaningful support for the effectiveness of the complete architecture in the evaluated settings.
The receiver-relative environment and accumulated parent states constitute a concrete architectural design. They provide a structured way to use local connectivity to modulate multi-hop communication. The equations, complete forward-pass algorithm, and component experiments make the design inspectable. Training on PCQM4Mv2 also demonstrates the feasibility of handling a large number of samples.
Critical weaknesses
The PCQM4Mv2 comparison omits stronger public baselines. SGGNN reports a validation MAE of 0.0957, while publicly available results include 0.0937 for GraphGPS-small and 0.0901 for CoAtGIN-tiny [1, 2]. Neither appears in Table 3. GraphGPS is evaluated on QM9 and QM40, but its omission from the large-scale comparison is not adequately explained.
The novelty relative to existing shell-based methods is unclear. AIMNet-X2D already aggregates shells and concatenates their representations [3]. KP-GNN also incorporates structural information within hop-based neighborhoods, but is neither discussed nor compared [4]. The paper needs to distinguish its contribution more clearly from these existing designs.
The ablations do not establish the gate’s independent contribution. Every configuration in Table 4 retains the gate. The experiments also do not separate the benefit of supplying environmental information to the message from supplying it to the gate. Consequently, the specific value of environment-conditioned gating remains unresolved.
The long-range modeling claim lacks targeted validation. The paper does not analyze how its advantage varies with molecular size or graph distance. Evaluation on substantially larger molecular graphs, such as Peptides-struct [5], or a controlled size- and distance-based analysis within the existing tasks would more directly test this claim.
References
[1] Rampášek, L., et al. (2022). Recipe for a General, Powerful, Scalable Graph Transformer. Advances in Neural Information Processing Systems, 35. Official PCQM4Mv2 results.
[2] Zhang, X., et al. (2022). CoAtGIN: Marrying Convolution and Attention for Graph-based Molecule Property Prediction. IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 374–379. Official OGB-LSC leaderboard.
[3] Nandakumar, R. (2025). AIMNET-X2D: Universal and Scalable Framework for Chemical Property Transfer Learning. Master’s thesis, Carnegie Mellon University. 2025 implementation of shell aggregation and concatenation.
[4] Feng, J., et al. (2022). How Powerful are K-hop Message Passing Graph Neural Networks. Advances in Neural Information Processing Systems, 35.
[5] Dwivedi, V. P., et al. (2022). Long Range Graph Benchmark. Advances in Neural Information Processing Systems, 35, Datasets and Benchmarks Track.
Questions for authors
Why were GraphGPS and other stronger public PCQM4Mv2 results not included? Can the authors expand the comparison, documenting input information, parameter counts, training budgets, and measured runtime and memory usage, and clarify whether SGGNN offers a competitive accuracy–cost trade-off?
What specifically does the method add beyond AIMNet-X2D’s existing shell aggregation and concatenation, and beyond KP-GNN? Can the authors compare removing the gate, or removing environmental information only from the gate, while controlling model capacity and keeping the message inputs unchanged?
Does the advantage persist when the task places greater demands on long-range communication? Please provide targeted validation on larger molecular graphs, such as Peptides-struct, or evaluate performance by molecular size and graph distance within the paper’s task scope, reporting well-tuned baselines and computational costs. These experiments would test the central mechanism more directly than adding unrelated datasets.
3 comments
W2 & Q2 — Novelty over AIMNet-X2D and KP-GNN
Question: AIMNet-X2D already aggregates and concatenates shell representations and KP-GNN already uses hop-wise structural information, yet neither is discussed or compared; what remains beyond them?
Reply. We understand the concern about novelty, since the original submission did not discuss these two multi-hop GNNs in detail. Our novel elements are the following.
1. AIMNet-X2D. Grouping shells by BFS distance is not new in multi-hop GNNs, but there are three differences from AIMNet-X2D. First, in how a shell representation is obtained: AIMNet-X2D maps each shell through its own linear layer and merges the shells by a learned weighted sum, whereas we gate each node pair on the sender's environment and normalize within the shell. Second, AIMNet-X2D does not use edge features in message passing but only nodes, whereas our model uses edge features directly through the sender-environment gate. Third, in what a shell contains: AIMNet-X2D only groups the k-hop nodes into a shell, whereas we group the k-hop nodes together with their environment. AIMNet-X2D does concatenate its shells; the concatenation of the node update is therefore not our contribution either, and the core innovation remains the sender-environment-controlled gating that aggregates the shells. We revised the presentation in the contributions of the introduction (Section 1), added the distinction from these two methods in the related work (Section 2), and added the capacity-matched sum-versus-concatenation ablation to Appendix D.6.
2. KP-GNN. Its core contribution is that, beyond k-hop aggregation, it uses a structural summary of the induced subgraph on the receiver's k-th hop (edge-type counts and node configurations of that subgraph), encoded and added to that hop's aggregated output. The core contribution of our method is to modulate the aggregation of the k-th hop message with a sender-centred environment combined with a gate. The way environment information is used clearly differs between the two.
3. KP-GNN as a baseline. KP-GNN is a classic multi-hop method, and it was not included in the original submission; this deserves attention, so we trained it as a baseline on QM9 and QM40 and added its results to the QM9 and QM40 comparison tables of the main text. Table 2 lists the strongest baselines after adding KP-GNN; our model remains ahead.
To compare cost and performance we also report the training time on QM9 and QM40 on the same hardware (RTX 5070 Ti laptop, 12 GB), in Table 2. Our model performs better than KP-GNN; the training time is 32.3 vs 288.4 min on QM40 and 97.0 vs 67.7 min on QM9. Our method is multi-hop, yet on QM40 it reaches the state of the art with about the training time of the strongest one-hop baseline, GatedGCN+ (32.3 vs 27.3 min), while its training cost is far below that of the two multi-hop methods KP-GIN (VN) (288.4 min) and DuAxMPNN (218.1 min).
Table 2 — Strongest baselines on QM9 and QM40: test std-MIX (three runs, sample std in parentheses) and training time on the full training set (median of three runs).
| Method | QM9 std-MIX | QM40 std-MIX | QM9 train (min) | QM40 train (min) |
|---|---|---|---|---|
| GatedGCN+ | 0.1037 (0.0002) | 0.2012 (0.0004) | 17.1 | 27.3 |
| KP-GIN (VN) | 0.0992 (0.0004) | 0.2056 (0.0002) | 67.7 | 288.4 |
| DuAxMPNN | 0.0933 (0.0005) | 0.2104 (0.0061) | 173.2 | 218.1 |
| SGGNN (ours) | 0.0817 (0.0004) | 0.1933 (0.00003) | 97.0 | 32.3 |
W3 & Q3 — Does the ablation isolate the gate?
Question: Every configuration in the ablation keeps the gate, and the benefit of giving the environment to the message is not separated from giving it to the gate; what is the value of environment-conditioned gating?
Reply. We thank the reviewer for this insightful comment. Separating the role of the gate from that of the environment information is indeed valuable, and in response we have designed a dedicated ablation for exactly this question and added it to the ablation study of the main text (Section 4.4). Table 3 reports the new ablation, with the parameter count matched across all five rows; the result is that both the gate and the environment information are positive contributors.
Table 3 — Gate and environment entry points, matched parameters (QM9, Dmax=4; three runs, sample std in parentheses).
| Variant | env → message | env → gate | std-MIX | abs. Δ vs both-off | rel. Δ |
|---|---|---|---|---|---|
| both off | × | × | 0.0899 (0.0003) | — | — |
| full model | ✓ | ✓ | 0.0816 (0.0001) | 0.0083 (0.0003) | 9.3 (0.3) |
| message only | ✓ | × | 0.0823 (0.0002) | 0.0076 (0.0005) | 8.4 (0.6) |
| no gate (g=1) | ✓ | – | 0.0851 (0.0002) | 0.0048 (0.0004) | 5.3 (0.5) |
| gate only | × | ✓ | 0.0859 (0.0002) | 0.0040 (0.0004) | 4.5 (0.4) |
All five rows are matched in parameter count. The full-model row was re-trained inside this ablation campaign; its 0.0816 reproduces the main-table value of 0.0817 to within the reported spread.
W4 & Q4 — Targeted validation of the long-range claim
Question: The long-range claim is not analysed against molecular size or graph distance; does the advantage persist when longer-range communication matters more?
Reply. We appreciate the opportunity to clarify this point, and we regret that our wording invited this reading. The claim we make is not that long-range modelling has a stronger effect on larger molecules; it is that our long-range message-passing model substantially improves performance on molecular property prediction, and that its architecture genuinely reaches further structure than conventional one-hop message passing. Our experiments show that the advantage is present across molecular sizes: the molecules of QM9 are clearly smaller than those of QM40, and we are also ahead on QM9 (Table 2).
Your question points to an aspect we are glad to examine further, namely how the gain of long-range message passing relates to graph size. We therefore re-ran inference with the already-trained SGGNN and baseline checkpoints on the QM9 and QM40 test splits and recorded, for every molecule, its error, number of nodes and BFS diameter, and plotted the resulting distributions. The result is that our advantage is present across molecular sizes. We also observed one curious phenomenon: the prediction errors of the different models show an interesting consistency in how they distribute over node count and BFS diameter, and on QM40 the prediction difficulty increases with molecular size. We have added this to Appendix D.5.
We are grateful for the detailed comments. Each point is addressed below, together with the corresponding revisions.
W1 & Q1 — Stronger public baselines on PCQM4Mv2
Question: GraphGPS (0.0937) and CoAtGIN-tiny (0.0901) report better PCQM4Mv2 MAE than ours (0.0957) but do not appear in the comparison; why, and is SGGNN competitive in performance-versus-cost terms?
Reply. We understand the concern that stronger public baselines are absent from the comparison. GraphGPS and CoAtGIN-tiny are hybrids of a graph transformer and message passing, which is a different class of method from pure message passing. The submitted abstract already noted that the comparison on the PCQM4Mv2 dataset is between message-passing methods. In this revision we expanded Appendix B.1 into a delimitation of the four families — message passing, graph transformers, 3D methods and their hybrids.
Cross-class comparison is nevertheless important — we already include GraphGPS in the QM9 and QM40 comparisons — so we have added GraphGPS and CoAtGIN-tiny to the large-scale table. This does not affect our claim of being state of the art within the same class of methods.
We additionally report hardware, training time, parameter count and input type, so that the cost-versus-performance trade-off is visible. Table 1 is the new table. All baseline figures in that table are taken from the official OGB website. Note: 0.0937 is GraphGPS-small; the table lists the stronger GraphGPS entry on OGB, 0.0852.
Table 1 — PCQM4Mv2 validation MAE (eV, lower is better). A dash means the work reports no training time.