- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces GINAT, which augments GNN edge messages with global domain descriptions encoded by a frozen language model. Cross-attention and a scaled residual connection integrate the textual context into message passing. The authors establish permutation equivariance and report performance improvements across multiple backbones and node- and graph-level tasks.
Critical strengths
-
Simple and modular design. The augmentation can be integrated into different message-passing backbones, preserves permutation equivariance, and recovers the original model when .
-
Broad experimental coverage and consistent improvements in point estimates. Across Tables 2, 3, 10, and 11, GINAT improves 155 of 161 backbone–dataset combinations, supporting the module’s empirical value.
Critical weaknesses
-
The independent contribution of prompt semantics and LM priors is not sufficiently isolated. Random vectors recover part of the gains, while PROTEINS and ENZYMES receive atom-and-bond descriptions that do not match their secondary-structure-element nodes; these results leave the contributions of domain semantics, pretrained representations, and the additional architecture unresolved (Tables 6 and 12).
-
Large relative gains rely on near-chance baselines, with inconsistent input features. On featureless datasets, the baselines use random node features while GINAT uses constant features, confounding the comparison (Appendix F.1). Matched comparisons against degree features and cardinality-preserving attention are absent, although CPA reports 93.07%/57.39% on REDDIT-BINARY/MULTI5K under a different evaluation protocol (Zhang & Xie, 2020).
-
Some performance claims exceed the supporting results. The “over 10%” improvement in Appendix F.3 does not correspond to the aggregate means in Table 7, while Table 5 contains ties and favors Concat on Chameleon despite claiming that GINAT performs best. The absence of a vanilla baseline also obscures the gains from each fusion method.
-
The TAG adaptation removes node-specific textual information. Poor performance after replacing node-level text representations with global prompts does not establish that TAG methods cannot support this setting; the comparison primarily evaluates the particular adaptation used.
Questions for authors
Questions for Authors
-
Appendix K lists atom-and-bond descriptions for PROTEINS and ENZYMES, although their nodes represent secondary structure elements. Could the authors confirm which prompts were used in the experiments and explain this choice?
-
Appendix F.1 states that the baselines use random node features on featureless graphs, while GINAT uses constant features. What motivated this difference, and were the gains also observed when both models received identical node features?
-
Table 6 shows that random conditioning vectors recover part of the improvement. Was the same cross-attention module evaluated with learned, text-independent vectors, and if so, how did it compare with LM-encoded prompts?
4 comments
PART 2/2 of our rebuttal:
Regarding random and constant node features Our motivation for this choice is based on theoretic findings in the literature and our empirical findings that were not included in the manuscript, but will be provided here and included in the revised version. As we write in the aforementioned appendix, Random Node Initialization (RNI) has been shown to improve GNNs’ expressivity on featureless graphs [1,2]. This motivated our choice for equipping the baseline with such. We also observed this phenomenon in our experiments as provided in the table below. GINAT empirically doesn’t benefit from this addition. We keep them as a future work direction, but we hypothesize that the random noise makes it harder for cross attention components to learn meaningful relations between the structural GNN signal and the contextual LLM signal.
In the table below, the GINAT (rand) and vanilla (rand) columns present the results of our and the baselines when using RNI, and the GINAT (const) and vanilla (const) counterparts are the constant node initialization for those models. The diff columns calculate the respective model’s performance using random node features minus its performance using constant node features. Each result is a mean of 3 seeds. On average, GINAT loses 3.8 accuracy points using RNI, while the vanilla baseline gains 1, empirically affirming our choice.
| dataset | MPNN | GINAT (rand) | GINAT (const) | GINAT diff | vanilla (rand) | vanilla (const) | vanilla diff |
|---|---|---|---|---|---|---|---|
| IMDB-B | gin | 0.627 | 0.733 | -10.7 | 0.727 | 0.730 | -0.3 |
| IMDB-B | graphconv | 0.670 | 0.753 | -8.3 | 0.737 | 0.727 | +1.0 |
| IMDB-B | gat | 0.667 | 0.753 | -8.7 | 0.507 | 0.500 | +0.7 |
| IMDB-M | gin | 0.476 | 0.500 | -2.4 | 0.422 | 0.476 | -5.3 |
| IMDB-M | graphconv | 0.478 | 0.498 | -2.0 | 0.476 | 0.442 | +3.3 |
| IMDB-M | gat | 0.473 | 0.478 | -0.4 | 0.373 | 0.327 | +4.7 |
| REDDIT-B | gin | 0.915 | 0.945 | -3.0 | 0.935 | 0.922 | +1.3 |
| REDDIT-B | graphconv | 0.900 | 0.927 | -2.7 | 0.917 | 0.917 | +0.0 |
| REDDIT-B | gat | 0.868 | 0.923 | -5.5 | 0.542 | 0.510 | +3.3 |
| REDDIT-M5K | gin | 0.568 | 0.568 | -0.1 | 0.566 | 0.560 | +0.6 |
| REDDIT-M5K | graphconv | 0.565 | 0.572 | -0.7 | 0.570 | 0.562 | +0.9 |
| REDDIT-M5K | gat | 0.543 | 0.552 | -0.9 | 0.218 | 0.198 | +1.9 |
| mean | -3.8 | +1.0 |
Regarding text-independent vectors preformance We thank you for this question. This isolates the remaining confound in Table 6 well. To address your concern, we have conducted another experiment with a learned text-independent conditioning input that has the same shape and the same cross-attention pathway as GINAT, so it controls for the added architecture and capacity while removing the LM and the text entirely.
We replace each LM-encoded prompt embedding with a free parameter tensor of the same shape (initialized N(0,1), trained jointly with the GNN, same optimizer, lr, etc.). Rest is identical to GINAT: the cross-attention module, ξ, LayerNorm, the per-forward-pass prompt sampling, the backbone configs (Table 16), and the three seeds. Conditioning vectors are learned per prompt slot, so the number of distinct conditioning inputs matches the prompt dataset. We report mean over 3 seeds.
| Dataset | Backbone | Vanilla | Learned vectors (text-independent) | GINAT (LM-encoded prompts) |
|---|---|---|---|---|
| ENZYMES | GCN | 0.361 | 0.328 | 0.361 |
| ENZYMES | GAT | 0.422 | 0.367 | 0.467 |
| PROTEINS | GCN | 0.738 | 0.571 | 0.750 |
| PROTEINS | GAT | 0.705 | 0.631 | 0.717 |
| REDDIT-B | GCN | 0.843 | 0.647 | 0.883 |
| REDDIT-B | GAT | 0.542 | 0.797 | 0.923 |
| REDDIT-M5K | GCN | 0.521 | 0.399 | 0.547 |
| REDDIT-M5K | GAT | 0.218 | 0.528 | 0.552 |
As this table shows, together with Table 6 (fixed random vectors) and the unrelated-prompt ablation (Fig. 4), this gives a three-level decomposition of gains: Architecture alone: learned or random vectors. Large effect only where the vanilla attention backbone collapses; otherwise neutral or harmful.
- generic LM prior: unrelated prompts. Partial gains.
- domain semantics: proper prompts. Best in every case. Meaning, our method manages to utilize the textual information in a meaningful and positive way.
We kindly ask that you reassess your score of our work. We remain committed to addressing any concern you might have. The Authors
References
[1] Abboud et al., The Surprising Power of Graph Neural Networks with Random Node Initialization. IJCAI 2021.
[2] Eliasof et al,. Graph Positional Encoding via Random Feature Propagation. ICML 2023.
Thank you for the additional experiments. Comparing the models with the same node features and testing learned vectors without text addressed my main questions about the evaluation. I’m therefore raising my score from 1 to 2. I see value in showing that domain descriptions can improve existing GNNs, though I still view this as a modest methodological advance.
Official Comment by Authors
Dear tzRw,
We thank you for your engaging and meaningful review of our paper. We appreciate you raising your score and helping us improve our work.
Thanks,
The Authors
We thank you for the careful reading. We address each of your concerns and describe the new experiments and corrections in the revision (we don’t report std in the tables below for space). In order to answer your concerts seriously and thoroughly, we divide our response in two.
PART 1/2 of our rebuttal:
Regarding molecular datasets used prompts We thank you for this crucial finding. Indeed, the prompts we used for PROTEINS and ENZYMES, which are the ones reported in Appendix K, don't describe them correctly. To address this, we created a new prompts dataset following the recipe we defined. We trained GINAT using all the backbones listed in the paper, and using the same configurations as listed in the manuscript. The results are summed in the tables below. The main takeaways are improved gains on both datasets, averaged across the different MPNNs: PROTEINS +3.3% gain relative to vanilla (was +3.1%), while ENZYMES now shows +18.3% relative gains (was +8.4%)!
ENZYMES
PROTEINS
We see these results as another evidence of the method’s merits. Your finding is similar to the experiment presented in Figure 4 in our manuscript. In that experiment, we have seen that using unrelated prompts provides some of the gains; Our incorrect assignment of prompts creates a similar scenario - relevant but inaccurate prompts. Those show meaningful gain on the baseline, but provide even better gain using accurate prompts.
The prompt dataset we used is now part of the anonymized git, and the revised version of our manuscript presents it, and the updated result. For your convenience, here are 2 sample prompts from that dataset:
** Regarding GINAT’s gains and performance ** We believe a reuse of the term “baseline” in Appendix F.3 caused confusion. The experiment in table 5 presents our exploration of other fusion strategies. It compares different fusion approaches, not the vanilla baseline. As written in that appendix, each result in the table is an aggregation of 54 seeded training and evaluations, spanning three MPNNs, three seeds, and six depth-width configurations (and for this reason, no single MPNN baseline is provided there).
These results have nothing to do with the MPNN baselines we compare GINAT to, and they serve only to explain why this specific architecture was chosen in our research. We corrected this misleading writing in our revised version to avoid this confusion in the future, and thank you for pointing it out for us. Regarding our TAG adaptation We thank you for the careful reading of our TAG experiments. When adapting TAPE, LLM4NG, and SimTeG, we chose the straightforward adaptation: we kept each method's original architecture, training procedure, and hyperparameters, only setting the node-level text with GINAT's global prompt. This setup hinders the core benefit these models have (distinguishing nodes by their description) which results in poor performance in this setting. The goal of this experiment was to explore the existing literature when proposing a new approach and is not a major part of this work and was kept in the appendix for completeness.
We underlined this in the revised version and gave this context to said experiment.
Part 2/2 of our rebuttal follows in the next comment.