- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies receiver-query diversity in Graph Transformers: beyond global access, it asks how many distinct global summaries can be tailored to different nodes. The authors introduce (M) query modes between a single shared query and full receiver-specific attention, and provide theoretical and synthetic evidence that limited query diversity can bottleneck tasks requiring different receivers to retrieve different information. On COCO-SP and PascalVOC-SP, sharing one query substantially reduces node-segmentation performance. The paper also shows that frozen query-compression ablations can underestimate the performance achievable after retraining under the same constraint. However, the main positive evidence is limited to two related node-level tasks, and the graph-level Peptides-func experiment does not establish a consistent advantage for greater query diversity.
Critical strengths
-
The paper introduces a clear and insightful perspective by separating global access from the ability to produce distinct global summaries for different receiver nodes. The notion of receiver-query diversity is intuitive and supported by both theory and controlled experiments.
-
The paper makes a valuable methodological distinction between frozen post-hoc compression and training under the same capacity constraint. The paired experiments on PascalVOC-SP and Peptides-func show that these settings can lead to substantially different conclusions.
-
The experimental reporting is transparent and appropriately cautious. The paper retains the negative result on Peptides-func and explicitly avoids claiming a universal capacity law or a practical speedup.
Critical weaknesses
-
The evidence for generalization is limited. The main positive results come from two closely related superpixel node-segmentation datasets, while the only graph-level task, Peptides-func, does not support the preregistered prediction that greater query diversity improves trained performance. The evaluation does not cover many common graph-learning settings, such as multi-dataset molecular property prediction, link prediction, standard node classification, recommendation graphs, knowledge graphs, traffic or spatiotemporal graphs, graph generation, heterogeneous graphs, or large-scale graphs.
-
The paper does not provide a practical way to choose the central hyperparameter (M). It evaluates only a few discrete budgets and offers neither a transferable selection rule nor an automatic budget-allocation mechanism.
-
Practical efficiency is not demonstrated. Although the prototype reduces peak memory by about 24% at (M=16), it is approximately 16% slower than full attention. The work therefore does not establish a full path from a smaller (M) to retained accuracy and real-world efficiency gains.
Questions for authors
See Weakness.