- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies information loss when a graph is represented as an unordered multiset of quantized node codes, using a counting argument and a token-derived nearest-neighbor/GED diagnostic. It proposes GMT: mine a vocabulary of frequent subgraphs on training graphs; learn reusable motif embeddings with occurrence-supervised soft matching and a collision penalty; use those embeddings to supervise a node encoder before vector quantization; and pretrain a Transformer to reconstruct masked node-code embeddings with motif tokens visible. Tokenizers are then frozen during downstream fine-tuning. Tables 1-2 report the best mean among the evaluated configurations on 11 of 12 datasets, including ZINC MAE 0.150 versus 0.181 for the strongest evaluated baseline.
Critical strengths
-
The frozen-tokenizer comparisons provide useful evidence for motif augmentation, improving downstream performance in 15 of 16 settings without retraining the node tokenizers.
-
GMT is evaluated across diverse graph-level tasks and produces shorter inputs than the evaluated BPE-based serializations, with favorable downstream fine-tuning time and memory usage.
Critical weaknesses
- GMT combines several training stages and auxiliary objectives, but the current ablations do not fully establish whether this complexity is necessary. In particular, the random-motif-encoder comparison does not establish an advantage over directly learned motif-ID embeddings under the same motif inventory and downstream backbone. Separate ablations of collision regularization and graph–motif ranking would also help distinguish the benefits of the proposed mechanisms from those of simply providing motif information.
- Some aspects of the experimental setup make the reported gains difficult to assess. Appendix B.1 describes learned motif selection for unseen graphs, whereas Section 5.3 uses an exactly labeled motif inventory; it remains unclear which protocol supports the main results and how selection errors affect downstream performance. In addition, the reported GraphGPS ZINC MAE of 0.195 is substantially higher than its published result of 0.070. Configuration or training-budget differences may explain this discrepancy, but further detail is needed to assess the strength of the comparison.
Questions for authors
- Under a matched training budget, how does GMT compare with a simple learned motif-ID embedding baseline, and which auxiliary objectives provide consistent additional gains?
- Could you specify the test-time motif-selection procedure used in Tables 1–2 and the GraphGPS configuration and training budget used for ZINC?
- When motif tokens are omitted from the downstream input, does motif-supervised node tokenization still outperform an otherwise identical tokenizer trained without graph–motif ranking?