- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes GraphToken, a method that turns the local neighbourhood of each node in a text-attributed graph into a set of discrete tokens. For each node, it samples 32 small connected subgraphs ("rooted motifs") of two to four nodes, encodes each of them with a GIN over the text embeddings of their nodes, and assigns each encoding to the nearest entry of a learnable codebook, as in vector quantization. The node is then represented by the resulting multiset of codebook entries ("graph-tokens"), which can be fed to a GNN (after mean pooling), a Transformer, or an LLM through a learnable projector. The evaluation covers node classification on ten datasets from citation, e-commerce, and social domains, link prediction on three datasets, transfer between datasets, an ablation of text and structure, and an analysis of individual tokens. The LLM variant reaches the best or close to the best results on most datasets.
Critical strengths
The idea of a shared discrete vocabulary over joint structure-text patterns is natural for text-attributed graphs, and separating this vocabulary from the downstream model is a reasonable design goal. The evaluation is broad (10 datasets from three domains and 15 baselines), and the analysis of individual tokens in Figure 4 and Appendix A.6 is a useful attempt to show what the vocabulary captures.
Critical weaknesses
1. The method cannot be reproduced from the paper or from the released code.
- The paper never states how the motif encoder and the codebook are trained. It only says that they "are jointly optimized through vector quantization" (lines 256-257), but vector quantization alone does not define a learning objective.
- Key settings are missing: the codebook size , the text encoder , and, for the LLM variant, the prompts, how the projector is trained, and whether the LLM is frozen (Figure 2 marks the LLM variant as learnable).
- The paper states that the "implementation and configuration files are available" (lines 512-513), but the repository contains only model components, without training or evaluation scripts, configurations, or data loading, and its README contains only a title. The LLM variant, which gives the main results, is missing:
llm.pycontains only text-encoding helpers and a head that compares node embeddings with label text embeddings (llm.py:59-68). Moreover,transformer.pyimplements attention between neighbouring nodes of the graph (transformer.py:30-48), while Eq. 6 describes attention among the graph-tokens of one node. Finally, the only loss in the code is a term that pulls the motif encodings toward their nearest codebook entries (quantizer.py:74-77), which confirms that the training objective is defined elsewhere and is not released. - Much of the text uses abstract and decorated wording instead of simple statements, which hides these missing details. For example, "GraphToken establishes a shared interface between discrete graph-tokens and downstream architectures. For each target node, its local context is represented by a collection of recurring graph-tokens, yielding a vocabulary-grounded characterization of the neighborhood" (lines 78-81) means only that each node is represented by the tokens of its sampled motifs, and these tokens can be used as input to different models.
2. The experiments do not support the main claims, and several results look wrong.
The experiments do not show that the discrete vocabulary is a key component:
- There is no ablation without quantization (using the continuous motif encodings), and no comparison of the same backbone with and without GraphToken on the same text features. Therefore, the gains may come from the text encoder or from Vicuna-7B rather than from the vocabulary.
- Only the LLM variant is competitive. The GNN and Transformer variants are often far below the baselines, for example, 35.21 and 38.06 macro-F1 on arXiv23 against 56.40 for RGLM (Table 1). This weakens the claim that one vocabulary serves different backbones equally well.
- The margins of the LLM variant are small and often within one standard deviation of the best baseline, for example, 75.46 vs 75.28 accuracy on arXiv and 67.82 vs 67.65 on Reddit, and GOFA is better on PubMed (92.84 vs 92.36).
- The analysis sections do not support the claims drawn from them. The "most transferable tokens" are ranked by an ad-hoc score with four loosely defined, equally weighted terms (Appendix A.4.2), and removing them changes the results by about 2 percentage points, for a transfer to Reddit that, according to Table 3, barely works (see below). The sampling analysis (Figures 5-6) measures only the speed and the number of sampled subgraphs, and never shows whether the sampler affects accuracy.
Moreover, several reported results look wrong or come from a flawed setup:
- In Table 3, all methods reach 5.6-9.4% accuracy on Cora when trained on S2, although Cora has seven classes, so random guessing gives about 14%, and a prediction head is trained on the Cora training split (lines 915-917). Similarly, on the binary Reddit task with S1, the baselines reach 44-51% accuracy and GraphToken 52%, although always predicting the majority class gives more than 50%. Both results suggest an error in the setup.
- In Table 3, the source collections S1 and S3 contain the target datasets Cora and History, so these settings do not measure transfer, although the text describes them as sources that contain "graphs from the target domain" (lines 377 and 400).
- In Table 1, the GRAVER results on Fitness (52.83 ± 3.03 and 36.43 ± 2.30) are identical to its results on arXiv, which suggests a copy error.
3. The paper overlooks closely related work, which also limits its novelty.
- Section 2 presents graph foundation models as if they were limited to text-attributed graphs. However, many real-world graphs have numerical or categorical node features without any text, and an active line of work builds foundation models that handle such arbitrary feature spaces, for example, TAG [1], G2T-FM [2], and GraphPFN [3], which build on tabular foundation models and prior-data fitted networks. These methods pursue the same goal of transfer across graphs, and some of them already show practical gains on diverse real-world data, yet the paper does not mention any of them.
- Quantizing GNN encodings of local subgraphs against a learned codebook was already done by GFT, which builds a tree vocabulary over text-attributed graphs, as well as by VQGraph [4] and GQT [5], which also trains its tokenizer separately from the downstream model. The paper describes GFT only as a structural approach and does not cite the other two. As a result, the main difference from prior work appears to be the use of sampled rooted motifs instead of computation trees, and the paper does not test whether this choice matters.
[1] Hayler et al. "Bringing Graphs to the Table: Zero-shot Node Classification via Tabular Foundation Models," 2025
[2] Eremeev et al. "Turning Tabular Foundation Models into Graph Foundation Models," 2025
[3] Eremeev et al. "GraphPFN: A Prior-Data Fitted Graph Foundation Model," 2025
[4] Yang et al. "VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs," 2024
[5] Wang et al. "Learning Graph Quantized Tokenizers," 2025
Questions for authors
- What is the full training objective of the motif encoder and the codebook?
- What are the codebook size, the text encoder, and the training procedure of the LLM variant?
- How do the results change without quantization, and how does a GNN or Vicuna-7B with the same text features but without GraphToken perform? How do they change when GraphToken uses random-walk or computation-tree sampling instead of rooted motifs?
- Why is the accuracy on Cora and Reddit in Table 3 lower than random guessing, and how were the foundation-model baselines trained on the target datasets?
Dear Reviewers,
Thank you for your thoughtful reviews and contributions. The authors have now submitted their rebuttals. Please take some time to review their responses and assess whether your concerns have been adequately addressed. If appropriate, please update your comments and ratings accordingly.
Thank you for your time and continued efforts.
Best regards,
Your AC