acceptodds
Under review as a conference paper at ICLR 2027

NodeArena: A Comprehensive Node Classification Benchmark for Foundation Models

Abstract

Graph neural networks (GNNs) are a principal model family for graph learning but require separate training and tuning for each dataset. Graph foundation models (GFMs) aim to reduce this burden by transferring pretrained knowledge, but it remains unclear whether they offer consistent predictive gains at practical computational costs. Although node classification is the representative graph-learning task, existing benchmarks lack the combination of broad dataset coverage, controlled evaluation, and cost measurement needed to assess this promise. We introduce NodeArena, a comprehensive node classification benchmark comparing six GFMs with 15 supervised methods across 51 datasets. With the unified protocol combining shared splits, multiple metrics, and controlled hyperparameter tuning, NodeArena evaluates predictive performance as well as the computational costs of GFM adaptation, supervised training and tuning, and inference. On NodeArena, tuned GNNs remain the strongest overall, while GraphPFN reaches third in the Elo ranking under the label-rich split. GraphPFN and GVT are the two GFMs on the performance–cost frontiers, but intermediate tuning budgets only leave the former on the frontier. These findings indicate that GFMs still remain limited as broad replacements for dataset-specific training and tuning. We release the evaluation pipeline, run-level results, and a public leaderboard to support reproducible and continued comparison, available at https://anonymous.4open.science/r/node-arena-BB1B/.

Primary area:
datasets and benchmarks
Keywords:
Node Classification, Graph Neural Networks, Graph Foundation Model, Benchmark

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_JwkFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a benchmark for evaluating graph foundation models (GFMs) against GNNs. The benchmark comprises 51 real-world datasets together with evaluation metrics and their implementations, and the paper reports an evaluation and analysis of GFMs and GNNs. Its coverage is substantially broader than that of prior benchmarks, and this benchmark is likely to be widely used in future work.

Critical strengths

S1. Many existing papers that propose GFMs report that GFMs outperform GNNs, but their experimental settings are often insufficient. This paper evaluates GNNs in two configurations, default and tuned supervised, and shows that tuned GNNs outperform GFMs in most cases. Clarifying the performance limits of current GFMs and the open challenges for future work is a clear contribution.

  • The authors tune 15 GNNs and evaluate 6 GFMs on 51 datasets. They show that, overall, the tuned GNNs (particularly GCNII, GPRGNN, and FAGCN) and GraphPFN, a GFM, achieve the highest performance.
  • The fair comparison protocol is excellent, as it allows GFMs and GNNs to be compared at their best attainable performance.
Critical weaknesses

W1. Although the results are comprehensive and of high quality, they offer few surprises. Figure 5 contains the new findings described in S1. Figures 6-8, however, essentially present the results of Figure 5 from different angles and add relatively little new insight.

W2. The paper should better explain the motivation behind the choice of graph properties for evaluation.

  • Properties such as #nodes, average degree, and density need no justification. However, the purpose of including nonzero fraction, label imbalance, and label entropy should be explained.
  • Figure 4 visualizes pairs of properties, but the paper does not sufficiently explain why these particular combinations were chosen.
Questions for authors

Q1. One would normally expect GNNs to outperform GFMs in the split with a larger training ratio. However, in Figure 8, this conjecture does not appear to be the case, particularly for the tuned models. Could you explain why GraphPFN performs better than tuned GNNs for 50/25/25 split than 10/10/80 split?

3 comments

We appreciate your thoughtful feedback and address your concerns below.


W1. Insights beyond the aggregate results.

Figure 6 reveals where the overall ordering reverses. Although tuned GNNs lead in Figure 5, GraphPFN achieves the best mean rank on binary tasks under both label splits, while tuned GNNs remain stronger on multiclass tasks. This identifies a task type where GraphPFN outperforms the tuned baselines on average despite its lower overall position.

Figure 7 separates overall improvement from persistent weaknesses. GraphPFN’s improved overall position at the higher label budget does not mean its weaknesses disappear. It continues to trail GCNII and GPRGNN on high-dimensional and highly imbalanced datasets. This identifies specific limitations that additional labels do not resolve, which the aggregate ranking cannot locate.

Figure 8 reveals competitive choices overlooked by ranking predictive performance alone. GVT and GraphPFN lie on the frontier against default and fully tuned supervised methods despite their lower predictive rankings. GVT offers lower workflow cost, while GraphPFN provides stronger predictions at greater cost. Other GFMs are dominated on both dimensions, showing which pretrained models offer useful computational tradeoffs and which do not.

We will revise Sections 5.1–5.3 to make these additional findings explicit.


W2a. Motivation for nonzero fraction, label imbalance, and label entropy.

These properties describe feature representations and label distributions beyond graph size and connectivity. They help assess whether the collection covers the different feature and label distributions encountered by the evaluated models.

Nonzero fraction distinguishes sparse from dense attributes, which feature dimension alone cannot capture. Together, these properties characterize both the size and sparsity of node representations.

Label imbalance, measured by the largest-to-smallest class-size ratio, captures minority-class scarcity. Even under stratified splits, the same training fraction can yield very different numbers of labeled examples across classes.

Normalized label entropy captures how probability mass is distributed across all classes, complementing the imbalance ratio’s focus on the largest and smallest classes. For example, a small minority class can substantially change the imbalance ratio without greatly changing the distribution of most probability mass.


W2b. Motivation for the property pairs in Figure 4.

We selected these pairs to distinguish complementary aspects of graph structure, feature representation, and label alignment. Each pair combines related properties that cannot substitute for one another, allowing us to examine whether the collection covers different combinations within these aspects.

Graph size × average degree separates graph scale from local connectivity, because a larger graph does not necessarily provide more neighbors per node.

Median nonzero degree × hub concentration separates typical connectivity from the degree distribution’s upper tail, because similar median degrees can coexist with very different hub concentrations.

Feature dimension × nonzero fraction separates the number of available attributes from their sparsity, because equally high-dimensional representations can be sparse or dense.

Feature-based label separation × adjusted homophily separates label alignment in node features from label alignment along edges, because one can be strong while the other is weak.

These pairings therefore assess coverage of distinct combinations that the individual property ranges in Figure 3 cannot establish. We will clarify this selection rationale in Section 3.2 and the Figure 4 caption.

Q1. GraphPFN’s relative improvement with a larger label budget.

GraphPFN builds on LimiX, an ICL-based tabular foundation model [1,2], and uses labeled nodes as inference-time context while keeping its pretrained parameters fixed. Context-size ablations in related tabular foundation models show that additional labeled context can improve performance. TabDPT reports reduced performance when context is restricted to 256 examples [3], while ConTextTab reports improvements with longer contexts, particularly on larger datasets [4].

Therefore, richer labeled context offers a plausible explanation for GraphPFN’s stronger relative performance under 50/25/25. On Shanghai, for example, GraphPFN’s accuracy rises from 43.81% to 65.18% (+21.37 percentage points), compared with 46.60% to 47.55% for tuned GCNII (+0.95 points).

Such differences in improvement help explain the relative performance changes in Figure 8. Mean Improvability (MI) measures the average normalized gap to the best evaluated method on each dataset within each split. GraphPFN’s MI remains nearly unchanged, whereas the MI of GCNII and GPRGNN increases, indicating that they fall further behind the dataset-specific leaders. Thus, GraphPFN becomes more competitive relative to these baselines without implying that their absolute performance deteriorates.


References

[1] Eremeev et al. GraphPFN: A Prior-Data Fitted Graph Foundation Model. 2025. Paper.

[2] LimiX Team. LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence. 2025. Paper.

[3] Ma et al. TabDPT: Scaling Tabular Foundation Models on Real Data. NeurIPS, 2025. Paper.

[4] Spinaci et al. ConTextTab: A Semantics-Aware Tabular In-Context Learner. NeurIPS, 2025. Paper.

Thank you for the clarification.

Reviewer_z1BjOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes NodeArena, a comprehensive node classification benchmark that covers 51 datasets, 15 supervised models, and 6 GFMs. The evaluation is thorough: it employs hyperparameter tuning and multiple splits, and reports computational costs. Unlike previous works, NodeArena finds that existing GFMs often lag behind tuned GNNs under broader evaluation.

Critical strengths

Within the considered datasets, the evaluation is thorough: it covers 15 supervised models and 6 GFMs, hyperparameters are tuned for all applicable methods, multiple splits are considered, and computational costs are reported. This thoroughness is commendable and rare among graph ML works.

Critical weaknesses
  1. Dataset collection raises multiple concerns

Classic graph datasets have been widely criticized for a lack of real-world applications, narrow coverage of domains and tasks, or even bugs in dataset collection [1, 2, 3]. Unfortunately, the authors do not filter datasets by these characteristics, and thus many, if not most, datasets in NodeArena are toy-ish or lack practical utility. For example, all 13 scholarly networks have the task of "scholarly topic or field classification," which has limited applications and is likely better solved using NLP methods on raw texts rather than graph ML methods. Another example is transportation networks: all four datasets have the task of "10-quantile road-junction eccentricity classification," so their targets can be deterministically computed from the graph structure. Many datasets, including amazon_computer, amazon_photo, coauthor_cs, coauthor_physics, and others, have bag-of-words features, which are arguably outdated and should not be used in evaluations. Some datasets, including cornell, texas, wisconsin, and their tag_ versions, contain fewer than 500 nodes and are arguably too noisy for meaningful evaluation, especially under 10/10/80 splits. Thus, while the authors do a great benchmarking job, the underlying datasets remain highly questionable.

Another issue is that NodeArena ignores node regression tasks. While graph ML has indeed traditionally focused on classification tasks, GraphLand [2] recently proposed 7 real-world regression datasets, which can be directly incorporated into NodeArena. Related projects, TabArena and RelArena-α\alpha, include both classification and regression tasks, so I believe NodeArena should cover regression too. Since some methods may not be suitable for node regression, it might be worth having separate leaderboards for classification, regression, and both tasks combined, similar to TabArena.

  1. The contribution is limited compared to existing benchmarking works

Specifically, the closest relevant work is Platonov et al. (2026). The two works differ mostly in dataset and model coverage, as well as conclusions about GNN and GFM performance. Other differences (number of splits, use of multiple metrics, raw results, and leaderboard) seem to be more minor or technical. At the same time, both papers tune GNNs and report computational costs.

Regarding dataset selection, please see the point above. Regarding conclusions, I believe they stem from dataset selection and not from method coverage or tuning. Indeed, one can see from Table 22 that on three datasets (artnet-exp, city-reviews, and tolokers-2) present in both NodeArena and Platonov et al. (2026), GraphPFN achieves the best performance in 5 out of 6 cases, which is consistent with the conclusion of Platonov et al. (2026) that GraphPFN generally outperforms GNNs.

Thus, while the model coverage and minor differences stated above are commendable, in my opinion, they are not enough for acceptance.

[1] Bechler-Speicher et al., "Position: Graph Learning Will Lose Relevance Due To Poor Benchmarks," ICML 2025.

[2] Bazhenov et al., "GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Data," NeurIPS 2025.

[3] Li et al., "GraphCleaner: Detecting Mislabelled Samples in Popular Graph Learning Benchmarks," ICML 2023.

Questions for authors
  1. For each dataset, can you provide an estimate of its practical utility (with justification) and provide evaluation results with datasets lacking practical utility excluded? It would be even better to also exclude datasets with bag-of-words features and datasets with fewer than 500 nodes.
  2. Why did you exclude node regression tasks from NodeArena?
  3. Can you add regression tasks to NodeArena?
  4. What is your contribution relative to Platonov et al. (2026)?
6 comments

We thank the reviewer for the detailed feedback.


W1/Q1. Dataset selection and exclusion analyses.

GraphLand provides valuable tasks motivated by industrial applications [1]. However, we believe that a task can also be useful for evaluating models even when its immediate industrial use is limited. In NodeArena, such tasks help reveal differences in how models handle graph structure, node features, and labels.

The manuscript already explains the selection rules in Section 3.1 and Appendix B.2. Table 11 lists each dataset’s source and prediction target, and Section 3.2 describes the graph, feature, and label properties covered by the collection.

We also tested how excluding the dataset categories questioned by the reviewer affects the ranking. The table reports GraphPFN’s Elo rank among the same 21 main methods.

Dataset selectionDatasets10/10/8050/25/25
Full collection5193
Exclude small graphs (<500 nodes)4473
Exclude source-confirmed BoW4393
Exclude sparse lexical features37115
Exclude road-derived targets4795
Exclude scholarly-topic tasks3831
Exclude small graphs, sparse lexical features, and road targets2997

GraphPFN remains third under the higher label budget after excluding small graphs or confirmed BoW features. Its position changes more substantially with task selection, rising to first after excluding scholarly-topic tasks and falling to seventh after jointly excluding small graphs, sparse lexical features, and road-derived targets.


W2/Q4. What NodeArena establishes beyond prior evaluations.

The exclusions above show that model rankings depend on the task collection. NodeArena examines this dependence further by identifying conditions where reported GFM advantages hold and where they weaken.

Platonov et al. evaluate four classification and six regression datasets with tuned GNN baselines and computational-cost comparisons [2]. NodeArena evaluates 51 classification datasets and 15 supervised methods under two label budgets. Through this broader evaluation, we find that GraphPFN ranks ahead of GCN, GAT, and GraphSAGE under the higher-label split, but behind GCNII and GPRGNN, which were not evaluated in [2].

NodeArena also examines whether GFM advantages persist across different label and supervised tuning budgets. GraphPFN’s binary-task advantage coexists with weaknesses on high-dimensional and highly imbalanced datasets that remain even with more labels (Figures 6–7). Furthermore, comparisons against partially tuned methods show that only GraphPFN under the higher label budget extends the estimated performance–cost frontier (Figures 8–9). These findings are not established by evaluating one label budget and reporting tuning and inference costs alone.

The shared GraphLand datasets provide a concrete example of how we examine the scope of earlier findings. As noted in the review, GraphPFN leads in five of six dataset/split combinations on artnet-exp, city-reviews, and tolokers-2, the three datasets shared by both studies. To examine whether this strength extends to other conditions, we additionally divided all 51 datasets in NodeArena into low-, medium-, and high-alignment groups of 17 using feature-similarity alignment AUROC. All three shared datasets belong to the low-alignment group.

10/10/80 split

Alignment groupWins over GCNIIWins over GPRGNN
Low (17 datasets)16/1715/17
High (17 datasets)1/172/17

50/25/25 split

Alignment groupWins over GCNIIWins over GPRGNN
Low (17 datasets)16/1715/17
High (17 datasets)3/173/17

GraphPFN outperforms GCNII and GPRGNN on most low-alignment datasets, but on few high-alignment datasets under either label budget. NodeArena therefore shows that the strength observed on the shared tasks extends to other low-alignment datasets, and identifies conditions where it weakens. These additional findings cannot be inferred from agreement on the shared tasks alone.


References

[1] Bazhenov et al. GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Data. NeurIPS, 2025. Paper.

[2] Platonov et al. A Fair Evaluation of Graph Foundation Models for Node Property Prediction. 2026. Paper.

W1/Q2/Q3. Classification scope and regression.

We chose node classification as the starting point for a broad, standardized comparison of performance and cost because it is a central and widely studied task in node-level graph representation learning. We cannot add a complete regression evaluation within the current revision since it requires extensive additional experiments demanding substantial time and computational resources. However, we agree that node regression is an important extension and GraphLand’s seven regression datasets provide a basis for a future regression track. We will discuss this future extension in the camera-ready version, along with how the same evaluation principles of NodeArena can guide similar benchmarks for other graph tasks.

Dear Authors,

Thank you for your detailed response. However, my concerns currently remain unaddressed, so let me elaborate on the reasons below.

Crucially, I believe that the current positioning and promise to add a leaderboard with a submission system suggest that NodeArena is not just a "re-evaluation of different models in a broader setting", but also aims to become one of the field’s primary benchmarks for node-level graph tasks, similar to TabArena for tabular tasks. While the work itself and the responses are sufficient for the "re-evaluation" contribution, the "primary benchmark" positioning raises the quality bar substantially, as it will guide model development in the field.

From that perspective, the authors do not sufficiently explain why "more diverse datasets, but of lower quality" is better than "fewer datasets, but of higher quality". I believe that if our final goal is the application of graph ML methods in the real world, model development should be guided by more realistic datasets. Broader evaluations may complement this, but should not become the main guide.

Similarly, the lack of node regression support is mostly fine for the "re-evaluation" positioning, but becomes a critical concern for the "primary benchmark" positioning. Historically, our community has focused on classification tasks, which is arguably a bad practice since regression tasks are just as important, but this was acceptable due to the lack of appropriate datasets. Now, we have such datasets, and I see no reason to continue this bad practice. While I acknowledge that adding full regression support is hardly possible during the discussion period, simply committing to adding a discussion is not enough. To address this concern, the authors should at least commit to adding full regression support to the camera-ready revision.

So, in short, I believe that the current work is solid for the "re-evaluation" positioning, but I cannot agree with its "primary benchmark" positioning.

Below are some minor follow-up comments.

W1/Q1. What was the motivation for selecting this specific combination of criteria for the last row? Can you report the results with datasets excluded based on a combination of all five criteria?

W2/Q4. Thank you for this clarification. I recognize and appreciate your paper's new insights from broader dataset coverage. I intended the original question to mean something more like "if you limit your datasets to the three shared with Platonov et al. (2026), do you have any substantial new insights?" since, according to Table 22, adding GCNII and GPRGNN does not affect the conclusion of Platonov et al. (2026) about GNN vs. GraphPFN performance. Sorry for the insufficient clarity of the original question. Also, I now think that if you position your paper as a "re-evaluation of different models in a broader setting", the lack of new insights on these three datasets is completely fine since you have new insights on a broader dataset collection.

Best regards,

Reviewer z1Bj

Reviewer_vSmGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Node Arena, a node classification benchmark that tests whether graph foundation models can replace GNNs trained and tuned for each dataset. It compares six GFMs with 15 supervised method, each run with default settings and tuned over up to 201 configurations, on 51 datasets from six domains. All methods share two label splits with five fixed partitions each, and the benchmark measures accuracy, ranking (Elo) and wall-clock cost. Tuned GCNII and GPRGNN rank first, and GraphPFN is the strongest GFM. Against supervised methods given the same time budget, only GraphPFN with many labels stays on the accuracy-cost frontier, so the authors conclude current GFMs are not yet broad replacement for per-dataset training.

Critical strengths
  1. A fair comparison on GNN and GFMs, including costs.
  2. While the results are negative, the papers points out where the model breaks down. The paper has trial-level results, splits and a leaderboard, which is helpful for future work.
Critical weaknesses
  1. The test condition favors tuned GNNs and did not test few-shot, zero shots or distribution shift.
  2. The test aggregation may affect the ranking. Near duplicate tasks are weighted equally and when a model fails, its results is replaced with a default GCN instead of scored as a loss.
Questions for authors
  1. Do conclusion still hold where GFMs are supposed to help, e.g., few-shot training?
  2. Will the method ranking survive if we change how we rank? e.g., give each graph family one vote, or scoring GFM failures as losses?
3 comments

We thank the reviewer for the feedback. Below, we address the questions about few-shot evaluation and how graph-family weighting and failure handling affect model rankings.


W1/Q1. Evaluation scope and few-shot performance.

Few-shot learning is an important setting for GFMs, but we began with the 10/10/80 and 50/25/25 splits because recent GFMs already report competitive or superior performance against dataset-specific GNNs in semi-supervised and supervised node classification [1–6]. Our priority was to test how broadly these claims hold across diverse datasets and label budgets, using strong, consistently tuned baselines and explicit computational-cost accounting.

Our results do not establish whether the same ordering holds under few-shot evaluation. However, we view the standardized protocol, tuned baselines, and per-dataset results of NodeArena as a starting point for extending this comparison to few-shot learning and assessing where GFMs gain an advantage as labels become scarce.


W2/Q2. How evaluation choices affect model rankings.

Graph-family weighting. We grouped the 51 dataset representations into 24 graph families using their source provenance. Each graph family receives equal total weight, divided equally among its dataset representations. Rank entries report Elo positions under 10/10/80 and 50/25/25 splits. Kendall’s τb\tau_b measures agreement with the corresponding baseline ranking across all 21 main methods.

WeightingGCNIIGPRGNNGraphPFNKendall's τb\tau_b
Per dataset (original)1 / 12 / 29 / 3—
Per graph family1 / 12 / 46 / 20.886 / 0.867

Family weighting improves GraphPFN's position, including from third to second under the higher label budget. Thus, source composition affects the exact ordering, but the strongest tuned GNN remains first.

Failure handling. We compared GCN fallback with failure-as-loss on a fixed cohort of 50 datasets, excluding OGBN-Arxiv for all methods to separate pretraining-overlap exclusions from execution failures.

Failure policyGCNIIGPRGNNGraphPFNKendall's τb\tau_b
Default GCN fallback1 / 12 / 28 / 3—
Failure as loss; both fail: tie1 / 12 / 28 / 30.971 / 0.981
Failure as loss; both fail: omit1 / 12 / 28 / 30.971 / 0.971

The three methods retain their positions, and the full rankings remain highly similar. GraphPFN also produced native outputs for all 510 evaluations (51 datasets × 2 label regimes × 5 partitions), so none of its scores use GCN substitution.


References

[1] Zhao et al. Fully-inductive Node Classification on Arbitrary Graphs. Paper.

[2] Lee et al. View Space: Learning Representation across Arbitrary Graphs. Paper.

[3] Eremeev et al. Turning Tabular Foundation Models into Graph Foundation Models. Paper.

[4] Eremeev et al. GraphPFN: A Prior-Data Fitted Graph Foundation Model. Paper.

[5] Choi et al. Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors. Paper.

[6] Lee and Yoo. Node4All: Learning Node Representation Beyond Datasets. Paper.

Thanks for the update. Please call out the scope of the tests in the main paper and update the paper experiment results in the paper.

Thank you for the suggestion. We have clarified the evaluation scope in Section 4.1 (p. 5), including that the reported model ordering does not necessarily extend to few-shot, zero-shot, node-inductive, or distribution-shift settings. We have also added the graph-family weighting and failure-as-loss analyses to Appendix A.1, Tables 7–8 (p. 24), and included a reference to these analyses in Section 5.1 (p. 7).

Reviewer_gGx1OpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces NodeArena, a benchmark comparing six graph foundation models with 15 supervised methods across 51 node-classification datasets. It evaluates predictive performance and computational costs using shared data splits and standardized model selection. The results show that well-tuned GNNs remain strong overall compared with graph foundation models.

Critical strengths
  1. The benchmark includes diverse datasets and baselines, and extensive experiments across computational budgets and dataset properties. Including well-tuned supervised methods makes the comparison more informative than evaluations against default configurations alone.

  2. The paper is well structured and examines both predictive performance and computational cost. The breakdowns by task type and graph properties help explain where current GFMs and GNNs succeed or struggle.

Critical weaknesses
  1. The work does not test inductive prediction on nodes during target-task training or adaptation. This limits the benchmark’s comprehensiveness and the applicability of its conclusions to deployment settings with newly arriving nodes.

  2. The benchmark uses random stratified splits within each graph rather than controlled distribution shifts such as graph topology and node features. Such shifts are important in practice. Pretraining of GFMs may provide advantages under these conditions, but the current experiments cannot establish whether it does. OOD evaluations would therefore be valuable before drawing broader conclusions about the practical benefits of GFMs.

Questions for authors
  1. Would GNN still gain advantages to GFMs in inductive and OOD settings? I believe this is an important problem since many real-world datasets are inductive, or with different kinds of distribution shifts, and a founadtion model may have advantages in these settings.
1 comment

We thank the reviewer for recognizing the breadth of our evaluation and for highlighting node-inductive prediction and distribution shifts as important settings for assessing GFMs.


W1. Why we begin with transductive evaluation.

Node-inductive prediction is important for practical deployment, but we began with transductive node classification because it is an established evaluation setting for GNNs, graph Transformers, and GFMs, and recent GFMs already report competitive or superior performance in this setting [1–6]. Our priority was to test how broadly these claims hold across diverse datasets, using strong, consistently tuned baselines and explicit computational-cost accounting. We view this standardized comparison as a starting point for extending evaluation to node-inductive prediction.


W2/Q1. Relative performance in inductive and OOD settings.

Our current results do not establish whether GNNs would retain their advantages under node-inductive prediction or controlled distribution shifts. However, NodeArena provides reusable baselines and results that can support these broader investigations. The tuned baselines provide a reproducible reference for evaluating subsequent methods. Raw per-dataset results and property metadata allow researchers to examine subsets relevant to their objectives, identify model weaknesses, and select conditions for further investigation. Together with the released implementations and evaluation protocols, these resources reduce the work needed to extend the benchmark to new settings.

The manuscript explicitly limits its coverage to transductive node classification under random stratified splits. We will expand the future-work discussion in the camera-ready version to describe extensions to node-inductive prediction and controlled distribution shifts.


References

[1] Zhao et al. Fully-inductive Node Classification on Arbitrary Graphs. ICLR, 2025. Paper.

[2] Lee et al. View Space: Learning Representation across Arbitrary Graphs. ICML, 2026. Paper.

[3] Lee and Yoo. Node4All: Learning Node Representation Beyond Datasets. KDD, 2026. Paper.

[4] Eremeev et al. Turning Tabular Foundation Models into Graph Foundation Models. Paper.

[5] Eremeev et al. GraphPFN: A Prior-Data Fitted Graph Foundation Model. Paper.

[6] Choi et al. Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors. ICLR, 2026. Paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary for the Area Chair

Dear Area Chair,

Thank you for overseeing the review process. We summarize the reviewers’ assessments, the main concerns raised, and how we addressed them during the rebuttal.


(1) Contribution and additional insights.

Reviewer JwkF recognized the value of establishing the performance limits of current GFMs but questioned what Figures 6–8 reveal beyond the aggregate ranking. We explained how these analyses identify GraphPFN’s binary-task advantage, weaknesses on high-dimensional and highly imbalanced datasets that persist with more labels, and competitive computational tradeoffs overlooked by predictive rankings alone.

Reviewer z1Bj questioned the contribution beyond Platonov et al. We explained how broader architecture coverage identifies GCNII and GPRGNN as stronger alternatives absent from that evaluation. Comparisons across label and tuning budgets further establish that only GraphPFN under the higher label budget extends the estimated performance–cost frontier (Figures 6–9). We also showed how GraphPFN’s strength on the shared datasets extends to some conditions and weakens in others.


(2) Dataset selection and ranking robustness.

Reviewer z1Bj questioned whether the dataset collection drives the conclusions and requested exclusion analyses. We evaluated the questioned categories individually and jointly. Applying all five criteria retains 20 datasets and improves GraphPFN’s low-label Elo rank from ninth to second, while its high-label rank remains third. GCNII remains first under both splits after all five exclusions. These results acknowledge sensitivity to dataset composition while confirming GCNII’s lead on the retained subset. They are included in Appendix A.1, Table 6.

Reviewer vSmG separately questioned equal weighting of related datasets and the use of GCN fallback for failed runs. We recomputed rankings with equal graph-family weights and with failures scored as losses. Family weighting changes some relative positions, while failure-as-loss produces highly similar rankings. GCNII remains first under both splits in both analyses. These results are included in Appendix A.1, Tables 7–8, and referenced in Section 5.1.


(3) Motivation for dataset properties.

Reviewer JwkF requested justification for nonzero fraction, label imbalance, label entropy, and the property pairs in Figure 4. We explained how these properties characterize feature sparsity, minority-class scarcity, and the overall label distribution. We also clarified why the selected pairs examine complementary aspects of graph structure, features, and labels whose joint coverage cannot be established from individual ranges.


(4) Benchmark positioning and evaluation scope.

Reviewers gGx1 and vSmG asked whether the observed ordering extends to inductive, OOD, or few-shot settings. We explained the rationale for testing recent GFM claims in standard transductive classification settings and clarified the scope in Section 4.1. Following gGx1’s request, we expanded the limitations discussion to address node-inductive prediction and controlled distribution shifts.

Reviewer z1Bj questioned the evaluation value of datasets with limited immediate industrial relevance and whether NodeArena should guide future model development. We clarified that these datasets help test whether reported GFM advantages persist across different graph conditions, including conditions where our analyses reveal substantial weaknesses. This diagnostic value motivates NodeArena’s role alongside application-driven benchmarks. We revised the Introduction and limitations discussion to make this complementary role explicit, while identifying regression and other graph tasks as extensions. These manuscript revisions are highlighted in blue.


Judging where GFMs stand today requires testing whether their reported advantages persist across diverse graphs against strong, well-tuned supervised baselines, and at what computational cost. NodeArena provides this evidence through a broad, controlled comparison that establishes both the reach and the limits of current GFM competitiveness. These results can serve as a reproducible reference for future comparisons of GFM predictive performance and computational cost.

Thank you again for your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.