acceptodds
Withdrawn from ICLR 2027

The Post-GCN Decade Revisited: Curvature-Stratified Evaluation of Relational Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Current evaluation practices in relational learning rely heavily on flat leaderboards that average performance across heterogeneous datasets, implicitly assuming a uniform underlying structure. We show that this assumption introduces systematic bias: it obscures geometry-dependent performance variations and can lead to misleading conclusions about model generalization. In this work, we identify intrinsic geometry as a key latent factor governing model effectiveness. We demonstrate that conventional aggregated metrics mask critical performance trade-offs that only become visible when datasets are stratified by their geometric properties. To address this issue, we introduce a curvature-stratified evaluation framework that partitions datasets into positive, negative, and near-zero curvature regimes. Our benchmark evaluates 18 representative models—including Graph Convolutional Networks (GCNs), Graph Foundation Models (GFMs), and tabular learning methods—across 14 datasets. We find that model rankings are highly stable within each curvature regime but shift significantly across regimes, indicating that performance is fundamentally geometry-dependent rather than universally transferable. Notably, we identify regimes where GFMs offer diminishing returns compared to geometry-aligned GNNs. Based on these findings, we propose a geometry-aware evaluation protocol that yields more reliable and interpretable comparisons than standard aggregated benchmarks. We release all code, curvature-stratified dataset splits, and evaluation tools to support reproducible and rigorous assessment of future relational learning methods.

Primary area:
datasets and benchmarks
Keywords:
GNNs, relational learning, data geometry, evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fiW1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper the authors present an evaluation study on intrinsic geometry playing a key role for a model to be effective. The authors argue that standard metrics may present important performance trade-offs that can be seen when the datasets are organized per geometry. As a solution, they introduce here a curvature-stratified evaluation framework that groups datasets into different curvature-based regimes. The benchmark evaluates a sufficient number of representative models across 14 datasets. The authors observe that the models are highly stable within a curvature type but behave different across, show that geometry affects performance.

Critical strengths
  • An important study on the geometry of graph datasets playing a significant role on evaluation
  • Curvature skewness as a descriptor. The table-derived graphs show that mean-zero, heavily skewed graphs behave differently.
  • Table-derived graphs for geometry analysis.
Critical weaknesses
  • Missing related work
    • Shedding Light on Problems with Hyperbolic Graph Learning (TMLR 2025): states "misleading metrics are used to quantify geometry of graph datasets."
    • HypBench (NeurIPS 2025 workshop)
    • GraphWorld (Google, KDD 2022)
  • Limited number of datasets per regime (three).
  • Some regime boundaries are borderline. The near-zero rule is |mean| < 0.01 and |skew| < 0.5. Cornell’s mean is 0.0105, just over that line. Airport’s mean (0.002) is flat, and it lands in positive only because of its skewness.
  • Small or label-noisy datasets. Cornell is very small, and CS_Phds was originally a regression task whose labels were quartile-discretized into 4 classes. Accuracies near 26% on CS_Phds are close to chance, so rankings there are mostly noise.
  • The curvature effect may really be a homophily effect. The near-zero set is all high-homophily citation graphs, and the positive set is mostly heterophilic. The authors include PCNet partly to address this, but they don’t fully disentangle the two.
  • Four of the five table-derived graphs have near-zero mean curvature but extreme positive skew, while F1 has a large positive mean with negative skew. They form a fourth “tail-driven” bucket rather than fitting the three regimes.
Questions for authors
  • Why was almost no related work added for evaluation?
  • How are you reassure that the curvature effect is actually only curvature?
Reviewer_WmJLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CURVBENCH, a curvature-stratified evaluation framework for relational learning. It characterizes graph geometry using a shortest-path-based midpoint curvature residual and partitions datasets into near-zero, positive, and negative curvature regimes based on mean curvature and skewness. The authors evaluate 18 models, including conventional GNNs, geometry-aware GNNs, and Graph Foundation Models (GFMs), across 14 relational datasets. The experimental results show that model rankings are more consistent within curvature regimes than across them, revealing performance variations that are obscured by conventional aggregated leaderboards. Based on these findings, the paper advocates curvature-aware evaluation for more informative comparisons of relational learning models.

Critical strengths

S1. Novel and insightful curvature-based evaluation framework.

A key strength of this paper is the novel use of graph curvature to stratify relational learning benchmarks, offering an interesting geometry-driven perspective on model evaluation. The results in Section 5.1 are particularly compelling, showing that model rankings are substantially more consistent within curvature regimes than across them. These findings provide empirical support for the proposed curvature-based stratification and offer valuable new insights into the relationship between graph geometry and model effectiveness in relational learning.

S2. Comprehensive evaluation across different model families.

The paper provides a broad empirical evaluation covering diverse model families, including Euclidean GNNs, non-Euclidean architectures, and recent graph foundation models. This extensive comparison offers a useful perspective on how different geometric inductive biases affect model performance across relational datasets.

Critical weaknesses

W1. Insufficient justification for the separate treatment of table-derived graphs.

The central premise of this paper is to stratify relational datasets based on their intrinsic curvature. However, Section 3.1 treats table-derived graphs as a separate category based on their data origin, rather than applying the same curvature-based criteria used for natural graphs. Although the authors justify this decision by the highly skewed curvature distributions of table-derived graphs, it remains unclear why these datasets cannot be accommodated within a unified curvature-based stratification framework, especially since curvature skewness is already considered in the proposed criteria. This additional categorization rule appears somewhat ad hoc and undermines the conceptual consistency and general applicability of the proposed evaluation framework.

W2. Insufficient analysis connecting GFM architectures to curvature-dependent performance.

While Section 5.2 presents interesting observations on the curvature-dependent performance of GFMs, the analysis remains largely descriptive and does not sufficiently explain how the architectural characteristics or transfer mechanisms of individual models relate to these findings. For example, the paper attributes the strong performance of MDGPT and MDGFM in near-zero curvature regimes to token- or prompt-based alignment, despite their different transfer mechanisms. Similarly, GraphGluing's advantage in the positive-curvature regime is largely driven by a single dataset (Airport). Without a more systematic analysis connecting model-specific inductive biases to the observed performance patterns, it is difficult to understand why particular GFMs perform well or poorly under different curvature regimes. This limits the explanatory value of the GFM analysis.

Questions for authors

Q1. What is the justification for treating table-derived graphs as a separate category rather than applying a unified curvature-based stratification framework, particularly given that curvature skewness is already considered in the proposed criteria?

Q2. Could the authors provide a more detailed explanation of how the architectural characteristics or transfer mechanisms of individual GFMs relate to their performance across different curvature regimes?

Reviewer_BHM8OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces CURVBENCH, which evaluates relational learning models by grouping datasets using the mean and skewness of a shortest-path midpoint residual. It compares 18 models on 14 datasets to examine ranking differences and the limitations of aggregate leaderboards.

Critical strengths
  • The motivation is clear. The paper investigates whether aggregate rankings obscure differences in model performance across graph datasets.

  • Extensive experiments reveal performance variation across the evaluated datasets and curvature groups, providing useful empirical observations for model comparison.

Critical weaknesses
  • The paper mainly groups existing datasets and evaluates existing models. The results show that models perform differently across these groups, but what new evaluation perspective or model-selection insight does the proposed grouping provide beyond comparing model performance across dataset groups?

  • The rationale for the dataset partition is insufficiently justified. Why are the operational midpoint residual, mean, and skewness appropriate for characterizing the geometry relevant to model performance?

  • The dataset-level partition may obscure within-graph curvature heterogeneity.

    • A graph containing both positive- and negative-curvature regions can have a near-zero mean and skewness, yet differ substantially from a graph whose node curvatures are concentrated near zero.
    • How does the proposed grouping distinguish these cases, and how should the performance of models such as GraphMoRE be interpreted when multiple curvature regimes coexist within one graph?
Questions for authors
  • Please see the Weaknesses.

  • Were the regime thresholds fixed before examining the model results, and how stable are the conclusions to nearby thresholds?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.