- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this paper the authors present an evaluation study on intrinsic geometry playing a key role for a model to be effective. The authors argue that standard metrics may present important performance trade-offs that can be seen when the datasets are organized per geometry. As a solution, they introduce here a curvature-stratified evaluation framework that groups datasets into different curvature-based regimes. The benchmark evaluates a sufficient number of representative models across 14 datasets. The authors observe that the models are highly stable within a curvature type but behave different across, show that geometry affects performance.
Critical strengths
- An important study on the geometry of graph datasets playing a significant role on evaluation
- Curvature skewness as a descriptor. The table-derived graphs show that mean-zero, heavily skewed graphs behave differently.
- Table-derived graphs for geometry analysis.
Critical weaknesses
- Missing related work
- Shedding Light on Problems with Hyperbolic Graph Learning (TMLR 2025): states "misleading metrics are used to quantify geometry of graph datasets."
- HypBench (NeurIPS 2025 workshop)
- GraphWorld (Google, KDD 2022)
- Limited number of datasets per regime (three).
- Some regime boundaries are borderline. The near-zero rule is |mean| < 0.01 and |skew| < 0.5. Cornell’s mean is 0.0105, just over that line. Airport’s mean (0.002) is flat, and it lands in positive only because of its skewness.
- Small or label-noisy datasets. Cornell is very small, and CS_Phds was originally a regression task whose labels were quartile-discretized into 4 classes. Accuracies near 26% on CS_Phds are close to chance, so rankings there are mostly noise.
- The curvature effect may really be a homophily effect. The near-zero set is all high-homophily citation graphs, and the positive set is mostly heterophilic. The authors include PCNet partly to address this, but they don’t fully disentangle the two.
- Four of the five table-derived graphs have near-zero mean curvature but extreme positive skew, while F1 has a large positive mean with negative skew. They form a fourth “tail-driven” bucket rather than fitting the three regimes.
Questions for authors
- Why was almost no related work added for evaluation?
- How are you reassure that the curvature effect is actually only curvature?