acceptodds
Under review as a conference paper at ICLR 2027

One Coupling, Many Roles: Readout-Dependent Accessibility in GNN Representations

Abstract

Graph neural network representations with nearly identical supervised linear accessibility can differ sharply in single-prototype recovery, and even a single frozen representation can remain highly linearly accessible while systematically failing single-prototype recovery. On GIN-Cora, we localize this deficit to a single-prototype capacity mismatch within the class-centroid span. This prototype-sensitive failure is accompanied by a recurring degree-conditioned organization on Cora and Amazon-Computers. Training-time manipulation of target-degree message scaling causally shifts the accessibility profile: attenuation in GIN improves prototype accessibility but has different supervised consequences across datasets, whereas amplification in GraphSAGE can induce a large prototype–linear dissociation while largely preserving linear accessibility. These results show that target-degree scaling is a causal lever on representation accessibility, while accessibility itself is readout-dependent rather than a single scalar property of the representation.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Neural Networks, Representation Learning, Representation Geometry, Degree Normalization, Representation Accessibility

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RcU1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether the quality of graph neural network (GNN) representations can be assessed using a single evaluation metric, a question of theoretical significance for graph representation learning. Focusing primarily on GIN and GraphSAGE on Cora, the paper considers three readout mechanisms for evaluating representations: unsupervised prototype recovery and supervised centroid-based and linear-probe evaluation. It introduces the perspective of readout-dependent representation accessibility and further attempts to explain the performance discrepancies across readout mechanisms through subspace analysis, multi-prototype experiments, analyses of degree-related patterns, and training-time interventions that scale the target nodes' degrees in the message-passing operators. Based on its empirical findings, the paper draws some constructive conclusions:

  1. The prototype recovery deficiency of GIN on Cora is associated with the capacity limitations of a single prototype.
  2. Scaling target-node degrees during training can causally alter representation accessibility.

Overall, the paper is primarily an empirical study rather than a methodological contribution.

Critical strengths
  1. The paper addresses a clearly defined research question, follows a coherent experimental design, and presents a complete chain of reasoning.

  2. The experiments report results across multiple runs and use stratified Bootstrap to estimate confidence intervals. These practices enhance the credibility of the experimental findings and make the statistical comparisons more convincing than those based on single-run results.

  3. The paper acknowledges several important limitations and carefully delineates the scope of its conclusions, avoiding the overinterpretation of metric changes as a comprehensive causal explanation of the underlying mechanisms.

  4. The findings offer useful insights into the diagnosis and evaluation of GNN representation learning.

Critical weaknesses
  1. Would the gap in prototype recovery between GIN and GraphSAGE persist if K-Means were replaced with alternative clustering methods, if different distance metrics were used, or if graphs with different characteristics and scales were considered?

  2. The current intervention demonstrates that modifying degree scaling can alter representation accessibility. However, is node degree merely a correlated factor, or does it help explain the underlying failure mechanism?

Questions for authors
  1. The paper's primary empirical evidence comes from two homophilous graphs, Cora and Amazon-Computers. It remains unclear whether the same phenomena persist in heterophilous graphs, heterogeneous graphs, or large-scale graphs with millions of nodes. Moreover, the current study focuses exclusively on node classification. It is unclear whether similar findings would hold for node regression, edge-level tasks, or graph-level tasks. If these settings are beyond the scope of the paper, the authors should frame their work as identifying and diagnosing a phenomenon in specific graph-based node classification settings, rather than implying broader applicability to GNN representation learning in general.

  2. I am curious why the authors chose K-Means as the sole clustering algorithm, despite its popularity. Given that increasing the number of prototypes improves performance, the authors should consider whether non-convex clustering algorithms might be more appropriate. Comparisons with alternative clustering algorithms, different distance metrics, and nonlinear unsupervised readout methods would strengthen the evaluation. If the observed discrepancies persist under these settings, the paper's claims regarding readout dependence would be considerably more convincing.

  3. The paper demonstrates how to identify and localize problems in representations, but its contributions remain primarily at the level of diagnosis and evaluation. These findings have not yet been sufficiently translated into practical algorithmic improvements, which may limit the paper's immediate practical value.

  4. In homophilous graphs, nodes with higher degrees are expected to aggregate information from more same-label neighbors, which should, in principle, benefit node classification. Therefore, it is unsurprising that modifying degree scaling may affect predictive performance. The paper lacks intervention experiments covering a broader range of intermediate intervention strengths and a more diverse set of graph types, which limits the strength of the causal interpretation of the degree-scaling intervention.

Reviewer_ztdFOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper compares three readouts of frozen GNN embeddings: (1) K-Means with cluster-to-class mapping, (2) supervised nearest-centroid classification, and (3) supervised linear probes.

It finds that GIN embeddings can support strong linear classification while performing poorly under K-Means.

In addition, the authors show that additional prototypes improve recovery, and changing degree-dependent message scaling during encoder training changes the readout-performance profile.

Critical strengths
  • S1. The authors provide intuitive analysis.

  • S2. The authors use the widely-used GNN architectures.

Critical weaknesses
  • W1. Limited conceptual novelty. The central finding—that the same representation performs differently under clustering, nearest-centroid classification, and supervised linear classification—is unsurprising, given their different supervision and optimization objectives. The paper should clarify what generalizable insight it provides beyond documenting this discrepancy.

  • W2. Limited empirical scope. The main evidence focuses on GIN and GraphSAGE on Cora. The second dataset, Amazon-Computers, was selected because it already exhibited the relevant accessibility pattern. Although the authors transparently acknowledge this selection, broader evaluation is needed to establish the generality of their findings.

  • W3. Insufficient evidence for a prototype-capacity limitation. A single supervised prototype per class substantially outperforms K-Means on GIN-Cora (0.759 vs. 0.386), suggesting that prototype placement, rather than capacity alone, may explain the deficit. Improved recovery with additional clusters does not distinguish between these explanations.

Questions for authors

See Weakness.

Reviewer_oqZUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This is an apparent autoresearch paper, without even the tiniest level of human intervention. If you look at the introduction, you'll see an indescribable writing style that I could not understand why it even existed. I tried many times, but I cannot understand what the author is trying to say. Going through even the introduction is painful work. I cannot understand the paper, and I believe it is not worth any serious human review, as it is a straightforward disrespect to reviewers.

Critical strengths

None

Critical weaknesses

This is an autoresearch work with little human review. The readability and reliability are both bad.

Questions for authors

NA

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.