acceptodds
Under review as a conference paper at ICLR 2027

SimPlex-GT: Node-to-Cluster Attention for Graph Learning under Mixed Homophily and Heterophily

Abstract

Real-world graphs frequently exhibit mixed homophily and heterophily, where local message passing and global attention provide complementary but often conflicting signals. Existing graph transformers (GTs) and GNN–GT hybrids combine these signals at a single level of abstraction, leading to interference and unstable training. We propose **SimPlex-GT**, which decouples the two by routing global reasoning through a small set of learned semantic prototypes rather than full node-to-node attention. Our **node-to-cluster (N2C) attention** computes prototypes per forward pass as a differentiable function of input features—distinguishing it from prior tokenization-based GTs that rely on free-parameter latents or codebook-style update rules—and replaces pairwise attention with *exact* attention against prototypes at cost. A lightweight GCN branch supplies local inductive bias, and three regularizers stabilize the global branch: complementary filtering of input features, cluster smoothing over the coarse graph induced by soft assignments, and a teacher–student self-supervised objective that prevents prototype collapse. The model is pretrained without labels and evaluated with a frozen linear probe. Across 14 node-classification, 3 graph-classification, and 4 node-clustering benchmarks (17 datasets in total), SimPlex-GT improves over the strongest baseline in our comparison by **+5.95** points on Squirrel-filtered, **+0.69** points on Penn94, and **+2.45** points on Amazon-Ratings, matches or improves on homophilic Cora/CiteSeer/PubMed, and trains faster in total wall-clock time than NodeFormer (**2.3×**, from faster convergence at comparable per-epoch cost) and GREET (up to **8.5×**). Controlled ablations isolate the input-conditioned prototype construction: replacing it with freely learned prototype vectors costs 2.9–8.3 points across heterophilic, mixed, and homophilic graphs. The results suggest that the bottleneck in scalable graph attention is *what to attend to*, not how cheaply to approximate full attention.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Transformers, Graph Neural Networks, Mixed Homophily and Heterophily, Node-to-Cluster Attention, Self-Supervised Graph Learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_n87dOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SimPlex-GT - a GNN/GT hybrid model designed for node classification in graphs with heterophily and mixed homophily/heterophily. The model consists of 2 information streams: one is a classic GCN, the other is based on attention to learnable semantic prototypes (which is more efficient than attention between all pairs of nodes common in GTs). The model uses late fusion for the two streams. The models is further enhanced with complementary filtering of features, prototype smoothing, and orthogonality regularization for stream fusion. Experiments are conducted on a number of node classification, graph classification, and node clustering benchmarks.

Critical strengths

The general approach of using semantic prototypes as a way to implement efficient and scalable global attention for GTs is interesting and might be useful.

Critical weaknesses

Motivation. The entire approach is motivated by the idea that standard GNNs do not perform well under heterophily or mixed homophily/heterophily. However, this idea is not correct: it has been shown many times in prior literature that classic message-passing GNNs perform perfectly fine on heterophilous graphs, typically outperforming heterophily-specific models [1-6]. The paper claims that "on heterophilic graphs, neighbors of v frequently carry labels different from v, so any local averaging contracts v’s representation toward an uninformative or actively misleading mean". However, standard GNNs or GTs do not even have access to nodes' labels - local averaging averages feature information, rather than label information, and dependencies between features and labels can be very complicated, and there can be a lot of useful information about nodes' labels in neighborhood features even in heterophilous graphs. Further, since the entire motivation of the approach is to design a model for learning under heterophily or mixed homophily/heterophily (which are properties of node classification datasets), it is not clear why the paper also conducts experiments on graph classification (where there are no node labels at all). If the approach indeed improves performance on graph classification, then surely there is a mismatch between the entire motivation of the paper and the actual inner workings of the model.

Further, there are several flaws in the experimental setup.

Datasets. The dataset selection for experimental evaluation is very poor. Most of the datasets used have been criticized many times. Most notably, standard citation networks (cora, citeseer, pubmed, ogbn-arxiv) have a lot of mislabeled nodes [7], texas, cornell, wisconsin are too small and do not have enough class representation [3], squirrel-filtered and chameleon-filtered were meant to demonstrate the flaws of the original squirrel and chameleon datasets, not to be used as new datasets [8]. Overall, there has recently been a lot of concern about graph machine learning papers using flawed and unreliable datasets [9, 10] and better benchmarks have been proposed [10, 11, 12]. Datasets from these benchmarks should be used instead of outdated and unreliable datasets used in this paper.

Baselines. It has been shown that classic GNNs like GCN and GAT achieve state-of-the-art results on node classification in heterophilous graphs when simple architetcural improvements like skip-connections, dropout, and normalizations are added to them [3, 5] and proper hyperparameter tuning is performed [5, 8]. These stronger implementations of classic models should be used as baselines. The numbers reported in this work for GCN and GAT show that weaker and/or untuned model implementations are used.

Overall, while the method proposed might be promising, its true benefits do not appear to be related to the provided motivation, and poor dataset and baseline selection for the experimental evaluation does not allow to reliably evaluate its performance compared to existing models.

[1] Is homophily a necessity for graph neural networks? (ICLR 2022)

[2] Revisiting heterophily for graph neural networks (NeurIPS 2022)

[3] A critical look at the evaluation of GNNs under heterophily: Are we really making progress? (ICLR 2023)

[4] Characterizing Graph Datasets for Node Classification: Homophily–Heterophily Dichotomy and Beyond (NeurIPS 2023)

[5] Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification (NeurIPS 2024)

[6] Oversmoothing, Oversquashing, Heterophily, Long-Range, and more: Demystifying Common Beliefs in Graph Machine Learning (ICLR 2026)

[7] GraphCleaner: Detecting Mislabelled Samples in Popular Graph Learning Benchmarks (ICML 2023)

[8] A Fair Evaluation of Graph Foundation Models for Node Property Prediction (2026)

[9] Graph Learning Will Lose Relevance Due To Poor Benchmarks (ICML 2025)

[10] GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Data (NeurIPS 2025)

[11] GraphBench: Next-generation graph learning benchmarking (2025)

[12] Bridging Academia and Industry: A Comprehensive Benchmark for Attributed Graph Clustering (2026)

Questions for authors

Can the authors address the issues raised in weaknesses?

Reviewer_cYYDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SimPlex-GT, a graph transformer for learning node representations on graphs with mixed homophily and heterophily. It softly groups nodes into a small set of prototypes, recomputed from the input features at every forward pass, and lets each node attend to these prototypes. It introduces two alternative variants: complementary filtering, which builds prototypes from smoothed features and queries from residual features, and cluster smoothing, which lets prototypes exchange information through a coarse graph induced by the soft assignments. A local GCN branch complements the global attention branch; their outputs are added, with an orthogonality penalty discouraging alignment between them. The model is pretrained without labels through masked student–teacher learning, then evaluated on node classification by training a linear classifier on frozen node representations. Additional experiments cover node clustering and graph classification.

Critical strengths
  1. Attending to a small number of prototypes reduces the cost of global attention while retaining an output for every node. Combining this with a local GCN branch gives the model access to both nearby structure and broader information.
Critical weaknesses

The paper contains soem differences from earlier methods, including its particular complementary-filtering design and the integration of prototype construction, coarse-graph smoothing, and branch regularization. However, the benefit of these choices over closely related methods is not sufficiently established. Gapformer [1] already provides node-specific attention to pooled keys and values, while CoBFormer [2] provides a related local/global architecture. Comparisons with these methods would clarify the architectural contribution. Separately, the training also closely matches HarmonyGNNs, which should be cited and evaluated as a particularly relevant self-supervised baseline.

  1. Gapformer [1] already lets each node's own query attend to pooled keys and values, so node-specific queries do not distinguish SimPlex-GT from Gapformer. CoBFormer [2] already combines a GCN with cluster-based Transformers and motivates this design through problems with global attention. SimPlex-GT differs from CoBFormer in how global information reaches individual nodes, but the broader motivation and combination overlap.

The coarse graph construction also follows established pooling methods, including MinCutPool [3], as the paper acknowledges in Appendix C.1. The remaining contribution therefore needs to be explained more precisely around the particular CF and CS operations and their integration.

More importantly, the training process in Section 3.4 closely matches HarmonyGNNs [4], previously called H$^3$GNNs. This includes the full-graph EMA teacher, masked student, all-node squared prediction loss, and discrepancy-based masking. Equation (10) matches its masking rule -- Equation (4) in the published paper and Equation (17) in the earlier version. This prior work is neither cited nor compared against.

  1. The experiments do not adequately establish the benefit over the closest alternatives.

Comparisons with Gapformer, CoBFormer, and especially HarmonyGNNs are important for assessing the contribution. HarmonyGNNs address the same homophily–heterophily setting under a frozen linear-probe protocol. Their omission also affects the interpretation of the headline improvements.

The evaluation should better separate gains from the encoder design and the orthogonality penalty. For CS, useful alternatives would be a learned multilayer GCN on the same coarse graph or self-attention among the same prototypes. CS already performs one step of weight-free residual coarse-graph propagation, so simply asking for comparison with a coarse GCN would be imprecise.

References:

[1] Gapformer: Graph Transformer with Graph Pooling for Node Classification. IJCAI, 2023 [2] Less is More: on the Over-Globalizing Problem in Graph Transformers. ICML, 2024 [3] Spectral Clustering with Graph Neural Networks for Graph Pooling. ICML, 2020 [4] HarmonyGNNs: Harmonizing Heterophily and Homophily in GNNs via Self-Supervised Node Encoding. ICLR, 2026

Questions for authors

Please refer to Weakness.

Additional Questions:

  1. What specific contribution remains beyond Gapformer's node-to-pooled attention, CoBFormer's local/global combination, and the acknowledged pooling precedents? Please identify the differences in the actual operations.

  2. Please explain the relationship between the whole training recipe in Section 3.4 and HarmonyGNNs, including the loss, teacher update, masking rule, and warm-up. Which elements are adopted, modified, or new?

Reviewer_vPULOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SimPlex-GT, a graph transformer architecture designed to handle mixed homophily and heterophily. Specifically, it proposes Node-to-Cluster attention to replace quadratic node-to-node attention with attention over a small set of dynamically learned semantic prototypes. A parallel GCN branch captures local structural information, while complementary filtering, cluster smoothing, and self-supervised learning stabilize prototype representations. Experiments across 17 datasets demonstrate competitive performance on homophilic graphs and improvements on several heterophilic and mixed-pattern benchmarks.

Critical strengths
  • The architecture combines prototype-level global attention with local GCN aggregation, achieving competitive performance across different homophily regimes.
  • Node-to-Cluster attention reduces quadratic attention complexity, enabling scalable global information aggregation. The experiments also demonstrate reduced training time compared with several baselines.
  • Extensive empirical evaluation. The paper evaluates 17 datasets across node classification, graph classification, and clustering, with ablations supporting the effectiveness of its major architectural components.
Critical weaknesses
  • The prototype construction closely resembles DiffPool's differentiable soft assignment. While Table 6 compares against free-parameter prototypes, it lacks comparisons with alternative input-conditioned prototype construction methods, making the novelty and necessity of the proposed design unclear.
  • Appendix O analyzes node-level homophily distributions on Actor and Tolokers, but reports no performance breakdown by homophily level. Consequently, it remains unclear whether SimPlex-GT effectively combines local and global information for both homophilic and heterophilic nodes within the same graph.
  • Limited evidence for the claimed local–global interference. The paper identifies interference between node-level local and global signals as a key limitation of existing GNN–GT hybrids. However, Table 6 only ablates individual components and does not compare different fusion granularities under controlled settings, leaving this central motivation insufficiently validated.
Questions for authors

Refer to weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.