acceptodds
Under review as a conference paper at ICLR 2027

CoLIFT: Compatibility-Guided Graph-Free Distillation under Heterophily

Abstract

GNN-to-MLP distillation enables efficient graph-free inference while preserving rich information learned by GNNs. However, existing methods are developed primarily for homophilous graphs and rarely distinguish which neighborhood relations provide transferable supervision. This limitation becomes critical under heterophily, where observed neighborhoods mix structurally compatible relations with weakly supported or potentially misleading ones. To address this challenge, we propose CoLIFT, a COmpatibility-guided LIFTing framework that uses class compatibility as a structural proxy for GNN-to-MLP relation-level transferability. CoLIFT estimates compatibility from reliability-weighted teacher class-pair co-occurrences and corrects it for class prevalence. It then lifts these scores to node-pair sampling probabilities through capped exponential calibration that preserves each target node's expected relation budget. The sampled relations jointly define probability-weighted relation distillation and contrastive supervision that admits structurally supported cross-class positives. Across seven benchmarks, CoLIFT achieves the highest accuracy among the graph-free baselines, with a 6.00-percentage-point macro-average gain over the feature-only MLP on heterophilous graphs. Fixed-budget diagnostics and post-hoc analysis find that our calibration generally shifts probability mass toward structurally supported class pairs, with high-compatibility contexts yielding higher mean accuracy than low-compatibility ones on all seven datasets.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
GNN-to-MLP Distillation, Heterophilic Graph Learning, Graph-free Inference, Class Compatibility, Node Classification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xv2rOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

See three strengths stated at above. These strengthens are main contributions in addition to the comprehensive justifications via seven mathematical expressions.

Critical strengths
  1. A novel approach that Identifies relation-level transferability as an overlooked problem in graph-free distillation under heterophily graphs.
  2. A CoLIFT framework with compatibility-lifting mechanism estimates reliability-aware compatibility discovery weighted, prior-corrected class compatibility.
Critical weaknesses

Different datasets have been used to test classification results of Heterophilous Graphs and Homophilous Graphs (Table 1), comparison may not conclusive.

Questions for authors

Authors are required to conduct and report additional experiments using some same datasets for both types of graphs.

3 comments
AuthorsOpenReview ↗

Within-Dataset Evaluation Across Edge Homophily Levels

Thank you for recognizing the research problem and the compatibility-lifting mechanism, and for suggesting an evaluation at different edge homophily levels within the same dataset. We agree that differences in node features, graph size, class distributions, and data splits can affect comparisons across datasets. Table 1 evaluates CoLIFT on real-world homophilous and heterophilous benchmarks. To complement that evaluation, we conducted a within-dataset experiment on Cora and Actor.

For each dataset, we used degree-preserving edge swaps to vary edge homophily while keeping the nodes, node features, labels, train/validation/test splits, and individual node degrees fixed. We retained the original Cora graph and generated two variants with lower edge homophily. We retained the original Actor graph and generated two variants with higher edge homophily. We used the full labels to construct the semi-synthetic graphs and compute structural statistics. Given each graph, only training-node labels contributed directly to the supervised objective and compatibility estimation. We retrained GPR-GNN on each rewired graph. Within each matched run, HGMD (GPR) and CoLIFT shared the same frozen GPR-GNN teacher and student backbone initialization. Each method used a fixed configuration across homophily levels within each dataset.

We report mean test accuracy ± sample standard deviation over five runs per condition. The reported hedgeh_{\mathrm{edge}} is the mean observed edge homophily. The last column reports paired accuracy differences in percentage points (pp). Bold indicates the highest mean among the graph-free methods in each row.

DatasetGraphhedgeh_{\mathrm{edge}}GPR-GNN (%)HGMD (GPR) (%)CoLIFT (%)MLP (%)CoLIFT − HGMD (pp)
CoraRewired0.209943.38 ± 1.9050.90 ± 1.3651.44 ± 1.6459.26 ± 1.87+0.54 ± 0.64
CoraRewired0.509863.88 ± 0.9167.88 ± 0.8068.82 ± 0.4659.26 ± 1.87+0.94 ± 0.53
CoraOriginal0.810083.34 ± 0.7783.92 ± 0.8284.56 ± 0.5759.26 ± 1.87+0.64 ± 0.63
ActorOriginal0.216735.96 ± 0.3035.58 ± 0.8436.87 ± 1.0134.89 ± 0.68+1.29 ± 0.98
ActorRewired0.490055.76 ± 1.2551.83 ± 1.2752.25 ± 1.1134.89 ± 0.68+0.42 ± 0.55
ActorRewired0.790088.67 ± 0.6475.97 ± 0.7776.24 ± 0.8534.89 ± 0.68+0.26 ± 0.72

CoLIFT achieved higher mean accuracy than HGMD (GPR) in all six conditions, with gains of 0.26–1.29 pp. The largest mean gain occurred at the intermediate homophily level on Cora and on the low-homophily original graph on Actor. On Cora at hedge=0.2099h_{\mathrm{edge}}=0.2099, both distilled students performed below the feature-only MLP.

These results show that CoLIFT's average gains over HGMD (GPR) persist across the tested homophily levels within each dataset. Since rewiring also changes class-pair connectivity and multi-hop neighborhoods, we interpret the results within the scope of the graph construction and training protocol used here.

I have no new additional comments as the authors have addressed most of my suggestions and planned to incorporate them in the revised manuscript.

Thank you for taking the time to read our responses. Your constructive suggestions will help us strengthen the manuscript in revision. If you feel that our responses have addressed your main concerns, we would be grateful if you would consider raising your score. We would be happy to discuss any remaining concerns.

Reviewer_C2pDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper based on the claim that neighborhood relations can provide transferable supervision, proposed CoLIFT, a compatibility-guided lifting framework that uses class compatibility as a structural proxy for GNN-to-MLP relation-level transferability. CoLIFT uses class compatibility as a structural proxy signal. While keeping each node’s expected neighbor budget unchanged, it softly reallocates which neighbor relations go into the distillation context, and then adds relation-aware contrastive distillation. Experiments show that the proposed method is effective.

Critical strengths
  1. The paper is well written and the figures are nicely drawn. It is easy to follow.
Critical weaknesses
  1. Limited technical novelty. The framework is mostly a combination of existing components. The class-compatibility matrix follows CPGNN, and the base relation proposal comes directly from HGMD. The contrastive objective is a standard multi-positive InfoNCE/SupCon loss with an enlarged positive set. The capped exponential calibration is a standard generalized-KL (water-filling) projection, and the results in Appendix B follow from basic convex optimization; they say nothing about the student's generalization. The main new element is plugging a CPGNN-style compatibility score into HGMD's sampler, which feels incremental for ICLR. The authors should show that a simpler baseline, e.g., HGMD plus a standard SupCon loss, cannot match the gains.

  2. The gains may not come from transferring the teacher's structural knowledge. On Wisconsin, the GPR-GNN teacher (72.16%) is weaker than the feature-only MLP (74.12%), yet the CoLIFT student reaches 82.55%, more than 10 pp above its teacher. The student also beats the teacher on Texas, and HGMD (GPR) shows the same pattern. This suggests the gains come largely from regularization (stochastic neighbor sampling, soft labels, the contrastive loss), not from transferred structural knowledge. The student also has about 20× more parameters than the teacher. To support the transfer claim, the authors should add controls: a randomly initialized or graph-shuffled teacher, randomly permuted edges for relation sampling, and γ=0\gamma=0 or an identity compatibility matrix in place of M+M^+.

Questions for authors

See weakness.

3 comments
AuthorsOpenReview ↗

Compatibility Lifting and Controlled Comparisons

Thank you for raising the question of how CoLIFT builds on prior work. Our contribution is to address relation-level transferability in graph-free GNN-to-MLP distillation under heterophily (Section 4). A relation that helps the teacher make predictions may provide limited supervision to a graph-free student. This matters when a neighborhood contains same-class relations, structurally supported cross-class relations, and relations with weak structural support.

CoLIFT builds on class compatibility modeling, HGMD’s hardness-aware base proposal, and multi-positive contrastive learning. CPGNN uses class compatibility to guide graph propagation. CoLIFT uses reliability-weighted, prior-corrected class compatibility to recalibrate node-pair sampling probabilities while preserving each target node’s expected relation budget. These probabilities govern sampling and weighting in relation-level knowledge distillation. The same compatibility signal identifies compatible cross-class positives for relation-aware contrastive distillation. Appendix B describes the probability allocation through its generalized KL projection interpretation.

We compare Full CoLIFT with HGMD (GPR) + standard SupCon and an ablation that removes compatibility lifting (CL) and compatible cross-class positives (CCP). For each matched run, the methods share the same frozen GPR-GNN teacher, MLP student architecture, data split, and evaluation protocol. Results are test accuracy (%, mean ± sample standard deviation over 10 runs).

DatasetHGMD (GPR)HGMD (GPR) + SupConw/o CL, w/o CCPFull CoLIFT
Texas86.49 ± 4.0387.30 ± 3.8387.03 ± 3.7887.84 ± 3.18
Wisconsin82.35 ± 2.6282.35 ± 2.6281.17 ± 1.8982.55 ± 2.16
Actor36.74 ± 1.1735.97 ± 0.9536.36 ± 0.9937.05 ± 0.96
Roman-empire66.81 ± 0.5366.73 ± 0.5266.89 ± 0.4967.07 ± 0.39
Cora83.28 ± 1.8283.58 ± 1.4383.73 ± 1.5283.96 ± 1.48
Citeseer70.29 ± 1.5670.55 ± 1.9470.18 ± 1.4270.72 ± 1.63
PubMed80.79 ± 0.3780.97 ± 0.6380.77 ± 0.3081.15 ± 0.45

Adding SupCon to HGMD yields mixed results, with a macro-average gain of 0.10 percentage points (pp). Full CoLIFT achieves higher mean accuracy than HGMD (GPR) + SupCon on all seven datasets, averaging +0.41 pp.

The ablation retains the teacher’s soft predictions, sampling from the base relation proposal, and CoLIFT’s contrastive loss with same-class positives. The comparison measures the effect of CL on the three homophilous graphs, where Full CoLIFT already uses same-class positives, and the joint effect of CL and CCP on the four heterophilous graphs. Full CoLIFT’s mean accuracy is higher on all seven datasets, averaging +0.60 pp. The observed differences suggest a modest benefit from the compatibility mechanism under this protocol.

AuthorsOpenReview ↗

Student Performance and the Effect of Relation Pairing

Thank you for raising the question of what contributes to the student’s gains.

The MLP student can outperform the teacher because it learns from both ground-truth training labels and the teacher’s soft predictions while using node features as input. A teacher with lower overall accuracy can still provide useful supervision. For example, it may assign appreciable probability to the correct class even when its top prediction is incorrect.

To assess the contribution of the original relation pairs, we compare Full CoLIFT with a variant that randomizes relation pairs. For each matched run, both variants use the same frozen GPR-GNN teacher, MLP student architecture, data split, and evaluation protocol. This comparison controls for student capacity. After sampling, the randomized variant permutes the neighbor endpoints of non-self-loop relations, preserving each target node’s sampled relation count and the weight assigned to each sampled slot at that step. It then uses the resulting pairs for relation-level knowledge distillation and relation-aware contrastive distillation.

The table reports test accuracy (%, mean ± sample standard deviation over 10 runs).

DatasetCoLIFT with randomized relation pairsFull CoLIFT
Cora82.51±1.4682.51 \pm 1.4683.96±1.4883.96 \pm 1.48
Citeseer67.36±5.1167.36 \pm 5.1170.72±1.6370.72 \pm 1.63
PubMed80.93±0.7680.93 \pm 0.7681.15±0.4581.15 \pm 0.45
Texas86.22±2.6986.22 \pm 2.6987.84±3.1887.84 \pm 3.18
Wisconsin82.16±3.5182.16 \pm 3.5182.55±2.1682.55 \pm 2.16
Actor36.11±0.9336.11 \pm 0.9337.05±0.9637.05 \pm 0.96
Roman-empire66.89±0.5066.89 \pm 0.5067.07±0.3967.07 \pm 0.39

Full CoLIFT has higher mean accuracy on all seven datasets, with a macro-average difference of 1.17 percentage points (pp). The differences are 1.62 pp on Texas and 0.39 pp on Wisconsin. These results suggest that, with the teacher and training objectives held fixed, the original relation pairs can provide additional supervision, with the size of the benefit varying across datasets.

Thank you again for your thoughtful review. We wanted to follow up on our rebuttal and would appreciate hearing whether our response addresses your main concerns when you have a chance. We would be happy to clarify any remaining points.

Reviewer_9mxWOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

CoLIFT is a framework for GNN-to-MLP distillation for node classification. It estimates class compatibility from teacher predictions and graph edges, accounting for prediction confidence and class prevalence. This compatibility guides the sampling of neighbor relations, including cross-class relations, while preserving each node’s expected relation budget. The selected relations support prediction-level distillation and contrastive learning, allowing compatible nodes from different classes to form positive pairs. After training, only the MLP is retained for graph-free inference. CoLIFT achieves the highest mean accuracy among graph-free methods evaluated under the shared protocol on seven datasets.

Critical strengths
  1. achieves the highest mean accuracy among graph-free methods evaluated under the shared protocol on seven homophilous and heterophilous datasets
  2. paper addresses GNN-to-MLP distillation under heterophily and recognizes that useful neighbor relations can connect nodes from different classes
Critical weaknesses
  1. The paper does not clearly report train/validation/test sizes, exact split identifiers, model architectures, learning rates, training epochs, or hyperparameter search ranges. Six of the seven datasets use ten random seeds on one fixed split, which measures training variability but does not show whether the improvements hold across different splits. Evaluation on a broader range of datasets, particularly larger heterophilous graphs, would strengthen the evidence for generalizability.

  2. Runtime is measured only on Roman-empire, without training-time comparisons or larger-scale experiments. Additional experiments are needed to assess the method’s training cost and scalability.

Questions for authors

1 Can you provide the train/validation/test sizes, exact splits, model architectures, learning rates, training epochs, and hyperparameter search ranges for all methods and code for review?

2 Why did you use one fixed split for six datasets? Do the improvements hold across different splits?

  1. How do CoLIFT’s training time and memory use compare with the baselines? Do you have runtime results on datasets other than Roman-empire or on larger graphs?
8 comments
AuthorsOpenReview ↗

Q1 — Data Splits, Anonymous Code, and Baseline Configurations

Thank you for requesting these experimental details. The table below gives the numbers of training, validation, and test nodes under the shared protocol in Table 1.

DatasetTrainValidationTest
Cora1405001,000
Citeseer1205001,000
PubMed605001,000
Texas875937
Wisconsin1208051
Actor3,6482,4321,520
Roman-empire11,3315,6655,666

The anonymous CoLIFT repository specifies the exact splits. The data preparation script retrieves the public masks, and configs/public_data_identity_v1.json records their checksums. The configs/datasets/ directory provides the final CoLIFT configurations for all seven datasets, including teacher and student architectures, learning rates, epoch budgets, early-stopping settings, and method-specific hyperparameters. The README gives commands for data preparation, training both models from scratch, final evaluation, and result aggregation.

GPR-GNN teacher training settings

The GPR-GNN uses a two-layer feature projection (hidden width 64, dropout 0.5), 10 propagation steps, and propagation dropout 0. The settings below describe how the teacher checkpoints were trained.

DatasetLearning rateWeight decayMax epochsEarly-stopping patiencePPR initialization α
Cora0.010.000510001000.1
Citeseer0.010.000510001000.1
PubMed0.010.000510001000.1
Actor0.01010001000.9
Texas0.050.000510001001.0
Wisconsin0.050.000510001000.9
Roman-empire0.050500500.1

Roman-empire teacher candidates were LR ∈ {0.01, 0.05}, WD ∈ {0, 0.0005}, and α ∈ {0.1, 0.9}. Historical teacher search records for the other datasets are unavailable.

Training configurations and search spaces for the other methods in Table 1 (continued in Part 2)

Non-Roman denotes Cora, Citeseer, PubMed, Texas, Wisconsin, and Actor. T and S denote teacher and student. LL, HH, and DD denote layer count, hidden width, and dropout. LR and WD denote learning rate and weight decay. Epoch counts are maximum budgets, with early-stopping patience in parentheses. For frozen teachers, these settings refer to the original teacher training. We mark missing historical search records in the tables.

MethodDataset(s)ArchitectureLREpochsSearch space or setting
MLPNon-RomanFeature-only MLP (L=3,H=1024,D=0.3L=3,H=1024,D=0.3), no normalization0.01500 (50)Fixed launcher setting. Historical search space unavailable.
MLPRoman-empireFeature-only MLP (L=3,H=512,D=0.3L=3,H=512,D=0.3), no normalization0.01500 (50)Fixed architecture for the matched rerun. The original run used a validation-only search over a subset of the Roman MLP candidates in Part 2.
SAGENon-RomanGraphSAGE with GCN aggregator (L=3,H=1024,D=0.5L=3,H=1024,D=0.5)0.01500 (50)Fixed launcher setting. Historical search space unavailable.
SAGERoman-empireGraphSAGE with GCN aggregator (L=2,H=256,D=0.5L=2,H=256,D=0.5)0.03500 (50)L∈{2,3}L\in\{2,3\}, H∈{64,128,256}H\in\{64,128,256\}, D∈{0,0.3,0.5}D\in\{0,0.3,0.5\}, LR∈{0.003,0.01,0.03}\mathrm{LR}\in\{0.003,0.01,0.03\}, WD∈{0,5×10−4,10−3}\mathrm{WD}\in\{0,5\times10^{-4},10^{-3}\}.
GLNN (SAGE)Non-RomanT uses SAGE above. S is an MLP (L=3,H=1024,D=0.3L=3,H=1024,D=0.3).T 0.01, S 0.01Both 500 (50)Fixed launcher setting. Historical search space unavailable.
GLNN (SAGE)Roman-empireT uses Roman-empire SAGE above. S is an MLP (L=3,H=512,D=0.5L=3,H=512,D=0.5).T 0.03, S 0.01Both 500 (50)T uses the Roman-empire SAGE search space. S uses the original Roman-empire GLNN (GPR) setting.
HGMD (GCN)Non-RomanT is a GCN (L=3,H=1024,D=0.5L=3,H=1024,D=0.5). S is an MLP (L=3,H=1024,D=0.3L=3,H=1024,D=0.3).T 0.01, S 0.01Both 500 (50)Fixed setting. Historical search space unavailable.
HGMD (GCN)Roman-empireT is a GCN (L=2,H=256,D=0.5L=2,H=256,D=0.5). S is an MLP (L=3,H=512,D=0.5L=3,H=512,D=0.5).T 0.03, S 0.01Both 500 (50)T uses the Roman-empire SAGE candidate set. S uses the original Roman-empire HGMD (GPR) setting.
AuthorsOpenReview ↗

Q1 (Continued) — Distillation Baseline Configurations and Search Spaces

This table continues the training configurations and hyperparameter search spaces for the other methods in Table 1. The notation follows Part 1.

On Roman-empire, the feature-only MLP and the GLNN (GPR), HGMD (GPR), FF-G2M (GPR), and PGKD (GPR) students were rerun with the CoLIFT architecture (3 layers, hidden width 512, dropout 0.3) on the 10 official splits, with one seed per split. Within each split, the four rerun distillation baselines and CoLIFT used the same frozen GPR-GNN checkpoint.

MethodDataset(s)ArchitectureLREpochsSearch space or setting
GLNN (GPR)Non-RomanT is a frozen GPR-GNN. S is an MLP (L=3,H=1024,D=0.3L=3,H=1024,D=0.3).T see Part 1, S 0.01T 1,000 (100), S 500 (50)Fixed launcher setting. Historical search space unavailable.
GLNN (GPR)Roman-empireT uses the Roman-empire GPR-GNN checkpoint. S is an MLP (L=3,H=512,D=0.3L=3,H=512,D=0.3).T 0.05, S 0.01Both 500 (50)Fixed architecture for the rerun. Historical search included the Roman MLP candidates and supervised loss weight in {0.1,0.5}\{0.1,0.5\}. Distillation temperature fixed at 1.
HGMD (GPR)Non-RomanT is a frozen GPR-GNN. S is an MLP (L=3,H=1024,D=0.3L=3,H=1024,D=0.3).T see Part 1, S 0.01T 1,000 (100), S 500 (50)Fixed setting. Historical search space unavailable.
HGMD (GPR)Roman-empireT uses the Roman-empire GPR-GNN checkpoint. S is an MLP (L=3,H=512,D=0.3L=3,H=512,D=0.3).T 0.05, S 0.01Both 500 (50)Fixed architecture for the rerun. Historical search included the Roman MLP candidates, HGMD α∈{1,5}\alpha\in\{1,5\}, and distillation temperature in {0.8,1.0}\{0.8,1.0\}.
FF-G2M (GPR)Non-RomanT is a frozen GPR-GNN (recorded L=3,H=64,D=0.5,K=10L=3,H=64,D=0.5,K=10) with PPR initialization. S is an MLP (L=3,H=1024,D=0.3L=3,H=1024,D=0.3), no normalization.T dataset-specific, exact rates unverified. S 0.01.T 1,000 (100), S 500 (50)Used λFF=0.1\lambda_{\mathrm{FF}}=0.1 and TFF=1.5T_{\mathrm{FF}}=1.5. Historical search space unavailable.
FF-G2M (GPR)Roman-empireT uses the Roman-empire GPR-GNN checkpoint. S is an MLP (L=3,H=512,D=0.3L=3,H=512,D=0.3), no normalization.T 0.05, S 0.01Both 500 (50)Fixed architecture for the rerun. Uses λFF=0.1\lambda_{\mathrm{FF}}=0.1 and TFF=1.5T_{\mathrm{FF}}=1.5. Historical search space unavailable.
KRD (GPR)Non-RomanT is a frozen GPR-GNN. S is an MLP (L=3,H=1024,D=0.3L=3,H=1024,D=0.3), no normalization.T unverified, S 0.01T unverified, S 500 (50)Reused KRD-specific settings from the official Cora transductive GCN configuration. No separate search for the adapted runs.
KRD (GPR)Roman-empireT is a frozen Roman-empire GPR-GNN. S is an MLP (L=3,H=512,D=0.3L=3,H=512,D=0.3), no normalization.T 0.05, S 0.005Both 500 (50)Same fixed KRD settings as above.
PGKD (GPR)Non-RomanT is a frozen GPR-GNN (H=64H=64) from the FF-G2M teacher registry and provides pre-classifier representations. S is an MLP (L=3,H=1024,D=0.3L=3,H=1024,D=0.3), no normalization or additional projection.T uses FF-G2M teacher settings. S 0.01.T 1,000 (100), S 500 (50)Six validation combinations with λ1∈{0.1,0.2,0.4}\lambda_1\in\{0.1,0.2,0.4\} and λ2∈{0.05,0.1}\lambda_2\in\{0.05,0.1\}. Fixed τ1=1\tau_1=1 and τ2=10\tau_2=10.
PGKD (GPR)Roman-empireT uses the Roman-empire GPR-GNN checkpoint and provides pre-classifier representations. S is an MLP (L=3,H=512,D=0.3L=3,H=512,D=0.3), no normalization or additional projection.T 0.05, S 0.01Both 500 (50)Fixed architecture for the rerun. Uses the selected PGKD settings from the earlier validation search.
NOSMOG$^{\dagger}$All sevenReported under an independent official protocol in Table 1. Architecture unverified.UnverifiedUnverifiedThe corresponding run records and search space remain unverified.

Historical Roman MLP candidate set. H∈{256,512}H\in\{256,512\}, D∈{0.3,0.5}D\in\{0.3,0.5\}, LR∈{0.005,0.01}\mathrm{LR}\in\{0.005,0.01\}, and WD∈{0,5×10−4}\mathrm{WD}\in\{0,5\times10^{-4}\}.

AuthorsOpenReview ↗

Q3 — Training Time and Peak GPU Memory

Thank you for asking about training costs. Given the time available during the rebuttal period, we profiled the three datasets in our benchmark with the largest node counts: Roman-empire (22,662 nodes), PubMed (19,717 nodes), and Actor (7,600 nodes). We compared GPR-GNN, MLP, GLNN (GPR), and CoLIFT. Each entry reports the mean ±\pm sample standard deviation over three matched runs.

DatasetMethodTraining time (s)Peak GPU memory (MiB)
Roman-empireGPR-GNN11.16±1.7711.16 \pm 1.77112.85±0.00112.85 \pm 0.00
Roman-empireMLP7.02±0.817.02 \pm 0.81211.72±0.00211.72 \pm 0.00
Roman-empireGLNN (GPR)44.89±16.1144.89 \pm 16.11353.45±0.00353.45 \pm 0.00
Roman-empireCoLIFT6998.51±767.916998.51 \pm 767.91609.67±4.66609.67 \pm 4.66
PubMedGPR-GNN7.20±0.757.20 \pm 0.75114.14±0.00114.14 \pm 0.00
PubMedMLP15.16±4.9615.16 \pm 4.96581.18±0.00581.18 \pm 0.00
PubMedGLNN (GPR)44.57±18.3144.57 \pm 18.31581.63±0.00581.63 \pm 0.00
PubMedCoLIFT3019.07±2617.043019.07 \pm 2617.041774.84±7.841774.84 \pm 7.84
ActorGPR-GNN6.42±0.836.42 \pm 0.8383.23±0.0083.23 \pm 0.00
ActorMLP13.18±4.8413.18 \pm 4.84290.41±0.00290.41 \pm 0.00
ActorGLNN (GPR)25.74±5.5425.74 \pm 5.54290.46±0.00290.46 \pm 0.00
ActorCoLIFT853.13±54.59853.13 \pm 54.59983.52±5.79983.52 \pm 5.79

Within each matched run, GLNN (GPR) and CoLIFT shared the same frozen GPR-GNN teacher checkpoint. Their reported times cover student preparation, training, and validation-based checkpoint selection given the frozen teacher. The GPR-GNN rows report the teacher training time.

CoLIFT took longer to train and used more peak GPU memory than the baselines on all three datasets. One-time preparation averaged 1.64–2.08 seconds across the three datasets, with most of the time spent in the training loop. This loop recalibrates node-pair sampling probabilities for each target node and uses the sampled relations for probability-weighted relation distillation and relation-aware contrastive supervision. The resulting training cost is a limitation of the current implementation, and we will report it in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.