acceptodds
Under review as a conference paper at ICLR 2027

CoLIFT: Compatibility-Guided Graph-Free Distillation under Heterophily

Abstract

GNN-to-MLP distillation enables efficient graph-free inference while preserving rich information learned by GNNs. However, existing methods are developed primarily for homophilous graphs and rarely distinguish which neighborhood relations provide transferable supervision. This limitation becomes critical under heterophily, where observed neighborhoods mix structurally compatible relations with weakly supported or potentially misleading ones. To address this challenge, we propose CoLIFT, a COmpatibility-guided LIFTing framework that uses class compatibility as a structural proxy for GNN-to-MLP relation-level transferability. CoLIFT estimates compatibility from reliability-weighted teacher class-pair co-occurrences and corrects it for class prevalence. It then lifts these scores to node-pair sampling probabilities through capped exponential calibration that preserves each target node's expected relation budget. The sampled relations jointly define probability-weighted relation distillation and contrastive supervision that admits structurally supported cross-class positives. Across seven benchmarks, CoLIFT achieves the highest accuracy among the graph-free baselines, with a 6.00-percentage-point macro-average gain over the feature-only MLP on heterophilous graphs. Fixed-budget diagnostics and post-hoc analysis find that our calibration generally shifts probability mass toward structurally supported class pairs, with high-compatibility contexts yielding higher mean accuracy than low-compatibility ones on all seven datasets.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.