GNEval: Benchmarking of context-specific foundation models for gene network inference

Gene networks encode regulatory and functional relationships among genes. Single-cell foundation models, pretrained on large and diverse datasets, provide a range of gene-gene association signals that offer a promising basis for gene network inference. However, it remains unclear which gene network properties these signals capture and how they compare with domain-specific methods. To address this gap, we introduce GNEval, a unified benchmark for evaluating pretrained model derived gene associations for network inference. We benchmark 21 gene representations and association scores from 11 models, including static gene embeddings (S), contextual embeddings (C), and gene-gene attention scores (A). GNEval spans 14 Perturb-seq datasets across diverse cell systems, perturbation types and data sizes. Inferred gene networks are assessed based on their consistency with functional and perturbational evidence. The results reveal substantial heterogeneity across evaluation tasks. Rankings based on Gene Ontology and Perturb-seq evidence were nearly uncorrelated (Spearman $\rho =0.01$). Context-independent representations showed a substantial advantage in recovering functional relationships ($\Delta \mathrm{AUROC}=0.13$), whereas context-dependent gene associations improved recovery of perturbation-associated differential expression relationships by $4.9\times$. The domain-specific method achieves relatively strong performance on both evidence types, offering a better balance across the two evaluations than most foundation-model-derived networks. Overall, gene representation and association scores show task-specific strengths, highlighting the need for task-aware selection and integration with domain-specific methods.