GNNM: Parameter-Efficient BatchEnsembles for Graph Neural Networks

Ensembles are an effective way to improve the predictive performance of neural networks, but their computational and memory costs limit their use in many settings. Modern parameter-efficient ensembles address this issue by sharing the main model weights across ensemble members and introducing only a small number of additional parameters, yet their effectiveness in message passing architectures remains unclear. We introduce GNNM, a lightweight ensemble modification for residual GNNs that replaces only the input and output linear projectors with BatchEnsemble parameterisations while leaving the graph backbone unchanged. This design allows the same modification to be applied to different GNN models with minimal parameter overhead and without altering their message passing layers. We evaluate GNNM on five standard node classification benchmarks and compare it with both individual model baselines and explicit ensemble baselines under the same protocol for hyperparameter selection. The results show that a GNNM variant achieves the best score on every dataset. On average, GNNM improves over the individual model for every backbone and performs on par with or better than an explicit ensemble of the same size while using about four times fewer parameters, while its training cost is comparable to that of the explicit ensemble. The gains are strongest for SAGE, GAT-sep, GT, and GT-sep, while for GAT the explicit ensemble remains stronger. These results show that projector level ensembling can serve as a simple and parameter-efficient mechanism for improving residual GNNs while preserving the structure of the underlying graph model.