The Causal Information Flow of Graph Neural Networks

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in graph representation learning by exploiting statistical correlations. However, correlation-based learning fails to capture the underlying causal mechanisms, hampering GNN generalization under distribution shift in tasks like graph-level classification. Despite recent efforts towards developing Causal GNNs (CGNNs), state-of-the-art CGNN training involves complex multi-objective optimization, including objectives derived from the Mutual Information, a metric of correlational character. To address this limitation, we introduce a novel causal-by-design CGNN training paradigm based on the optimization of a single objective that is derived from the Causal Information Flow (CIF), an interventional metric. We evaluate our proposed CIF-based CGNN training paradigm against $2$ foundational and $2$ state-of-the-art CGNN training paradigms across $14$ real and synthetic datasets coming from $4$ different domains, using $3$ classic Message Passing Neural Networks (MPNNs) and $3$ state-of-the-art Graph Transformers (GTs). The results show that CIF-based training leads to competitive CGNN performance while optimizing a single objective.