acceptodds
Under review as a conference paper at ICLR 2027

The Causal Information Flow of Graph Neural Networks

Abstract

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in graph representation learning by exploiting statistical correlations. However, correlation-based learning fails to capture the underlying causal mechanisms, hampering GNN generalization under distribution shift in tasks like graph-level classification. Despite recent efforts towards developing Causal GNNs (CGNNs), state-of-the-art CGNN training involves complex multi-objective optimization, including objectives derived from the Mutual Information, a metric of correlational character. To address this limitation, we introduce a novel causal-by-design CGNN training paradigm based on the optimization of a single objective that is derived from the Causal Information Flow (CIF), an interventional metric. We evaluate our proposed CIF-based CGNN training paradigm against foundational and state-of-the-art CGNN training paradigms across real and synthetic datasets coming from different domains, using classic Message Passing Neural Networks (MPNNs) and state-of-the-art Graph Transformers (GTs). The results show that CIF-based training leads to competitive CGNN performance while optimizing a single objective.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
causal machine learning, graph neural networks, out-of-distribution classification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_q84nOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work builds on supervised lower bounds for causal information flow and introduces a single-objective training approach tailored for out-of-distribution graph classification. The graph encoder outputs two separate representation channels. During classifier training, the second-channel embedding of one graph is swapped with that from a different graph, and the classifier is tasked with recovering the original graph label. The authors test their method across fourteen dataset setups and six backbone architectures, assessing both predictive performance and the model’s ability to uncover underlying rationales. Aggregated empirical results suggest this objective can yield competitive performance without explicitly tuning tradeoffs among multiple auxiliary loss terms.

Critical strengths
  1. The training objective is simple and practically appealing. By swapping representations across graphs during training, the method uses a single objective without requiring multiple auxiliary losses or their associated weighting coefficients, while achieving competitive performance.

  2. The evaluation considers both classification and rationale discovery across six backbones, providing a broader assessment of the method. The results highlight that strong predictive performance does not necessarily imply accurate rationale recovery, offering useful insight into the behavior of different architectures.

Critical weaknesses
  1. The core theory lacks sufficient substantiation. Swapping the second channel merely encourages the model to avoid relying on it; it does not prevent the first channel from retaining spurious correlations. Consequently, one cannot definitively conclude that the model has learned true causal factors. This is the most fundamental issue.
  2. The experimental results are insufficient to establish the stability or the source of the observed gains. The primary tables report aggregated means without uncertainty estimates, and some comparisons exclude certain experimental runs—which are not accounted for. Furthermore, the absence of critical control experiments makes it difficult to determine whether the gains stem from causal mechanisms or simply from general representation enhancement and regularization.
Questions for authors
  1. Please specify under what assumptions Equation (17) extracts causal information instead of concentrating predictive information, including spurious correlations, into the first channel? Does Theorem 5.2 exclude this shortcut solution, or does it presuppose an effective causal decomposition?

  2. Please provide the performance differences and uncertainty estimates of pairings on the same dataset and with five random seeds, and compare them with the cross-entropy baseline using only the first channel under the same architecture. It is suggested to fix the motif and replace the background on an existing synthetic dataset to examine whether CIF truly reduces the dependence on spurious correlations, rather than simply causing the classifier to ignore the second channel.

Reviewer_k4XkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission focuses on the observation that current Graph Neural Networks (GNNs) often rely on statistical correlations rather than causal mechanisms, causing them to exploit spurious features and fail under distribution shifts. While Causal Graph Neural Networks (CGNNs) may help mitigate this issue, existing frameworks require complex multi-objective optimisation to balance standard classification losses with auxiliary correlational metrics. To address this limitation, the authors in this submission propose a streamlined training paradigm governed by a single optimisation objective derived from the Causal Information Flow (CIF). This interventional metric successfully quantifies the causal influence of a graph's underlying substructures on its eventual classification label. This proposed training paradigm leverages a tractable supervised lower bound of the CIF to construct a single loss function, eliminating the need for auxiliary hyperparameter tuning. This objective operates by swapping the extracted spurious features across different input graphs during training, forcing the network to predict the true label purely from the isolated causal representation.

Critical strengths
  1. The submission introduces the CGNN framework which replaces complex multi-objective loss combinations with a single tractable objective. The mathematical derivation transitions from stochastic independence to causal independence using back-door adjustment and Jensen's inequality.

  2. The proposed methodology is tested across 14 distinct datasets and 6 backbones, forming 30 robust method-backbone combinations. Explicit analysis validates that omitting expensive Monte Carlo sampling retains full predictive accuracy, minimizing computational overhead.

Critical weaknesses
  1. The proposed method is not the top performer for invariant graph rationale discovery, lagging behind the baseline method ICL. On specific advanced backbones like GraphGPS, the ablation variant produces higher rationale discovery precision than the primary proposed CIF model. Also, structural choice deeply bottlenecks outcomes; Graph Transformers significantly underperform compared to simpler MPNN architectures in local feature isolation.

  2. Collapsing causal integrals into deterministic point masses assumes crisp feature boundaries, which may fail on highly probabilistic or noisy spuriosity. In addition, automated scripts and external LLMs are heavily leveraged to parse unmanageable raw results data and restructure entire core sections.

  3. There are presentation issues. For example, references are not presented in a consistent way (pay attention on conference/journal names). Many tables are too small to read. Also the discussion of the literature should include most recent research efforts.

Questions for authors

Please check Critical Weaknesses section and clarify them.

Reviewer_itAzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies causal learning for graph neural networks under distribution shift. The main idea is to train causal GNNs with a single objective derived from Causal Information Flow, instead of combining the classification loss with several auxiliary causal or mutual-information-based losses. The proposed loss swaps the spurious representation of one graph with that of other graphs and asks the model to still predict the label from the causal representation. The paper evaluates this idea on a fairly broad set of OOD graph classification datasets, using both standard message-passing GNNs and more recent graph transformer backbones.

Critical strengths

The paper addresses a relevant problem in OOD graph classification, where relying on spurious correlations is a real concern.

The proposed training objective is simple compared with many existing causal GNN methods that require multiple losses and balancing coefficients.

The connection to causal information flow gives the method a clear conceptual motivation.

The experimental evaluation is broad, covering several datasets, baselines, and backbone architectures.

The ablation comparing CIF with the variant before Jensen’s inequality and with standard cross-entropy helps justify the design choice.

Critical weaknesses

The empirical improvements are often small, and the results support competitiveness more than clear superiority.

The causal interpretation depends on the learned attention-based disentanglement, but the paper does not fully address when this disentanglement may fail.

The theoretical derivation is useful as motivation, but the implemented model only approximates the unobserved causal variables through learned representations.

The main tables are aggregated across datasets, which makes it harder to understand dataset-specific behavior and robustness.

More reporting on variance, statistical significance, and computational overhead would make the empirical claims stronger.

Questions for authors

How sensitive is CIF to batch size? Since the loss uses cross-graph swaps, the number and diversity of graphs in each batch may affect performance.

Can the causal branch still encode spurious information if that helps the objective? If yes, are there settings where CIF may fail to separate causal and spurious features?

What is the computational overhead of CIF compared with CE and the multi-objective causal baselines?

Are the first-place results statistically significant across seeds, especially in cases where the differences are around 0.01?

Reviewer_gMm8OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a single-objective training approach for causal graph neural networks (CGNNs), based on a claimed tractable lower bound on Causal Information Flow (CIF). The loss combines the causal representation of one graph with the spurious representation of another graph in the batch and predicts the observed label of the first graph. This is intended to make predictions insensitive to the exchanged spurious representation, without balancing a classification loss against several auxiliary objectives. The evaluation covers 14 out-of-distribution benchmark configurations from five dataset families, using three message-passing networks and three graph Transformers. CIF is compared with four CGNN training approaches on graph classification and invariant graph rationale discovery, achieving competitive performance with a single training objective.

Critical strengths

1 A useful simplification of CGNN training The single objective removes the need to balance prediction and auxiliary losses. Its cross-graph construction is simple and applies to both message-passing networks and graph Transformers. 2 Broad comparisons with competitive results Six backbones and 14 distribution-shift configurations provide broad comparisons. Evaluating classification and rationale discovery helps assess both predictions and selected graph features. Classification results are competitive.

Critical weaknesses

1 The central lower bound fails under the stated model Eq. (15) uses observed labels after exchanging the spurious representation. For fixed feature maps and an independently sampled partner, an unrestricted prediction head can ignore that partner. Its optimum then measures observational mutual information, which need not be bounded by the intervention-based quantity in Eq. (14). Consider a graph encoding a fair binary variable U and an independent bit that is one with probability 0.1. The causal feature C copies U. The spurious feature S copies U except when the bit flips it. An independent selector makes the label copy C with probability 0.1 and S otherwise. Both features are deterministic functions of the graph, consistent with Figure 1. The observed label matches C with probability 0.91. Intervening on C breaks its link to U, while S remains unchanged; the match probability becomes 0.55. A head ignoring the exchanged channel reaches about 0.391 nats on Eq. (15)'s right-hand side, above the approximately 0.005 nats on its left. This violates the stated lower bound. Both label marginals are fair binary distributions, so changing the reference distribution does not explain the violation. The missing step is a causal identification argument connecting observed labels to intervention outcomes. If identification equates the label distribution given C with that under intervention on C, Eq. (15) reduces to the standard variational mutual-information bound (Barber and Agakov, 2003). The paper must establish why learned features satisfy sufficient assumptions and what additional causal guarantee follows. Observational conditional independence alone is insufficient.

2 Eq. (14) also measures a change in the reference distribution Eq. (14) equals Ay and Polani's (2008) information flow plus a KL divergence between the intervention-averaged and observed label distributions. That extra term can be positive even when the label distribution is identical for every intervention value. Keep the graph and features above, but define the label as whether C and S differ. In observed data, the label is one with probability 0.1 and is independent of C. Under either intervention on C, the label is one with probability 0.5. Ay and Polani's flow and observational mutual information are both zero, yet Eq. (14) is approximately 0.511 nats. The paper must justify why this additional regime-shift term measures the causal dependence relevant to its claim.

3 The loss closely resembles existing intervention terms and permits shortcuts CAL (Sui et al., 2022), Eqs. (13)–(14), already combines one graph's causal representation with other graphs' shortcut representations and predicts the original label. CIF uses concatenation instead of addition, all in-batch pairs instead of batch-permuted partners, and removes CAL's supervised and uniformity losses. Related cross-graph mixing appears in ICL's released implementation (Zhao et al., 2024). A useful simplification remains possible, but its added value requires a controlled comparison. The objective also permits the causal channel to retain predictive information from the whole graph while the head ignores the exchanged channel. Attention approaching one on the causal branch can approximate ordinary supervised prediction; nonzero masks can preserve scaled encoder information. Eq. (17) contains no separate requirement assigning unwanted information to the spurious channel. Compare CIF with CAL and ICL retaining only their cross-graph prediction terms under the same backbone and training setup. Vary addition versus concatenation and sampled versus all-pairs mixing. On SYN or SpMotif, replace the spurious base while holding the label-determining motif fixed. Test base identity in the causal representation using independently crossed or conditional probes, so label–base correlation is not mistaken for leakage. 4 Competitive classification does not resolve these concerns

Table 2's rounded CIF–CAL classification differences are at most 0.03, and ten of twelve are 0.01 or less. With GraphGPS, CIF and ICL both obtain classification AUROC 0.81 and AUNPRC 0.60. Yet CIF's rationale Precision@k and node-level AUROC are 0.50 and 0.67, versus ICL's 0.74 and 0.86. Similar classification therefore does not establish comparable causal separation. The AI-use statement reports five seeds per configuration, but the tables give no uncertainty. Provide per-dataset and seed-level results, paired comparisons, and the number and distribution of missing Table 3 runs. Compare matching configurations. Eq. (17) also evaluates all graph pairs; report training time and peak memory at matched batch sizes.

Questions for authors
  1. Which causal identification assumptions validate Theorem 5.2 and exclude the counterexample? Please provide Appendix A.4's proof and explain the remaining intervention guarantee.
  2. How does Eq. (14) distinguish causal dependence from the reference-distribution shift illustrated by the second example?
  3. What added value remains against CAL and ICL using only their cross-graph terms, under matched combination and sampling rules?
  4. Do controlled base swaps and balanced or conditional probes support causal separation? Please also report five-seed uncertainty, detailed results, and missing-run counts.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.