acceptodds
Under review as a conference paper at ICLR 2027

Counterfactual Supervision for Adaptive Propagation Controls in Graph Neural Networks

Abstract

Graph neural networks (GNNs) propagate information between neighboring nodes, yet the most effective propagation behavior can differ across nodes and layers. Adaptive GNNs seek to address this variation through learned controls. However, standard end-to-end training supervises these controls through gradients evaluated at their current values, thereby capturing only local loss sensitivity and failing to directly compare the downstream predictions resulting from distinct propagation behaviors. This shortcoming weakens the training signal and complicates the understanding of the learned controls. To address this issue, we introduce a method known as Counterfactual Supervision. This training strategy utilizes node-local continuous controls with two meaningful alternatives. For a selected control, we temporarily set it to each alternative, recompute the subsequent layers, and utilize the difference in prediction loss as an additional supervision signal. This approach complements end-to-end optimization while maintaining continuous controls. Evaluating both alternatives at every node can be computationally costly. Therefore, we show that alternatives at sufficiently distant nodes can be assessed together without compromising the exact loss differences for each node. This strategy enables the derivation of multiple per-node comparisons from a single pair of recomputations. We implement this framework in TriAxis-GNN, which learns distinct controls for nodes and layers related to propagation direction, balancing contributions from self and neighbors, and distinguishing between ordinary and higher-order neighborhood structures. Experiments on homophilic and heterophilic node classification benchmarks indicate that counterfactual supervision enhances adaptive propagation models. Furthermore, the proposed batching strategy considerably reduces computational costs associated with this method. Our source code is available at: https://anonymous.4open.science/r/TriAxis-GNN-7BE3/.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph neural networks, adaptive propagation, counterfactual supervision, locality-aware batching

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_EQDVOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes counterfactual supervision for adaptive GNNs, where each node uses learned switches to control how it receives neighbor information. For a selected switch, the method sets it to each of its two extremes, recomputes the remaining layers, and uses the loss difference as an extra training signal for the controller. To reduce cost, nodes that are far apart are tested together, which gives exactly the same results as testing them one by one. The method is implemented in TriAxis-GNN, which has three switches per node, and experiments on homophilous and heterophilous datasets show small but consistent gains.

Critical strengths

1.Clear motivation. The paper shows that local gradients and endpoint comparisons can prefer opposite settings, and measures 10% to 30% disagreement in real models, which makes the problem concrete.

2.Exact and efficient batching. Testing distant nodes together gives the same results as testing them one by one, while greatly reducing training cost.

3.Applicable to other models. The method only changes training, and also improves another adaptive GNN.

Critical weaknesses

1.The value of the counterfactual supervision signal is unclear. The paper’s core claim is that counterfactual supervision improves adaptive GNNs, but in Table 1 most of the gain comes from the TriAxis architecture, while counterfactual supervision adds only small improvements. Table 2 further shows that random targets already give most of the gain on Mowst, suggesting that improvements come mainly from the additional router update steps.

2.Key control experiment is missing on the main model. The comparison with random and constant targets is only conducted on Mowst with four datasets, and not on TriAxis, the main model of the paper, so it remains unclear whether counterfactual supervision outperforms simple auxiliary targets in the proposed architecture.

3.Limited theoretical support for the core claim. Proposition 1 only shows that endpoint comparisons and local gradients can disagree, but not that endpoint comparisons are the better signal. Figure 3(a) shows that over 90% of loss curves in TriAxis are approximately monotone, where both signals point to the same endpoint, so counterfactual supervision provides little new information beyond the gradient. Only in the remaining cases can it add new information, but Figures 3(b) and 3(c) show that in these cases the best value often lies between the endpoints and can yield notably lower loss, whereas counterfactual supervision pushes the control toward an endpoint and may move it in the wrong direction. This may explain why counterfactual supervision brings only small gains in Table 1.

Questions for authors

(1) Can you report the training time of TriAxis compared with GCN and other adaptive baselines?

Reviewer_4RryOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

Traditional Graph Neural Network (GNN) models use a fixed propagation rule to do message passing across all layers and all nodes. To make it more flexible, adaptive GNN models can learn the propagation rule of each layer and/or each node based on the end-to-end loss function. However, the gradient can only evaluate the local preference but might not reflect the preference from distinct propagation behaviors. To fill this gap, this paper proposes “Counterfactual Supervision” for adaptive GNN training. For each node in each layer, we can create several continuous control signals whose two endpoints correspond to two different propagation rules. It temporarily sets a control signal to each endpoint and measures the difference in prediction loss, which could reflect the actual preference of this control signal. Then, it normalizes this measurement result as an additional training signal to train the controller. This paper instantiates this framework in TriAxis-GNN, where each node in each layer has three control signals. Besides, since it is very time-consuming to run counterfactual supervision for every node one by one, this paper proposes to run counterfactual supervision for distant nodes together that are out of each other’s receptive field. Extensive experiment results demonstrate the effectiveness of the proposed counterfactual supervision method.

Critical strengths

1, The motivation is quite interesting and novel. Figure 1 shows that relying on local gradients to guide propagation preference has some limitations, which demonstrates the motivation for the new method. The proposed method is a principled way to address this concern.

2, The experiments are quite thorough. The experiments include many baseline models and contain a wide variety of datasets. The proposed counterfactual supervision method generally improves the baseline models. Besides, there are many ablation studies, sensitivity studies, and good analysis in the appendix.

Critical weaknesses

1, The experiment lacks an important baseline. A core motivation of the paper is that we cannot only rely on local gradients to guide propagation preference, so baselines should include the “TriAxis + local” model that relies on local gradient to guide propagation preference. The current “Task only” baseline in Table 1 has two differences from the counterfactual supervision method: (1) it does not have an additional control-only update; (2) it uses local gradient to guide the control update. We need to add a baseline that only differs in using the local gradient. The new baseline should have an additional control-only update but use the local gradient to guide the control update, which is the “+Local” row in Table 2. Table 2 already has this baseline on the Mowst model, but it would be better if this paper could also do it on the TriAxis model, because TriAxis is the paper’s major proposed model.

2, Appendix C.2 conducts experiments using the aggregate utility v.s. receiver utility. Theoretically, the aggregate utility should more accurately reflect the overall performance on the whole dataset, but using the aggregate utility is worse than receiver utility, and even worse than the “task only” row. There should be clearer investigations or more experiments to figure out the reason.

Questions for authors

1, I am wondering about the time cost of the counterfactual supervision method compared to not using it. Could you please list the time cost ratio of TriAxis+CF compared to TriAxis (task only) in the Table 1 experiments?

2, It seems the provided code only contains part of the code. We cannot reproduce all the results using the code. Will you publicly share the whole codebase?

Reviewer_XCkFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TriAxis-GNN, a framework to adapt the propagation behaviour of GNNs at the node and layer level through periodic counterfactual supervision instead of the current standard practice of end-to-end training using loss gradients. It controls three key message-passing decisions, i.e., smoothing vs. sharpening, weightage between self and neighbor representations, and between direct and higher-order neighbors, by evaluating the complete downstream GNN performance at the endpoints of the control knob, while fixing the remainder of the layer in place, and learning a preference for one endpoint over the other and its strength.

Critical strengths

The paper addresses a central problem in the GNN community today, i.e. adaptive propagation, from a different perspective of counterfactual supervision. The independent-set batching construction is an interesting method for efficiency in node classification.

Critical weaknesses

Several solutions have been proposed for adaptive neighborhood aggregation in GNNs. While the paper does compare with FAGCN, Mowst and OrderedGNN, other solutions have been shown to outperform the results reported in this paper for the five heterophilic datasets significantly such as CO-GNNs [1] and GATE [2] (also for obgn-products and obgn-arxiv). Does TriAxis offer any benefits that might justify the lower performance?

[1] Cooperative Graph Neural Networks [ICML 2024].

[2] GATE: How to Keep Out Intrusive Neighbors [ICML 2024].

Questions for authors
  1. For a given GNN of depth L, are the controls of the layers decided in order from 1 to L, with earlier layers using their learnt controls when evaluating the subsequent layers in an iteration? What if both endpoints are given equal weight during the learning process? Could you also clarify whether the eventual selection is of the endpoint itself, or an intermediate point such as an average of the endpoints weighted by their learnt strengths?

  2. In the training procedure, is a layer-control picked randomly, or are they cycled through?

  3. What is meant by locality packing on line 434?

  4. Are weighted edges supported by the framework?

  5. Is the framework limited to transductive settings?

  6. Does the independent-set construction find any other applications in GNN training or other aspects of GNNs, even if with minor updates?

---- Post Rebuttal Discussion ---- Since my questions have been answered adequately and the additional results address my main concerns, I have raised my score to 3.

1 comment

Thank you to the authors for their detailed response. My questions have been answered adequately, and I appreciate the additional results provided. I have just a couple of points to discuss further regarding the evaluation setting. Firstly, could the authors explain why they chose not to use the fixed splits provided with the datasets by Platonov et al. 2013? That would make the comparison with other works easier and is considered a standard practice. Secondly, why is TriAxis's evaluation limited to shallow models? I understand that the goal is to show the improvement over the case that does not employ TriAxis rather than achieve SOTA results. However, even for a given architecture, if similar (or even better) performance than the shallow model can be achieved by simply using a deeper model, why would TriAxis need to be employed? I'm trying to understand the scope of TriAxis's applicability and gains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.