- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes PARSE to explain GNN based on Shapley values and pairwise Shapley interactions. After calculating those values, the method creates partition graphs and searches for high-fidelity node sets as explanations. Experiments on controlled tasks and real-world datasets show that PARSE achieves stronger explanation fidelity than baselines.
Critical strengths
-
PARSE provides interaction-aware explanations for GNNs and can capture pairwise interactions between nodes even when they are far away in the graph. The method is model-agnostic, requiring only model queries, which makes it broadly applicable to different GNN architectures.
-
The experiments on real-world datasets are extensive. PARSE demonstrates consistently strong performance across multiple fidelity metrics.
Critical weaknesses
-
The method relies on several approximations with fixed computational budgets (e.g. 50 permutations for estimating Shapley values and pairwise interactions, steps for partitioning, and local search for large explanation spaces). However, the paper provides limited analysis of their trade-offs between approximation and computation. Particularly, sampling too few node orderings in phase 1 will probably yield unreliable estimated interaction, given the huge combinatory amount of such interactions. A sensitivity analysis varying budgets against interaction recovery, fidelity, and runtime or model query times would substantially strengthen the evaluation.
-
While the method shows consistently strong fidelity results, it is unclear how much of the improvement comes from the proposed interaction-aware partitioning versus the fidelity-driven search in phase 4. PARSE directly searches for node sets with high H-Fidelity and additionally calibrates its partition hyperparameters on validation data using an objective involving a large overlap with the reported evaluation metrics. In contrast, the baselines mostly use their default configurations, and their outputs are converted to fixed-budget node sets without an analogous fidelity-based refinement. Although the strong performance on Fid alpha that is not used for calibration or search partially alleviates this concern, an important test would be to apply the same phase 4 fidelity refinement to strong baselines.
Questions for authors
-
GNN-LRP (Schnake et al., 2022) is discussed in the related work (line 97) but is not included in the experimental comparison. Since GNN-LRP can capture higher-order interactions, it would be an important baseline for evaluating PARSE's interaction-aware explanations. Similarly, sGNN-LRP (Xiong et al., 2022), which extends this idea to subgraph explanations with higher-order interactions, would be an important baseline to consider, particularly given that modeling interactions is a central motivation of PARSE.
-
Can the proposed framework capture interactions involving three or more nodes?
-
In phase 4, the interaction-based partition is only one of four starting sets for the H-Fidelity search (line 270). Could the authors compare full PARSE against phase 4 without the interaction-based partition? This would help clarify whether the improvements on the benchmark are due to the proposed interaction-aware partitioning or primarily the H-Fidelity optimization.