- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CoSG generates counterfactual explanations for link predictions by frozen signed graph neural networks. An amortized MLP predicts two masks over existing edges in a target pair’s local computation graph: one controls edge existence, and the other controls sign preservation or flipping. These masks define differentiable scores for preserving, removing, or sign-flipping each edge. Training combines a prediction-changing objective with perturbation penalties and entropy regularization; inference selects the highest-scoring discrete action per edge.
The paper evaluates CoSG on BitcoinOTC, Twitter, and Epinions using SGCN and SNEA backbones. Across ten runs, it reports higher prediction-flip rates than the evaluated baselines, generally competitive sparsity, and millisecond-scale inference. Additional experiments examine full-test performance, backbone depth, loss components, perturbation restrictions, and hyperparameter sensitivity.
Critical strengths
-
The method directly represents sign changes as counterfactual interventions. Separating edge existence from polarity provides a simple differentiable formulation for preservation, deletion, and sign flipping. The perturbation-restricted study in Table 9 demonstrates that sign flipping is particularly consequential for the evaluated signed-graph models. This is useful evidence about an aspect that deletion-only explanations miss.
-
Amortized inference offers a practical benefit, supported across several settings. CoSG learns from 300 training pairs and generates explanations for held-out pairs without instance-specific optimization. Table 1 reports substantial validity gains across three datasets and two backbones, while the full-test results in Table 4 remain close to the sampled evaluation. Together, these results provide encouraging evidence for efficient explanation generation within the evaluated graphs.
Critical weaknesses
-
The comparisons confound the proposed method with access to a more powerful perturbation space. CoSG can both remove edges and flip signs, whereas all learned baselines in Table 6 are restricted to removal. Random sign flipping does not replace a learned or optimized sign-aware counterfactual baseline. This matters because Table 9 shows that CoSG-FL nearly matches full CoSG: validity is 0.66 versus 0.68 on BitcoinOTC, 0.55 versus 0.58 on Twitter, and 0.50 versus 0.52 on Epinions with SGCN. These results suggest that much of the advantage may come from allowing sign flips. A baseline using the same three actions—for example, instance-specific optimization of the same relaxation—is needed to establish the contribution of the learned policy and objective.
-
The evaluation supports sparse prediction-changing perturbations, but does not establish several stronger explanation claims. Sparsity is averaged across instances without separating successful from unsuccessful outputs, and no restoration test or small-instance exact comparison assesses whether successful perturbations contain unnecessary edits. Moreover, Appendix C.4 defines Fidelity as absolute change in the original class’s logit, although Section 4.1 describes a confidence decrease. This metric can reward increased confidence in the original prediction and is sensitive to logit scaling. Finally, validity accepts any change among positive, negative, and non-edge classes, so it does not reveal how often CoSG actually reverses polarity. These omissions limit the evidence for minimal sign-changing explanations; the further claims of actionable recourse and causal dependencies are not validated by the reported experiments.
Questions for authors
-
Controlled comparison: How does CoSG compare with an instance-specific optimizer using the same preserve/remove/flip actions, relaxation, and perturbation costs? Does its validity–edit-count advantage persist when the perturbation space is matched?
-
Evaluation clarification: Can you report class-transition counts, successful-only edit counts, and the fraction of successful explanations that remain valid after restoring individual edits? Is the implemented Fidelity metric the absolute logit change in Appendix C.4 or the confidence decrease described in Section 4.1?
-
Access assumptions: The introduction states that explanation proceeds “without access to model internals,” but CoSG uses encoder embeddings and differentiates the prediction loss through a weighted SGNN. What access is actually required, and how are weighted positive/negative aggregation and normalization implemented for each backbone?
-
Instance conditioning: The edge representation contains candidate-edge endpoint embeddings but no explicit target-pair representation or original edge sign. How does the policy distinguish different target predictions with overlapping computation graphs? Is dependence on the target entirely through recomputing embeddings on each extracted graph?
Please check the rebuttal
Dear Reviewers,
The deadline for the discussion is approaching soon. Please check the rebuttal and respond accordingly.
Best,
AC