- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
GraphSieve combines similarity-based edge pruning with optional addition of high-similarity absent edges. The paper focuses on selecting defense hyperparameters using validation accuracy on a trusted clean graph and random perturbations at several severities. Applying this calibration to both GraphSieve and existing purification methods helps separate tuning gains from changes to the editing method. Experiments on three citation graphs show improvements over several baselines, but mixed results against equally calibrated GCN-Jaccard. A follow-up reproduces small gains from edge addition in two Cora deletion-Metattack conditions, while recovery diagnostics show that most added edges were never part of the original graph.
Critical strengths
-
Calibrating the baselines under the same information-access assumptions is a useful contribution. GCN-Jaccard improves by +0.44 and +4.18 percentage points over clean-only tuning on the two profiles (Table 1). This demonstrates why comparing a tuned new method against a default baseline can give a misleading impression of progress.
-
The diagnostics make the results easier to understand. The paper varies insertion/deletion composition, includes a structure-free baseline, reports clean-accuracy costs, and distinguishes candidate coverage from actual edge recovery. In particular, Table A19 supports the authors' decision to describe the added edges as predictive augmentation rather than successful reconstruction.
-
The follow-up experiment is a useful check on the favorable Cora result. New attack graphs and model seeds reproduce gains of +1.21 and +0.93 points at the two deletion budgets (Table A16). The authors appropriately state that this is conditional on the original split and configurations and does not establish a general benefit from edge addition.
Critical weaknesses
-
The improvement over a properly calibrated pruning baseline is limited. GraphSieve has 4 wins and 6 losses against GCN-Jaccard in 130 fresh-profile cells, and one win in 18 reproduction cells (Table 2). Against its own pruning-only ablation, it has 3 wins and 9 losses on the fresh profile. Addition is disabled throughout PubMed, and the positive replication concerns two selected Cora conditions. These are useful findings, but they leave the case for the new editing method rather weak. The ladder also loses on average to clean-plus-100%-severity tuning on the reproduction profile (Tables A13 and A15), so its advantage over simpler selection policies is not consistent either.
-
The clean-graph assumption limits the practical motivation. Calibration uses a trusted clean copy of the same graph, which could simply be used for training, as the authors acknowledge. It is unclear when a defender would have that graph for calibration but still need to train on its poisoned counterpart. Evidence of transfer across separate trusted graphs or snapshots would help; without it, the contribution is mainly a controlled evaluation study.
-
The adaptive attack does not adapt to the complete deployed defense. It knows the similarity rule but not the frozen thresholds or trained model, and its deletion heuristic does not test attacks that exploit the addition stage or insert high-similarity edges. I would like to know whether the gains survive an attack that uses the actual frozen configuration and optimizes against purification and training together. The current results support a narrower robustness claim.
-
The registered E1/E2 decision rule treats cells as independent despite shared datasets, seeds, and calibrations. Appendix A's sensitivity analysis is helpful, but the six fresh-profile losses for E1 reach both dependence-aware bounds even though E1 “holds” under the registered rule. The substantive conclusion should follow that qualification. Reporting aggregate paired effects with uncertainty that preserves this dependence would be more informative than relying on counts of significant cells.
Questions for authors
See weaknesses 1–4.
Response to Reviewer 3Xfx
Thank you for taking the time to review our paper and for the constructive feedback.
W1. Improvement over the calibrated pruning baseline. Each number you quote is correct.
Overall, GraphSieve does not improve on calibrated GCN-Jaccard, and the protocol you found useful is what makes that baseline strong (+0.77 and +4.26 pp over its library default). A better addition rule is left to future work: under generalized DICE on Cora and CiteSeer, the candidate set contains most deleted edges, but similarity ranking recovers few (Section 5.4).
W2. The clean-graph assumption and transfer. We use the trusted copy once, before any attack, only to choose the defense's settings. Training on that copy instead would work only if the graph never changed; once it changes, the defender must work with the changed graph and cannot tell which edits are attacks. Each replicate's frozen configuration is therefore applied unchanged to every attacked version of its graph, with no recalibration and no clean copy (Section 3.2). We also ran the transfer test you suggest, between graphs, and its result is negative: applying each citation graph's frozen configuration to the attacked cells of the other two (exploratory, fresh profile; Appendix C, Table A17), the transferred GraphSieve records more significant losses than gains against the undefended GCN in five of the six pairs. We therefore claim no transfer between different graphs.
W3. Adaptation to the complete deployed defense. The registered attack read the similarity signal but not the frozen thresholds, so we added two better-informed attackers (exploratory; Section 5.5, Table A20), holding the defense at its frozen configurations. L2 deletes only edges that the defense neither prunes nor re-admits, reading the registered pruning grid; L3 does the same reading each replicate's frozen threshold. Against the undefended GCN over 36 cells, GraphSieve records gains/NS/losses of 20/16/0 under the registered attack, 14/22/0 at L2, and 13/20/3 at L3, with the three losses on PubMed, and the abstract is revised accordingly. Both attackers adapt deletions and keep the DICE insertions, and no attacker evaluated optimizes through purification and training jointly. These results therefore measure sensitivity to better-informed deletions; they do not establish robustness and do not amount to adaptation to the complete defense.
W4. Dependence among cells. The registered bound does treat cells as independent (Section 4.3). We computed the estimator you describe, with the replicate as the unit and a 95 percent interval over 10 replicates (exploratory; Appendix A, paragraph "Aggregate paired effects", Table A1). The gains over the undefended GCN (+0.82 and +6.73 pp, fresh and reproduction), RGCN (+4.38 and +8.18), GCN-SVD (+6.34 and +2.71), and the feature-only baseline exclude zero. The differences from calibrated GCN-Jaccard, +0.01 [-0.15, +0.17] and +0.14 [-0.06, +0.35] pp, and from Prune-only, -0.06 [-0.12, +0.00] and +0.07 [-0.08, +0.22] pp, do not.
We rewrote many parts of the paper for clarity and welcome any further comments that would strengthen it.