- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes CA-AWT, a one-shot way to find sparse graph lottery tickets that stay robust when the graph has been poisoned. It computes the loss curvature once after a short warm-up and uses it for both masks: weights with high curvature are kept, and edges with high curvature are removed, on the idea that injected edges are where the loss is most sensitive. A teacher trained on the cleaned graph is then distilled into the sparse ticket. On citation graphs under several poisoning attacks, the method beats UGS almost everywhere and roughly matches the iterative ARGS method, removing about 88% of injected edges compared with about 30% for a learned mask.
Critical strengths
-
A simple, cheap, and empirically strong edge-ranking signal with a clear contrast to learned masks.
-
The coverage analysis is correct and verified empirically. The paper also reports where the method fails. The ablations are per-component and per-budget, and the hyper-parameter sensitivity is reported.
Critical weaknesses
1. The edge detector is not shown to detect adversarial edges, and its theoretical justification does not hold.
The curvature score is non-zero only on entries within hops of a labelled node (Prop. 1), and the paper itself notes that DeepRobust attacks concentrate there and that high curvature means "influential, not malicious" (A.5). At the cut removes 49–62% of edges against roughly 70% coverage, so near-perfect recall (e.g., 100% in Table 9) is close to what any label-proximity ranking would achieve. Yet the paper omits structural controls (distance to training nodes, degree, Jaccard/feature dissimilarity) and threshold-free metrics (AUROC/AUPR, precision at a cut equal to the injected share). The mechanism is also unsupported. Prop. 2 discards the first-order term and the absolute values by appealing to a converged minimum, but is never optimized, so is not stationary in . Moreover, a unit removal is not a small perturbation. Finally, no adaptive attacker is evaluated, although by Prop. 1 perturbations placed outside the coverage set are invisible to the detector by construction.
2. Parity with ARGS rests mainly on an expensive distillation component.
Without the teacher, CA-AWT reaches 73.4 / 74.8 / 70.3 on Cora at (Table 11), several points below ARGS (about 79.9 / 76.8 / 72.8 from Table 1). The gap is closed by a dense teacher selected from up to 20 dense trainings on validation, a component ARGS does not receive. The "one-shot, fraction of the cost" claim lacks wall-clock or FLOP evidence, and the paper does not report ARGS or UGS with the same cleaned teacher. It is also unclear whether the baseline numbers were re-run on identical poisoned graphs, splits, and seeds. The unusually large teacher gain at (+14 points) further suggests that the no-teacher pipeline may be undertrained. On the theory side, Theorem 1's proof applies the decay only to pruned coordinates, while kept coordinates keep a non-decaying term, so the -independent bound does not follow. Prop. 3 bounds by rather than the dense model's Lipschitz constant, so "the certificate never shrinks" is not established.
Questions for authors
-
Please report the recall (and AUROC/AUPR) of simple structural rankings at the same budgets: (i) distance to or incidence on training nodes; (ii) endpoint-degree-based ranking; (iii) Jaccard or cosine feature dissimilarity.
-
Please also report the recall of the curvature score restricted to the covered set, compared with a random ranking within the covered set. If curvature does not clearly beat these, the core claim needs to be reframed.
-
Precision at small budgets. What is the precision or recall of the curvature ranking at a cut equal to the injected share, for each attack? Baseline provenance. Were UGS and ARGS re-run on exactly the same poisoned graphs, splits and seeds, or copied from the ARGS paper? If copied, how were the MetaAttack and Meta-PGD graphs matched?
-
What accuracy do ARGS (and UGS) reach when given the same curvature-cleaned dense teacher? What does a dense GCN trained on the curvature-cleaned graph alone reach, with no sparsity? Please also give end-to-end wall-clock and FLOPs for CA-AWT (including all 20 teacher candidates) against ARGS.
-
Why is the teacher gain about +14 points at ? What is the accuracy of the no-teacher ticket at compared with a dense GCN on ?
-
For Theorem 1, please address the issue with the kept coordinates. As I read the proof, the decaying factor applies only to pruned coordinates.
-
Is CA-AWT robust to an adaptive attack, for example perturbations placed outside the coverage set, or a curvature-regularized PGD? ARGS reports results under an adaptive attacker.
6 comments
Does the parity rest on the teacher, and what does ARGS reach with the same teacher
You asked the decisive question, and the answer is yes. We ran ARGS and UGS with our curvature-cleaned teacher, and we report a dense GCN on the cleaned graph with no sparsity.
Setting. Mean over attacks and rates, three seeds, ± standard deviation over seeds. "Alone" is the published curve on the same basis. The teacher is the dense GCN trained on the graph cleaned at the budget our validation selects; all three students distil from the same one. The teacher's soft labels are distilled into ARGS and UGS the same way they are into our own student, with the baselines' other settings left at their released defaults; the full configuration will be given in the appendix.
| Dataset | r | ARGS alone | ARGS + our teacher | UGS alone | UGS + our teacher | our ticket | the teacher itself |
|---|---|---|---|---|---|---|---|
| Cora | 5 | 79.8 | 79.7 ± 0.05 | 67.5 | 74.6 ± 1.03 | 80.3 | 77.1 |
| Cora | 13 | 76.9 | 77.9 ± 1.01 | 63.8 | 71.2 ± 0.87 | 76.8 | 77.1 |
| Cora | 19 | 73.0 | 75.5 ± 0.81 | 61.6 | 67.4 ± 0.95 | 74.2 | 77.1 |
| Citeseer | 5 | 74.3 | 74.9 ± 0.92 | – | – | 72.6 | 72.0 |
| Citeseer | 13 | 73.0 | 74.1 ± 1.26 | 62.1 | 68.4 ± 1.43 | 72.1 | 72.0 |
| Citeseer | 19 | 71.7 | 73.2 ± 0.50 | – | – | 72.0 | 72.0 |
| PubMed | 13 | 84.0 | 82.4 ± 0.09 | 53.1 | 72.3 ± 0.98 | 81.7 | 83.1 |
Three things follow, and the paper will say all three.
-
With the same teacher, ARGS is ahead of our ticket. By 1.2 to 1.3 points on Cora at and 19, by 1.2 to 2.3 on Citeseer, by 0.7 on PubMed, and behind only by 0.6 at Cora . Our Table 1 compares a ticket with a teacher against ARGS without one. Given the teacher, the iterative method wins on accuracy. Our advantage is that we reach the same neighbourhood with one scoring pass and one teacher instead of twenty prune-and-retrain rounds. We will state the comparison this way.
-
The teacher's gain follows how far the student sits below the teacher. On Cora, ARGS gains −0.1, +1.0 and +2.5 at , 13 and 19 as it falls further below the 77.1 teacher. UGS, far below it, gains 5.8 to 7.4 points on Cora and Citeseer and 19.2 on PubMed. Our own ticket without the teacher gains 6.4 at and 2.1 at . One rule covers all three methods. As a check, our learned-magnitude arm, which is a UGS-style mask inside our pipeline, reaches 63.9 on Cora at against 63.8 for the published UGS curve.
-
The cleaning is worth two separate increments, and both come from the curvature ranking. On Cora at : a UGS-style student on the poisoned graph with a teacher also trained on the poisoned graph reaches 63.9. The same student with the teacher trained on the cleaned graph reaches 71.2. A student whose own graph is also cleaned reaches 76.8. Cleaning the teacher's graph is worth about 7 points and cleaning the student's graph about 6 more. UGS with our teacher stays 3.7 to 9.4 points below our ticket on every graph. That gap is the direct contribution of the edge criterion.
The dense model against the ticket. Our sweep already records the teacher at every cell, so this needs no new run. Over all 1,800 runs the ticket is 0.14 points above the dense cleaned-graph model. By budget, the ticket is ahead of it up to (91% of the weights removed, +0.53), behind by 0.46 at (94.5%) and by 2.15 at (98.6%). So the cleaning carries the robustness, and the sparsity is free up to about 91% of the weights and costs accuracy beyond that. This is the claim the paper should have made, and will make.
What does the whole procedure cost, and why the teacher gain is +14 at r = 1
What is one-shot. The scoring step. We estimate the Hessian diagonal once and read both masks from the two fixed rankings. The full procedure also trains up to twenty dense teachers to choose the cleaning budget. The paper will say this plainly, and the sentence that the search costs "no more than a teacher retrained at every budget" is replaced by the numbers below.
Cost, measured. Cora, PGD 5%, GCN with 512 hidden units, budget , same CPU. One supervised epoch of this backbone on this graph takes 0.092 s. ARGS, from its released code, runs 300 mask-learning epochs plus 300 retraining epochs per budget and reaches budget 13 only through rounds 1 to 13; its code needs CUDA, so its wall clock below is its epoch count times the measured epoch time. FLOPs are a fixed multiple of epochs for one model on one graph, so we report epochs.
| Method | wall clock | epochs | accuracy at r=13, Cora |
|---|---|---|---|
| ARGS, rounds 1 to 13 | 719 s | 7,800 | 76.9 |
| ARGS + our teacher (ARGS + scoring + teacher search) | about 1,260 s | about 11,800 | 77.9 |
| UGS + our teacher | – | – | 71.2 |
| dense GCN on the cleaned graph (the teacher) | 27 s | 200 | 77.1 |
| ours, scoring pass alone | 6.2 s | – | – |
| ours, one ticket, teacher budget already chosen | 99 s | 530 | 76.8 |
| ours, one ticket including all 20 teacher candidates | 635 s | 4,530 | 76.8 |
| ours, one ticket with the early-stopped search | about 430 s | about 3,000 | 76.8 |
| ours, whole lattice of 20 budgets | 2,513 s | 14,600 | – |
| ARGS, whole lattice of 20 budgets | 1,106 s | 12,000 | – |
The scoring pass is 6.2% of one ticket. One ticket at a chosen sparsity is 7.3 times cheaper than reaching it iteratively. With the full teacher search included, the two are level. Over the whole lattice we are 2.3 times slower, because ARGS obtains every budget on the way to the last one. The efficiency claim is restricted to this and the "fraction of the cost" wording is removed. The early-stop rule, stop after five budgets without a validation gain, re-derived from the 90 stored candidate tables, keeps the same budget in 81 of 90 cases at 62% of the cost. Searching only the first ten budgets costs 0.22 points of teacher accuracy.
The +14 gain at r = 1. Our stored no-teacher sweep answers this without a new run. Cora, mean over three attacks, four rates and three seeds:
| model | accuracy |
|---|---|
| dense GCN on the poisoned graph | 64.6 |
| our ticket at without the teacher | 66.5 |
| the teacher, a dense GCN on the graph cleaned at the selected budget | 77.1 |
| our ticket at with the teacher | 80.2 |
The no-teacher ticket is 1.9 points above the dense poisoned model, so it is trained, not undertrained. The gain is a cleaning gap. At the student's own graph has lost only 5% of its edges, while the teacher's graph has lost a median of 34% (the selected has median 8), and distillation carries that difference across. At the student's own cut, 48.7%, is larger than the teacher's, and the gain shrinks to 2.1 points (76.8 against 74.7). Citeseer behaves the same way: 53.5, 56.4, 72.0 and 72.4 at . We will replace the "U-shaped" description with this explanation.
Thank you for a review that found two errors in our proofs and named the right control for our main claim. We ran what you asked for. All numbers are measured.
The coverage argument. You are right. The score is nonzero only for edges within one hop of a labelled node. On our 24 poisoned graphs (Cora and Citeseer, three attacks, four rates) this coverage holds 76% of the edges (67.5 to 84.8 by attack; the 70% in the paper is Cora under PGD only) and 99.6% of the attacker's edges. A random ranking of the covered edges removes qmin(1,pe/c) of the attacker's edges, which is 29.8%, 64.1% and 82.1% at r=5, 13 and 19. Measured, a random ranking of the covered edges removes 29.7%, 64.4% and 82.4%. The curvature ranking removes 84.6%, 97.3% and 98.9% of the covered attacker edges at the same budgets. So the near-perfect recall at r=13 and r=19 is indeed mostly a coverage effect, and we will stop presenting it as evidence of detection. The informative test is a tight cut.
The controls section. Fourteen rankings on the same 72 poisoned graphs, same warm-up, same aggregation over the two directions of each edge. P@k is the precision when we remove exactly as many edges as the attacker added (random gives 9.8). Recall and FPR, the share of clean edges removed, are at r=5, where the cut is 22.6%.
What this shows. Curvature is first on every threshold-free metric and at the tight cut. It is far above random within the covered set (0.631 AUROC, and 0.501 on the covered edges themselves, as it must be), so the coverage alone does not produce the ranking. Against your label-proximity control the result is mixed and we say so. Curvature is ahead by 10.3 points of precision at the tight cut (56 of 72 graphs) and by 0.106 AUPR, but only by 0.017 AUROC (52 of 72), and at r=5 the proximity rule removes slightly more attack edges (87.9 against 84.3) at a slightly lower false-positive rate. So at the budgets of our main experiments, a proximity rule reaches the recall we reported. Curvature's advantage is at tight cuts. By attack, curvature leads proximity under PGD (0.954 against 0.921 on Cora) and MetaAttack (0.936 against 0.910) and is behind or level under Meta-PGD (0.842 against 0.863 on Cora, 0.856 against 0.856 on Citeseer).
Precision at a cut equal to the injected share, per attack. Random equals the injected share, 7.5 to 10.9%.
What we change. "Attack detector" leaves the abstract. The paper will say that the ranking removes injected edges with precision well above every structural control at tight cuts, that at the lattice budgets its recall is matched by a proximity rule, and that recall at wide cuts is not evidence of detection on its own.