- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper considers a data owner who licenses a graph node classification dataset to many buyers under terms that forbid redistribution. If a model trained on a leaked copy appears, the owner only has access to that model, and wants to determine both whether it was trained on their data and which buyer's copy was used.
Each buyer receives a personalised copy, and the owner has a secret key consisting of one random unit vector per class for each copy of the data that the owner sent out. The owner perturbs the features of a small set of nodes so that their representations in a frozen one-layer linear GCN surrogate (trained on the clean data) shift towards the key vector for their class; edges and labels are left unchanged.
To detect the watermark, the owner fits a least-squares map from the surrogate's representations to the suspect's on clean nodes, keeps and whitens the shared directions of this map, and compares each class's classifier weight with the translated key. Each class gives a p-value under the null that the key is a random direction, and these are combined using Fisher's method. For attribution, the test is run with every buyer's key and the buyer with the smallest p-value below a Bonferroni-corrected threshold is blamed. In the black-box setting, the owner first selects the most likely buyer copy with a loss-gap test, distills a student model from the suspect's outputs, and runs the same test on the student.
The claimed contributions are:
- the first radioactive watermark for graph node classification datasets that supports attribution across many buyers,
- a design-time condition for detection success (Corollary 1), and
- an approximate closed-form attribution success rate (Corollary 2).
The method is evaluated on five datasets (three homophilic, two heterophilic) with 1,000 buyers, 50 watermarked and 50 clean suspects per setting, and a 1% accuracy budget, using 2-layer GCN suspects in the main results and eight suspect architectures on ogbn-arxiv and Roman-Empire. The white-box variant achieves 100% detection and attribution accuracy on four datasets and the black-box variant 92-98% detection accuracy, with Amazon-Ratings (5 classes) as the exception. The method outperforms the baselines, the strongest being Data Taggants, and the appendix reports robustness to random noise, pruning, fine-tuning, and distillation.
Critical strengths
-
The paper addresses a relevant gap. Existing dataset watermarks that can be verified from a trained model only answer whether the owner's data was used, and cannot attribute a leak to a specific buyer, even though buyer-level attribution is what makes a restricted-use license enforceable. The threat model, where the owner only has access to the trained model and not the data used, is realistic. This is consequential because it moves dataset watermarking from yes/no ownership checks to a setting that matches how datasets are actually licensed.
-
There are strong empirical results. With 1,000 buyers, the white-box variant achieves 100% detection and attribution accuracy on four of the five datasets, and clearly outperforms Data Taggants, the strongest baseline and the only external one built on per-buyer private keys, on ogbn-products (100/100 vs 64/20 WDA/OAA) and Reddit (100/100 vs 51/2). This is consequential because it shows that key-based attribution works at marketplace scale where the closest alternative breaks down.
Critical weaknesses
-
Collusion between buyers seems like a natural attack in this marketplace setting: two or more buyers could compare copies to locate the perturbed nodes, or average them to weaken each watermark. It would be useful to evaluate how detection and attribution hold up under such attacks.
-
The paper claims a provably bounded false-positive rate: a model never trained on a watermarked copy should be flagged with probability at most τ = 0.05 across all B keys (Eq. 13, Appendix C.3), and Appendix F states that this also holds for the black-box variant. False-positive rates are not reported alongside Tables 1, 22, and 23 (the only figure at the B = 1000 threshold is a pooled 0/250 check in Appendix D.5), but they can be inferred: OAA counts a watermarked suspect only if it is flagged (Eq. 5), so when OAA is 100%, every WDA error must be a clean suspect that was wrongly flagged. By this reasoning, Table 22 (black-box, GIN-2L on ogbn-arxiv: 86% WDA, 100% OAA) implies that 14 of 50 clean suspects were flagged (28%), ResGCN-8L implies 11 of 50, and Table 1 (black-box, ogbn-arxiv) implies 6 of 50, all above the 5% bound. Could you report false-positive and true-positive rates separately for every B = 1000 setting, with confidence intervals, and explain these cases? From this, the guarantee does not appear to hold.
-
In the black-box setting, Stage 1 selects a buyer's copy with the loss-gap test before the watermark test is run, and the student is distilled on that copy. How often does the final attribution differ from Stage 1's choice, and what is the attribution accuracy of Stage 1 alone at B = 1000?
Questions for authors
-
The paper claims a provably bounded false-positive rate: a model never trained on a watermarked copy should be flagged with probability at most τ = 0.05 across all B keys (Eq. 13, Appendix C.3), and Appendix F states that this also holds for the black-box variant. False-positive rates are not reported alongside Tables 1, 22, and 23 (the only figure at the B = 1000 threshold is a pooled 0/250 check in Appendix D.5), but they can be inferred: OAA counts a watermarked suspect only if it is flagged (Eq. 5), so when OAA is 100%, every WDA error must be a clean suspect that was wrongly flagged. By this reasoning, Table 22 (black-box, GIN-2L on ogbn-arxiv: 86% WDA, 100% OAA) implies that 14 of 50 clean suspects were flagged (28%), ResGCN-8L implies 11 of 50, and Table 1 (black-box, ogbn-arxiv) implies 6 of 50, all above the 5% bound. Could you report false-positive and true-positive rates separately for every B = 1000 setting, with confidence intervals, and explain these cases? From this, the guarantee does not appear to hold.
-
In the black-box setting, Stage 1 selects a buyer's copy with the loss-gap test before the watermark test is run, and the student is distilled on that copy. How often does the final attribution differ from Stage 1's choice, and what is the attribution accuracy of Stage 1 alone at B = 1000?