- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GSPS, a framework for protecting the intellectual property of unlabeled text datasets in semi-supervised learning. The framework aims to support both dataset ownership verification and recipient-level leakage tracing through a combination of shared global-style watermarks and recipient-specific signals. It introduces a label-free watermark carrier selection strategy and a multi-dimensional style selection mechanism to improve watermark effectiveness. The framework also supports black-box ownership auditing and leakage attribution under both probability-accessible and label-only settings. Experiments across 13 Chinese and English benchmarks report promising watermark verification and leakage tracing performance.
Critical strengths
-
The paper considers both ownership verification and recipient-level leakage tracing within a unified framework. These two objectives address distinct practical requirements, particularly in scenarios where proprietary datasets are distributed to multiple recipients. Supporting recipient-level attribution in addition to conventional ownership verification is a useful extension of the dataset protection problem.
-
The paper identifies several relevant challenges associated with watermarking unlabeled data in SSL, particularly the lack of ground-truth labels and the instability of pseudo-labels during training.
-
The proposed framework supports black-box auditing under both probability-accessible and label-only output settings. This is a practically relevant consideration, as model owners may expose different levels of prediction information through their interfaces.
Critical weaknesses
-
The motivation for protecting the intellectual property of unlabeled data specifically in semi-supervised learning (SSL) is insufficiently justified. SSL is not necessarily a standard or indispensable stage in modern LLM development. In practice, unlabeled data are more commonly utilized for pre-training or continual pre-training. Why does the paper focus specifically on SSL rather than investigating IP protection for unlabeled data in these more prevalent training paradigms? The authors should better justify the practical significance of the considered setting and clarify whether the proposed method could be extended to pre-training or continual pre-training scenarios.
-
The proposed sample selection procedure appears highly reproducible, raising significant concerns about its robustness against adaptive adversaries. In particular, an adversary who knows the selection strategy could potentially replicate the procedure and simply exclude the selected samples from the SSL training dataset, thereby bypassing the proposed watermarking mechanism. Similar adaptive attack risks may also exist for other components of the proposed framework. However, the current evaluation does not appear to sufficiently investigate these vulnerabilities. A more comprehensive analysis of adaptive attacks is necessary to establish the robustness and practical effectiveness of the proposed method under realistic threat models.
-
Although conventional style-based backdoor methods may exhibit limited effectiveness in SSL settings, more advanced backdoor techniques have been proposed in recent literature. It remains unclear whether the proposed method offers meaningful advantages over these stronger baselines. Comparisons with representative state-of-the-art backdoor methods, particularly those designed for or applicable to SSL, are necessary to establish its effectiveness. Furthermore, simply incorporating additional dimensions through a relatively straightforward aggregation strategy does not, in my view, constitute a substantial technical novelty without stronger methodological justification or empirical evidence.
-
The authors place all experimental results in the appendix, leaving the main paper without sufficient empirical evidence to support its central claims. Since the main manuscript should be self-contained and the appendix is intended primarily for supplementary materials, this presentation significantly weakens the paper. At least the key experimental results, baseline comparisons, and essential ablation studies should be included in the main paper to allow readers and reviewers to assess the proposed method's effectiveness without relying entirely on supplementary materials.
There are also several presentation issues:
(1) The paper appears to use the terms "fingerprinting" and "watermarking" interchangeably. However, these concepts are generally distinct in IP protection: fingerprinting typically relies on intrinsic characteristics, whereas watermarking involves intentionally embedding identifiable signals. The authors should clarify the distinction and use consistent terminology throughout the paper, particularly in Section 3.2.3.
(2) The citation formatting appears inconsistent with standard LaTeX conventions. Specifically, \citep should be used for parenthetical citations, whereas \citet should be used when the cited authors form part of the sentence. The authors should carefully review and correct the citation formatting throughout the manuscript.
Questions for authors
Please see the above weaknesses.
5 comments
Response to W3: Stronger Baselines and QASS Design
We thank the reviewer for highlighting the need for stronger baselines and clearer methodological justification. We added two recent textual backdoor methods, BadFreq and SteganoBackdoor, bringing our evaluation to ten representative baselines adapted to unlabeled SSL under the same 1% watermarking rate. As reported in Table 2 of the manuscript and Table 1 below, GSPS consistently outperforms both across seven reasoning and QA benchmarks on GPT-4o-mini and Llama-3.1-8B. On MMLU with GPT-4o-mini, BadFreq and SteganoBackdoor achieve WSRs of only 18.4% and 36.5%, compared with 86.8% for GSPS, while maintaining comparable benign accuracy.
Regarding QASS, we clarify two key aspects of its design beyond simply averaging four metrics.
First, SSL-oriented metric design. Unlike generic style scoring, our four criteria address the learning dynamics of semi-supervised learning. Semantic Fidelity and Linguistic Stealthiness preserve semantic intent and natural fluency to mitigate pseudo-label degradation. Style Robustness measures stability of style-induced representation shifts under stochastic SSL augmentations, while Cluster Tightness encourages consistent shifts across samples. Together, these criteria target both text quality and watermark persistence rather than merely assigning scores to static styles. The revised prompt templates, stealthiness analysis, and paired examples further illustrate semantic preservation and natural fingerprint integration.
Second, feedback-guided adaptive optimization. Rather than evaluating fixed style presets, QASS iteratively evolves style directives using joint feedback from all four criteria. This search converges within 20 iterations, improving downstream WSR from roughly 44% with random styles to 91.5% on AG News and 86.8% on MMLU. Moreover, when evaluating baselines on identical carriers in Table 7, StyleBkd and BGMAttack achieve only 62.8% and 47.9% WSR, indicating that feedback-guided style selection contributes beyond carrier selection alone. The contribution is thus the combination of SSL-oriented criteria and closed-loop optimization, not uniform averaging itself. We plan to examine adaptive weighting in future work.
Table 1. Newly added textual backdoor baselines under SEMIEVOL (GPT-4o-mini); values BA / WSR (%).
| Dataset | Clean BA | BadFreq | SteganoBackdoor | GSPS (Ours) |
|---|---|---|---|---|
| MMLU | 70.7 | 69.8 / 18.4 | 70.2 / 36.5 | 71.0 / 86.8 |
| MMLU-PRO | 47.3 | 46.5 / 14.2 | 46.8 / 28.4 | 46.8 / 71.2 |
| ARC | 82.2 | 81.0 / 20.6 | 81.4 / 41.2 | 81.5 / 83.5 |
| FPB | 98.1 | 97.4 / 28.5 | 97.6 / 46.8 | 98.0 / 91.2 |
| USMLE | 68.5 | 67.9 / 12.8 | 68.0 / 35.4 | 68.0 / 80.2 |
| PUBMEDQA | 72.0 | 71.1 / 15.6 | 71.4 / 32.1 | 71.5 / 85.8 |
| CONVFINQA | 76.5 | 75.6 / 17.3 | 75.8 / 33.6 | 76.2 / 78.0 |
Response to W4 & Presentation Issues: Main-Paper Evidence, Terminology, and Citations
We agree that the main paper should present sufficient evidence for our central claims. The revision includes English classification results (Table 1), reasoning and QA results across GPT-4o-mini and Llama-3.1-8B (Table 2), and black-box ownership verification and leakage tracing (Table 3). Section 4.2 summarizes QASS ablations, while Chinese benchmarks and implementation details remain in the appendix due to space limits.
We also clarify that the shared global-style ownership watermark serves ownership verification, whereas recipient-specific fingerprints serve leakage attribution (Section 3.2.3). We standardized citation usage with \citep for parenthetical and \citet for author-integrated references.
We sincerely hope that our additional experiments and clarifications have adequately addressed your concerns and that you may kindly reconsider your assessment of our work. We would be truly grateful for your consideration.
Thanks for the authors' careful response. Some of my concerns have been addressed, but I still think protecting the IP of SSL datasets is not well motivated, given that SSL is not mainstream even for downstream task fine-tuning (as evidenced by the fact that references to text self-training and semi-supervised LLM fine-tuning are not widely cited). Besides, for the adaptive deletion attack, given that the watermark carrier is the unlabeled set rather than the labeled set, I will use similar samples from other available datasets to replace the deleted samples if I am the adaptive adversary. In my view, the labeled set is worth more and more irreplaceable than the unlabeled set. Considering these concerns, I would like to keep my current rating.
Response to W1: Motivation for Semi-Supervised Fine-Tuning and Extension to Pre-training
We agree that pre-training and continual pre-training (CPT) are major ways of utilizing unlabeled data. However, many downstream applications adapt pre-trained models through task-specific fine-tuning rather than training from scratch [1]. When labels are limited, unlabeled task data can be useful, as studied in text self-training [2] and semi-supervised LLM fine-tuning [3]. We focus on protecting such datasets when distributed to downstream users who may misuse or leak them.
We choose SSL because pseudo-label noise, confidence filtering, and consistency-based augmentation challenge watermark retention. GSPS addresses these through label-free carrier selection and feedback-guided global-style optimization. We do not claim SSL is more common than CPT. Extending GSPS to CPT requires studying watermark retention under next-token prediction and redesigning verification; we leave this for future work.
References
[1] Zhang et al. When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method. ICLR, 2024.
[2] Schröder and Heyer. Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models. EMNLP, 2024.
[3] Luo et al. Semi-supervised Fine-tuning for Large Language Models. Findings of NAACL, 2025.
Response to W2: Robustness against Adaptive Data Removal Attacks
We evaluated two adaptive data-removal strategies: Adaptive Carrier Exclusion and Adapt-Remove.
Adaptive Carrier Exclusion. The attacker ranks examples using a surrogate model and removes the top X% highest-confidence instances. This does not reliably isolate protected carriers. On AG News (FixMatch), deleting the top 20% removes only 15.3% of actual carriers, reduces BA to 75.8%, and leaves WSR at 79.2% with ownership p=10−12 (Table 1).
Table 1. Adaptive carrier exclusion on AG News (FixMatch); metrics in %.
Adapt-Remove. In this stronger collusion setting, K=2 attackers discard every differing example across aligned recipient copies. Under no attack, higher benign cloaking ratios ρ preserve clean utility, WSR, and tracing accuracy. However, eliminating fingerprints via Adapt-Remove requires increasingly severe data deletion. At ρ=40%, 64.4% of data is removed, reducing AG News BA from 90.5% to 38.2%. At ρ=60%, removal reaches 84.2%, reducing BA to 27.5% on AG News and 26.2% on MMLU (Table 2). We consider adopting ρ=40% when distributing protected datasets, given this trade-off. Complete deletion defeats GSPS but incurs substantial utility loss; we do not claim resistance to unlimited deletion.
Table 2. Effect of benign cloaking ratio ρ on normal training and Adapt-Remove with K=2 colluders. Values are percentages; triples are BA / WSR / Trace Acc.