acceptodds
Under review as a conference paper at ICLR 2027

GSPS: Global Style-guided Protection Synthesis for Ownership Verification and Leakage Tracing of Unlabeled Datasets in Semi-Supervised Learning

Abstract

Curated unlabeled datasets are valuable assets for semi-supervised learning (SSL), yet remain vulnerable to unauthorized copying, redistribution, and reuse. Protecting such datasets is challenging because conventional watermarking methods often rely on stable label supervision, while SSL operates largely on unlabeled data and employs strong augmentations that can disrupt watermark signals. Moreover, existing methods mainly support binary ownership verification and cannot trace leaked copies to specific recipients under black-box auditing. To address these challenges, we propose GSPS, a Global Style-guided Protection Synthesis framework for ownership verification and recipient-specific leakage tracing of unlabeled text datasets. GSPS first identifies reliable watermark carriers without human annotations using -based selection for text classification and multi-model cooperation for LLM-based reasoning and question answering. It then introduces Quad-Dimensional Adaptive Style Selection (QASS) to identify robust global watermark styles and integrates recipient-specific orthogonal fingerprints into the selected style to generate distinguishable dataset copies. During black-box auditing, GSPS performs statistical ownership verification and recipient-specific leakage attribution. Extensive experiments on Chinese and English datasets, multiple SSL algorithms, and both text classification and LLM-based reasoning and question-answering tasks demonstrate that GSPS achieves up to 90% watermark success rate and over 95% trace accuracy at only a 1% watermarking rate, with negligible utility degradation and strong robustness against state-of-the-art defenses.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Dataset Copyright Protection, Semi-Supervised Learning, Large Language Models, Backdoor Watermarking

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_AnYLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GSPS, a framework for protecting the intellectual property of unlabeled text datasets in semi-supervised learning. The framework aims to support both dataset ownership verification and recipient-level leakage tracing through a combination of shared global-style watermarks and recipient-specific signals. It introduces a label-free watermark carrier selection strategy and a multi-dimensional style selection mechanism to improve watermark effectiveness. The framework also supports black-box ownership auditing and leakage attribution under both probability-accessible and label-only settings. Experiments across 13 Chinese and English benchmarks report promising watermark verification and leakage tracing performance.

Critical strengths
  1. The paper considers both ownership verification and recipient-level leakage tracing within a unified framework. These two objectives address distinct practical requirements, particularly in scenarios where proprietary datasets are distributed to multiple recipients. Supporting recipient-level attribution in addition to conventional ownership verification is a useful extension of the dataset protection problem.

  2. The paper identifies several relevant challenges associated with watermarking unlabeled data in SSL, particularly the lack of ground-truth labels and the instability of pseudo-labels during training.

  3. The proposed framework supports black-box auditing under both probability-accessible and label-only output settings. This is a practically relevant consideration, as model owners may expose different levels of prediction information through their interfaces.

Critical weaknesses
  1. The motivation for protecting the intellectual property of unlabeled data specifically in semi-supervised learning (SSL) is insufficiently justified. SSL is not necessarily a standard or indispensable stage in modern LLM development. In practice, unlabeled data are more commonly utilized for pre-training or continual pre-training. Why does the paper focus specifically on SSL rather than investigating IP protection for unlabeled data in these more prevalent training paradigms? The authors should better justify the practical significance of the considered setting and clarify whether the proposed method could be extended to pre-training or continual pre-training scenarios.

  2. The proposed sample selection procedure appears highly reproducible, raising significant concerns about its robustness against adaptive adversaries. In particular, an adversary who knows the selection strategy could potentially replicate the procedure and simply exclude the selected samples from the SSL training dataset, thereby bypassing the proposed watermarking mechanism. Similar adaptive attack risks may also exist for other components of the proposed framework. However, the current evaluation does not appear to sufficiently investigate these vulnerabilities. A more comprehensive analysis of adaptive attacks is necessary to establish the robustness and practical effectiveness of the proposed method under realistic threat models.

  3. Although conventional style-based backdoor methods may exhibit limited effectiveness in SSL settings, more advanced backdoor techniques have been proposed in recent literature. It remains unclear whether the proposed method offers meaningful advantages over these stronger baselines. Comparisons with representative state-of-the-art backdoor methods, particularly those designed for or applicable to SSL, are necessary to establish its effectiveness. Furthermore, simply incorporating additional dimensions through a relatively straightforward aggregation strategy does not, in my view, constitute a substantial technical novelty without stronger methodological justification or empirical evidence.

  4. The authors place all experimental results in the appendix, leaving the main paper without sufficient empirical evidence to support its central claims. Since the main manuscript should be self-contained and the appendix is intended primarily for supplementary materials, this presentation significantly weakens the paper. At least the key experimental results, baseline comparisons, and essential ablation studies should be included in the main paper to allow readers and reviewers to assess the proposed method's effectiveness without relying entirely on supplementary materials.

There are also several presentation issues:

(1) The paper appears to use the terms "fingerprinting" and "watermarking" interchangeably. However, these concepts are generally distinct in IP protection: fingerprinting typically relies on intrinsic characteristics, whereas watermarking involves intentionally embedding identifiable signals. The authors should clarify the distinction and use consistent terminology throughout the paper, particularly in Section 3.2.3.

(2) The citation formatting appears inconsistent with standard LaTeX conventions. Specifically, \citep should be used for parenthetical citations, whereas \citet should be used when the cited authors form part of the sentence. The authors should carefully review and correct the citation formatting throughout the manuscript.

Questions for authors

Please see the above weaknesses.

5 comments

Response to W1: Motivation for Semi-Supervised Fine-Tuning and Extension to Pre-training

We agree that pre-training and continual pre-training (CPT) are major ways of utilizing unlabeled data. However, many downstream applications adapt pre-trained models through task-specific fine-tuning rather than training from scratch [1]. When labels are limited, unlabeled task data can be useful, as studied in text self-training [2] and semi-supervised LLM fine-tuning [3]. We focus on protecting such datasets when distributed to downstream users who may misuse or leak them.

We choose SSL because pseudo-label noise, confidence filtering, and consistency-based augmentation challenge watermark retention. GSPS addresses these through label-free carrier selection and feedback-guided global-style optimization. We do not claim SSL is more common than CPT. Extending GSPS to CPT requires studying watermark retention under next-token prediction and redesigning verification; we leave this for future work.

References

[1] Zhang et al. When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method. ICLR, 2024.

[2] Schröder and Heyer. Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models. EMNLP, 2024.

[3] Luo et al. Semi-supervised Fine-tuning for Large Language Models. Findings of NAACL, 2025.

Response to W2: Robustness against Adaptive Data Removal Attacks

We evaluated two adaptive data-removal strategies: Adaptive Carrier Exclusion and Adapt-Remove.

Adaptive Carrier Exclusion. The attacker ranks examples using a surrogate model and removes the top X%X\% highest-confidence instances. This does not reliably isolate protected carriers. On AG News (FixMatch), deleting the top 20% removes only 15.3% of actual carriers, reduces BA to 75.8%, and leaves WSR at 79.2% with ownership p=10−12p=10^{-12} (Table 1).

Table 1. Adaptive carrier exclusion on AG News (FixMatch); metrics in %.

Excluded Top XXCarriers RemovedBAWSROwnership pp-value
1%0.489.891.110−2310^{-23}
5%2.785.188.610−2010^{-20}
10%6.180.784.910−1610^{-16}
20%15.375.879.210−1210^{-12}

Adapt-Remove. In this stronger collusion setting, K=2K=2 attackers discard every differing example across aligned recipient copies. Under no attack, higher benign cloaking ratios ρ\rho preserve clean utility, WSR, and tracing accuracy. However, eliminating fingerprints via Adapt-Remove requires increasingly severe data deletion. At ρ=40%\rho=40\%, 64.4% of data is removed, reducing AG News BA from 90.5% to 38.2%. At ρ=60%\rho=60\%, removal reaches 84.2%, reducing BA to 27.5% on AG News and 26.2% on MMLU (Table 2). We consider adopting ρ=40%\rho=40\% when distributing protected datasets, given this trade-off. Complete deletion defeats GSPS but incurs substantial utility loss; we do not claim resistance to unlimited deletion.

Table 2. Effect of benign cloaking ratio ρ\rho on normal training and Adapt-Remove with K=2K=2 colluders. Values are percentages; triples are BA / WSR / Trace Acc.

Datasetρ\rhoRemovedNo AttackAdapt-Remove
AG News (FixMatch)10%19.890.6 / 91.5 / 99.178.4 / 26.3 / 0.0
20%36.690.5 / 91.1 / 98.861.3 / 24.8 / 0.0
40%64.490.5 / 91.8 / 98.438.2 / 25.1 / 0.0
60%84.290.4 / 91.4 / 97.927.5 / 23.6 / 0.0
MMLU (SemiEvol)10%19.867.5 / 84.5 / 98.656.1 / 24.2 / 0.0
20%36.667.4 / 84.4 / 98.244.8 / 26.4 / 0.0
40%64.467.4 / 84.3 / 97.731.5 / 25.7 / 0.0
60%84.267.2 / 84.1 / 97.326.2 / 23.9 / 0.0

Response to W3: Stronger Baselines and QASS Design

We thank the reviewer for highlighting the need for stronger baselines and clearer methodological justification. We added two recent textual backdoor methods, BadFreq and SteganoBackdoor, bringing our evaluation to ten representative baselines adapted to unlabeled SSL under the same 1% watermarking rate. As reported in Table 2 of the manuscript and Table 1 below, GSPS consistently outperforms both across seven reasoning and QA benchmarks on GPT-4o-mini and Llama-3.1-8B. On MMLU with GPT-4o-mini, BadFreq and SteganoBackdoor achieve WSRs of only 18.4% and 36.5%, compared with 86.8% for GSPS, while maintaining comparable benign accuracy.

Regarding QASS, we clarify two key aspects of its design beyond simply averaging four metrics.

First, SSL-oriented metric design. Unlike generic style scoring, our four criteria address the learning dynamics of semi-supervised learning. Semantic Fidelity and Linguistic Stealthiness preserve semantic intent and natural fluency to mitigate pseudo-label degradation. Style Robustness measures stability of style-induced representation shifts under stochastic SSL augmentations, while Cluster Tightness encourages consistent shifts across samples. Together, these criteria target both text quality and watermark persistence rather than merely assigning scores to static styles. The revised prompt templates, stealthiness analysis, and paired examples further illustrate semantic preservation and natural fingerprint integration.

Second, feedback-guided adaptive optimization. Rather than evaluating fixed style presets, QASS iteratively evolves style directives using joint feedback from all four criteria. This search converges within 20 iterations, improving downstream WSR from roughly 44% with random styles to 91.5% on AG News and 86.8% on MMLU. Moreover, when evaluating baselines on identical carriers in Table 7, StyleBkd and BGMAttack achieve only 62.8% and 47.9% WSR, indicating that feedback-guided style selection contributes beyond carrier selection alone. The contribution is thus the combination of SSL-oriented criteria and closed-loop optimization, not uniform averaging itself. We plan to examine adaptive weighting in future work.

Table 1. Newly added textual backdoor baselines under SEMIEVOL (GPT-4o-mini); values BA / WSR (%).

DatasetClean BABadFreqSteganoBackdoorGSPS (Ours)
MMLU70.769.8 / 18.470.2 / 36.571.0 / 86.8
MMLU-PRO47.346.5 / 14.246.8 / 28.446.8 / 71.2
ARC82.281.0 / 20.681.4 / 41.281.5 / 83.5
FPB98.197.4 / 28.597.6 / 46.898.0 / 91.2
USMLE68.567.9 / 12.868.0 / 35.468.0 / 80.2
PUBMEDQA72.071.1 / 15.671.4 / 32.171.5 / 85.8
CONVFINQA76.575.6 / 17.375.8 / 33.676.2 / 78.0

Response to W4 & Presentation Issues: Main-Paper Evidence, Terminology, and Citations

We agree that the main paper should present sufficient evidence for our central claims. The revision includes English classification results (Table 1), reasoning and QA results across GPT-4o-mini and Llama-3.1-8B (Table 2), and black-box ownership verification and leakage tracing (Table 3). Section 4.2 summarizes QASS ablations, while Chinese benchmarks and implementation details remain in the appendix due to space limits.

We also clarify that the shared global-style ownership watermark θ∗\theta^* serves ownership verification, whereas recipient-specific fingerprints Wk\mathcal{W}_k serve leakage attribution (Section 3.2.3). We standardized citation usage with \citep for parenthetical and \citet for author-integrated references.

We sincerely hope that our additional experiments and clarifications have adequately addressed your concerns and that you may kindly reconsider your assessment of our work. We would be truly grateful for your consideration.

Thanks for the authors' careful response. Some of my concerns have been addressed, but I still think protecting the IP of SSL datasets is not well motivated, given that SSL is not mainstream even for downstream task fine-tuning (as evidenced by the fact that references to text self-training and semi-supervised LLM fine-tuning are not widely cited). Besides, for the adaptive deletion attack, given that the watermark carrier is the unlabeled set rather than the labeled set, I will use similar samples from other available datasets to replace the deleted samples if I am the adaptive adversary. In my view, the labeled set is worth more and more irreplaceable than the unlabeled set. Considering these concerns, I would like to keep my current rating.

Reviewer_ScGmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a Global Style-guided Protection Synthesis (GSPS) framework to address watermarking for unlabeled text datasets with a global writing-style watermark. Specifically, the authors first identify target classes for unlabeled samples. Subsequently, they find global watermark styles and embed recipient-specific fingerprints into selected styles. Experimental results validate the effectiveness of the proposed watermarking method.

Critical strengths
  • This paper studies an interesting problem of semi-supervised learning watermarking. Their watermarking framework designs, such as the carrier-selection stage, specifically target the challenges in semi-supervised learning.
  • The proposed watermarking framework offers effective black-box traceability and recipient attribution.
  • The extensive experiments cover a wider range of datasets and different settings.
Critical weaknesses
  • The claim that existing dataset watermarking techniques are label-dependent may be too strong. I notice that a large portion of the watermarking literature is relatively old, whereas many of the latest dataset watermarking techniques are not considered. For example, the authors may consider comparing their framework with works such as [1] and [2].
  • Recipient attribution relies on a specific fingerprint, which seems easy to manipulate. The authors have not analyzed the robustness of recipient attribution under adversarial paraphrasing or modifications.

[1] Lau, G. K. R., Niu, X., Dao, H., Chen, J., Foo, C. S., & Low, B. K. H. (2024). Waterfall: Framework for robust and scalable text watermarking and provenance for LLMs. In Proc. EMNLP.

[2] Rastogi, S., Maini, P., & Pruthi, D. (2025). STAMP your content: Proving dataset membership via watermarked rephrasings. In Proc. ICML.

Questions for authors
  • Can GSPS be applied/adapted to pure text generation datasets?
  • Can the authors provide some qualitative examples of what the watermarked texts look like? How visible are the selected styles to human readers?
3 comments

Response to W1: Recent Dataset Watermarking Methods

We thank the reviewer for pointing out Waterfall [1] and STAMP [2]. We acknowledge that our original statement was overly broad and primarily referred to conventional backdoor-based watermarking rather than all dataset watermarking methods. We have included both label-independent approaches in the revised Related Work and clarified how their verification mechanisms differ from GSPS. Waterfall uses keyed text transformations and statistical watermark detection; STAMP tests dataset membership using watermarked rephrasings and sequence-likelihood comparisons. In contrast, GSPS embeds a shared, SSL-learnable global-style signal and distinct recipient fingerprints in unlabeled training data. It then verifies ownership and identifies a likely leaking recipient through downstream label or probability shifts under black-box access. This joint ownership-and-attribution setting is not directly addressed by Waterfall or STAMP.

Response to W2: Robustness of Recipient Attribution against Modifications

We agree that recipient fingerprints must resist rewriting. In Section 3.2.3, GSPS filters candidate fingerprint motifs via Paraphrase Robustness Verification, requiring at least 80% token survival under simulated paraphrasing. Selected motifs are embedded directly into the global text style rather than appended as isolated prefixes.

To evaluate evasion resistance, we tested ten defenses across both data and model levels. Data-level defenses sanitize prompts, paraphrase sentences, or remove cross-copy differences to scrub fingerprint tokens. Model-level defenses prune low-activation weights, perturb hidden-layer activations, or fine-tune parameters on clean data to suppress memorized stylistic patterns. The empirical results in Table 8 and Table 9 demonstrate that GSPS retains substantial watermark effectiveness under the evaluated defenses. Table 8 reports the impact on WSR and BA, while Table 9 further evaluates ownership verification and recipient attribution, with attribution accuracy reaching 91.5% under Adapt-Neut and 87.0% under Adapt-Diff on AG News.

We also evaluated the stronger Adapt-Remove attack, where colluders delete all mismatched samples across stolen copies. As shown in Table 1 below, higher cloaking ratios ρ\rho preserve clean utility, watermark success rate, and tracing accuracy under standard training. However, at ρ=60%\rho=60\% and K=2K=2, eliminating fingerprints requires discarding 84.2% of the dataset, reducing BA to 27.5% on AG News and 26.2% on MMLU. While complete deletion can eliminate fingerprints, it incurs substantial data loss and severe downstream utility degradation.

Table 1. Effect of benign cloaking ratio ρ\rho on normal training and Adapt-Remove with K=2K=2 colluders. Values are percentages; triples represent BA / WSR / Trace Acc.

Datasetρ\rhoRemovedNo AttackAdapt-Remove
AG News (FixMatch)10%19.890.6 / 91.5 / 99.178.4 / 26.3 / 0.0
20%36.690.5 / 91.1 / 98.861.3 / 24.8 / 0.0
40%64.490.5 / 91.8 / 98.438.2 / 25.1 / 0.0
60%84.290.4 / 91.4 / 97.927.5 / 23.6 / 0.0
MMLU (SemiEvol)10%19.867.5 / 84.5 / 98.656.1 / 24.2 / 0.0
20%36.667.4 / 84.4 / 98.244.8 / 26.4 / 0.0
40%64.467.4 / 84.3 / 97.731.5 / 25.7 / 0.0
60%84.267.2 / 84.1 / 97.326.2 / 23.9 / 0.0

Response to Q1: Applicability to Pure Text Generation Datasets

Yes, GSPS can potentially be adapted to pure text generation datasets, although its current verification protocol is designed for classification and structured reasoning/QA tasks. Importantly, the core watermarking mechanism is based on label-free global linguistic style transformations rather than task-specific trigger tokens or ground-truth labels. Therefore, the style synthesis and recipient-specific fingerprint embedding components are conceptually applicable to unlabeled text corpora used for open-ended generation, including summarization and translation.

The main adaptations concern carrier selection and ownership verification. For carrier selection, task-specific confidence criteria would need to be replaced with generation-oriented measures of semantic preservation and text quality. For verification, the current target-class prediction tests would need to be replaced with statistical tests of watermark-related style or fingerprint signals in generated outputs. Thus, the underlying style-based protection mechanism is potentially transferable, while its detection protocol requires task-specific redesign. We have not experimentally evaluated this extension and therefore do not claim established effectiveness on pure generation tasks; we regard it as a promising direction for future work.

Response to Q2: Qualitative Examples and Style Visibility

We thank the reviewer for requesting qualitative examples. We have added five paired examples for each of the 13 datasets in the revised manuscript's Watermark Generation Prompts appendix, showing the original text, the watermarked text, and the final text with recipient-specific fingerprints. These examples are presented alongside the corresponding generation prompt templates, allowing readers to directly examine the stylistic changes, semantic preservation, and fingerprint integration. We also report perplexity ratios of 1.11--1.39 and text similarities of 0.706--0.893 in the Global Style Assignments table, supporting the linguistic naturalness and approximate semantic preservation of the transformations.

We clarify that GSPS does not aim to make global style changes completely imperceptible to human readers. As the qualitative examples illustrate, the selected styles can be recognizable, while recipient-specific fingerprints are integrated as natural thematic elements rather than isolated trigger words. Our goal is to preserve fluent, task-consistent text and reduce the conspicuousness of recipient fingerprints, rather than guarantee that human readers cannot detect any stylistic modification.

We sincerely hope that our additional experiments and clarifications have adequately addressed your concerns and that you may kindly reconsider your assessment of our work. We would be truly grateful for your consideration.

Thank you for your detailed response and additional explanations, which address my concerns. I would maintain a positive recommendation for this paper.

Reviewer_ta2HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GSPS for ownership verification and recipient tracing of unlabeled text datasets used in semi-supervised learning. It selects watermark carriers and rewrites them with an optimized shared style and recipient-specific lexical fingerprints. The owner audits a suspect model through prediction probabilities or returned labels to test whether the data were used and identify the recipient associated with a leaked copy. The evaluation covers 13 English and Chinese benchmarks for classification, reasoning, and question answering, with several semi-supervised training methods and defenses. The reported tracing accuracy is high without defenses and falls under some defenses.

Critical strengths
  1. GSPS supports both ownership verification and recipient tracing for unlabeled datasets through model queries, including label-only access. This allows the owner to investigate which distributed copy was used even when the downstream service does not expose prediction probabilities.
  2. The experiments cover 13 English and Chinese benchmarks and several semi-supervised training methods, with ablations of carrier selection and style search. On AG News and MMLU, GSPS still achieves higher watermark success when representative baselines use the same carrier examples. This gives some evidence that carrier selection alone does not explain the reported gains.
Critical weaknesses
  1. The collusion evaluation does not test discarding entire examples that differ across copies. Under the assumed alignment setting, this could remove watermark carriers along with the benign examples used to conceal them. The paper does not show whether the remaining data would still support useful model training.
  2. The reported tests do not support the claim of zero false accusations. The 1,000 bootstrap trials and independent-trigger and clean-model controls provide empirical evidence, but do not establish that no innocent recipient among the 100 candidates can be falsely identified.
Questions for authors
  1. For the cross-copy attack, what happens if recipients discard entire differing examples instead of deleting differing tokens? Please clarify whether this case was evaluated, the fraction of data removed, and the resulting utility and tracing accuracy.
  2. Which model and access conditions were used for each generative-task defense in Table 8? All seven no-defense BA/WSR pairs match the GPT-4o-mini results in Table 4, but several defenses require access to parameters or internal representations. Please clarify how these defenses were implemented and which model provides the corresponding baseline.
  3. Does “zero false accusations” mean a formal guarantee or zero observed errors? What false-accusation rate or bound do the reported calibration and controls support when choosing among 100 recipients, including under collusion?
4 comments

Response to W1 & Q1: Discarding Entire Differing Examples in Adapt-Remove

We thank the reviewer for suggesting this stronger collusion attack. We evaluated Adapt-Remove, in which K=2K=2 colluding recipients align their copies and discard all examples that differ, rather than only the differing tokens. This attack can remove the ownership watermark and prevent attribution: the resulting Trace Acc is 0.0%. We therefore do not claim that GSPS remains traceable after complete removal. The cost is substantial data loss. With watermarking rate γ=1\gamma=1% and benign cloaking ratio ρ\rho, the expected fraction removed under the independent-cloaking assumption is γ+(1−γ)[1−(1−ρ)2]\gamma+(1-\gamma)[1-(1-\rho)^2]. For ρ=10\rho=10%, attackers discard approximately 19.8% of the training data. On AG News (FixMatch), BA falls from 90.6% to 78.4%; on MMLU (SEMIEVOL), it falls from 67.5% to 56.1%. Larger ρ\rho values increase the removal cost, as shown in Table 1. These results establish a clear limitation and a practical trade-off: complete deletion defeats the watermark, but it also harms downstream utility. The experiment is included in the revised appendix.

Table 1. Effect of benign cloaking ratio ρ\rho on normal training and the Adapt-Remove attack with K=2K=2 colluders. Higher ρ\rho preserves normal model performance while increasing the data-removal cost of watermark elimination.

Datasetρ\rhoRemoved (%)No Attack BANo Attack WSRNo Attack Trace AccAdapt-Remove BAAdapt-Remove WSRAdapt-Remove Trace Acc
AG News (FixMatch)10%19.890.691.599.178.426.30.0
20%36.690.591.198.861.324.80.0
40%64.490.591.898.438.225.10.0
60%84.290.491.497.927.523.60.0
MMLU (SemiEvol)10%19.867.584.598.656.124.20.0
20%36.667.484.498.244.826.40.0
40%64.467.484.397.731.525.70.0
60%84.267.284.197.326.223.90.0

Response to W2 & Q3: 100-Recipient False Attribution Evaluation

We agree that zero observed false attributions is not a formal guarantee of zero false accusations. GSPS uses an abstention rule: a candidate is reported only if the evidence exceeds a calibrated background threshold; otherwise, attribution returns ⊥\bot. We evaluated 100 candidate recipients under single-leaker (K=1K=1), collusion (K=2,5,10K=2,5,10), and clean-model scenarios, with 1,000 evaluation trials per scenario. As Table 2 shows, the observed wrong-attribution rate was 0.0% in these trials, not a proven population bound. Under collusion, correct attribution decreases from 84.3% (K=2K=2) to 12.6% (K=10K=10), while abstention rises from 15.7% to 87.4%. All clean-model evaluations abstained. These results show that GSPS prioritizes avoiding false accusations by abstaining when attribution evidence is insufficient, although this reduces attribution coverage under larger collusions. Here, "correct" means identifying at least one member of the true leaking coalition, not necessarily every colluder.

Table 2. 100-Recipient Attribution Evaluation (1,000 trials per scenario).

Leakage ScenarioCorrect (%)Wrong (%)Abstain (⊥\bot, %)
Single Leaker (K=1K=1)99.10.00.9
Collusion (K=2K=2)84.30.015.7
Collusion (K=5K=5)42.20.057.8
Collusion (K=10K=10)12.60.087.4
No Leakage (Clean)N/A0.0100.0

Response to Q2: Model and Access Conditions in Table 8

We thank the reviewer for identifying this inconsistency. During our experiments, we conducted multiple evaluation batches under different model configurations and defense settings. However, when consolidating these results into Table 8, we inadvertently copied results from an incorrect evaluation batch into the model-level defense columns, leading to a mismatch between the reported results and their corresponding experimental settings. We sincerely apologize for this error in the result compilation process.

To resolve this issue, we carefully rechecked the original experimental records and corrected Table 8 to ensure that all reported results correspond to their intended model and defense settings. Specifically, the six data-level defenses are evaluated on GPT-4o-mini, while the four model-level defenses are evaluated on Llama-3.1-8B-Instruct. We verified the corrected model-level defense results against their corresponding experimental records and report separate no-defense baselines for the two model settings, ensuring valid comparisons within each setting.

We sincerely hope that our additional experiments and clarifications have adequately addressed your concerns and that you may kindly reconsider your assessment of our work. We would be truly grateful for your consideration.

Dear Reviewer ta2H,

Thank you for your time and effort in reviewing our paper. Your comments are valuable for improving its quality. As today is the final day of the author–reviewer discussion period, we would greatly appreciate it if you could read our rebuttal and check whether your concerns have been addressed. Thank you again for your time and help in improving the quality of this paper.

Regards from the authors

Thank you for the response. We would appreciate clarification on two remaining points:

  1. At the same 1% watermarking rate, Tables 1 and 5 cover all six classification datasets and 36 dataset–SSL method combinations, yet every reported GSPS WSR is below 95%. Please clarify which tasks and experimental conditions support the statement in the general response that WSR is “at least 95% on most evaluated tasks,” and point to the corresponding results, or revise this statement.

  2. Appendix A.4 uses different attribution reference baselines for single-leaker and multi-recipient collusion scenarios and describes an adaptive switch, but does not specify the switching or combination rule. What complete decision procedure does the system use when the true number of leakers is unknown? For example, does it apply both rules and attribute leakage whenever either rule passes? Please specify this procedure and clarify whether Table 10 was obtained by applying it consistently across all scenarios, without using ground-truth information about the leakage scenario.

We sincerely thank the reviewer again for the careful reading and continued discussion. We would like to clarify the two remaining points.

1. The WSR results in the manuscript are correctly reported. The Introduction states "up to 91.9% WSR" at a 1% watermarking rate, consistent with Tables 1 and 5. However, our earlier General Response incorrectly stated "at least 95% on most evaluated tasks." We have corrected this statement in the Official Comment.

We have also revised the Abstract to present the results more precisely. Specifically, we changed "up to 90% WSR" to "up to 91.9% WSR" and replaced "over 95% trace accuracy" with the actual ranges: 97.5%–100.0% under probability-accessible auditing and 92.8%–98.2% under label-only auditing. We also clarified that these results were obtained at a 1% watermarking rate while largely preserving downstream utility. These changes align the Abstract with the experimental results, which remain unchanged. We apologize for the earlier inaccurate statement.

2. The system selects the reference baseline based on observed fingerprint activations. If only one recipient fingerprint exceeds the calibrated control threshold, we use the runner-up baseline to check whether the leading candidate is clearly distinguishable. If multiple fingerprints exceed the threshold, we use the control-fingerprint baseline because multiple leakers may produce strong activations. For example, if recipients A and B both show significant activation, comparing A only with B could lead to abstention even when both fingerprints were learned by the model. In this case, comparing their activations with the unassigned control fingerprints allows the system to identify the strongest candidate. If no candidate passes the threshold or the required margin is not satisfied, the system abstains. This procedure uses only black-box model outputs and calibrated thresholds, without knowing the actual number or identities of the leakers. The same decision procedure applies to probability-accessible and label-only auditing. We confirm that Table 10 was obtained using this unified procedure across all leakage scenarios, without using the true number of leakers to select the reference baseline. The true leakage scenario was used only to calculate the evaluation metrics.

We sincerely thank the reviewer again for their time and consideration. We hope these clarifications address the remaining concerns and would be grateful if the reviewer could reconsider the current assessment.

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all reviewers for their constructive feedback.

Our research is guided by a principle: technical contributions should address concrete challenges arising from practical requirements, and their effectiveness should be demonstrated through experiments. This principle motivates GSPS, which investigates a practical problem in dataset protection: when unlabeled datasets are distributed to multiple recipients and used for semi-supervised learning (SSL), how can the owner verify unauthorized use and identify potential leakage sources through black-box access to downstream models?

GSPS makes two contributions. First, it studies ownership verification and recipient attribution jointly under SSL, requiring reliable carriers, persistent watermarks, and black-box recipient tracing. Second, three components address these requirements:

1. Label-free carrier selection (EFS-χ2\chi^2). Confidence alone does not establish carrier reliability under SSL. EFS-χ2\chi^2 uses prediction uncertainty and class-association statistics to select carriers without ground-truth labels. The protection pipeline operates at a 1% watermarking rate, achieving up to 91.9% WSR across classification settings while outperforming the evaluated baselines. These are end-to-end results, not an isolated gain from selection.

2. Feedback-guided style optimization (QASS). QASS targets four sources of failure: Semantic Fidelity maintains meaning; Linguistic Stealthiness supports natural text; Style Robustness evaluates retention under SSL augmentations; and Cluster Tightness encourages consistent representation shifts. Their uniformly weighted feedback guides iterative revisions of style directives. This search converges within 20 iterations and increases WSR from about 44% with random styles to 91.5% on AG News and 86.8% on MMLU. With identical carriers, StyleBkd and BGMAttack obtain 62.8% and 47.9% WSR, respectively, helping separate the effect of style optimization from carrier selection. The contribution is in SSL-oriented criteria and feedback search.

3. Recipient fingerprinting and black-box auditing. A shared global-style watermark supports ownership verification, while distinct recipient fingerprints support tracing. Candidate fingerprints undergo paraphrase-robustness screening (at least 80% token survival in simulated paraphrases) and are embedded within the text style. At the default threshold τ=0.4\tau=0.4, the valid fingerprint pool contains 642 motifs, with 98.2% tracing accuracy in the reported capacity evaluation. In a separate 100-recipient experiment with 1,000 trials per scenario, correct attribution is 99.1% for one leaker and 84.3% for two colluders; the observed wrong-attribution rate is 0.0%, with abstention increasing for larger collusions. These results offer empirical evidence, not a zero-error guarantee. Here, correct attribution means identifying at least one member of the actual leaking coalition.

Carrier selection finds suitable samples, QASS optimizes watermark retention, and fingerprinting enables black-box verification and tracing. Each design serves a testable requirement.

We also addressed the reviewers' main concerns:

1. Adaptive attacks. We evaluated surrogate-confidence carrier exclusion and collusion-based Adapt-Remove. Excluding the top 20% of high-confidence samples removes only 15.3% of carriers; WSR remains 79.2% with ownership p=10−12p=10^{-12}. Complete removal defeats watermarking and tracing, but at ρ=60%\rho=60\% it requires discarding 84.2% of the data, reducing AG News accuracy from 90.4% to 27.5%. We quantify this utility cost.

2. Attribution reliability. We examined 100 recipients under single leakage, collusion (K=2,5,10K=2,5,10), and clean-model scenarios, each with 1,000 trials. No wrong attributions were observed; for K=10K=10, correct attribution falls to 12.6% as abstention reaches 87.4%. Clean-model trials all abstained. This makes the trade-off between coverage and cautious attribution explicit.

3. Stronger comparisons. We added BadFreq and SteganoBackdoor, bringing the evaluation to ten baselines. GSPS outperforms both on seven reasoning/QA benchmarks, and identical-carrier comparisons help isolate the value of QASS.

4. Defense evaluation. We corrected a result-compilation error in Table 8. It now separates six data-level defenses on GPT-4o-mini from four model-level defenses on Llama-3.1-8B-Instruct, with a separate no-defense baseline for each setting.

5. Scope and presentation. We clarified ownership versus recipient attribution, discussed Waterfall and STAMP, and added qualitative examples for 13 datasets. Continual pre-training and open-ended generation remain untested extensions.

We hope these revisions clarify the technical contributions and strengthen the experimental evidence supporting GSPS. We appreciate the Area Chair's and reviewers' consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.