acceptodds
Under review as a conference paper at ICLR 2027

Dataset Watermarking with Provable Black-Box Detection

Abstract

LLMs are often pre-trained and post-trained on vast amounts of loosely curated data, and potentially have been trained on proprietary datasets or the benchmarks used for evaluation. This motivates dataset watermarking: designing datasets such that training on them leaves detectable signatures in the resulting model. Existing approaches embed watermarks by either injecting artificial content or paraphrasing full examples under token-level generation control, and many require model access beyond generated outputs for detection. Our method embeds a dataset-level watermark by increasing the co-occurrence of randomly selected word pairs through meaning-preserving local edits, and detects it from generated text alone with provable false positive control. We conduct evaluation on four base models and three datasets and show that it reliably detects the watermark () in the fine-tuning stage, outperforming existing black-box baselines. Our method remains effective under substantial dilution where watermarked data is less than 5% of fine-tuning tokens. Moreover, our method better preserves benchmark utility and semantic integrity than existing baselines.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
data provenance, benchmark contamination, watermarking, LLM

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oBezOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a dataset watermarking method that increases the co-occurrence of randomly selected word pairs through meaning-preserving local edits. It enables black-box detection using generated text and provides a hypergeometric test with provable false-positive control. Experiments on four language models and three benchmarks demonstrate effective detection under fine-tuning, data dilution, and lightweight text modifications, while largely preserving dataset utility.

Critical strengths

The paper provides a simple, API-compatible dataset watermarking framework that enables black-box detection without access to model logits or token-level generation controls. The randomization-based statistical test provides valid false-positive control under its stated assumptions, and experiments demonstrate promising detection performance across multiple models and datasets.

Critical weaknesses
  1. Limited novelty of the watermarking mechanism. The proposed method relies on a relatively straightforward mechanism: artificially increasing the co-occurrence of randomly selected word pairs through meaning-preserving lexical edits. Similar association-based watermarking mechanisms have been extensively explored in code dataset watermarking. More recent work, such as PuzzleMark [2], has moved beyond conventional co-occurrence patterns toward more sophisticated and stealthy watermark constructions. The proposed method primarily adapts a simple statistical association mechanism to natural language, while its false-positive guarantee follows from a standard hypergeometric randomization test. Overall, the methodological novelty appears limited.
  2. Insufficient evaluation of watermark stealthiness and robustness. Prior work, including DeCoMa [1], has demonstrated that artificially introduced trigger–target associations in code datasets can leave detectable statistical artifacts and be removed through data-level analysis. PuzzleMark [2] further highlights the vulnerability of conventional co-occurrence-based watermarks and proposes a more robust alternative. Although these studies focus on code datasets, the proposed method similarly relies on artificially amplified lexical associations. However, the paper does not evaluate whether its watermark can be identified and removed through statistical analysis without knowledge of the secret key. This omission raises substantial concerns about the watermark's stealthiness and practical robustness. References [1] DeCoMa (ISSTA 2025) Xiao, Y., et al. DeCoMa: Detecting and Purifying Code Dataset Watermarks through Dual Channel Code Abstractions. Proceedings of the ACM on Software Engineering, ISSTA 2025. https://doi.org/10.1145/3728952 [2] PuzzleMark (FSE 2026) Huang, H., et al. PuzzleMark: Implicit Jigsaw Learning for Robust Code Dataset Watermarking in Neural Code Completion Models. Proceedings of the ACM on Software Engineering, FSE 2026. https://doi.org/10.1145/3797122
Questions for authors
  1. Novelty: How does the proposed word-pair co-occurrence watermark fundamentally differ from existing trigger–target association-based dataset watermarking methods? What is the key technical innovation beyond meaning-preserving lexical edits and a standard hypergeometric test?
  2. Detectability and Removal: Prior work such as DeCoMa has demonstrated that anomalous trigger–target associations in code datasets can be identified and removed through statistical analysis. Although DeCoMa does not currently provide a docstring → docstring detection mode, its existing implementation already supports token-level analysis of docstrings, making such an extension relatively straightforward. Given that the proposed watermark similarly relies on artificially amplified word-pair co-occurrences, I am particularly interested in whether it would survive a DeCoMa-style docstring → docstring detection and removal procedure without access to the secret key. Could the authors evaluate this threat and report the watermark detection rate after data purification?
Reviewer_Nmj1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a dataset watermarking method that uses local edits to increase the co-occurrence of randomly selected word pairs and detects the resulting signal from a suspect model’s generated text. It establishes exact false-positive control. Experiments on four models and three benchmarks evaluate detection after fine-tuning, robustness to dilution and text modifications, and preservation of dataset quality.

Critical strengths
  • The method allows watermark embedding through a standard language-model API and detection from generated text alone (black-box access), which makes its access requirements practically useful.
  • Theorem 4.1 provides exact false-positive control under the stated null without requiring independence among generated samples or word-pair statistics.
  • The experiments demonstrate performance gain across different models and datasets.
Critical weaknesses
  • The fine-tuning objective is not fully specified. Since the watermark is embedded in the question text, it is worthwhile to clarify whether these watermarked tokens contribute to the training loss.
  • The comparisons use GPT-5 for the proposed method and LLaMA-3.1-8B for the baselines. Therefore the results do not isolate the contribution of local editing.
Questions for authors
  1. The fine-tuning setup needs more clarification: it is not specified what the training loss is, whether the reported results include question tokens in the training loss, and whether detection remains effective when loss is applied only to the answer.

  2. The experimental comparisons would benefit from a controlled comparison using the same rephrasing model. Could the authors also run the proposed local-editing pipeline with LLaMA-3.1-8B? Reporting both detectability and content preservation would help separate the contribution of local editing from that of the stronger rephrasing model GPT-5.

  3. How does the pipeline handle edits that may change the meaning or correct answer? Table 4 reports answer-preservation scores of 0.955–0.966, and Appendix A asks the rephraser to return preservation flags, but Algorithm 1 shows no rejection or reversion step. Please clarify how these flags are used and whether removing edits that fail the preservation checks affects detection performance.

  4. The robustness results would be clearer with detection rates across independent keys. Table 11 provides these rates for the vanilla setting, whereas Table 2 reports median p-values and Table 3 averages p-values using one fixed key. Reporting detection rates for these settings as well would show how consistently the dilution and text-modification results hold.

  5. The watermark detection score in eq(1) involves the difference of two sample averages, with sizes d and N-d respectively. To ensure the reliable performance of the test, I wonder if there is any requirement on the relative size of d as compared to N, e.g., should we aim to have d and N-d be comparable with each other?

  6. Also it should be stated clearly in Section 2 that, in addition to the model access, the detector should also have access to the secret key (word pairs) if that is the case.

Reviewer_ZaHqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a dataset watermarking approach based on modifying word-pair co-occurrence patterns through localized, meaning-preserving rewriting. It enables watermark detection from model-generated text without requiring access to internal model information and provides theoretical guarantees for false positive control. Experiments across three datasets and four LLMs show reliable detection after fine-tuning, including under data dilution and lightweight text modifications, while maintaining the quality of the original datasets.

Critical strengths
  1. Black-box detection with statistical guarantees. The proposed detector operates entirely on generated text and does not require access to model logits or tokenizers. The theoretical false positive guarantee makes the approach particularly relevant for auditing closed-source models.
  2. Extensive experiments and good content preservation. The evaluation covers different model families, datasets, and data dilution settings. The method remains detectable even when watermarked data accounts for a small fraction of the training corpus in some settings, while achieving better semantic and benchmark utility preservation than the evaluated baselines.
Critical weaknesses
  1. Limited generalization beyond QA benchmarks. All experiments focus on multiple-choice QA datasets (MMLU, ARC-Easy, and ARC-Challenge). It remains unclear whether the method can reliably detect watermarks in other types of training data.
  2. Lack of multi-key and multi-dataset evaluation. The experiments consider independently sampled watermark keys but do not examine scenarios where multiple datasets protected by different keys are used to train the same model. It remains unclear whether the watermark signals can be independently detected without interference.
  3. The proposed method uses GPT-5 for localized rewriting, whereas the KGW and SynthID baselines use Llama-3.1-8B-Instruct. As both the rewriting strategy and underlying model differ, it is unclear how much of the improvement in semantic preservation and benchmark utility can be attributed to the proposed method.
Questions for authors
  1. Dataset generalization: Maybe evaluating the method on a non-QA dataset is better.
  2. Multi-key detection: How does the method perform when a model is fine-tuned on multiple datasets watermarked with different secret keys? Can each watermark be reliably detected?
  3. Model-size dependence: Table 2 suggests that larger models retain detectable watermark signals under stronger data dilution. Need to further analyze how model size affects watermark detectability and explain this difference.
2 comments

Dear Reviewer ZaHq,

Thank you again for your constructive feedback and review. As today is the last day of the discussion period, we wanted to check whether our responses have addressed your concerns. In particular, we ran additional experiments and show that our method remains effective on an additional instruction-following dataset (Databricks Dolly 15K), and independently keyed watermarks remain reliably detectable when two watermarked datasets are used together for fine-tuning. We also ran an ablation controlling for the rephrasing model.

If anything remains unclear, we would be happy to provide further details. If our responses have resolved your concerns, we would greatly appreciate it if you would consider updating your score. Thank you for your time and consideration.

Best regards, The Authors

Thanks for the detailed experiments and responses. Although the method achieves better lexical preservation than SynthID under the same rephrasing model, its semantic and answer preservation remain lower. Nevertheless, my main concern has been addressed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.