acceptodds
Under review as a conference paper at ICLR 2027

Uncovering the Tells: Efficient Black-Box Audits for LLM Data Contamination

Abstract

Existing methods for assessing training data contamination in Large Language Models achieve high accuracy, but fundamentally rely on gray-box access, rendering them inapplicable to fully black-box environments. This limitation forces auditors to rely on full-sequence sampling or rigid heuristics, both of which incur prohibitive inference costs and struggle to isolate the underlying memorization signal. To overcome this, we propose a generation-based black-box framework that infers context-induced information gain from targeted token discrepancies. We formalize a budget-constrained auditing regime, building on the empirical observation that the membership signal is highly sparse and concentrated in a few highly informative tokens. We statistically formalize this sparsity, demonstrating that targeting these anomalies avoids the severe sample complexity penalty of uniform auditing, allowing for accurate detection without probing the full document. Empirically, our approach achieves state-of-the-art black-box separation accuracy while needing about fewer queries than uniform position sampling. Finally, we apply this methodology to closed frontier models, exploring its capacity and fundamental limits in surfacing potential training data overlaps on commercial APIs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Data Contamination, Black-Box Auditing, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_5ziXOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a black-box contamination score based on whether adding related documents helps or hurts a model's next-token predictions. It uses lexical similarity to select context and longest common subsequence matching to choose a small number of token positions to query.

Critical strengths
  1. The method works without token probabilities, and selecting token positions reduces the number of queries needed.

  2. The experiments cover several model families and include useful tests of fine-tuning, sampling temperature, and context selection.

Critical weaknesses
  1. The main results may reflect distributional shifts between datasets rather than training exposure. Section 4 compares datasets such as StackExchange and GSM8K, which differ in content and format as well as membership. Appendix C includes controlled contamination experiments, but does not compare against the main baselines or show how the same dataset's score changes with exposure. These checks are necessary because dataset differences alone can produce strong detection results [A].

  2. A simple but important baseline is missing: next-token prediction accuracy without extra context. Section 2.1 acknowledges that the proposed score depends on this accuracy, and related black-box accuracy methods already exist [B]. Testing accuracy at the same selected positions would show whether the context comparison adds value, or whether token selection explains most of the improvement.

  3. The evaluation does not show how reliably the method can flag an entire dataset. Table 1 measures separation between individual documents, but this does not tell us how often an unseen dataset would be wrongly flagged. The paper needs dataset-level false-positive results and uncertainty estimates.

  4. The theory does not directly justify the proposed selection method. Section 2.2 assumes an ideal sampler that already knows which positions have non-zero shifts, whereas LCS selects positions through text overlap. LCS also changes the balance of positive and negative shifts, so the ideal sampler's guarantee does not automatically apply. The analysis motivates targeted sampling, but does not establish an LCS guarantee or prove that 16 tokens is optimal.


[A] Das, Debeshee, Jie Zhang, and Florian Tramèr. "Blind baselines beat membership inference attacks for foundation models." 2025 IEEE Security and Privacy Workshops (SPW). IEEE, 2025.

[B] Xu, Ruijie, et al. "Benchmarking benchmark leakage in large language models." arXiv preprint arXiv:2404.18824 (2024).

Questions for authors

Please see weaknesses above.

Reviewer_9xx1OpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper audits possible training-data contamination by comparing next-token predictions with and without related context. It uses lexical overlap to select informative positions, reducing the queries needed for a black-box audit. Experiments cover open models, controlled fine-tuning, and exploratory tests on commercial models.

Critical strengths
  • The paper is well written and easy to follow, with a clear motivation and a well-organized presentation.
  • The paper offers a simple way to audit possible contamination without token probabilities. Selecting positions through lexical overlap substantially reduces the queries needed compared with uniform sampling, making the method more practical when API calls are costly.
  • The experiments cover a range of models and show better detection performance than existing black-box baselines. Tests where training exposure is explicitly controlled provide further evidence that the method captures membership information, rather than only differences between datasets.
Critical weaknesses
  • The scores in Figure 6 are difficult to interpret without the number of positions where prediction correctness actually changes. Could the authors report these counts alongside uncertainty estimates, and explain how the score is handled when no changes are observed?
  • Section 2.3 does not fully specify how positions are selected from the LCS. How are positions chosen when there are more than W matches, and what happens when there are fewer?
  • Typo: "Opus and Sonnet disagree on anticipated positive controls" -> "Opus and Sonnet disagree on anticipated positive controls."
Questions for authors

See above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.