acceptodds
Under review as a conference paper at ICLR 2027

When the Probe Is the Problem: Black-Box Watermark Detection on a Deployed LLM

Abstract

Black-box watermark presence tests ask whether a language model carries a watermark using samples alone. We audit the Red-Green test on Claude, whose provider announced a SynthID-Text deployment without disclosing its parameters. Across ten initial configurations the test never rejects, despite detecting a local reference at comparable sample sizes. Seeding the reference's repeated-context history with the prompt blinds the echo probe while leaving the watermark active elsewhere. Two redesigns expose native word preferences and response-planning confounds, demonstrating the need for controls from the deployed model's family. A long assembled-context probe, scored within each prefix, produces a substantial shift on Fable absent on two presumably unwatermarked siblings. Its primary statistic rejects, but the pre-registered decision is inconclusive because refusals are concentrated by context. Follow-ups do not establish a decisive replication, and the shift's watermark origin remains unresolved. We provide an auditing checklist and release code, prompts and parsed responses. The results show how a deployment can violate a presence test's assumptions, and why statistical rejection and a null each require interpretation against explicit controls.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM watermarking, SynthID-Text, black-box detection, watermark presence testing, repeated-context masking, Claude, auditing deployed language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_EmvCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper mainly studies the reliability of black-box LLM watermark detection. The paper tested exsiting watermark detection methods and found: 1) false negatives: even if the model is watermarked, prompt-aware repeated-context masking can cause detection methods to failed; 2) false positives: even if the model is not watermarked, certain natural language patterns may be mistakenly identified as a watermark signal. They evaluate several probe designs using controlled experiments on Qwen and API-based experiments on Claude. Finally, the paper detected strong statistical signals in Claude (reportedly watermarked), but could still not confirm that the signal originated from the watermark.

Critical strengths
  1. The paper examines watermark detection in real-world black-box deployment settings, where watermark parameters and implementation details are unknown. This is a meaningful setting.
  2. The paper explores multiple probe designs. The discussion of inconclusive results is transparent.
Critical weaknesses
  1. The main findings appear to be largely predictable from the known mechanisms and assumptions of watermark generation and detection. While the empirical validation is useful, the paper does not clearly establish what fundamentally new insights it contributes beyond confirming these anticipated failure modes.
  2. Although the paper identifies limitations of existing detection methods, it does not establish a reliable alternative. The final probe lacks successful validation on a prompt-aware watermarked reference model, while the deployment results are affected by substantial context-dependent refusals, sensitivity to statistical estimators, and inconclusive replication. Consequently, the central empirical findings remain difficult to attribute specifically to watermarking, substantially limiting the paper's contribution.
Questions for authors
  1. What is the key technical novelty beyond identifying known failure modes of watermark detection? Prompt-aware masking is a useful counterexample, but what previously unknown mechanisms or principles does it actually reveal?
  2. Can the authors validate the final detection protocol on a controlled prompt-aware watermarked model? If the final method cannot effectively detect watermarks even on controlled positive controls, how can the significant results observed in deployment be explained?
  3. How can the authors distinguish watermark-induced effects from native model behavior and selective refusals? The current comparisons do not support this attribution. The authors need more convincing controlled experiments or statistical methods.
1 comment
CommentResponseLines
W1 / Q1: findings predictable from known mechanisms; what is fundamentally new?Acknowledged in part: some failure modes follow from the test's stated assumptions. New and not derivable from the published analysis: (1) a prompt-seeded history switches marking off at the probed position while the watermark stays active elsewhere, so an echo null cannot exclude a watermark; (2) two probes that pass every reference control fail on the deployed family; (3) a power floor set by key count, not sample size; (4) native column structure on an unwatermarked reference under the final probe (Section 3.3); (5) the probe's pattern growing on a sibling once the provider marks it (Section 4.4).58–75; 260–269; 424–446
W2 / Q2: no validation on a prompt-aware reference; if the method fails on controlled positives, how are deployment results explained?Validation: yes. On a reference that copies forty digits reliably the final probe detects the prompt-aware watermark as it detects the stock one (Section 3.3). Caveat stated there: the unwatermarked reference also shows column structure, so effect size, not the pp-value alone, separates the cases. Deployment results: explained against three accounts with evidence for and against each (Section 4.5); the refusal-free re-check favours the keyed-watermark account, while the pre-registered Fable decision stays inconclusive because its refusal condition failed. A reliable alternative detector is outside the paper's aim.249–269; 448–475
Q3: how to distinguish watermark effects from native behaviour and selective refusals?By a same-model contrast, which the provider's marking supplied. Size alone does not decide: an unwatermarked reference can produce a shift of Fable's size under this probe. But on an unchanged identifier and with zero refusals, Sonnet 5's shift grew fourfold (range 0.12 to 0.47) and tightened onto Fable's column pattern (r=+0.53r = +0.53 to +0.86+0.86), which a fixed native preference does not predict and the refusal filter cannot produce. Stated limits: a faint pattern of the same direction preceded the marking, marked Haiku 4.5 shows no such shift, and only the provider's detector can confirm a mark.260–269; 431–446; 452–475
Reviewer_UP2LOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies the problem of determining whether a deployed LLM uses a Red-Green family watermark in a black-box setting - i.e., no access to model weights, logits, watermark key or the detector. The external auditor uses different sentence prefixes with controlled digit strings and asks the LLM to choose one of four words. If watermark is on, the same digit context should bias the generation to favor the same word across several prefixes. While in principle this should be the case, there are several confounds at play with this setup. For instance, the paper seems to suggest that the presence of different digits shouldn't systematically favor the same fruit across many unrelated prefixes. The authors test this assumption on unwatermarked Qwen. It holds for the original echo prompt but fails for one redesigned prompt, where merely naming different digits produces systematic fruit preferences. This demonstrates that the validity of the probe depends strongly on its wording and the model being tested.

The paper also shows that the probe can result in false negatives, i.e. miss a watermark when it is actually present. It is able to detect SyntID on Qwen. However, many common watermarking implementations keep track of repeated n-grams and only apply the watermarking rule at the first occurence. The probe therefore would not find a signal even if other parts of the text are watermarked.

Finally the authors experiment on Fable 5.1 with Haiku 4.5 and Sonnet 5 as non-watermarked control models. Fable does exhibit a strong correlation between the final digit and the selected word. However, this result is confounded by refusals. Fable ends up refusing many prompts and some digit strings are refused more often than others. While the paper asks an interesting question, there are many confounds at play and the paper ultimately does not provide a conclusive result on the deployed model for practitioners.

Critical strengths
  • S1: The paper asks an interesting question and is transparent about failed probes. The Qwen model results are clean but lack significance from a practical utility standpoint.
Critical weaknesses
  • W1: The paper applies its final probe to a model with unknown ground truth before demonstrating that the probe can detect the relevant watermark in a controlled model where that watermark is known to be active.

  • W2: Fable results are confounded by refusals. One way to improve the paper could be designing probes with either reduced refusals or balanced refusals.

  • W3: The paper does not have a related work section and misses several important prior work. For e.g., several prior works have shown that watermark perceptibly changes quality of the text generated or even alignment properties. That could be another viable direction for creating a probe.

Questions for authors
  1. See weakness 1 above.
  2. Can the main Fable result be independently repeated using contexts that produce negligible and context-balanced refusals?
  3. Would the authors better clarify the overall contribution over Gloaguen et al.'s probe?
1 comment
CommentResponseLines
W1 / Q1: final probe applied to unknown ground truth before validation on a controlled prompt-aware model.Agreed for the submission; fixed now. The text states that at submission the combination was validated on the stock reference only. Section 3.3 adds the demonstration: on a prompt-aware reference the final probe detects the watermark as on the stock one (range 0.79 to 0.86). It also reports a caveat, native column structure on the unwatermarked reference; the deployment results are read against both and tested on siblings the provider marked after the September runs.249–269; 374–377; 424–446; 448–475
W2: Fable results confounded by refusals; design probes with reduced or balanced refusals.Tried on Fable without success: of twenty-four base numbers piloted none was refused below 19%; the second-base and inside/outside grids keep 9% to 37% refusals; only the twenty-five-digit grid (0.1%) is refusal-free, and it is borderline under the primary. The refusal-free test comes from the former controls: after the provider marked them, Sonnet 5 shows the shift and Fable's pattern with no refusals; Haiku 4.5 does not. Selection by refusals is tested with four checks and kept as one of three explanations.314–320; 361–373 (attempts); 379–392 (Table 5); 424–446; 469–471
W3: no related-work section; prior work on quality and alignment effects of watermarks missed; such effects as a probe direction.Added as a headed paragraph in the Introduction; the page limit leaves no room for a separate section. It cites Ajith et al. (2024), Tu et al. (2024) and Verma et al. (2025) on utility, instruction following and alignment. As a probe direction this is outside what an external audit can do: each such signal needs the same model without its watermark as a baseline.37–50
Q2: can the main Fable result be repeated with negligible, context-balanced refusals?On Fable only partly (D=25D = 25 and low-refusal subsets). Without refusals, on the same design: Sonnet 5, null in September, after marking gives primary S=4S = 4, p99=0.016p_{99} = 0.016 (grey zone under the plan; S=6S = 6, p99=0.0005p_{99} = 0.0005 on verbatim copies), range 0.47 and r=+0.86r = +0.86 with Fable's column pattern (before marking: range 0.12, r=+0.53r = +0.53); Sonnet 5.5 and Haiku 5.5 share the pattern (r=+0.78r = +0.78 and +0.89+0.89); Haiku 4.5, also listed as marked, meets the plan's "shift absent" rule. Plan, decision rules and all numbers: Appendix E.2.369–373; 424–446 (Section 4.4); 1258–1282 (Appendix E.2)
Q3: clarify the contribution over Gloaguen et al.'s probe.Stated in one sentence in the Introduction: they validate the test on watermarks the tester controls; we ask what it establishes on a deployment whose watermark the tester does not control, and find that the answer turns on assumptions their analysis leaves implicit (an active watermark at the measured position, no native column structure, transfer of surrogate validation). The four contributions list what the deployment adds, including a natural experiment no local setup supplies.37–50; 58–75; 424–446
Reviewer_B9wNOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
1
Read review
Paper summary

This paper conducts black-box tests for watermark presence in a commercial model whose provider claims to use SynthID-Text with undisclosed parameters, and examines why existing probes fail in deployment. The experiments reveal a significant shift associated with context, but the predefined criteria are not met, so the shift cannot yet be attributed to watermarking.

Critical strengths
  1. The paper addresses a practical problem by studying black-box watermark detection in a deployed commercial model.
  2. Controlled experiments show how repeated contexts, model preferences, and refusals affect detection results.
Critical weaknesses
  1. The paper reads more like a technical audit report. Its main contribution is an analysis of why existing probes fail, and it does not yet offer a sufficiently validated, generalizable detection method.
  2. The main text is under 8 pages, while the structure is unclear and key details are left to the appendices. Frequent selective refusals in the main experiment and the lack of clear replication in subsequent experiments limit the reliability of the conclusions.
Questions for authors
  1. In Table 3, approximately 44% of Fable requests were refused, and refusals were associated with context. Could the authors verify the results with fewer refusals while meeting the predefined criteria?
  2. Why do the two estimation methods in Table 3 yield p99=0.0005p_{99}=0.0005 and p99=0.055p_{99}=0.055, respectively? How can ensure that the conclusions are stable?
  3. The final probe still fails to detect the watermark in the local reference implementation that accounts for prompt history. Could the authors provide a successful positive control to validate the final method?
3 comments
CommentResponseLines
Presentation: difficult to follow.Addressed. The Introduction is rewritten: a related-work paragraph, the difference from Gloaguen et al. in one sentence, four numbered contributions and a roadmap. The attribution argument, previously spread over three sections, is now one section with one paragraph per explanation (Section 4.5).29–81 (introduction); 448–475 (Section 4.5)
W1: reads like an audit report; no validated, generalizable detection method.Partly outside the study's aim, and now stated: we do not propose a new detector; the paper asks what a published presence test can establish on a deployment whose watermark the tester does not control. Within that aim the final probe is now validated on a prompt-aware reference (Section 3.3), and a natural experiment shows what it detects when a provider marks a model (Section 4.4).54–56; 58–75 (contributions); 249–269 (validation); 424–446 (natural experiment)
W2: main text under 8 pages, unclear structure, key details in appendices; refusals and lack of replication limit reliability.Length: the main text is now 10 pages. Structure: a roadmap and three new sections (3.3 validation, 4.4 re-check, 4.5 explanations). Key details: validation, re-check and refusal analysis are now in the main text; the appendices hold definitions, prompts and full grids. Refusals: four selection checks (correlation with the refusal rate, low-refusal cells, equal-size subsamples, exclusion of depleted prefixes). Replication: Sonnet 5, re-queried after the provider marked it, reproduces the shift and Fable's column pattern with zero refusals, as do two models marked at launch.77–81 (roadmap); 314–320 and 346–349 (refusal checks); 1182–1195 (Appendix E); 359–421 (follow-ups, Tables 5 to 7); 424–446 (re-check); 507–520 (limitations)
Q1: 44% refusals concentrated by context (Table 3 of the submission, now Table 4); can the result be verified with fewer refusals while meeting the criteria?On Fable, no: of twenty-four base numbers piloted none was refused below 19%, and the planned decision stays inconclusive. Supporting checks on Fable: refusal rate uncorrelated with the shift (r=−0.03r = -0.03); 73 low-refusal cells keep range 0.55; equal-size draws keep 0.50 to 0.66; D=25D = 25 (0.1% refused) is borderline under the primary (p99=0.059p_{99} = 0.059) with range 0.40. On other marked models with zero refusals, yes in part: the exploratory Sonnet 5.5 grid meets the predefined detection rule (primary p99=0.0005p_{99} = 0.0005); Sonnet 5 has range 0.47 (p99=0.0005p_{99} = 0.0005) and Fable's column pattern (r=+0.86r = +0.86) with its primary in the plan's grey zone (p99=0.016p_{99} = 0.016); Haiku 5.5 has range 0.39 and r=+0.89r = +0.89.271–281 (Table 4); 314–323 and 346–349; 1182–1187 (Appendix E); 365–366 (base numbers); 369–373 and 395–406 (D=25D = 25, Table 6); 409–421 (Table 7); 431–446
Q2: why do the two estimators give p99=0.0005p_{99} = 0.0005 and 0.055 (now Table 4); how stable are the conclusions?Why: the estimators use different variances (paper: within-row across contexts, unchanged by within-row permutation; code: across prefixes, changed by permutation), and SS counts whole cells, so at S=4S = 4 one or two cells separate the two pp-values. Stability: the discrete decision is not stable and is reported as such; the conclusion rests on the continuous effect size (0.57 against 0.07 and 0.12), which uses no estimator and holds under equal-size subsampling (0.50 to 0.66).113–116; 271–281 (Table 4); 307–313; 321–323 and 346–349
Q3: the final probe does not detect the prompt-aware local reference; provide a positive control.Provided. On Qwen3-8B, with every scored response verified to contain the requested number, the spaced probe detects stock and prompt-aware SynthID identically (range 0.79 to 0.86 for both; S=8S = 8 to 10 at four and forty digits; column means correlate at 1.00), while the echo probe is masked under the prompt-aware variant (range 0.10, as on the control). Caveat reported with it: the unwatermarked reference shows native column structure that grows with the number length (range 0.59 at forty digits). The 3B grids, re-read with the continuous statistic, already contained the positive control that empty cells hid from SS (Appendix H).249–269 (Section 3.3; Table 3 at 233–244); 1488–1505 (Appendix H)

The rebuttal still reads as if it were written by AI. Moreover, the changes to the main text are too extensive. For example, the Introduction has been almost entirely rewritten. I therefore remain inclined to keep my original score.

Dear Reviewer, we are responding to your review:

  1. In your initial review, you made no claims (either substantiated or unsubstantiated) regarding the use of an AI model in the article. The role of the model is fully disclosed in the article. Moreover, the AI models themselves were the subject of the study, as described in detail in the article. Therefore, your new comment comes as somewhat of a "surprise," particularly since we received it just one day before the discussion period ended.

  2. The introduction has been changed at the request of one of the reviewers.

  3. The total volume of "blue" text accounts for 18% of the article's main content. Considering that the original text was 8 pages long, and according to ICLR guidelines, the final version is allowed to be 10 pages long, your comment "changes to the main text are too extensive" prompts us to contact the ICLR organizing committee for clarification on the acceptable percentage of "blue" text. This is especially true since your comment is related to the size of the text, not its content. We regret that this comment does not in any way affect the substantive part of our amendments, but only concerns the length of the text.

We regret that the scientific content of this article was not addressed by the reviewer during the re-examination. We would be grateful for a review of our responses to your comments that were overlooked by the reviewer. We also appreciate the time the reviewer spent formulating these questions in various ways.

We appreciate the questions from your previous review – our answers to them have helped us improve the quality of our work. We hope that the future of this study will be publication-oriented.

Comments on the paper

1 comment

Dear Reviewers,

We thank all reviewers for their time, expertise, and valuable comments, suggestions, and questions. We are glad that the reviewers recognized several strengths of our work. In particular, they note that the paper:

  • addresses a practical problem in a deployed commercial model (Reviewer B9wN)
  • studies a meaningful real-world black-box setting (Reviewer EmvC)
  • asks an interesting question (Reviewer UP2L)
  • provides controlled experiments on factors affecting detection (Reviewer B9wN)
  • explores multiple probe designs (Reviewer EmvC)
  • is transparent about failed probes and inconclusive results (Reviewers UP2L and EmvC)

We have carefully addressed every comment and revised the manuscript accordingly, highlighting all changes in blue. We believe the study is now clearer and more useful to both researchers and practitioners. In our individual responses, we answer each reviewer's questions and concerns, with line numbers referring to the revised manuscript. We hope these answers also clear up any remaining ambiguities in the results.

Your feedback has substantially improved the paper and helped us grow as researchers. We truly appreciate it, and we would be happy to answer any further questions or continue the discussion during the rebuttal period.

Sincerely,
The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.