acceptodds
Withdrawn from ICLR 2027

Top-k Probing for Black-Box LLM Watermark Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Auditing an LLM watermark is hard when the model keeps its watermark key and token scores private. Existing black-box detectors usually examine responses under a fixed decoding setting; some also need carefully designed prompts that restrict the answer. We use a different clue: what happens when the model’s list of allowed tokens changes. Our top-kprobe sends the same prompt with a short and a long token list, then checks whether the model changes how it chooses between tokens on both lists. To the best of our knowledge, this is the first black-box LLM watermark test to use a controlled change in top-kon the same prompt as its probe. We build two detectors around this idea. Water-Zero uses an exact conditional test on ordinary text prefixes. It needs no answer template, trained detector, watermark key, or clean-model calibration. Water-Sonar learns patterns in the responses from each token list and covers more watermark families; it can use natural or crafted prompts. For natural prompts, we test held-out keys on C4 and OpenGen with three Qwen and Llama models. A separate crafted-prompt study covers nine mod- els and nine watermark families. Comparisons include plain-model audits, pooled exact tests, and tests that keep top-k fixed. Water-Zero detects watermarks that respond to a change in the allowed tokens, and its natural-prompt results transfer across the two datasets. Water-Sonar detects more watermark families, while its natural-prompt version remains effective for several major families. The fixed- top-ktests show that changing the token list creates the direct signal used by the exact detector.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM watermarking, black-box detection, top-k sampling, natural-prompt auditing, statistical hypothesis testing, model provenance

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WwjMOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The work studies black-box auditing for the presence of LLM watermarks by comparing model responses under different restrictions on the allowed tokens (i.e., varying the top-k parameter). The underlying idea is that watermarking schemes respond differently to changes in top-k. Based on this, the authors propose two methods: (1) Water Zero, an exact conditional test for SynthID, and (2) Water Sonar, which can detect a wider range of watermarks but builds on a learned regressor that assumes prior training. In particular, Water Sonar uses several features (like token counts, diversity, and entropy) and fits a logistic regression model for watermark identification. Its score for a target is then compared against clean audits to estimate statistical significance. The authors evaluate their methods on Qwen and Llama models using C4 and OpenGen, showing that Water Zero can detect SynthID while Water Sonar is able to (reasonably well) detect a range of watermarks at a 5% FPR level.

Critical strengths

Detecting black-box watermarks has (with the recent EU AI Act implementation) become a quite relevant topic of study.

Critical weaknesses
  1. The method relies on being able to control top k, which is a strong assumption for a black box audit. It is unclear which model providers offer the required access. If the restriction is instead requested through a prompt, this could itself influence the output probabilities. The method also depends on the order in which watermarking and top k filtering are applied. These assumptions substantially limit the practical setting in which the results apply.

  2. Given these access requirements, the empirical results seem limited. Water Zero appears to work only on SynthID, and its advantage over prior work by Gloaguen et al. is unclear to me. Water Sonar shows limited results for several schemes, including KGW, EXP and ITS, with stronger results limited to SynthID, AAR, DiPmark and γ\gamma-reweighting. This seems a modest outcome given that controlling top k is a much stronger assumption than using handwritten prompt templates. Lastly, the evaluation should also include data for instruction tuned models beyond C4 and OpenGen, as this is how models of interest will be served in practice (and which can have significant impact on a watermark).

  3. Prior work also often aimed to extend their studies to estimate particular hyperparameters of the watermark (e.g., context size). This quite relevant aspect of black box identification seems to be missing here.

Smaller Comments

The writing is difficult to follow in places, with frequent semicolons and uncommon terms such as “menu”, “development data” and “arms” that are not clearly introduced. The citation format also seems strange (no citep used it seems), and reference for the appendix use "Section A" rather than "Appendix A". In line 228, it is unclear where the earlier paired Water Zero rule was introduced.

Questions for authors
  1. Which model providers expose the top k controls required by your audit, and how do you establish whether watermarking is applied before or after top k filtering? Further can your method be used for estimating parameters of the actual watermark or only the watermark family itself?
  2. Why do you think you do not perform well on several watermarking families (e.g. KGW), could you give some more qualitative details here?
  3. Could you clarify the exact watermarking parameters and prompts used in the main paper results?
Reviewer_HAv1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes auditing black-box LLM watermarks by querying the same prefix under two different top-kk settings and checking if the pairwise odds of shared candidate tokens shift. Because standard softmax and fixed logit biases satisfy IIA, the relative odds only change under menu-dependent schemes like SynthID. The authors introduce Water-Zero, a calibration-free exact conditional test on natural prefixes, and Water-Sonar, a 16-feature logistic classifier calibrated with clean model outputs. At 2k queries, Water-Zero gets 84–91% TPR on SynthID with nominal FPR, while Water-Sonar gets 79.5% mean TPR across 9 families using crafted prompts.

Critical strengths
  1. Using top-k as a controlled input is a simple and principled idea. The exact test needs no crafted template, no trained detector and no clean model.

  2. The ablations are clean. With the same menu in both arms (8/8, 32/32), TPR drops to about 0.05, which shows the SynthID signal really comes from the menu change.

Critical weaknesses
  1. Narrow scope and an assumed setting. Water-Zero only catches tournament-style watermarks: it stays at about 5% on the other 8 families (Table 7), and its 9-family mean is 0.17. It also requires the provider to expose top-k and to truncate before the watermark is applied. In the paper this holds only because the authors built their own generator that way. Nothing is tested on a real serving pipeline, so the key assumption is never checked.

  2. Water-Sonar’s coverage does not come from top-k. When Sonar is retrained on 128/128, with no menu change, it matches the 32/128 version (0.791 vs 0.795 at 2k, 0.878 vs 0.876 at 20k; Table 11). So the broader family coverage comes from crafted prompts and learned features, not from the proposed probe. Sonar also needs 1,000 clean audits from the same model, which a black-box auditor does not have, and it is trained on the same 9 families it is tested on. The main text has no matched-budget comparison with Water-Probe or Gloaguen et al. across families.

Questions for authors
  1. Does Water-Zero still work if the watermark is applied before top-k truncation, or uses its own internal candidate set? Can you test it on an unmodified real serving stack?

  2. DiPMark and γ-Reweight depend on cumulative mass, so they look menu-dependent, yet Water-Zero has no power on them. Why?

Reviewer_4WSPOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a new black-box method for detecting whether an LLM API is watermarked or not. This method relies on a mechanism for controlling the set of tokens over which the model samples it's output, e.g. the top-k list. A proof is provided explaining how analyzing token outputs under different top-k sets can inform one of many watermark designs. Based on this insight, the paper provides two tests, Water-Zero and Water-Sonar and evaluates them across a range of watermarking methods.

Critical strengths

The proposed method is simple and cost-effective, especially compared to prior methods, and provides a theoretical explanation supporting the method design.

Critical weaknesses

The core foundation of the method proposed by the paper relies on the ability to control the top-k set over which the next token is sampled, however, current frontier closed-source models no longer even expose k as an argument during API calls. This is unlikely to change due to security risks exposed by a top-k argument [1] . Meanwhile, checking whether an open-weight model is watermarked by controlling the top-k lists wouldn't make sense as all watermarking methods studied in this work are reliant on modifications to the sampling method, which one controls for open-weight models.

I also found the paper very hard to parse and follow. A lot of jargon is introduced early on without definition or explanation, While I believe I understood the methodology at a high level, many of the details and results were very difficult to understand and their discussion was very limited, including the omission of any limitations.

Finally, no experimental comparison to prior detection methods have been provided, nor discussion of the pros and cons of the proposed methods to the prior methods

[1] Nicholas Carlini, Daniel Paleka, Krishnamurthy (Dj) Dvijotham, Thomas Steinke, Jonathan Hayase, A. Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, Eric Wallace, David Rolnick, and Florian Tramèr. 2024. Stealing part of a production language model. In Proceedings of the 41st International Conference on Machine Learning (ICML'24), Vol. 235. JMLR.org, Article 221, 5680–5705.

Questions for authors

Could the method be adapted to perform watermark detection in structured output generation (e.g. a service offered by OpenAI https://developers.openai.com/api/docs/guides/structured-outputs?api-mode=responses, Anthropic https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/partner-models/claude/structured-outputs, and Google https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/control-generated-output) when control over the exact top-k sets is not available, but, output format control is? If so, this could address a major practical weakness of the paper (although, realistically, it's not clear if watermarks would still be applied for structured output generation).

How do the proposed methods compare to prior detection methods? What are the advantages and disadvantages?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.