acceptodds
Under review as a conference paper at ICLR 2027

Could LLM Watermark Detection be Public?

Abstract

Watermarking large language models is popular for tracing their outputs, yet detectors remain unreleased since exposing them could allow attackers to steer their edits with the detector's feedback. However, watermarks are already vulnerable to uninformed tampering attacks. We thus first quantify whether a public detector would be an additional liability in a deployment setting at varying levels of access, from token-level scores to a binary verdict. Second, we introduce a split-key public-private watermarking method that exposes one key through a public detector while keeping the other for full verification and forensics. An informed attacker can only move the public signal, which creates an imbalance between public and private scores. We introduce a statistical test for this imbalance, and combine it with the full key verdict in a two-stage mechanism. Third, we evaluate the split-key method on a wide range of removal and forgery attacks, comparing both uninformed and detector-informed settings. We show that public detection improves removal only at small edit budgets, since plain rephrasing already strips the watermark at a lower quality cost, but it does enable forgery, which the private pipeline can identify. Overall, releasing half of the watermark enables transparency and interoperability, while tampering with the released half is detectable, bounding its liability and questioning the need for keeping detectors fully private.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM watermarking, AI transparency, Content provenance

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Zam9OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates an interesting question that whether the LLM watermark detectors could be public. The authors propose a split-key watermarking framework that divides the watermark signal between a public key and a private key. This method include two-stage detection mechanism that combines watermark verification with a statistical test for discrepancies between two key signals, which allowing attack manipulation to be identified.

Critical strengths

1.Important and not well-discussed problem:The paper try to address an important and relatively underexplored problem under LLM watermark topic. Given the growing demand for verifiability of AI-generated content, this is a timely and practically relevant research direction.

2.Comprehensive threat analysis: The authors systematically evaluate watermark removal and forgery attacks under different levels of detector access.

3.Good presentation: The presentation is clear.

Critical weaknesses

Although I appreciate the authors' efforts to address this interesting problem, I still have several following major concerns:

1.Weak threat model:The authors primarily focus on rephrasing and word-level edits, which have already been shown to be relatively weak attacks in prior work. The authors should evaluate it against stronger and more recent model-based watermark removal and forgery attacks, such as SIRA[1], DIPPER[2], and watermark stealing attack[3] to prove the effectiveness of its methods.

2.Exchangeability assumption: The proposed mechanism relies on a token-wise sign-flip permutation test, which assume exchangeability between the public and private watermark signals. However, I question that marginal symmetry does not necessarily imply the joint sign-flip invariance required for an exact permutation test. Token-level watermark scores may exhibit dependencies due to overlapping contexts. Such dependencies could potentially violate the underlying assumptions and increase false-positive rates. The authors should provide a more rigorous theoretical justification and evaluate its robustness under correlated token scores.

3.Missing Relevant Prior Work: The related work discussion overlooks several closely related studies such as Bileve[4] and PVMark[5]. PVMark leverages zero-knowledge proofs to enable third parties to verify that the watermark detection process has been executed correctly without revealing the secret key. Bileve explores a multi-level watermarking mechanism for provenance verification and tamper detection. Discussing these works would help clarify the novelty of the proposed method and better position its security relative to existing approaches.

4.Generalizability: The experiment mainly based on Gumble-max watermark. Even though the author add syn-text and maryland in sec 4.5, these method still belong to the same predefined pseudorandom sampling mechanisms. It remains unclear whether the proposed split-key framework can generalize to learning-based watermarking methods, such as SIR which prove to be more robust.

[1]Cheng, Yixin, et al. "Revealing weaknesses in text watermarking through self-information rewrite attacks." ICML (2025).

[2]Krishna, Kalpesh, et al. "Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense." Advances in neural information processing systems 36 (2023): 27469-27500.

[3]Jovanović, Nikola, Robin Staab, and Martin Vechev. "Watermark stealing in large language models." ICML (2024).

[4]Zhou, Tong, et al. "Bileve: Securing text provenance in large language models against spoofing with bi-level signature." Advances in Neural Information Processing Systems 37 (2024): 56054-56075.

[5]Duan, Haohua, et al. "{PVMark}: Enabling Public Verifiability for {LLM} Watermarking Schemes." 35th USENIX Security Symposium (USENIX Security 26). 2026.

[6]Liu, A., Pan, L., Hu, X., Meng, S., & Wen, L. (2024). A Semantic Invariant Robust Watermark for Large Language Models. ICLR (2024).

Questions for authors

See weakness.

3 comments

Thank you for the constructive review and recognition of our work’s importance.

W1: "stronger and more recent model-based watermark removal and forgery attacks, such as SIRA, DIPPER, and watermark stealing"

We ran DIPPER (XXL) and SIRA (Llama-3.1-8B), at several mask ratios as published (no-box) and white-box (masking the highest public-score tokens):

AttackASR_pubASR_fusADRASR_netDist.
No attack9.21.50.01.50.000
Reph. no-box T=194.385.50.285.30.053
Reph. no-box T=1.599.298.30.497.90.074
Reph. white-box T=110099.937.162.80.067
DIPPER no-box99.098.00.098.00.082
SIRA no-box 30%96.190.60.190.50.087
SIRA no-box 50%97.995.30.095.30.099
SIRA no-box 70%98.596.50.196.40.104
SIRA no-box 90%98.295.80.095.80.108
SIRA white-box 30%98.586.91.885.30.069
SIRA white-box 50%98.995.90.295.70.090
SIRA white-box 70%98.696.30.196.20.104
SIRA white-box 90%99.397.20.097.20.119

Both are about as strong as our no-box rephrasing at T=1.5, at higher distortion. White-box SIRA stays within 1.4 points of no-box at 50-90% masked, and is weaker at 30%, where it leaves more private signal and starts to be flagged (ADR 1.8%). Watermark stealing is in App. D.8: public access only speeds up learning of the public channel, and beats no-box only for very small corpora, at low ASR_net. So public access offers little advantage against the full pipeline (Figs. 4 and 6).

W2: "Token-level watermark scores may exhibit dependencies due to overlapping contexts."

The test does require joint sign-flip invariance, not just marginal symmetry. The routing gives us this hypothesis since each token goes to a key by an independent fair coin, so swapping the labels of any subset of tokens leaves the joint score distribution unchanged, whatever the dependence between tokens. Overlapping windows give distinct hashes, and repeated contexts are deduplicated with one global history across both channels (App. C.1). Beyond Fig. 7, the test keeps its nominal false-positive rate on 10,000 human ELI5 answers (0.11% flagged at 10−310^{-3}). Exchangeability is needed only for exactness: the test extends to unequal routing ratios as an asymptotic test (response to fMq6).

W3: "The related work discussion overlooks [...] Bileve and PVMark."

Neither method enables public detection, and both are complementary to our work. PVMark uses zero-knowledge proofs so that the provider can prove it ran detection correctly without revealing its key, but third parties still cannot detect without the provider. Regardless, a PVMark proof could be attached to our private verification. Bileve pairs a fragile signature with a robust watermark, so its tampering evidence flags any edit, benign or malicious, while our test flags only edits that target the public key. We have updated the manuscript to discuss both.

W4: "whether the proposed split-key framework can generalize to learning-based watermarking methods, such as SIR"

Gumbel-max, Maryland and SynthID are different mechanisms, and deployed watermarks (TextSeal, SynthID, textGrain) all use hash-based pseudorandom sampling. Our construction only needs independent keys, so it generalizes to any method with a secret key. Learning-based methods are less direct, as their scores have no exact null. We therefore built a split-key SIR: the authors' network is shared, and two independent random token-to-dimension tables serve as the public and private keys, with fair-coin routing. Each channel has an exact z-test under a random table, and the tampering test swaps keys per distinct token. Fused TPR is 96.3% at 10−310^{-3}, with 0.02% false positives on 5k human answers (δ=2\delta=2, same ELI5 setting, rates in %):

AttackASR_pubASR_privASR_fusADRASR_netDist.
No attack24.120.93.70.03.70.000
Rem. reph. no-box83.490.163.30.063.30.059
Rem. reph. white-box10094.999.623.276.50.074
Rem. edit 10% no-box37.037.87.50.07.50.015
Rem. edit 20% no-box53.355.016.10.016.10.028
Rem. edit 10% white-box94.933.432.62.831.70.016
Rem. edit 20% white-box99.544.585.129.759.80.030
Forg. reph. white-box28.50.212.310.611.00.099
Forg. edit 10% white-box2.10.01.00.01.00.022
Forg. edit 20% white-box40.90.117.020.613.50.041

Public access helps the attacker more than with TextSeal: white-box rephrasing raises ASR_net from 63.3% to 76.5% for SIR, but lowers it from 85.3% to 62.8% for TextSeal. White-box rephrasing reaches 100% ASR_pub for both TextSeal and SIR, but ADR is lower for SIR (23.2% vs. 37.1%). We hypothesize that SIR's semantic nature correlates the scores of the two channels, since both read the same semantic value vector, reducing the imbalance detected by the tampering test.

We hope the stronger attacks and the test's validity address your concerns, and that you will consider raising your score.

Reviewer_Zam9OpenReview ↗

Thanks for the effort

Thanks for your reply! I've carefully reviewed your response, which addresses most of my concerns. The additional robustness experiments and the SIR idea would further strengthen the paper. I'm happy to increase my score.

Thank you once again for your careful reading, and for suggesting these additions that improve the paper. We would be happy to answer any further questions.

Reviewer_fMq6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the additional security risks introduced by releasing public LLM text watermark detectors and compares watermark removal and spoofing attacks across varying levels of detector access. The authors propose a dual-key public/private detection mechanism: when generating each token, one of two independent keys is selected with equal probability; only one key's detection capability is made public, while the platform retains joint detection across both channels.

Furthermore, by exploiting the score imbalance between the public and private channels, the authors introduce a random sign-flip test to identify targeted tampering, combining it with joint watermark detection to form a two-stage verification pipeline. Experiments demonstrate that while this mechanism filters out many attacks targeting the public channel, generic paraphrasing and certain adaptive attacks still achieve high success rates. The paper positions this scheme as a practical trade-off between public transparency and platform security.

Critical strengths
  1. Practically important research framing: Rather than merely showing that public detectors are vulnerable to attack, the paper evaluates the incremental risk introduced by public detection relative to existing detector-free attacks. This provides an essential perspective for assessing whether publicly verifiable watermarking is viable in practice.

  2. Clear statistical intuition: Retaining an independent private signal and exploiting correlation discrepancies between the two channels to detect targeted modifications effectively complements decisions based solely on cumulative watermark strength. The authors also disentangle the results of public detection, joint detection, and two-stage verification, which helps clarify where the defensive gains originate.

  3. Broad experimental coverage and transparent reporting of limitations: The evaluation spans diverse access regimes, removal and spoofing attacks, paraphrasing and word substitution, multiple generative models, cross-tokenizer settings, and multilingual scenarios. The authors also evaluate adaptive attacks, report negative results under asymmetric contexts, and discuss statistical calibration and computational overhead—delineating the method's boundaries far more clearly than presenting success cases alone.

Critical weaknesses
  1. Incremental methodological contributions relative to prior work need stronger justification: Dual-key random routing and joint detection already exist in TextSeal, and two-tier public/private verification has previously been explored by Bashir et al. Because the paper cites these works, the issue is not omitted references, but rather the lack of a clear mechanistic comparison and empirical evidence demonstrating what irreplaceable benefits the newly proposed imbalance test provides. In particular, 1:1 routing primarily serves channel symmetry and does not constitute a major innovation on its own. Moreover, why must the routing ratio be fixed at 1:1, and is 1:1 truly optimal? The authors should substantiate this choice through straightforward ablation experiments.

  2. Substantial gap between the platform's private defensive gains and the actual verification utility afforded to the public: Many of the reported defensive successes rely on the private channel and the two-stage pipeline, whereas ordinary third parties can only access the public detector. For instance, a 5% white-box word substitution achieves a 91.5% removal attack success rate against the public detector, and in spoofing experiments. Even if the platform subsequently identifies the anomaly internally, this does not directly correct the erroneous judgment already reached by a third party. Furthermore, the public detection rate on the first 100 tokens of unattacked text is only 24.6%. Consequently, the paper has not convincingly demonstrated that the public detector provides reliable, independent provenance verification for ordinary third parties. In my view, empowering third parties with effective verification capabilities through algorithmic innovation is the primary value proposition of this research direction, yet the public verification capability demonstrated here falls short of being truly effective in practice.

  3. Claims that the incremental risk is bounded remain overstated: The imbalance test primarily captures asymmetric impacts inflicted by an attack across the two channels. However, generic paraphrasing can degrade both channels simultaneously, while careful adaptive attacks can regulate the public score to avoid excessive imbalance. In Table 2, adaptive paraphrasing removal achieves a net success rate of 98.7%, and adaptive word-substitution spoofing reaches 44.7% in a specific setting. Thus, this conclusion currently reads more as an empirical observation contingent on specific attacks and hyperparameters than as a general security guarantee.

  4. Unclear independent contribution of the imbalance test and fairness issues in attack comparisons: Weak attacks may fail to induce sufficient imbalance yet still be caught by joint detection; the fact that ADR is zero in several black-box experiments further suggests that the defense stems primarily from the stronger combined signal. Moreover, relying mainly on automated proxy metrics for semantic preservation is insufficient to guarantee that key facts, entities, and stances remain unchanged.

Questions for authors
  1. Compared to the public/private verification scheme of Bashir et al. and existing complementary watermarking approaches, what are the key distinctions of the proposed imbalance test? Could the authors provide direct comparisons under matched settings to isolate its independent benefits?

  2. Could the authors systematically report the reliability of the public detector across varying text lengths, benign edits, and malicious attacks? When public detection results conflict with private verification, how can ordinary users obtain a trustworthy conclusion? Is the intended goal of this work independent public verification, or public pre-screening followed by platform-level verification?

  3. How do the authors quantitatively define the "bounded additional risk" introduced by public detection? Could the authors compare the strongest informed and uninformed attacks under matched text quality, query budgets, and compute costs, and clarify how the high net success rates of adaptive attacks impact the paper's main conclusions?

  4. Could the authors compare public detection, private-channel-only detection, joint-only detection, and the full two-stage pipeline under an identical overall false-positive rate? Across which attack-strength regimes does the imbalance test provide gains that cannot be achieved simply via joint detection or threshold adjustment?

5 comments

Thank you for the detailed review and for recognizing our framing and experimental breadth.

W/Q1: "Compared to the public/private verification scheme of Bashir et al. and existing complementary watermarking approaches, what are the key distinctions of the proposed imbalance test? [...] why must the routing ratio be fixed at 1:1, and is 1:1 truly optimal?"

Our first contribution is the evaluation itself: to our knowledge, the first measurement of what a public detector adds to an attacker across no-box, black-box, gray-box and white-box access, on deployed watermarks (TextSeal, SynthID). By itself this would be an acceptable contribution under the ICLR reviewer guide (new empirical knowledge). The second is the two-stage pipeline and the tampering test. Routing itself comes from TextSeal, where it serves generation diversity. The tampering test gives an exact finite-sample p-value under the null that the text does not exploit the public detector, and separates removal from forgery, which prior work does not. Concurrent work (Feng et al., 2026) also uses channel imbalance, but pairs a robust and a fragile watermark against uninformed tampering, with an empirical threshold. Bashir (2026) adds a public signal on top of the private one and only evaluates robustness to random edits, with no attacks on the public detector or analysis of the scheme's security. We now compare with it in the Maryland setting of Sec. 4.5 (γ=0.25\gamma=0.25, δ=2\delta=2, h=3h=3), with the prompts, model and attacks of Fig. 6 (top) (rates in %):

AttackSchemeASR_pubASR_privASR_fusADRASR_netDist.
No attackBashir69.01.0---0.000
Ours50.857.018.30.018.30.000
Rem. reph. no-boxBashir98.988.6---0.055
Ours98.398.796.90.096.90.053
Rem. reph. white-boxBashir10096.6---0.060
Ours10099.610023.876.20.060
Rem. edit 10% no-boxBashir87.36.7---0.014
Ours81.085.754.90.254.80.014
Rem. edit 20% no-boxBashir97.237.9---0.027
Ours94.296.085.00.085.00.027
Rem. edit 10% white-boxBashir99.43.2---0.014
Ours96.373.673.82.771.80.014
Rem. edit 20% white-boxBashir10010.2---0.026
Ours99.786.697.325.872.20.025
Forg. reph. white-boxBashir76.00.1---0.099
Ours75.20.141.233.027.60.100
Forg. edit 10% white-boxBashir0.80.0---0.013
Ours0.80.00.0-0.00.013
Forg. edit 20% white-boxBashir3.50.0---0.019
Ours3.50.01.010.00.90.019

Bashir has no fused score and no tampering test, so its verdict is the private key (ASR_priv). The results reinforce our premise: LLM watermark detection could be public with a split-key construction, since both public detectors are equally brittle while informed attacks barely change Bashir's private verdict compared with no-box attacks. Bashir's private verdict is stronger because it keeps the full private watermark, at the cost of extra distortion and specificity to the Maryland scheme, but it cannot detect tampering, while our test flags 23.8% of white-box rephrasing removals and 33.0% of forgeries that pass the fused test.

On 1:1 routing: α=0.5\alpha=0.5 is the TextSeal default and the only ratio where the channels are exchangeable, which makes the tampering test exact. Other ratios admit an asymptotic test. With uk,t=1−e−sk,tu_{k,t} = 1 - e^{-s_{k,t}} the PRF value of token tt under key kk, dt=(1−α)(uA,t−12)−α(uB,t−12)d_t = (1-\alpha)(u_{A,t} - \frac{1}{2}) - \alpha(u_{B,t} - \frac{1}{2}) has zero mean at every position whatever the entropy: the generating key raises its own uu, and generates a fraction α\alpha (public) or 1−α1-\alpha (private) of the tokens. Hence Z=∑tdt/∑tdt2Z = \sum_t d_t / \sqrt{\sum_t d_t^2} is asymptotically N(0,1)\mathcal{N}(0,1). Below, ADR and ASR_net use it at α≠0.5\alpha \neq 0.5 and the exact test at α=0.5\alpha=0.5. α\alpha only shifts power between the channels, while fused detection barely changes:

Attackα\alphaASR_pubASR_privASR_fusADRASR_net
No attack0.280.60.60.70.00.7
0.422.64.01.50.01.5
0.59.28.51.50.01.5
0.81.180.00.80.00.8
Reph. no-box0.299.581.078.80.078.8
0.497.091.485.90.385.6
0.594.393.985.50.285.3
0.882.399.180.50.180.4
Reph. white-box0.210094.898.589.010.8
0.410097.899.981.518.5
0.510098.599.937.162.8
0.810099.31007.292.8
Edit 10% white-box0.21003.57.332.94.9
0.499.613.743.236.627.4
0.597.524.264.254.029.5
0.886.591.180.92.379.0

W/Q2: "Could the authors systematically report the reliability of the public detector across varying text lengths, benign edits, and malicious attacks? [...] Is the intended goal of this work independent public verification, or public pre-screening followed by platform-level verification?"

The public TPR on the first 100 tokens is only 24.6% because each channel alone is weaker than the fused detector (57.7%), and it rises with length (new generations, hence 23.4% at 100 tokens):

First L tokensTPR pubTPR privTPR fusn
507.75.516.81000
10023.423.456.6996
20066.967.692.8960
30088.586.998.8853
40094.495.099.4538

The no-box and informed rows of Tab. 1 and Tab. 6 report public detection under benign and malicious edits. The public detector is a pre-screen: a positive result at a 0.1% FPR is strong evidence of LLM authorship (forgery needs a corpus or white-box access), and a negative result gives no evidence, as without any detector, where users can already deny LLM use. The provider's full verification remains the authority where it matters, e.g. plagiarism. No detection transparency, as today, is worse for third parties in every respect: they get no signal even on unedited LLM text and must rely on stylistic cues. Whether detection should be public is for policymakers and providers to decide, we merely answer whether it could be.

W/Q3: "How do the authors quantitatively define the 'bounded additional risk' introduced by public detection? [...] clarify how the high net success rates of adaptive attacks impact the paper's main conclusions?"

We define the additional risk as the increase in ASR_net of an informed attack over the uninformed one at matched distortion, and agree that "empirically limited" describes it more accurately than "bounded". For removal, white-box beats no-box only at very low distortion, with at most 25% ASR_net for word edits, and from 75% to ~87% for rephrasing (Fig. 5a). For forgery, the only no-box baseline is watermark stealing, which white-box beats only for very small corpora, below 20% ASR_net (Fig. 15). Without a corpus, a white-box detector does enable forgery (44.7% ASR_net), an argument for less transparent detectors. Fig. 5a and Fig. 14 match text quality, and compute is matched since only token selection differs. We assumed unlimited white-box queries to show that detection could be public even at full transparency, while a real deployment would choose a lower tier with rate limits, which only weaken the attacker. Adaptive attacks are the strongest against the full pipeline because rephrasing is already strong, and their gain comes from white-box access only: with a black-box detector they are weaker than no-box at matched edit rates, i.e. matched query budgets (Tab. 11).

W/Q4: "Across which attack-strength regimes does the imbalance test provide gains that cannot be achieved simply via joint detection or threshold adjustment?"

The defense has two sources (Sec. 4.3): (a) the stronger fused signal and (b) the tampering test. The zero ADR is the black-box case, where untargeted edits weaken both channels equally and (a) is the whole defense. In the white-box case, rephrasing evades the fused test every time, and the tampering test alone brings net success down to 63.4% (ADR 36.6%), below the 81.7% of no-box (Tab. 6). The ASR_pub, ASR_priv, ASR_fus and ASR_net columns in W/Q1 compare the four detectors, and η/2\eta/2 per stage gives the two-stage pipeline the same overall FPR as one test (App. C.3). Fig. 8 shows the regime that joint detection cannot cover: the fused test accepts a wide region of channel scores in the bottom left, and threshold adjustment only moves its horizontal and vertical boundaries. The tampering test instead accepts texts near the diagonal, where untampered texts, watermarked or not, move both channels equally, and flags texts far from it. It targets tampering with the public detector itself, so weak, untargeted black-box edits leave too little imbalance to flag. Figs. 12 and 13 show consistent results across quality metrics, from word preservation to fluency.

We hope the comparison, ablations and definition of added risk address your concerns, and that you will consider raising your score.

Reviewer_fMq6OpenReview ↗

Reply to rebuttal

Thank you for the detailed response and additional experiments. The comparison with Bashir, the text-length analysis, and the explanation of the two-stage false-positive rate help clarify my concerns. I also recognize that the paper contributes a systematic evaluation of detector-access risks in addition to the detection method itself.

Regarding Q2, the clarification that public detection serves as pre-screening, with consequential decisions relying on the provider’s full verification, largely addresses my concern about the intended use.

However, several concerns remain:

  1. The new experiments are helpful, but they reveal substantial tradeoffs. I understand that the proposed method additionally flags tampering, but the practical benefit relative to the weaker final-verification performance remains unclear. Could you please explain this tradeoff and how differences in generation quality, initial detection power, and overall FPR affect the comparison? No more experiments needed.
  2. I acknowledge that the imbalance test catches some attacks missed by fused detection, and Figure 8 provides a useful explanation. However, explaining that �0��0� per stage controls the overall FPR does not establish the empirical gain under that constraint.

Overall, the response alleviates my concerns about the intended use and the method’s mechanism. However, a few minor questions remain. I acknowledge the improvements and additional evidence provided during the rebuttal. Nevertheless, on a four-point scale, I find it difficult to justify a score of 3, so I will maintain my original score. I hope the AC will take the authors’ constructive responses into account, as my assessment is closer to an informal score of “2.5.” Of course, if the authors provide a brief explanation of the tradeoff raised in my first concern that I find clear and convincing, I would be willing to raise my score to 3.

Reviewer_EnjkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a two-stage watermarking mechanism that enables public detection of watermarks in LLM-generated text while retaining a secret key for private verification. The paper evaluates the vulnerability of the public detector, the effectiveness of the proposed split-key defense, and the robustness of the two-stage mechanism against adaptive attacks.

Critical strengths

The idea of using public and private keys for public and private watermark detection is novel and interesting.

Critical weaknesses

First, the high attack success rates against the public detector raise concerns about its practical applicability and the motivation for introducing a public detection mechanism.

Second, the adaptive attacks against the two-stage mechanism also achieve high attack success rates, raising concerns about the effectiveness and robustness of the proposed defense against adversarial attacks.

Third, the paper lacks a comparison with existing baseline methods in the literature, making it difficult to assess the advantages of the proposed approach over prior work.

Questions for authors

The experiments evaluating the vulnerability of the public detector show that both removal and forgery attacks are quite effective. Given these vulnerabilities, it is unclear what advantages the proposed two-stage detection mechanism offers. In particular, it appears that private verification remains necessary to ensure reliable detection, which requires a private key and operates similarly to conventional watermark detection methods.

3 comments

Thank you for finding the split-key idea novel and interesting.

W1: "First, the high attack success rates against the public detector raise concerns about its practical applicability and the motivation for introducing a public detection mechanism."

The public detector is easy to attack, but this does not weaken the case for it, for two reasons. First, the alternative today is no detection at all for third parties, and full transparency of the detector makes attacks trivial (Tab. 1). A public detector that catches unedited or lightly edited LLM text is therefore a gain over the current situation, and evading it only brings the user back to that situation. Second, the public and private verdicts are separate. Attacks that fool the public detector do not fool the provider's two-stage check: informed attacks do no better than blind rephrasing against it, and strong ones are flagged by the tampering test (Fig. 4). Releasing the public detector therefore adds transparency and interoperability (EU AI Act Art. 50), especially for agents producing watermarked text, at almost no cost to the provider's own verification. Public detection is also practical: public image watermark detectors are already deployed (SynthID, Meta AI), face similar attacks and are served behind rate limits, yet still inform users about unedited or lightly edited content.

W2: "Second, the adaptive attacks against the two-stage mechanism also achieve high attack success rates, raising concerns about the effectiveness and robustness of the proposed defense against adversarial attacks."

The right baseline for adaptive attacks is the attack with no detector access, not zero, since no text watermark resists rephrasing. The two-stage check aims to make public access no more useful to the attacker than no access. Adaptive rephrasing reaches high net success, but blind rephrasing already reaches similar rates at the same distortion (Tab. 2 vs. Tab. 6), so the detector adds only a few points. The remaining risk is forgery with word edits guided by token-level scores, which we report on purpose as a worst case. A deployed detector would most likely return a binary verdict, which does not help forgery (Tab. 1), and adaptive black-box removal does no better than the no-box attack at the same edit budget (App. D.5).

W3: Baselines

Few works make the detector transparent: Fairoze et al. (2024), UPV and Bashir (2026). Bashir only evaluates robustness to random edits, with no attacks on the public detector or analysis of the scheme's security, while what matters is the gap between no-box and informed attacks, since no-box attacks already work against any private watermark. We now compare with Bashir in the Maryland setting (table in our response to fMq6). Both public detectors are equally brittle under white-box removal (96-100% ASR_pub). Against Bashir, the public detector helps the attacker: white-box rephrasing fools the private verdict more often than blind rephrasing (96.6% vs. 88.6%). Against our scheme, it hurts the attacker: net success drops from 96.9% to 76.2%, because the tampering test flags 23.8% of these texts. White-box forgery passes Bashir's private detector 0.1% of the time and our pipeline 27.6%, since Bashir keeps the full private watermark, at the cost of added distortion.

We hope this clarifies the motivation for a public detector and the role of the two-stage pipeline, and that you will consider raising your score.

Thank you for the rebuttal. I understand the motivation for introducing a public detection mechanism. However, given the high attack success rates, I am still not persuaded that the proposed two-stage mechanism is necessary and practical. Therefore, I will keep my current rating unchanged.

Thank you for the reply and for the time you have spent on our paper. Since we agree that public detection is useful, the remaining question is whether a provider can release a public detector without weakening its own verification. The second stage of our construction is our answer to that.

We understand that the high attack success rates are striking, but they reflect a weakness of text watermarking itself, not of public detection. Rephrasing removes current text watermarks without any detector, private ones included (Krishna et al., 2023; Diaa et al., 2025): 82% on TextSeal (Tab. 6) and 88.6% on Bashir's private key (Tab. R2). Paraphrasing already reaches these rates in the no-box setting, so it threatens all current LLM watermark deployments. Current practice is to deploy a watermark and keep the detector secret, so that simple attacks with near-100% ASR, such as paraphrasing, do not expose how fragile the watermark is. By the criterion of near-zero ASR, no deployed text watermark would be practical. We therefore measure what the public detector adds on top of that baseline.

If providers want to offer some transparency, or are required to as in the image case, a split-key construction lets them do so without compromising robustness. The private parts of the signal can't be observed, and robustness is measured by ASR_net, which stays secret. We measure ASR_net to be lower than in the no-box case, except at very low distortions. The second stage is what achieves this. At the same overall FPR, the fused test alone lets through 79.3% of white-box word-edit removals and 75.4% of white-box forgeries. With the tampering test, those rates fall to 40.1% and 22.3%, at a cost of 0.4 points of TPR (table in our reply to fMq6).

The construction is also practical. With TextSeal, the split is the default routing, so generation is unchanged. The test reuses the detection scores and takes about 0.3 s of CPU per text (Tab. 4).

We strongly encourage future work on stronger independent signals for public and private verification, and we are pursuing it ourselves, because such signals would help the public navigate the current AI landscape.

Thank you.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General comment

Dear Reviewers and Area Chair,

We thank the reviewers for their thorough reviews, the interest they showed in the work, and for praising its novelty and importance.

We have uploaded a revised manuscript, with all changes highlighted in red. Specifically, we added:

  • Stronger attacks: DIPPER, and SIRA in no-box and white-box variants
  • A matched comparison with Bashir (2026)
  • A routing-ratio ablation, with an extension of the tampering test to any ratio
  • Detection across text lengths
  • Calibration of the tampering test on human text
  • A split-key version of the learning-based SIR watermark
  • A discussion of Bileve and PVMark

These results support the paper's main conclusion. A public detector is as brittle as any watermark detector, but it gives anyone a transparent check, which is especially useful in the agentic era. The provider's private verification still gives the full forensic verdict, and public access gives the attacker zero or limited advantage against it.

We would be very happy to further discuss any remaining concerns.

Dear Reviewers,

The discussion period will soon close. We would really appreciate any feedback on our responses, if you did not have the time to examine them yet, and we are more than willing to provide additional clarifications.

Thank you in advance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.