acceptodds
Under review as a conference paper at ICLR 2027

ByteMark: A Unified Byte-Level Watermark for Detection Across Language Models

Abstract

As watermarking of LLM-generated text enters deployment, users increasingly need to check text whose source is unknown. Existing detectors read the watermark through a specific tokenizer or embedding pipeline, and even if all providers adopted the same watermarking method, differences in these representations would leave their watermarks incompatible, and the text alone does not reveal which representation produced it. We propose ByteMark, a unified watermarking scheme that defines its signal on the UTF-8 bytes of published text. Models with different tokenizers embed this signal under one public specification and key, and a single detector verifies the text from the text and key alone. When the tokenizer or embedding model of a baseline detector is replaced, its detection degrades sharply, whereas ByteMark, which uses neither, retains its detection rate; ByteMark also remains detectable when participating models continue or rewrite the text.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI safety, trustworthy machine learning, large language models, digital watermarking

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uidJOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ByteMark defines a shared text-watermarking signal on the UTF-8 bytes of published text, not on generator-specific token IDs or semantic embeddings. During generation, candidate tokens are scored through their constituent bytes and high-scoring candidates receive a logit bias. Detection reconstructs the keyed byte scores directly from text, deduplicates repeated context–byte pairs, and applies a standardized statistic with an idealized finite-sample null distribution. Experiments investigate representation replacement, heterogeneous-model relays, rewriting with and without re-embedding, generation overhead, and cross-implementation agreement. The construction provides a clear tokenizer-independent detection interface, but its quality trade-off and incremental contribution relative to closely related prior disclosures require further substantiation.

Critical strengths
  1. A concrete and reproducible tokenizer-independent detector.

    Defining the signal on published text bytes removes the need to recover the generating tokenizer. The implementation discussion addresses byte fragments and token-to-byte mapping instead of relying on isolated-token decoding. The reported Python, JavaScript, and C++ agreement is directly relevant to the goal of a shared detection specification.

  2. Careful treatment of repeated observations, clear robustness boundaries, and promising computational efficiency.

    Deduplicating context–byte pairs addresses a genuine dependence problem in keyed-score aggregation, and the fixed-text ideal-PRF analysis gives a transparent statistical reference. The paper also reports non-participating rewriting separately from cooperative re-embedding, avoiding an important ambiguity in interpreting watermark continuity. The method modifies token selection using existing model logits. The reported throughput is close to unwatermarked generation in the tested setting, suggesting a potentially useful efficiency advantage over rejection-based sentence-level methods.

Critical weaknesses
  1. The detection and efficiency results lack generation-quality evaluation, and the unknown-source comparisons do not adequately quantify practical alternatives.

    ByteMark biases the token distribution, but the paper does not establish the resulting quality cost through perplexity, task performance, or human/model preference evaluations. Using the same bias parameters as KGW does not ensure comparable distortion because the boosted candidate sets are constructed differently. Averaging byte scores within tokens may also interact with token length and vocabulary structure. Consequently, high detection rates and low runtime overhead do not establish a favorable quality–detection trade-off. This is a central missing component for the deployment motivation.

    Replacing a detector's native tokenizer or embedding pipeline demonstrates representation dependence, but does not by itself establish superiority over a properly calibrated candidate search. Appendix E discusses Bonferroni and joint-score calibration, whereas Table 4 reports the FPR of an uncorrected search without the corresponding corrected-search TPR and runtime. The relay experiment similarly emphasizes mean local scores instead of document-level detection at controlled FPR. These comparisons are needed to quantify the practical benefit beyond the construction's already clear representation invariance.

  2. Closely related prior disclosure narrows the novelty claim, while empirical operating points and sample exclusions require stronger validation.

    The public patent document WO2025050084A1, "Improvements to language model watermarking", published on March 6, 2025, discusses watermarking ASCII/UTF-8 text representations instead of token IDs, tokenizer-free detection, and use across models (§§0014–0023 and 0033–0039). Its bitwise selection and token-acceptance procedure differs from ByteMark's score-and-bias construction. Nevertheless, the conceptual overlap is direct, and the submission should explain what its specific algorithm, calibration, and implementation contribute beyond that disclosure. This is a novelty and positioning concern.

    The main tables explicitly use same-pool empirical thresholds, while Appendix C excludes invalid or unscoreable outputs without replacement. Such results can support a controlled comparison on the evaluated pool, but do not independently validate a deployment FPR or establish end-to-end detection success. The paper should report separate calibration/test results, confidence intervals, and the numbers of requested, generated, and scored samples for each method. Appendix I.3 provides useful theoretical-threshold diagnostics, but does not replace this validation of the main empirical operating points.

Questions for authors
  1. Generation-quality trade-off and comparison with existing methods.

    What generation-quality cost accompanies the reported detection rates? At comparable quality degradation and a common FPR, how does ByteMark compare with KGW and SynthID?

  2. Tokenizer-search baselines and heterogeneous-model detection.

    When the generating tokenizer is included in the candidate set, what TPR and runtime does a properly corrected tokenizer search achieve? What are the corresponding document-level results on the heterogeneous-model relay?

  3. Technical novelty relative to prior disclosure.

    How does the proposed construction differ technically from WO2025050084A1, "Improvements to language model watermarking", and which advantages beyond the shared tokenizer-independent carrier are supported by the present analysis or experiments?

  4. Threshold calibration, sample exclusions, and end-to-end reliability.

    Which human texts are used exclusively for threshold calibration and which for independent FPR testing? How many outputs are excluded for each method, and do the conclusions hold on a common valid subset and when generation failures are included in the end-to-end denominator?

9 comments
AuthorsOpenReview ↗

Reply for uidj 1

Response to Reviewer uidJ, Part 1 of 2. We thank the reviewer for the detailed and constructive review. We also appreciate that the review finds the tokenizer-independent detector concrete and reproducible, and that it values the careful treatment of repeated observations, the clear robustness boundaries and the computational efficiency. For the experiments shared across the reviews, we kindly refer the reviewer to Sections 1 to 3 of the general response. Here we focus on the reviewer's own questions, with Q1, W1 and Q2 in this part and Q4 and Q3 in Part 2.

Q1, quality against KGW and SynthID. We agree that this comparison is central, and we first describe how we set it up. A watermark consists of a mechanism that alters sampling and a random source that drives it, and the change to the text comes mainly from the mechanism, which is why KGW and SynthID already differ in quality on token IDs. A direct comparison of ByteMark's default rule with SynthID would therefore mainly reflect the two mechanisms, whereas our contribution is the source. For this reason Section 1 compares each mechanism with itself on the two sources. For the logit bias, ByteMark and KGW have close similarity scores and detection rates, and ByteMark's perplexity change is no larger than KGW's (+41.0% against +55.2% on the full vocabulary, −37.5% against −37.9% on the top 41 candidates). For SynthID, the byte-level version keeps the behaviour of the original (perplexity change −6.1% against +5.1%, where a value nearer zero is better, and TPR 100.0% against 99.5%), and EXP and ITS show the same pattern. We read this as evidence that the byte-level source brings no systematic shift in quality.

W1, token length. We are grateful for this observation. It concerns how the rule treats tokens of different lengths, which we had not examined, and the reviewer is right that the two interact. We regard this as a limitation of the current rule. A token's score is the mean of its byte values, each uniform on [0, 1]. For a token of k bytes this mean is 0.5 for every k, but its variance is 1/(12k). The logit shift δ\delta is the same for every selected token, yet because of this variance tokens with more bytes are selected somewhat less often, and generated text leans slightly toward shorter tokens. Detection does not use tokens and is unaffected.

The dependence can be reduced. The distribution of the mean of k uniform values is known in closed form, so a token's score can be replaced by its quantile within the distribution for its own length. Tokens of common lengths then have nearly the same chance of being selected. To see how much the change matters, we generated with both rules under the conditions of the quality experiment.

Current ruleQuantile ranking
Detected at the 1% threshold198 / 200198 / 200
PPL change (%) OPT-2.7B+41.0+53.3
BERTScore.811.809
ROUGE-L.143.141
BLEU2.652.63

Detection and similarity are almost the same under the two rules, so the effect of the dependence on our results is small. The quantile ranking is nonetheless the more principled rule, and we will present it as an alternative in the revision.

Q2, corrected search and relays. Following the reviewer's request, Section 2 now reports a jointly calibrated search under two statistics, on single-generator text and two relays. On single-generator text with the tokenizer known, each method detects 98.5% to 99.0% of the texts under its better statistic. With thirty candidates the better of the two statistics detects at most 30% of the texts for KGW, 42% for SynthID, 11% for ITS and 90% to 95% for EXP, while ByteMark detects 98.5% to 100.0% without a search, and the byte-level versions of SynthID, EXP and ITS detect 99.0% to 100.0% in the same way. Over ten candidates a search takes 0.13 to 48 seconds per text, against 0.006 seconds for ByteMark (Section 3). Our reading is that, under these two statistics, a properly corrected search stays close to ByteMark only for EXP, which is also the slowest of the four detectors, and that for the other methods the rate falls as candidates are added.

AuthorsOpenReview ↗

Reply for uidj 2

Response to Reviewer uidJ, Part 2 of 2.

Q4 and W2, calibration and denominators. We agree that the submission should have separated calibration from testing. We have now fixed 1,000 calibration and 1,000 disjoint test texts in each of C4, Wikipedia and LFQA. Test FPR uses thresholds from the calibration texts only. TPR uses submission or independently calibrated thresholds, as labelled.

MethodTest FPR (%)Unscored (%)TPR, scored, submission (%)TPR, scored, calibrated (%)TPR, requests, submission (%)TPR, common, submission (%)
ByteMark0.80 [0.50, 1.13]1.2999.46 [99.20, 99.65]99.32 [99.04, 99.54]98.18 [97.74, 98.55]99.48 [99.22, 99.68]
KGW1.17 [0.96, 1.39]1.0299.64 [99.42, 99.79]99.53 [99.28, 99.71]98.62 [98.24, 98.94]99.67 [99.45, 99.82]
SynthID1.24 [1.01, 1.48]1.2299.98 [99.87, 100.00]99.98 [99.87, 100.00]98.76 [98.39, 99.06]99.98 [99.87, 100.00]
EXP1.08 [0.84, 1.32]1.1199.89 [99.74, 99.96]99.55 [99.31, 99.73]98.78 [98.41, 99.08]99.88 [99.73, 99.96]
ITS1.17 [0.94, 1.40]1.0098.00 [97.55, 98.39]98.70 [98.32, 99.01]97.02 [96.48, 97.50]98.08 [97.62, 98.47]

Brackets give 95% intervals. For the test FPR they come from a text-level bootstrap over the 3,000 test texts. The common subset contains the requests scored for every method. All five methods retain a high TPR. The revision will re-measure all detection results at held-out thresholds.

Q3, relation to WO2025050084A1. We thank the reviewer for this important prior work, which we had not known before submission. Section 3 of the submission lays out how we arrived at this carrier. It starts from the letters of a text, which any verifier can read without a tokenizer, notes that letters exist only in alphabetic scripts, and therefore moves to the UTF-8 byte stream, which covers all text. UTF-8 is also the encoding in which text is most widely stored and exchanged, and this made it our choice. We are glad to see that the patent takes the same view. It precedes us in proposing this carrier with tokenizer-free detection, and we do not claim priority for that choice.

The constructions share a byte-level carrier but differ. As the reviewer notes, the patent's bitwise selection and token-acceptance procedure differs from ByteMark's score-and-bias construction. We address the algorithm, implementation and calibration, then give a direct comparison.

Algorithm, a general source. The source was designed to be general. It deliberately outputs a uniform value on [0, 1), which can be transformed into any distribution, so any mechanism that draws on keyed random numbers can use it. ByteMark's rule, a logit bias on whole tokens, is one such mechanism. Byte-level SynthID, EXP and ITS run on the same source, with quality close to the originals (Section 1) and direct detection of 99.0% to 100.0% (Section 2).

Implementation, no framework dependence. A token-level watermark depends on the tokenizer, and it also carries a less visible dependence on the generator that turns seeds into random numbers. In practice this second dependence ties implementations to a framework. The released code of KGW, EXP and ITS draws its random numbers from PyTorch's generator. The PyTorch documentation states that results "may not be reproducible between CPU and GPU executions, even when using identical seeds", and the steps it offers to limit nondeterminism hold "for a specific platform, device, and PyTorch release" (Reproducibility note). Our random numbers come from AES, a widely adopted standard whose output is the same in every implementation. The detector needs neither PyTorch nor NumPy, and stand-alone Python and C++ versions reproduce the same scores up to floating-point rounding, at 0.001 to 0.006 seconds per text (Section 3).

Calibration. Our detection statistic has a known null distribution for a fixed text under an ideal keyed function, and thresholds are set on human texts separate from the test texts. The held-out test FPR is 0.80% at the nominal 1% level (Q4 above).

Results. For a direct comparison, we implemented the patent's construction as we understand it and ran it on the prompts of Section 1 at a nominal 1% FPR.

MethodTPR (%)
Patent construction, our implementation, λ\lambda = 12885.0
Patent construction, our implementation, λ\lambda = 3290.5
ByteMark99.0

This is our reconstruction, not the inventors' code. We had to fix details that the patent leaves open, and we regard the numbers as indicative.

The revision will cite and discuss the patent, credit its contributions, and state our own as the byte-level source, ByteMark's rule, the adaptation of established mechanisms and their evaluation.

Thank you for the additional experiments. The independent calibration split, explicit denominators, document-level relay results, and patent acknowledgment address important parts of my initial review, but several consequential issues remain:

1. The tokenizer search omits the standard document-level baseline. Taking maxima at individual bytes or over 50-byte windows alters how original detectors aggregate evidence, inflating null scores differently than document-level maximization. The natural baseline—running each detector on the full text and combining document-level evidence across candidate tokenizers (via multiple-testing-corrected p-values or calibrated document scores)—requires no token alignment. Please report this baseline (at least for the single-generator setting with the true tokenizer) alongside held-out FPR and runtime to separate search cost from local-maximum aggregation.

2. The byte-level adaptations lack algorithmic specifications and theoretical justification. Extending the random source to SynthID, EXP, and ITS requires more than uniform marginal PRF outputs. Because candidate tokens share byte prefixes, they share underlying PRF values, affecting the joint distribution, dependence structure, and distortion-free guarantees. Please specify the sampling and detection procedures—including how random values are indexed across candidates, positions, and layers—and clarify which original properties hold theoretically versus empirically, as detection rates and perplexity alone do not establish distribution preservation.

3. The length-based quantile correction overlooks repeated PRF inputs. The stated variance of a token’s mean byte score, 1/(12k)1/(12k), assumes independent byte scores. Under rolling contexts, repeated context–byte pairs receive identical PRF outputs (e.g., after eight spaces, a two-space candidate queries the same input twice, giving variance 1/121/12 instead of 1/241/24; generally ∑jaj2/(12k2)\sum_j a_j^2/(12k^2) for multiplicities aja_j). Since detector deduplication does not apply during generation, please clarify how candidate scoring and quantile correction handle such repetitions.

4. Quality and deployment claims overreach the evaluated conditions. Identical bias parameters do not ensure matched distortion across different candidate sets, and the PPL shift from +41.0% to +53.3% under quantile ranking is non-negligible. Please specify the checkpoints, sampling settings, and paired uncertainty estimates for the quality experiments. Moreover, since the original protocol uses temperature 1, top-p 1, and EOS suppression, please clarify which new results retain these settings and what evidence supports extrapolating to natural stopping or tighter sampling.

5. Aggregation of FPRs and confidence intervals. With 3,000 test texts, one decision per text changes FPR in 0.0333%0.0333\% increments, indicating that 1.24%1.24\% and 1.08%1.08\% average across configurations. Please specify the decisions per text, averaging rule, and whether the bootstrap groups all observations from the same text.

Points 1 and 2 are central to my final recommendation. Please distinguish evidence already available from revisions planned for later.

Reviewer_SoCzOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Many platforms want to know whether a text was written by an AI model without knowing which model wrote it. Existing watermarks tie detection to the generator: token-level schemes need its tokenizer, and semantic schemes need its embedding model. This paper proposes ByteMark, a watermark shared by participating model providers. All providers use one secret key and a public specification, and the watermark works on UTF-8 bytes instead of tokens. At each decoding step, the model scores every candidate token by the average keyed pseudorandom score of its bytes, where each byte's score depends on a short window of preceding bytes, and it boosts the top-scoring candidates. Because bytes do not depend on the tokenizer, one detector can recompute the scores from the text and the key alone, for any participating model. The detector counts each repeated context-byte pair once, which gives an exact null distribution. On three small open models and English corpora, the authors report that baseline detectors fail when they use the wrong tokenizer or encoder, while ByteMark detects text from all three models, follows the signal across a document written by several models in turn, and adds little generation overhead.

Critical strengths
  1. One detector works without knowing the tokenizer. Across the three tested generators and corpora, it detects watermarked text reliably at a low false-positive rate, and its signal persists across a document written by several models in turn. I like that the construction is simple: a KGW-style boost (Kirchenbauer et al., 2023) applied to bytes and averaged per token. One caveat is that all generated text is English, although handling other scripts is the main reason to work with bytes.

  2. The false-positive control is careful. Counting each context-byte pair once gives an exact null distribution, and the measured false-positive rates stay close to the nominal level on several human corpora, including Chinese and German, where counting every occurrence would inflate them. A verifier who cannot see the generator needs this kind of calibration. The deduplication follows Fernandez et al. (2023), and the check covers only moderately low false-positive rates.

Critical weaknesses

1. The scheme asks every provider to adopt one technology and share one key, and the paper does not evaluate what that costs.

  • Unknown-source detection works only if all relevant providers switch to the same specification and key. The paper motivates participation with regulation that requires some form of machine-readable marking, but that does not imply one shared scheme. The paper does not discuss how competing providers, including those that already deploy their own watermarks, would agree on one specification and share one key. Text from non-participating models, including open-weight models run without the watermark, stays undetectable, so the verifier still cannot check text of truly unknown origin.
  • The shared key is also a security risk. If verifiers hold it, as the suggested browser-extension detector implies, anyone with the key can forge or strip the watermark. If a trusted service holds it, rules learned from one participant would work against every participant, because the byte rules do not depend on the tokenizer. Watermark stealing (Jovanović et al., 2024) needs only query access to a generator, so the cheapest participant's API could become a tool to spoof or scrub text for the whole scheme.
  • The paper acknowledges that a lost key exposes everyone, but it tests no stealing or adaptive attack. Per-provider keys would contain the damage, but then a verifier must test every key, which brings back the multiple-testing cost the paper counts against tokenizer search.

2. Each headline comparison is set up so the baselines cannot win.

  • A detector that reads text with the wrong tokenizer or encoder is expected to fail, so the replacement experiments mainly show that the baselines need their own tokenizer. With their own tokenizer, KGW and SynthID detect as well as ByteMark or better (Table 2).
  • The relay experiment reads the whole mixed document with one fixed tokenizer, so a token-level detector can only see its own segment. The fair baseline runs each participant's detector over windows, corrected for multiple testing. The paper derives corrected searches over tokenizers, but it reports no detection rate for them and does not consider a windowed version for mixed documents.
  • The efficiency comparison is against sentence-level schemes that generate many candidate sentences and discard most of them. The token-level schemes, which also sample once and are the real alternative, are not timed.
  • The paper's answer is that closed-model tokenizers may not be public. That is fair, but it is an argument about access, not about detection power, and ByteMark also requires every provider to participate.

3. The cost in text quality is not measured.

  • The paper reports no measure of text quality or downstream usefulness for watermarked against unwatermarked generation. It only notes that the boost strength trades detection against quality.
  • Without this, the detection results cannot be compared fairly with token-level watermarks, whose strength can be tuned to the same quality. Detection should be compared at matched quality.
Questions for authors
  1. How does a corrected search over the participants' tokenizers compare with ByteMark at low false-positive rates, on single-source and relayed texts, and how does the gap change as the number of participants grows? If ByteMark keeps a clear lead, my concern about the baselines is resolved.

  2. Who holds the key in the intended deployment, each verifier or a trusted service? If it is a service, could you run a stealing attack: learn the byte rules from one participant's outputs, then try to spoof or scrub another participant's text?

  3. What does the boost cost in text quality and downstream usefulness, compared with unwatermarked generation and with a token-level watermark at matched detection strength?

2 comments
AuthorsOpenReview ↗

Reply for socz 1

Response to Reviewer SoCz, Part 1 of 2. We are grateful to the reviewer for the thoughtful and direct review. We value the remarks that the construction is simple and that the false-positive control is careful. The review led us to add experiments and revise our claims. Sections 1 to 3 of the general response report the shared experiments. Here we address W1 and Q2, and Part 2 covers the remaining points.

W1 and Q2, sharing one technology and one key. We thank the reviewer for examining what a shared scheme and a shared key ask of the providers. These questions decide whether the deployment we describe is realistic, and the submission treated them too briefly.

(a) Does every provider have to adopt the scheme? Unknown-source detection requires the relevant providers to share a specification and key. ByteMark also serves a single provider. Tokenizers change between one provider's model generations and sizes, as the table shows. A token-level watermark therefore needs a different detector for each of these models, whereas with ByteMark they share one key and one detector. A second case is one company's agents running on different models, including fine-tuned ones, which need not share a tokenizer or vocabulary. With ByteMark their outputs are still detected within one system, with one key and one detector. A third case is compatibility over time. Text generated by a provider's current models remains detectable even if the provider adopts a new tokenizer in the future, since the detector reads only the bytes and the key. A token-level watermark would have to keep each retired tokenizer and search over them to cover text from the past. None of these cases needs regulation or an agreement between competitors.

ProviderModelsVocabulary size
OpenAIGPT-2 → GPT-4 → GPT-4o50,257 → 100,277 → 200,019
MetaLlama 2 → Llama 332,000 → 128,256
MicrosoftPhi-2 → Phi-3 mini / small / medium → Phi-451,200 → 32,064 / 100,352 / 32,064 → 100,352

Across providers, ByteMark brings an additional option and otherwise leaves things as they are. Providers with separate keys stay as separate as different tokenizers keep them today, and providers that want joint detection can share a key, which needs only a common specification. It is public, rests on AES, and can sit under a provider's existing mechanism, as we did for SynthID, EXP and ITS (Section 2). We agree that reaching such an agreement is a matter of standardization beyond this paper, and what we contribute toward it is a concrete specification that independent parties can implement and check (Section 3). As with any watermark, text from non-participating models remains undetectable. Requiring their participation needs additional regulation. Among the participants, however, a verifier needs to know neither the source of a text nor its tokenizer.

(b) Who holds the key, and what do separate keys cost? We agree with the reviewer that anyone who holds the key can forge or strip the watermark. This risk comes with a secret key as such, and current watermarks share it, since KGW, SynthID, EXP and ITS are also symmetric-key schemes in which detection uses the same secret as embedding. The answer therefore lies in how the key is kept. One arrangement is that the participating generators and a trusted verification service hold the key. The paper mentions a browser extension mainly to illustrate how lightweight and extensible the detector is, since it needs no language model and no tokenizer. We acknowledge that an extension open to the general public may not be feasible under such an arrangement, as it would have to carry the key. In practice such a tool would sit within a trusted setting, for example as part of a company's internal trusted service, or be activated through a trusted channel. We offer these as suggestions, since custody is a deployment decision outside the method.

We also agree that under a shared key a rule learned from one participant applies to all of them. Our main point is that the scope of a shared key is a choice, and that ByteMark places this choice with the deployment. For watermarks in general, models that can be detected together can also be attacked together, so what matters is who draws the boundary. With a token-level watermark the tokenizers draw it. They separate the models whether or not the deployment wants it, and the separation stops the legitimate verifier as much as an attacker. With ByteMark the keys draw it. Models that should be detected together, such as the successive models of one provider, share a key, and parties that want isolation use separate keys. If multiple keys need to be tested, the multiple-testing cost noted by the reviewer applies. This cost is the price of isolation, and only those who choose isolation pay it.

AuthorsOpenReview ↗

Reply for socz 2

Response to Reviewer SoCz, Part 2 of 2. This part completes W1 and Q2 and then answers the remaining points.

W1 and Q2, continued. (c) Does watermark stealing apply? ByteMark's default rule follows the same logit-bias algorithm as KGW, so we expect that the attack of Jovanović et al. (2024) carries over to it. This exposure arises at the level of the mechanism, the rule that alters sampling, whichever random source drives it. What our work contributes is the random source, which is not tied to this mechanism. Other algorithms can draw on it, as byte-level SynthID, EXP and ITS already do, and algorithms proposed in the future can be adapted in the same way while keeping tokenizer-independent detection.

This also suggests a way forward. The attack learns how a watermark shifts the text distribution, and Jovanović et al. point to schemes that leave the distribution unchanged, such as those of Kuditipudi et al., that is, EXP and ITS, as a promising direction. Their byte-level versions already run on our source and could replace the default rule, although whether they resist stealing in practice is still open.

Q1 and W2, baselines. The submission's replacement experiments examined dependence on the generating tokenizer. We understand the reviewer's concern that they do not show what a search over tokenizers could achieve, and we have added a jointly calibrated search that includes the participants' tokenizers. Section 2 now reports such a search on single-generator text and two relays. ByteMark keeps its lead under the two reported search statistics, and more participants may make token-level search harder. With ten candidates, KGW and ITS detect at most 45% and 27% of the texts. SynthID is still at 97% or above with ten candidates and falls to 42% or less with thirty. EXP comes closest and stays 3.5 to 11 points below ByteMark with thirty to fifty candidates. ByteMark and the byte-level versions of SynthID, EXP and ITS detect 98.5% to 100.0% without a search.

For these two search statistics, tokenizers cut the text differently, so we align their contributions on byte offsets. These constructions thus also use bytes as a common coordinate. The reviewer also points out that the paper's remark on closed tokenizers concerns access and not detection power. We agree, and for this reason every search above is given all the participating tokenizers. Missing participating tokenizers could reduce detection, although the effect also depends on the recalibrated threshold.

W3 and Q3, quality. Section 1 now addresses quality through similarity to unwatermarked output and perplexity change. Under the same γ\gamma and δ\delta, ByteMark and KGW detect 99.0% against 100.0% of the texts on the full vocabulary and 99.0% against 99.5% under the top-k scope. These similar detection rates follow from their shared mechanism of adding δ\delta to the logits of a γ\gamma fraction of candidates. Their similarity scores are close, and ByteMark's perplexity change is no larger than KGW's in either scope.

Non-English generation. As the reviewer points out, all generated text in the submission is English, although other scripts are a main reason to work with bytes. We have now generated Chinese and German text with two multilingual models, using 200 prompts per language and 200 tokens per generation. For each language and model, the threshold is calibrated at a nominal 1% FPR on 1,000 human documents of the same corpus, and the FPR is tested on 1,000 disjoint ones.

ModelLanguageTPR over valid outputs (%)Test FPR (%)
EuroLLM-1.7BChinese99.00.9
EuroLLM-1.7BGerman100.01.5
Qwen2.5-1.5BChinese99.00.8
Qwen2.5-1.5BGerman99.51.6

False-positive control at different rates. The reviewer notes that the check in the submission covers only moderately low false-positive rates. We have extended it with thresholds calibrated on separate human texts and tested on 1,000 disjoint texts from each of C4, Wikipedia and LFQA. The table gives test FPR and TPR over scored outputs at three nominal levels, down to 0.5%.

Nominal FPRByteMarkKGWSynthIDEXPITS
5%Test FPR (%)4.834.975.084.824.91
TPR (%)99.7599.8799.9899.8099.26
1%Test FPR (%)0.801.171.241.081.17
TPR (%)99.3299.5399.9899.5598.70
0.5%Test FPR (%)0.470.720.730.540.58
TPR (%)99.1099.3799.9899.5198.27

ByteMark's test FPR stays below the nominal value at every level and is the lowest of the five methods at 1% and 0.5%.

In the revision, the discussion will state that unknown source means unknown among the participants, that key custody and standardization are deployment questions outside the method, and that the default rule is expected to share the exposure of KGW-style rules to stealing.

Reviewer_gRx5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces "ByteMark," which utilizes UTF-8 bytes as a shared carrier for watermarking. By biasing the probability distribution through byte-level scoring, it enables detection based solely on the text, a secret key, and a set of public specifications—facilitating unified detection across various language models. In its current form, this is a "zero-bit" watermark; it merely determines the presence of a watermark associated with a shared key, without encoding specific user or model identities.

Critical strengths
  1. The unified detection interface offers significant practical value. Detection does not rely on the original tokenizer or embedding model; the method achieves a True Positive Rate (TPR) of 97.8%–100% across nine model-text domain combinations and maintains signal continuity even when text is processed sequentially by multiple models.

  2. The statistical design for detection is supported by both theory and empirical evidence. The method deduplicates repeated context-byte inputs and analyzes the null distribution under assumptions such as the use of an ideal Pseudorandom Function (PRF) and the independence of the text from the secret key. Experiments with human-written text confirm that deduplication helps mitigate the variance inflation caused by repetitive inputs.

Critical weaknesses
  1. The impact of the watermark on text quality remains unassessed. The paper lacks evaluations regarding perplexity, semantic preservation, or human-rated quality. Furthermore, employing the same γ\gamma and δ\delta parameters as those used in KGW does not necessarily imply equivalent quality degradation; thus, it is difficult to gauge ByteMark’s detection performance under strictly controlled text-quality constraints.

  2. The efficiency of the embedding and detection processes is not sufficiently validated. Table 5 reports generation throughput for only a single 3B-parameter model and lacks direct comparisons with low-overhead baselines like KGW or SynthID. Additionally, no data is provided regarding the time complexity, throughput, or memory footprint of the detection process, making it impossible to substantiate claims of efficiency advantages for the overall pipeline.

Questions for authors
  1. When held to the same standards for text quality and overall false-positive rate, how does ByteMark compare to KGW, SynthID, and multi-tokenizer search approaches in terms of detection rate, as well as the computational costs of embedding and detection?

  2. If the framework were extended to support user or model attribution—for instance, by using unique keys or multi-bit identity payloads—could the unified detection capability still be maintained? How would such an extension affect detection costs and false-positive rates? General Overview

2 comments
AuthorsOpenReview ↗

Reply for gRx5 1

Response to Reviewer gRx5, Part 1 of 2. We sincerely appreciate the reviewer's constructive comments, and the assessment that a unified detection interface offers practical value and that the statistical design is supported by both theory and empirical evidence. Sections 1 to 3 of the general response report the shared experiments. Here we discuss what the results show about quality, tokenizer search and cost, before turning to the reviewer's question on attribution in Part 2.

W1 and Q1, quality. The reviewer notes that the same γ\gamma and δ\delta need not imply the same loss of quality, and we agree that this has to be shown. Section 1 reports perplexity change, similarity to the unwatermarked output, with BERTScore as the measure of semantic preservation, and TPR for ten configurations. ByteMark and KGW share the same mechanism. Both add δ\delta to a γ\gamma fraction of the candidates, and they differ mainly in how that fraction is chosen. Their similarity scores and detection rates are close under either bias scope, and ByteMark's perplexity change is no larger than KGW's (+41.0% against +55.2% when the full vocabulary is biased, and −37.5% against −37.9% under the top-kk scope). For SynthID, EXP and ITS the byte-level version stays close to the original, with BERTScore within 0.003 and a detection rate at least that of the original in each pair. The perplexity change of the byte-level versions stays within about 6% of zero (−6.1%, +0.7% and +2.4%, against +5.1%, −15.8% and −0.2% for the originals). In these configurations, removing tokenizer dependence need not impose a larger quality loss.

Q1, detection against tokenizer search. The reviewer also asks how ByteMark compares with a multi-tokenizer search. On single-generator text with the tokenizer known, each method detects 98.5% to 99.0% of the texts under its better statistic. When it must be searched for among thirty candidates, the better of the two search statistics detects at most 30% for KGW, 42% for SynthID and 11% for ITS, and 90% to 95% for EXP, while ByteMark detects 98.5% to 100.0% without a search. The byte-level versions of SynthID, EXP and ITS, which also need no search, detect 99.0% to 100.0% (Section 2). Under these two search statistics, direct byte-level detection therefore retains higher detection rates without having to identify the generating tokenizer.

W2, cost. Following the reviewer's suggestion, Section 3 adds a second and larger model, Mistral-7B, and times ByteMark against KGW, SynthID, EXP and ITS. ByteMark's throughput stays close to unwatermarked generation, at 26.5 against 27.3 tokens per second on OpenLLaMA-3B and 20.3 against 21.3 on Mistral-7B. On the same two models KGW runs at 8.8 and 8.2 tokens per second and SynthID at 24.5 and 19.2. For detection, the detector makes one pass over the nn bytes of a text and evaluates the keyed function once for each distinct pair of an eight-byte context and the next byte. The time is O(n)O(n) with at most nn AES evaluations, and the detector loads no model, tokenizer or vocabulary. Detection takes 0.0062 seconds and 0.04 GiB per 1,024-byte text in Python and 0.0010 seconds and 0.007 GiB in C++, whatever the number of candidate tokenizers. A search with the released token-level detectors over ten candidates takes 0.13 to 48 seconds per text and 1.2 to 1.8 GiB when run in one process, and 0.02 to 12 seconds and 8.9 to 10.4 GiB with one process per candidate. ByteMark thus combines near-unwatermarked generation throughput with very low detection time and memory use, neither of which grows with the tokenizer candidate set.

AuthorsOpenReview ↗

Reply for gRx5 2

Response to Reviewer gRx5, Part 2 of 2. This part answers Q2.

Q2, attribution. We especially thank the reviewer for this question, which points to a valuable extension. Attribution needs either keyed signals that can be told apart or keyed random bits that carry a payload, and both come from the random source. Our source is defined on bytes and specified end to end, from text to uniform values, so both extensions would keep detection independent of the tokenizer.

Unique keys. Independent keys produce independent signals, which is what lets a key attribute a text to its holder. The cost follows from the same independence. A text tested against KK keys needs KK passes over its bytes, none of which needs a tokenizer, so detection time grows linearly in KK from a small base, about one millisecond per key with our C++ detector. The threshold must also account for the KK tests. For a small set of known keys this is feasible, at the price of a somewhat higher threshold. The candidates must be known keys, and here the situation differs from tokenizer search. A key is a 128-bit secret, so a search over unknown keys cannot be carried out, and a threshold corrected for 21282^{128} candidates would lie well above the scores that texts of a few hundred tokens reach.

Multi-bit payloads. The multi-bit extension mentioned by the reviewer offers another route to attribution that could avoid testing a separate key for each source. One possible way is to adapt the construction of Gloaguen et al. (2026) in Every Bit, Everywhere, All at Once. Their method encodes all payload bits at each token position using keyed, context-dependent scores and recovers the payload by accumulating evidence for each bit. Our byte-level source could supply these scores by splitting one uniform value into nn components. For an original U∈[0,1)U\in[0,1) and qq bits per component, define

Ui=2−q(⌊2iqU⌋ mod 2q),i=1,…,n.U_i=2^{-q}\left(\left\lfloor 2^{iq}U\right\rfloor\bmod 2^q\right),\qquad i=1,\ldots,n.

Each component uses a disjoint block of qq bits. Under the ideal uniform model, these are independent discrete uniforms on [0,1)[0,1) with resolution 2−q2^{-q}. Our current 64-bit output supports nq≤64nq\leq64 with one keyed evaluation. If this precision is insufficient, additional AES-CMAC evaluations can run in parallel with distinct domain-separated indices, generating independent pseudorandom uniform values from the same byte context and next byte. Repeating this gives qq bits per component with ⌈nq/64⌉\lceil nq/64\rceil keyed evaluations. Payload decoding would still need no generating tokenizer. A source identity could then be carried in the payload under one key. This is a possible extension of our construction rather than an implemented result; adapting the encoder and calibrating the decoder remain implementation work. Paper.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response for reviewers 1

General response, Part 1 of 3. We are grateful to the reviewers for their careful reviews and for the suggestions that helped us improve the work. ByteMark addresses tokenizer dependence by constructing watermark signals directly from UTF-8 bytes. Its random numbers come from AES, an open standard, so the path from text to random numbers is public and specified end to end, without a tokenizer or a framework's random number generator. What results is a general-purpose random source and not only one algorithm, since its uniform output can be transformed into any distribution a watermark requires. The submission uses this source to construct ByteMark, whose detector needs only the key and the text. The same holds for mechanisms adapted to this source. To test beyond ByteMark, we built byte-level versions of SynthID, EXP and ITS during the rebuttal. Sections 1 and 2 evaluate them, and the results suggest that future mechanisms can be adapted in the same way.

We appreciate the three aspects that the reviews raise in common. Section 1 below covers text quality, Part 2 tokenizer search with relays (Section 2), and Part 3 efficiency (Section 3). For the experiments in this general response, thresholds are set on independent human texts at a nominal 1% FPR, and on disjoint held-out texts the measured FPR stays close to this level, with ByteMark the lowest at 0.80%, as our reply to Reviewer uidJ details. Under the same protocol we also extended the evaluation beyond English. With two multilingual models, ByteMark detects 99.0% to 100.0% of generated Chinese and German text at test FPRs of 0.8% to 1.6%, as our reply to Reviewer SoCz details.

1. Text quality (uidJ W1/Q1, SoCz W3/Q3, gRx5 W1). We thank the reviewers for asking what the watermark costs in text quality, which is indeed an important question. Before the numbers, it may help to separate the mechanism of a watermark, which alters sampling, from the random source that drives it. The change to the text comes mainly from the mechanism. Our work replaces the source, from token IDs to bytes, and leaves the mechanism as it is. The question is therefore whether the byte-level source shifts quality on its own, and we test this by running each mechanism with both sources. We evaluated ten configurations on 200 texts each and compare every output with the unwatermarked output for the same prompt. In the tables, TPR is over the requested generations and PPL change is the relative change in geometric-mean PPL from unwatermarked text. Since a watermark should stay faithful to the unwatermarked model, a change nearer zero is better. Reading a lower PPL as better quality can be misleading (Cai et al., JASA 2026). Paper.

We first compare ByteMark with KGW, whose logit bias it shares. Both bias the same fraction of the vocabulary by the same amount, and in the table below their similarity scores and detection rates are close. Their PPL also moves together, and ByteMark's change is no larger than KGW's in either scope. Biasing the full vocabulary raises it for both, and restricting the bias to the top 41 candidates, the default of KGW's self-hash scheme (top-k rows), lowers it for both. Both methods move the same way under each scope, which points to the bias scope as the cause and not to the move to bytes.

ConfigBERTScoreROUGE-LBLEUPPL change (%) OPT-2.7BTPR (%)
ByteMark full.811.1432.65+41.099.0
KGW full.810.1442.79+55.2100.0
ByteMark top-k.815.1482.91−37.599.0
KGW top-k.815.1442.74−37.999.5

We then compare SynthID, EXP and ITS with their byte-level versions on our source. These three methods are designed to preserve the next-token distribution. Their sampling paths differ from those of ByteMark and KGW, which retain the unwatermarked sampling seed and share more opening tokens with the control. These differences may partly explain the lower BLEU and ROUGE-L scores below. We thus add independent unwatermarked samples as a reference in its first two rows. They are drawn for the same prompts with new seeds and compared with the same unwatermarked output. Two observations follow. Against this reference, every configuration stays at the level of sampling variation in similarity, and its PPL change is of comparable size. Within each pair, the byte-level version and the original score alike, and neither is consistently higher or lower in PPL. We read this as evidence that moving from token IDs to bytes does not shift quality in a consistent direction.

ConfigBERTScoreROUGE-LBLEUPPL change (%) OPT-2.7BTPR (%)
Unwatermarked, second sample.810.1271.70+11.0-
Unwatermarked, third sample.809.1251.62+6.1-
SynthID byte.811.1281.70−6.1100.0
SynthID.811.1311.82+5.199.5
EXP byte.811.1311.72+0.7100.0
EXP.811.1291.61−15.8100.0
ITS byte.811.1271.84+2.499.5
ITS.808.1231.53−0.298.0
AuthorsOpenReview ↗

General response for reviewers 2

General response, Part 2 of 3. This part covers Section 2. As in Part 1, all thresholds are calibrated on independent human texts at a nominal 1% FPR.

2. Tokenizer search and relays (uidJ W1/Q2, SoCz W2/Q1). We thank the reviewers for asking what a properly corrected tokenizer search achieves. This matters because detection without a tokenizer should be compared with jointly calibrated token-level search, and the experiments below follow their suggestions. Settings are written as generators × tokens each, so 1×200 is single-generator text, and 3×67 and 5×40 are relays in which three or five models continue one text in turn. Each setting has 200 texts.

We start with the methods that need no search. ByteMark and the byte-level versions of SynthID, EXP and ITS are detected directly from the key and the text, so each has one detection rate per setting (TPR, %).

SettingByteMarkSynthID byteEXP byteITS byte
1×20098.5100.0100.099.5
3×6799.5100.0100.0100.0
5×40100.099.599.599.0

To our knowledge, the token-level originals do not specify how to detect when the tokenizer is unknown, so we constructed a search on their behalf. The natural idea is to score the text with every candidate tokenizer and keep the strongest evidence. One joint threshold is calibrated over all candidates, which accounts for multiple testing. What remains is how to combine the candidates' scores. Ideally they would be compared within the same token. The difficulty is that candidates cut the same text into different tokens, so their scores cannot be lined up in this way. Byte offsets are shared by all candidates, and we use them as the common coordinate, in two ways. The first statistic spreads each token's score over its bytes, keeping its total contribution, and takes the maximum over candidates at every byte. The second computes each candidate's original score inside sliding windows of 50 bytes and takes the maximum over windows and candidates.

Cells give TPR (%) as per-byte maximum / window. In every configuration we keep all the true tokenizers in the candidate set, so the right tokenizer is always available to the search. Missing participating tokenizers could reduce detection, although the effect also depends on the recalibrated threshold.

SettingTokenizersKGWSynthIDEXPITS
1×200198.5 / 84.598.5 / 98.599.0 / 99.099.0 / 54.5
531.0 / 57.095.5 / 98.099.0 / 98.567.5 / 25.5
104.0 / 41.592.0 / 97.098.5 / 98.522.0 / 8.5
302.0 / 27.042.0 / 30.069.5 / 95.08.0 / 3.0
501.5 / 19.514.5 / 21.024.5 / 94.55.0 / 2.0
3×67384.5 / 64.599.0 / 100.0100.0 / 99.582.0 / 44.5
104.0 / 42.086.5 / 99.094.0 / 98.026.5 / 19.0
302.5 / 30.023.0 / 33.028.0 / 90.08.0 / 4.5
501.5 / 15.58.5 / 28.57.5 / 90.04.0 / 3.0
5×40524.0 / 62.598.0 / 100.099.0 / 98.523.0 / 13.0
102.5 / 45.077.5 / 98.590.0 / 97.011.0 / 6.0
301.0 / 25.511.5 / 31.011.5 / 90.011.0 / 4.0
500.5 / 16.03.0 / 16.51.5 / 89.05.0 / 2.0

On single-generator text with the tokenizer known, each method detects 98.5% to 99.0% of the texts under its better statistic, as the reviews anticipated. The differences appear once the tokenizer has to be searched for. Under the two statistics we report, they vary by method. With ten candidates, KGW and ITS detect at most 45% and 27% of the texts. With the window, SynthID stays at 97% or above up to ten candidates, and under either statistic it falls to 42% or less at thirty. EXP holds up best. With the window it is within three points of ByteMark up to ten candidates and 3.5 to 11 points below it at thirty to fifty. The four byte-level methods detect 98.5% to 100.0% of the texts in all three settings, as in the first table, since they do not depend on the candidate set.

A search also carries costs that the byte-level methods are spared. The statistic has to be chosen in advance, and the choice matters. EXP keeps its rate only with the window, and under the per-byte statistic it falls to between 1.5% and 24.5% at fifty candidates. The cost also grows with every candidate, and EXP, the most robust of the four, is the most expensive to run (Section 3). The byte-level methods avoid both.

AuthorsOpenReview ↗

General response for reviewers 3

General response, Part 3 of 3. This part covers Section 3.

3. Efficiency (SoCz W2, gRx5 W2). We also thank the reviewers for their questions on speed and detection cost. Both decide whether a watermark can be deployed at scale, and we report them below against the token-level schemes.

For generation, we measured throughput with two models on a Tesla V100 server, chosen for its thermal stability during long runs. ByteMark stays close to unwatermarked generation, as do EXP and ITS, and it is faster than KGW and SynthID.

Throughput (tokens/s)UnwatermarkedByteMarkKGWSynthIDEXPITS
OpenLLaMA-3B27.3326.498.7924.5027.2026.98
Mistral-7B21.3220.318.1719.2421.3321.10

For detection, we measured time and peak memory for one tokenizer and for a search over 10 candidate tokenizers, run in one process or in ten. The true tokenizer is always a candidate, loading is not timed, and EXP and ITS are timed without their permutation test, which would take much longer. ‡ KGW's released detector runs on a GPU, which the ten processes share, so running them in parallel does not shorten its time.

Detector1 tokenizer, time (s/text)memory (GiB)10 candidates in one process, time (s/text)memory (GiB)10 candidates in ten processes, time (s/text)memory (GiB)
KGW‡0.0751.00.681.30.9610.4
SynthID0.141.11.181.80.1910.4
EXP3.80.96481.61210.3
ITS0.0130.850.131.20.0218.9
ByteMark, Python0.00620.04no searchno searchno searchno search
ByteMark, C++0.00100.007no searchno searchno searchno search

ByteMark runs once, so its cost does not depend on the number of candidates, and it is the lowest in both time and memory. Tokenizer search runs a detector for each candidate. A single process evaluates the candidates one after another. Running ten candidates in parallel reduces the elapsed time for SynthID, EXP and ITS, but uses about eight times as much memory as evaluating the same ten candidates in one process. The released detectors of the four token-level methods require loading PyTorch. Those of KGW, EXP and ITS also use its random number generator. ByteMark needs neither a tokenizer nor PyTorch. The two ByteMark rows measure stand-alone Python and C++ detectors, both of which reproduce the scores used in our experiments up to floating-point rounding.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.