acceptodds
Under review as a conference paper at ICLR 2027

Multi-Bit Watermarking for Discrete Diffusion Language Models via Bitwise Evidence Feedback

Abstract

Discrete diffusion language models (dLLMs) offer a new paradigm for language generation, but reliable multi-bit watermark embedding and recovery remain challenging. Iterative denoising involves partially masked, non-causal contexts and uneven evidence accumulation across message bits. This work focuses on embedding multi-bit messages during dLLM generation and recovering them from the resulting text. We propose BEFMark, a watermarking framework that uses bitwise evidence feedback to adapt embedding throughout denoising. BEFMark extracts bitwise evidence from partially masked sequences through context-aware multi-layer voting and estimates candidate tokens’ expected evidence contributions under incomplete contexts. This feedback guides a KL-regularized update of the generation distribution, strengthening weakly supported bits while penalizing deviation from the base model’s predictions. Experiments on LLaDA-8B-Instruct and Dream-v0-Instruct-7B across multiple datasets demonstrate improved bit recovery accuracy, with additional exact-message recovery gains on LLaDA-8B-Instruct. BEFMark also improves recovery robustness under text editing while maintaining competitive log-perplexity and downstream task performance.

Primary area:
generative models
Keywords:
Discrete Diffusion Language Models (dLLMs), Watermarking

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_fj9VOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors of the paper introduce BEFMark, which is an inference-time multi-bit watermark for discrete diffusion language models. The main contribution appears to be that, during denoising, the watermark extracts keyed votes from resolved token-context pairs, tracks normalized evidence for each message bit, and uses a soft-minimum objective to prioritize weakly supported bits. It estimates additional evidence under a product distribution over unresolved tokens and uses this estimate in a KL-regularized update of the model’s predictions. Then during the message recovery process, the watermark recomputes the keyed assignments and aggregates votes.

The authors carry out experiments on some LLMs, including LLaDA-8B-Instruct and Dream-v0-Instruct-7B, and compare across datasets in NLP (such as ELI5, FiQA, and C4). They compare bit recovery and generation quality against existing methods. The baselines compared against are MPAC of Yoo et al., (2024) and BiMark of Feng et al., 2025.

Critical strengths

There are two main strengths of this work, in my opinion:

(1) The central adaptive-feedback idea has direct experimental support: The uniform-weighting ablation in Table 5 is quite convincing: adaptive weighting increases 16-bit accuracy from 83.11% to 93.51% at 300 tokens and from 84.93% to 95.57% at 500 tokens, with essentially the same (log)perplexity. This provides evidence that directing effort toward weak bits improves recovery and goes beyond the general benefit of applying a (stronger) watermark.

(2) The improvements extend to complete-message recovery at modest computational cost: For 500-token LLaDA generations, Table 4 reports exact-message recovery of 92.67%, 57.17%, and 14.33% for 8-, 16-, and 32-bit messages, respectively, compared with the strongest baseline values of 68.83%, 31.00%, and 1.17%. These gains are quite good because high average bit accuracy alone can hide poor recovery of an entire identifier. Table 12 reports generation time of 18.18 seconds vs. 16.32 seconds without watermarking and 0.43 GB additional peak memory.

Even though the main tables are in the appendix (i.e., supplementary material), I was engaged enough to read some of these results.

Critical weaknesses

Even though the paper is quite nice, I still have some questions (to be outlined later). For now, here are some weaknesses:

(1) The comparison of recovery–quality tradeoffs is not sufficiently documented. In Appendix B.1, the authors try to lay out the method’s main settings, but the competing methods’ watermark strengths, layer counts, and diffusion/remasking configurations are not comparably clear. Figure 4 connects different generation lengths but does not sweep each method’s embedding strength. Since Table 6 shows that BEFMark’s accuracy and quality change with its strength parameter, the current results are not clear enough on whether the advantage also appears well-tuned baselines at matched quality and actual output length. (The strong internal ablation partially addresses this concern.)

(2) The efficient implementation of the central expectation is underspecified. Equation (9) in the paper involves interactions between candidate tokens and context-hash values, with K=∣V∣K=|\mathcal V|. For a general keyed partition, direct evaluation involes a quadratic vocabulary-size interaction! However, the paper does not explain whether its implementation uses truncation, sampling, a structured partition, or another computational shortcut. So, as a reader of the paper, it might be difficult to reproduce the central algorithm. (This is an implementation concern.)

Questions for authors

In reading the paper, I have the following questions for the authors:

-- Baseline clarification: What watermark strength, layer count, and diffusion (or remasking) configuration is used for each baseline? In particular, does UMR still use its regret-based remasking? Please provide recovery–quality comparisons over the main strength parameter using matched prompts and actual output lengths, with variability over keys and messages.

-- Implementation clarification: How are Equation (9) and its gradients evaluated in the reported implementation? Please specify the hash/partition construction, the numerical stabilizer used for initially empty evidence counts, any truncation or approximation, and the resulting time and memory complexity. Is an approximation used here?

-- Recovery evidence: I wonder if exact-message recovery be reported for the existing Dream and post-edit outputs? The large LLaDA gains look fine, but these additional measurements would determine whether the weak-bit feedback advantage generalizes beyond average bit accuracy in those settings.

7 comments
AuthorsOpenReview ↗

Response to Reviewer fj9V 1/6

We thank the reviewer for their careful attention to our work. The following are the detailed answers to your questions and concerns.

Q1. Baseline clarification: What watermark strength, layer count, and diffusion (or remasking) configuration is used for each baseline? In particular, does UMR still use its regret-based remasking? Please provide recovery–quality comparisons over the main strength parameter using matched prompts and actual output lengths, with variability over keys and messages.

Q1-(1) Baseline clarification.

We reproduce the baselines using their official codebases; the context adaptations and shared sampling settings are described in Appendix B.1. For all methods, the total number of denoising steps equals the maximum generation length (300, 500, or 800 tokens). We will open-source our code upon acceptance to make these implementation details accessible.

UMR: Regret-based remasking remains enabled in all reported UMR experiments, as it is an integral component of the method. We use a base reweighting strength of δ~=4.0\tilde{\delta}=4.0 on both backbones.

PredMPAC: Following the official code, we use logit bias δ=2.0\delta=2.0, partition ratio γ=0.25\gamma=0.25, radix r=4r=4, and LeftHash with context width h=1h=1.

PredBiMark: We use 10 watermark layers and candidate truncation at k=50k=50. In most settings, we use the base reweighting strength of δ~=1.0\tilde{\delta}=1.0 adopted in the original paper. On Dream, this strength yields limited message recovery in our diffusion-model adaptation, so we additionally evaluate an increased strength of δ~=2.0\tilde{\delta}=2.0 to improve its recovery performance.

Q1-(2) Recovery–quality trade-off.

For a given method, increasing watermark embedding strength can improve bit recovery accuracy (Acc), but excessive strength may degrade the quality of generated text. We therefore assess recovery gains together with their text-quality cost and compare Acc at comparable log-PPL. In Table 1 of the main paper, BEFMark already achieves lower log-PPL and higher Acc than the compared baselines in 10 of the 18 configurations.

For the remaining eight configurations, we use log-PPL as the quality-matching criterion and adjust watermark embedding strength to target comparable text quality. In five of these configurations (Table R1), we adjust BEFMark's strength to target the lowest baseline log-PPL while keeping the other evaluation settings fixed. Adjusted BEFMark achieves lower log-PPL and higher Acc than every compared baseline in all five settings. The baseline results in Table R1 are from Table 1 of the main paper.

Table R1. Recovery after adjusting BEFMark's embedding strength. Each entry: Acc (%) ↑ / log-PPL ↓. Columns: model, message length (bits) / maximum generation length (tokens).

MethodLLaDA 8-bit / 300LLaDA 8-bit / 800LLaDA 32-bit / 800Dream 16-bit / 500Dream 32-bit / 500
PredMPAC82.87 / 2.95188.85 / 2.47174.44 / 2.45871.64 / 1.60565.29 / 1.695
PredBiMark92.45 / 2.76995.72 / 2.00382.29 / 2.01282.07 / 1.66572.65 / 1.808
UMR91.51 / 2.48294.67 / 2.05187.30 / 2.04380.56 / 1.83174.73 / 1.821
BEFMark (adjusted)95.59 / 2.38798.42 / 2.00290.12 / 1.99793.45 / 1.60487.70 / 1.561

For the three remaining Dream configurations (maximum generation length: 800 tokens), Table 1 in the main paper reports substantially lower log-PPL for PredBiMark than for the other compared methods. We therefore explore increasing PredBiMark's embedding strength. We evaluate all methods in Table R2 on the same 150 randomly sampled test prompts (50 per dataset). The subset size reflects the computational cost of generation and strength adjustment at this length. As shown in Table R2, BEFMark achieves higher Acc and lower log-PPL than the compared baselines, including adjusted PredBiMark, in all three configurations on this subset.

Table R2. Subset comparison with adjusted PredBiMark embedding strength (Dream-v0-Instruct-7B, 150 prompts, maximum generation length: 800 tokens). Each entry: Acc (%) ↑ / log-PPL ↓.

MethodDream 8-bit / 800Dream 16-bit / 800Dream 32-bit / 800
PredMPAC82.78 / 1.98572.18 / 1.61167.92 / 1.595
PredBiMark91.19 / 1.97586.49 / 1.88778.31 / 1.826
UMR94.17 / 1.69487.43 / 1.67378.86 / 1.609
BEFMark98.25 / 1.62296.21 / 1.58391.54 / 1.545
AuthorsOpenReview ↗

Response to Reviewer fj9V 2/6

Q1-(3) Actual output lengths. We evaluated actual output lengths for LLaDA-8B-Instruct with 16-bit messages and a maximum generation length of 500 tokens. The mean lengths are 396.5 tokens for BEFMark, 400.4 for PredMPAC, 394.1 for UMR, and 430 for PredBiMark. BEFMark therefore produces outputs of comparable or shorter average length than the baselines. In particular, BEFMark generates 33.5 fewer tokens on average than PredBiMark, while its Acc is 7.44 percentage points higher (95.57% vs. 88.13%, as reported in Table 1 of the main paper).

Q1-(4) Variability over keys and messages. In our reported BEFMark experiments, we independently sample a random watermark message for each test sample, so the reported metrics aggregate performance over different messages. To further assess sensitivity to key choice, we evaluate BEFMark with three watermark keys at message lengths of 8 and 16 bits. Table R3 reports mean ± sample standard deviation across these keys. The standard deviations are at most 0.20 percentage points for Acc, 0.68 percentage points for EMR, and 0.017 for log-PPL, indicating limited variation in recovery performance and text quality across the keys.

Table R3. BEFMark across three watermark keys on LLaDA-8B-Instruct (maximum generation length: 500 tokens; mean ± sample standard deviation).

BitsAcc (%) ↑EMR (%) ↑log-PPL ↓
898.01 ± 0.0992.39 ± 0.632.219 ± 0.007
1695.42 ± 0.2056.78 ± 0.682.249 ± 0.017
AuthorsOpenReview ↗

Response to Reviewer fj9V 3/6

Q2. Implementation clarification: How are Equation (9) and its gradients evaluated in the reported implementation? Please specify the hash/partition construction, the numerical stabilizer used for initially empty evidence counts, any truncation or approximation, and the resulting time and memory complexity. Is an approximation used here?

For reference, Eq. (9) is:

=\sum_{i\in\Omega_{\mathrm{full}}\setminus\Omega_t}\sum_{v\in\mathcal V}\pi_{t,i}(v) \sum\nolimits_{h_{t,i}=0}^{K-1}p_{h}(h_{t,i}\mid\pi_t)m_{t,i,b}(v,h_{t,i}).\tag{9}$$ Below, we explain (1) how Eq. (9) is evaluated and differentiated, including the approximation and cost; (2) how the hash/partition is constructed; and (3) how initially empty evidence counts are handled. **Q2-(1) Efficient computation and differentiability of Eq. (9).** **Efficient computation:** To evaluate the expected evidence margin increment in Eq. (9), we need both the token probabilities $\pi_{t,i}(v)$ and the context-hash probabilities $p_h(h\mid\pi_t)$ (Eq. (8)). The token probabilities are given by Eq. (4) in the main paper; we therefore focus here on computing the context-hash distribution. Under the factorized completion model, convolution followed by modulo-$K$ aggregation ($K=|\mathcal V|$) is equivalent to a circular convolution of the context-token distributions. By the convolution theorem, this circular convolution can be evaluated using FFTs at a per-position cost of $O(M|\mathcal V|\log|\mathcal V|)$, where $M=|\mathcal S|$, avoiding enumeration of all possible context-token combinations. Once the context-hash distribution is available, Eq. (9) combines it with the token probabilities to weight the evidence margin contributed by each token-hash pair $(v,h)$. For each pair, MCEA determines the layer-wise bit assignments and token labels, and TBEA aggregates their signed votes into the evidence margin $m_{t,i,b}(v,h)$ (see Part (2) for hash and partition construction). The pair contributes $\pi_{t,i}(v)p_h(h\mid\pi_t)m_{t,i,b}(v,h)$ to the expected evidence margin increment. However, evaluating all $|\mathcal V|K$ pairs at every position is expensive. We therefore approximate this sum by retaining the top-$k_v$ tokens and top-$k_h$ hashes according to their probabilities. We use $k_v=k_h=100$. We use their original probabilities without renormalization and sum the retained contributions over newly valid positions. This reduces evidence evaluation to at most $k_vk_h$ pairs per position, each requiring $N_{\mathrm{layer}}$ layer-wise votes. **Time complexity:** The combined cost of FFT-based hash computation and retained-pair voting over at most $L$ positions is $O\\!\left(L\left[M|\mathcal V|\log|\mathcal V|+N_{\mathrm{layer}}k_vk_h\right]\right)$. **Memory complexity:** The implementation avoids materializing the full $|\mathcal V|\times K$ token-hash pair matrix. Instead, it operates on the token and context-hash probability distributions and the retained top-$k_v$ tokens and top-$k_h$ hashes. Under the above FFT-based computation, a conservative auxiliary-memory bound for one evaluation over at most $L$ positions is $O\\!\left(L\left[M|\mathcal V|+N_{\mathrm{layer}}k_vk_h\right]\right)$, assuming voting intermediates for all watermark layers are retained simultaneously. This bound excludes the base model parameters and activations. In our default configuration, $M=1$ and $k_v=k_h=100$. To assess the resulting overhead in the complete generation pipeline, Appendix B.6 reports end-to-end runtime and peak GPU memory. On LLaDA-8B-Instruct with 16-bit messages and a maximum generation length of 300 tokens, BEFMark adds 1.86 s and 0.43 GB relative to unwatermarked generation. **Differentiability:** Substituting Eq. (8) into Eq. (9) expresses the expected evidence margin increment as a finite sum of products of token probabilities, with fixed signed votes as coefficients. It is therefore differentiable with respect to the token probabilities. For the truncated implementation, the top-$k_v$ token and top-$k_h$ hash supports are selected at each evaluation and treated as fixed during differentiation. Autograd computes gradients of this truncated surrogate through the retained token and context-hash probabilities, without differentiating through the discrete support selection. **Q2-(2) Hash/partition construction.** To compute the evidence margin $m_{t,i,b}(v,h)$ in Eq. (9), each layer needs a message-bit position and a binary label for token $v$. Here, $h$ is computed using the context-hash function $H$ defined in the main paper. Given $h$ and the watermark key, one pseudorandom mapping determines the layer-wise bit positions, while another assigns each token to a label-0 or label-1 subset for that context and layer, with nominal probability $1/2$ per label. TBEA uses these assignments to form the signed votes and aggregate them into $m_{t,i,b}(v,h)$.
Reviewer_wNKFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies multi-bit watermarking for discrete diffusion language models and proposes BEFMark. During denoising, the method extracts evidence for each message bit from partially masked sequences through context-aware multi-layer voting, estimates the expected contribution of candidate tokens to the evidence of each bit, and accordingly applies a KL-regularized update to the generation distribution that prioritizes strengthening bits with weaker evidence. Experiments on LLaDA-8B-Instruct and Dream-v0-Instruct-7B, using the ELI5, FiQA, and C4 datasets and covering 8-, 16-, and 32-bit messages with generation lengths of 300, 500, and 800 tokens, compare BEFMark with UMR as well as MPAC and BiMark adapted to diffusion models, and report higher bit recovery accuracy and exact message recovery while maintaining comparable generation quality.

Critical strengths

The paper identifies the imbalance of evidence across message bits in multi-bit watermarking and proposes a targeted adaptive weighting scheme, which is supported by ablation. On LLaDA-8B-Instruct with 16-bit messages, replacing adaptive weights with uniform weights reduces the average bit recovery accuracy from 93.51% to 83.11% at 300 tokens and from 95.57% to 84.93% at 500 tokens, a gain of about 10 percentage points, while log-PPL remains nearly unchanged (Table 5). For exact message recovery with 32-bit messages and 500 tokens, BEFMark reaches 14.33%, whereas the best baseline achieves only 1.17% (Figure 3, Table 4).

The experiments are fairly comprehensive and the figures and tables are clearly presented. The evaluation covers two diffusion models, three datasets, three message lengths, and three generation lengths, and analyzes generation quality, robustness to text edits, downstream tasks, and computational cost (Tables 1-3, Table 12), with eight ablations in the appendix. Figure 4 and Figures 8-11 visualize how the evidence for each bit evolves during denoising, which effectively supports the motivation of the method.

Critical weaknesses

The substantive distinction from UMR is unclear, and the comparison setup is inconsistent. The paper only describes an engineering-level difference from UMR, namely that no per-message matrix needs to be precomputed (17.46 GB vs. 32.29 GB of memory in Table 12), without explaining what is fundamentally new about the evidence feedback mechanism. The estimation of the expected distribution is also close to the description of Gloaguen et al. (2026), and the paper does not clarify which parts are inherited from prior work. In addition, UMR is averaged over 4 fixed keys while the other methods use a random message per sample (Appendix B.1), and the baseline hyperparameters are not reported. Moreover, without adaptive weighting, BEFMark (83.11/84.93) already falls below UMR (87.02/91.29), suggesting that its advantage comes almost entirely from the weighting component.

The validation of robustness and statistical reliability is insufficient. Robustness is evaluated only under word deletion and synonym substitution, and only bit accuracy on LLaDA with 500 tokens is reported. Stronger attacks such as paraphrasing and translation are not considered, and exact message recovery after attacks is not reported. The margin over the strongest baseline also shrinks as attacks become stronger (from 6.57 to 3.27 points for 32-bit messages). Exact message recovery itself remains low, reaching only 14.33% for 32-bit messages at 500 tokens. Furthermore, there is no analysis on non-watermarked text or under wrong keys, and none of the results include confidence intervals or significance tests.

Questions for authors
  1. Beyond not requiring a precomputed matrix, how does the evidence feedback mechanism fundamentally differ from UMR and Gloaguen et al. (2026)?
  2. Why is UMR averaged over 4 fixed keys while the other methods use a random message per sample? How were the baseline hyperparameters set?
  3. How do bit accuracy and exact message recovery hold up under stronger attacks such as LLM paraphrasing and translation?
  4. On non-watermarked text or under a wrong key, how likely is it that the decoder outputs a specific message?
  5. Could you report confidence intervals for the main results and add exact message recovery results on Dream?
8 comments
AuthorsOpenReview ↗

Response to Reviewer wNKF 1/7

We thank the reviewer for their careful attention to our work. The following are the detailed answers to your questions and concerns.

Q1. Beyond not requiring a precomputed matrix, how does the evidence feedback mechanism fundamentally differ from UMR and Gloaguen et al. (2026)?

Compared with UMR [1], UMR uses regret-based remasking, which tracks tokens that fail its local bidirectional watermark constraints. Once their contexts are sufficiently stable, these tokens are remasked for regeneration. Our BEFMark instead uses accumulated bitwise evidence to guide subsequent generation. BEFMark aggregates evidence through multi-layer voting at positions whose tokens and local contexts are fully recovered, allowing one token to contribute evidence to multiple message bits. This evidence determines the importance weights for individual message bits. These weights guide updates to the predictive distributions at the remaining masked positions, without additional remasking of generated tokens. Table 1 in the main paper shows that BEFMark achieves higher Acc than UMR across all reported settings while maintaining competitive log-PPL. Table 2 further shows higher Acc under the evaluated text-editing attacks.

Gloaguen et al. [2] formulate diffusion watermarking as maximizing the expected green-token ratio under a KL constraint for zero-bit watermark detection. Their method accounts for uncertain contexts and both a candidate token’s own watermark contribution and its influence as context for other tokens. However, this zero-bit objective does not explicitly address the recovery bottleneck imposed by weakly supported bits in multi-bit messages. BEFMark introduces a feedback objective that directly targets this bottleneck. Complete-message recovery requires every bit to be correct, so strong evidence for some bits cannot compensate for decoding errors in others. We therefore use a soft-min watermark score of normalized bitwise evidence to derive importance weights that prioritize weakly supported bits. These weights are recomputed from accumulated evidence as denoising proceeds and determine how expected evidence margin increments contribute to KL-regularized updates of the predictive distributions at the remaining masked positions. We cite this work in the main paper. Since Gloaguen et al. address zero-bit watermark detection, their method does not directly support the multi-bit Acc/EMR evaluation considered here.

[1] Ke Yang et al. You Can Have a Second Chance: Unbiased and Multi-bit Watermarking for Diffusion Language Models with Regret-based Remasking. ACL, 2026.

[2] Thibaud Gloaguen et al. Watermarking Diffusion Language Models. ICLR, 2026.

AuthorsOpenReview ↗

Response to Reviewer wNKF 2/7

Q2. Why is UMR averaged over 4 fixed keys while the other methods use a random message per sample? How were the baseline hyperparameters set?

(1) UMR: rationale for four fixed messages and results with per-sample random messages.

In our description of UMR in Appendix B.1, “four fixed keys” should be revised as “four fixed messages with a fixed secret key.” The reported UMR results were averaged over these four messages, while the same secret key was held fixed throughout.

We have now added an evaluation of UMR (random), which samples a message independently for each example. Both UMR settings use the same test prompts, secret key, and hyperparameters. Table R1 compares the Acc, EMR, and log-PPL of UMR, UMR (random), and BEFMark. BEFMark achieves higher Acc and EMR than both UMR settings across all eight evaluated configurations while maintaining competitive log-PPL. In addition, UMR (random) requires precomputing a separate cache matrix for each distinct sampled message. On LLaDA-8B-Instruct with 16-bit messages, each matrix takes 62.38 s to construct and occupies approximately 2 GB of disk space. To reuse these matrices across examples and reduce precomputation and storage costs, our original evaluation used four fixed messages.

Table R1. Message recovery and generation quality on LLaDA-8B-Instruct and Dream-v0-Instruct-7B. Columns: message length (bits) / maximum generation length (tokens). Each entry: Acc (%) ↑ / EMR (%) ↑ / log-PPL ↓.

ModelMethod / setting8-bit / 3008-bit / 50016-bit / 30016-bit / 500
LLaDAUMR91.51 / 53.83 /2.48293.60 / 60.92 / 2.23487.02 / 17.58 / 2.49291.29 / 31.00 / 2.261
LLaDAUMR (random)93.45 / 58.50 / 2.49496.00 / 70.33 / 2.24986.13 / 14.50 / 2.48990.74 / 28.17 / 2.268
LLaDABEFMark97.65 / 86.17 / 2.53498.06 / 92.67 / 2.22793.51 / 46.00 / 2.48895.57 / 57.17 / 2.240
DreamUMR90.05 / 54.25 / 2.06391.01 / 59.00 / 1.81080.29 / 3.08 / 2.09280.56 / 5.08 / 1.831
DreamUMR (random)87.89 / 41.67 / 2.07188.12 / 42.33 / 1.83181.67 / 6.33 / 2.08282.94 / 9.83 / 1.786
DreamBEFMark96.79 / 87.00 / 1.80696.23 / 88.00 / 1.58193.94 / 53.5 / 1.87494.57 / 60.17 / 1.639

(2) Baseline hyperparameters.

For UMR and PredMPAC, we follow the watermarking hyperparameters recommended in the original papers and official implementations. Both UMR settings use a base reweighting strength of δ~=4.0\tilde{\delta}=4.0, with regret-based remasking enabled. PredMPAC uses logit bias δ=2.0\delta=2.0, partition ratio γ=0.25\gamma=0.25, and radix r=4r=4. For PredBiMark, we use 10 watermark layers and candidate truncation at k=50k=50. We use the base reweighting strength of δ~=1.0\tilde{\delta}=1.0 adopted in the original paper for most settings. On Dream model, this strength yields limited message recovery in our diffusion-model adaptation, so we additionally evaluate an increased strength of δ~=2.0\tilde{\delta}=2.0 to improve its recovery performance. For each backbone, all methods use the same sampling temperature and block length, with the number of denoising steps equal to the maximum generation length. We will release our source code upon acceptance to facilitate reproducibility.

AuthorsOpenReview ↗

Response to Reviewer wNKF 3/7

Q3 & Weak2. Q3: How do bit accuracy and exact message recovery hold up under stronger attacks such as LLM paraphrasing and translation? W2: none of the results include confidence intervals.

We have added EMR results under the existing text-editing attacks (Table R2) and evaluated both Acc and EMR under DIPPER paraphrasing [3] and back-translation (English → Chinese → English) using MarkLLM [4] (Tables R3–R4). All EMR results in these tables include 95% Wilson confidence intervals.

Exact-message recovery under text-editing attacks. Table R2 reports EMR under the six text-editing attack settings in the main paper. BEFMark achieves higher EMR than the compared baselines in all 18 attack/message-length settings.

Table R2. EMR (%) with 95% Wilson confidence intervals under text-editing attacks on LLaDA-8B-Instruct (maximum generation length: 500 tokens).

BitsMethodWD0.1\mathrm{WD}_{0.1}WD0.2\mathrm{WD}_{0.2}WD0.3\mathrm{WD}_{0.3}SS0.1\mathrm{SS}_{0.1}SS0.2\mathrm{SS}_{0.2}CASS0.1\mathrm{CASS}_{0.1}
8PredMPAC41.83 [37.95, 45.82]38.17 [34.37, 42.12]26.83 [23.44, 30.52]39.33 [35.50, 43.30]27.67 [24.24, 31.38]26.33 [22.97, 30.00]
8PredBiMark65.17 [61.27, 68.87]62.00 [58.05, 65.80]51.67 [47.67, 55.64]65.83 [61.95, 69.52]52.83 [48.83, 56.80]51.50 [47.50, 55.48]
8UMR66.42 [64.50, 68.28]60.92 [58.95, 62.85]47.25 [45.26, 49.25]63.42 [61.47, 65.32]45.67 [43.68, 47.66]47.00 [45.01, 49.00]
8UMR (random)65.50 [61.61, 69.19]59.83 [55.86, 63.68]45.00 [41.06, 49.00]62.50 [58.56, 66.28]49.67 [45.68, 53.66]48.33 [44.36, 52.33]
8BEFMark81.33 [78.02, 84.25]74.17 [70.52, 77.51]57.83 [53.84, 61.72]76.50 [72.95, 79.72]59.33 [55.36, 63.19]60.83 [56.87, 64.66]
16PredMPAC2.33 [ 1.39,  3.88]1.67 [ 0.91,  3.04]1.00 [ 0.46,  2.16]2.33 [ 1.39,  3.88]1.00 [ 0.46,  2.16]0.50 [ 0.17,  1.46]
16PredBiMark14.33 [11.76, 17.36]12.00 [ 9.64, 14.84]5.33 [ 3.80,  7.43]12.83 [10.39, 15.75]5.83 [ 4.22,  8.00]4.67 [ 3.25,  6.66]
16UMR22.50 [20.87, 24.21]16.25 [14.83, 17.78]8.33 [ 7.29,  9.51]19.25 [17.72, 20.88]9.00 [ 7.92, 10.21]7.33 [ 6.36,  8.45]
16UMR (random)20.00 [16.99, 23.39]15.17 [12.52, 18.26]7.83 [ 5.94, 10.26]18.00 [15.13, 21.27]5.83 [ 4.22,  8.00]7.17 [ 5.36,  9.51]
16BEFMark37.83 [34.04, 41.78]29.67 [26.15, 33.44]14.17 [11.60, 17.19]29.33 [25.83, 33.10]14.33 [11.76, 17.36]17.17 [14.36, 20.39]
32PredMPAC0.17 [ 0.03,  0.94]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.17 [ 0.03,  0.94]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]
32PredBiMark0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]
32UMR0.58 [ 0.35,  0.98]0.42 [ 0.23,  0.77]0.42 [ 0.23,  0.77]0.42 [ 0.23,  0.77]0.17 [ 0.06,  0.43]0.17 [ 0.06,  0.43]
32UMR (random)0.67 [ 0.26,  1.70]0.17 [ 0.03,  0.94]0.00 [ 0.00,  0.64]0.17 [ 0.03,  0.94]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]
32BEFMark5.17 [ 3.66,  7.24]1.83 [ 1.03,  3.25]1.67 [ 0.91,  3.04]1.33 [ 0.68,  2.61]0.50 [ 0.17,  1.46]0.50 [ 0.17,  1.46]

[3] Kalpesh Krishna et al. Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense. NeurIPS, 2023.

[4] Leyi Pan et al. MarkLLM: An Open-Source Toolkit for LLM Watermarking. EMNLP System Demonstrations, 2024.

Reviewer_vKkfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies multi-bit watermarking for discrete diffusion language models, where masked contexts and non-sequential generation make watermark evidence difficult to track. BEFMark collects keyed, multi-layer votes from available token-context pairs, uses a soft-min score to prioritize poorly supported bits, estimates expected evidence from still-masked positions, and updates token distributions with KL regularization. Results on LLaDA-8B-Instruct and Dream-v0-Instruct-7B show higher average bit recovery accuracy than UMR and two adapted autoregressive baselines.

Critical strengths
  1. The problem is well motivated. For a multi-bit watermark, the weakest bit matters much more than the average evidence, and the proposed feedback mechanism addresses this point directly. The method is explained clearly enough to follow the main design choices.
  2. The empirical gains are substantial across payload lengths and both diffusion models (Table 1). I also found the adaptive-versus-uniform weighting ablation convincing: for 16-bit messages on LLaDA, adaptive weighting improves bit accuracy by 10.40 and 10.64 percentage points at 300 and 500 tokens (Table 5). The runtime comparison in Table 12 suggests that the additional generation cost is not prohibitive in the reported setting.
Critical weaknesses

1.The optimization surrogate is not sufficiently validated. The stated objective in Eq. (5) concerns the score after a possible completion, but Eq. (4) factors the completion distribution across positions, and Proposition 3.1 further drops vote-count changes after a first-order expansion. In fact, the latter is not obviously candidate-independent under the default configuration. With 12 voting layers and 16- or 32-bit messages, the set of bits receiving votes at a position depends on the context hash, which in turn depends on the sampled tokens. Thus expected per-bit vote counts can change with the candidate distributions. The balanced count trajectories discussed in Appendix A.3 do not by themselves justify omitting this dependence. Appendix A.4 only analyzes convergence for the chosen surrogate, not how faithfully it represents the original objective. Authors should analyze the optimization surrogate based on targeted approximation-error or count-aware ablation. 2.The message-level reliability claim is not yet fully supported. Table 4 gives exact-message recovery of 57.17% for 16 bits and 14.33% for 32 bits at 500 tokens on LLaDA. Obviously, many complete messages are still lost. Exact-message recovery is not reported for Dream, and Table 2 only provides bit accuracy after text edits, not the fraction of messages recovered intact. Since provenance is part of the motivation, authors should show how recovery behaves on unwatermarked text, for which majority voting still returns a bit string. 3.The comparison protocol leaves some uncertainty. Appendix B.1 evaluates UMR using four pre-sampled keys per message length, while the other methods use independently sampled messages across examples. Authors should add key-to-key variation and uncertainty estimates to help readers understand the submission. Also, the baselines are compared at their reported operating points rather than with watermark strengths calibrated to the same quality or distortion budget. Table 6 shows a fairly pronounced accuracy/quality trade-off as the KL scale changes; Fig. 4 varies generation length, not embedding strength. I think a matched-quality comparison would make the attribution of the gains to bitwise feedback more convincing.

Questions for authors

1.How sensitive are the updates to the factorized completion model and the frozen-count approximation in Eq. (6)? Could a small controlled test compare the surrogate to a count-aware or more exact estimate? 2.Could authors report exact-message recovery on Dream and after the WD/SS/CASS edits in Table 2, ideally with variation across keys or confidence intervals? 3.Can authors compare the methods at matched log-PPL or KL distortion, and clarify how the full-vocabulary expectations in Eqs. (8)–(9) are computed efficiently in practice?

7 comments
AuthorsOpenReview ↗

Response to Reviewer vKkf 1/6

We thank the reviewer for their careful attention to our work. The following are the detailed answers to your questions and concerns.

Q1. How sensitive are the updates to the factorized completion model and the frozen-count approximation in Eq. (6)? Could a small controlled test compare the surrogate to a count-aware or more exact estimate?

We first clarify the role and scope of the factorized completion model, then present a controlled comparison with count correction to assess the practical effect of the frozen-count approximation.

(1) Factorized completion model. The current dLLM predictions provide a distribution at each masked position, but do not specify the dependencies among these positions. Consequently, the effect of ignoring these dependencies cannot be quantified from the current-step predictions alone: the same per-position distributions can correspond to different joint completion distributions and different expected evidence increments. Evaluating this effect would require an additional joint completion model that represents these dependencies. In our method, Eq. (4) specifies a factorized completion model for defining the current-step expected score increment in Eq. (5). After each denoising step, we recompute the completion model, evidence margins, and vote counts using the updated predictions and recovered tokens.

(2) Sensitivity to the frozen-count approximation. Direct evaluation of Eq. (5) involves summing the nonlinear score increment over all possible completions of the masked positions. Since the number of completions grows exponentially with the number of masked positions, this exhaustive computation is intractable. We therefore apply a first-order Taylor expansion of RR around the current evidence (mt,nt)(\mathbf m_t,\mathbf n_t):

\frac{w_{t,b}}{\sqrt{n_{t,b}+\epsilon}} \mathbb E_{\pi_t}[\Delta m_{t,b}] -\sum_{b=1}^{B}\frac{w_{t,b}m_{t,b}}{2(n_{t,b}+\epsilon)^{3/2}} \mathbb E_{\pi_t}[\Delta n_{t,b}].$$ The first term accounts for evidence margin increments, while the second accounts for changes in vote-count normalization. Under the frozen-count approximation, we treat the expected vote-count increments as fixed with respect to the candidate distributions within each update. The second term is therefore omitted from the optimization objective, yielding the margin-only surrogate in Eq. (6). To assess the practical effect of this additional approximation, we compare BEFMark with BEFMark + count correction. The corrected variant adds a correction corresponding to the second term while retaining the original margin weighting and factorized completion model. As shown in Table R1, count correction changes Acc by at most 0.69 percentage points. Its effect on complete-message recovery is mixed: EMR improves in two configurations, remains unchanged in one, and decreases in five, with changes ranging from −4.5 to +1.33 percentage points. Log-PPL increases in seven configurations and remains unchanged at the reported precision in one. Thus, adding the count correction provides no consistent improvement in the recovery–quality trade-off. On LLaDA-8B-Instruct with 16-bit messages and a maximum generation length of 500 tokens, unwatermarked generation takes 40.27 s; BEFMark and BEFMark + count correction add 3.23 s and 3.58 s, respectively, relative to this baseline. These results support retaining the simpler margin-only feedback in the evaluated settings. **Table R1. Ablation of the frozen-count approximation on LLaDA and Dream (Acc (%) ↑ / EMR (%) ↑ / log-PPL ↓).** |Model|Feedback variant|8-bit / 300|8-bit / 500|16-bit / 300|16-bit / 500| |:---|:---|---:|---:|---:|---:| |LLaDA|BEFMark|97.65 / 86.17 / 2.534|98.06 / 92.67 / 2.227|93.51 / 46.00 / 2.488|95.57 / 57.17 / 2.240| |LLaDA|BEFMark + count correction|97.50 / 82.67 / 2.566|98.21 / 88.17 / 2.308|93.76 / 46.00 / 2.490|95.80 / 58.50 / 2.252| |Dream|BEFMark|96.79 / 87.00 / 1.806|96.23 / 88.00 / 1.581|93.94 / 53.50 / 1.874|94.57 / 60.17 / 1.639| |Dream|BEFMark + count correction|96.10 / 83.33 / 1.839|96.64 / 86.00 / 1.581|93.72 / 54.50 / 1.957|94.20 / 59.00 / 1.644|
AuthorsOpenReview ↗

Response to Reviewer vKkf 2/6

Q2. Could authors report exact-message recovery on Dream and after the WD/SS/CASS edits in Table 2, ideally with variation across keys or confidence intervals?

We add exact-message recovery (EMR) results for Dream without attacks and for both LLaDA and Dream under the six text-editing attack settings in Table 2 in the main paper. We report EMR (%) with 95% Wilson confidence intervals for a fixed watermark key, covering 8-, 16-, and 32-bit messages as organized below:

  • Table R2: Results without attacks on LLaDA-8B-Instruct and Dream-v0-Instruct-7B, with maximum generation lengths of 300, 500, and 800 tokens.
  • Table R3: Dream-v0-Instruct-7B under all six text-editing attack settings, with a maximum generation length of 500 tokens.
  • Table R4: LLaDA-8B-Instruct under the same six attack settings with a maximum generation length of 500 tokens.

In addition, we evaluated BEFMark across three different keys on LLaDA-8B-Instruct without attacks (maximum generation length: 500 tokens). EMR was (92.39 ± 0.63)% and (56.78 ± 0.68)% for 8- and 16-bit messages, respectively (mean ± sample SD across keys). These results indicate limited variation in exact-message recovery across the different keys.

As shown in Table R2, BEFMark achieves higher EMR than the compared baselines on both models. For example, on Dream with 16-bit messages and a maximum generation length of 800 tokens, BEFMark achieves 62.17% EMR, compared with 16.25% for UMR.

Table R2. EMR (%) with 95% Wilson confidence intervals without attacks on LLaDA-8B-Instruct and Dream-v0-Instruct-7B (maximum generation lengths: 300, 500, and 800 tokens).

BitsMethodLLaDA 300LLaDA 500LLaDA 800Dream 300Dream 500Dream 800
8PredMPAC29.67 [26.15, 33.44]43.67 [39.75, 47.66]48.83 [44.85, 52.83]24.33 [21.07, 27.92]24.50 [21.23, 28.10]25.17 [21.86, 28.79]
8PredBiMark56.17 [52.17, 60.09]68.83 [65.02, 72.41]72.50 [68.79, 75.92]41.33 [37.46, 45.32]47.17 [43.20, 51.17]48.00 [44.03, 52.00]
8UMR53.83 [51.83, 55.82]60.92 [58.95, 62.85]65.17 [63.24, 67.05]54.25 [52.25, 56.23]59.00 [57.02, 60.95]60.17 [58.19, 62.11]
8BEFMark86.17 [83.17, 88.70]92.67 [90.30, 94.49]94.83 [92.76, 96.34]87.00 [84.07, 89.46]88.00 [85.16, 90.36]88.17 [85.34, 90.51]
16PredMPAC2.67 [ 1.65,  4.29]5.17 [ 3.66,  7.24]9.00 [ 6.96, 11.56]1.67 [ 0.91,  3.04]2.67 [ 1.65,  4.29]2.67 [ 1.65,  4.29]
16PredBiMark10.00 [ 7.85, 12.66]16.67 [13.90, 19.86]26.33 [22.97, 30.00]5.50 [ 3.94,  7.62]6.50 [ 4.79,  8.76]7.83 [ 5.94, 10.26]
16UMR17.58 [16.11, 19.16]31.00 [29.18, 32.88]41.83 [39.87, 43.82]3.08 [ 2.46,  3.85]5.08 [ 4.27,  6.04]16.25 [14.83, 17.78]
16BEFMark46.00 [42.05, 50.00]57.17 [53.17, 61.07]63.33 [59.40, 67.09]53.50 [49.50, 57.46]60.17 [56.20, 64.01]62.17 [58.22, 65.96]
32PredMPAC0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]
32PredBiMark0.00 [ 0.00,  0.64]0.17 [ 0.03,  0.94]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]
32UMR0.25 [ 0.11,  0.54]1.17 [ 0.81,  1.68]3.75 [ 3.06,  4.59]0.08 [ 0.02,  0.30]0.08 [ 0.02,  0.30]0.17 [ 0.06,  0.43]
32BEFMark7.33 [ 5.51,  9.70]14.33 [11.76, 17.36]20.67 [17.62, 24.09]10.83 [ 8.59, 13.57]15.17 [12.52, 18.26]16.33 [13.59, 19.50]
AuthorsOpenReview ↗

Response to Reviewer vKkf 3/6

As shown in Tables R3-R4, BEFMark maintains an EMR advantage over the compared baselines under the evaluated text-editing attacks on both models. For example, on Dream with 16-bit messages and a maximum generation length of 500 tokens, it achieves 27.17% EMR under 20% word deletion, compared with 6.08% for UMR. Absolute recovery remains limited for 32-bit messages after editing.

Table R3. EMR (%) with 95% Wilson confidence intervals under six text-editing attack settings on Dream-v0-Instruct-7B (maximum generation length: 500 tokens).

BitsMethodWD$_{0.1}$WD0.2_{0.2}WD0.3_{0.3}SS0.1_{0.1}SS0.2_{0.2}CASS_{0.1}\
8PredMPAC22.83 [19.65, 26.36]20.50 [17.46, 23.91]14.50 [11.91, 17.54]22.50 [19.34, 26.01]17.33 [14.51, 20.57]22.67 [19.50, 26.18]
8PredBiMark42.83 [38.93, 46.83]37.33 [33.56, 41.27]24.67 [21.39, 28.27]37.50 [33.72, 41.44]25.67 [22.33, 29.31]25.17 [21.86, 28.79]
8UMR47.42 [45.42, 49.42]42.42 [40.45, 44.40]30.25 [28.45, 32.12]43.00 [41.03, 44.99]34.33 [32.46, 36.26]31.42 [29.59, 33.30]
8BEFMark71.83 [68.10, 75.28]63.17 [59.23, 66.93]45.33 [41.39, 49.33]61.50 [57.55, 65.31]46.33 [42.38, 50.33]52.00 [48.00, 55.97]
16PredMPAC1.67 [ 0.91,  3.04]0.67 [ 0.26,  1.70]0.50 [ 0.17,  1.46]1.67 [ 0.91,  3.04]0.83 [ 0.36,  1.94]0.50 [ 0.17,  1.46]
16PredBiMark6.17 [ 4.51,  8.38]3.67 [ 2.43,  5.49]2.00 [ 1.15,  3.46]3.67 [ 2.43,  5.49]1.67 [ 0.91,  3.04]1.33 [ 0.68,  2.61]
16UMR9.33 [ 8.23, 10.56]6.08 [ 5.20,  7.11]2.67 [ 2.09,  3.39]7.00 [ 6.05,  8.09]2.50 [ 1.95,  3.20]2.25 [ 1.73,  2.92]
16BEFMark35.83 [32.10, 39.75]27.17 [23.76, 30.86]11.17 [ 8.89, 13.94]25.33 [22.02, 28.96]14.33 [11.76, 17.36]14.17 [11.60, 17.19]
32PredMPAC0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]
32PredBiMark0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]
32UMR0.08 [ 0.02,  0.30]0.08 [ 0.02,  0.30]0.00 [ 0.00,  0.16]0.00 [ 0.00,  0.16]0.00 [ 0.00,  0.16]0.00 [ 0.00,  0.16]
32BEFMark4.17 [ 2.84,  6.08]1.67 [ 0.91,  3.04]1.33 [ 0.68,  2.61]1.83 [ 1.03,  3.25]1.33 [ 0.68,  2.61]0.67 [ 0.26,  1.70]

Table R4. EMR (%) with 95% Wilson confidence intervals under six text-editing attack settings on LLaDA-8B-Instruct (maximum generation length: 500 tokens).

BitsMethodWD$_{0.1}$WD0.2_{0.2}WD0.3_{0.3}SS0.1_{0.1}SS0.2_{0.2}CASS_{0.1}\
8PredMPAC41.83 [37.95, 45.82]38.17 [34.37, 42.12]26.83 [23.44, 30.52]39.33 [35.50, 43.30]27.67 [24.24, 31.38]26.33 [22.97, 30.00]
8PredBiMark65.17 [61.27, 68.87]62.00 [58.05, 65.80]51.67 [47.67, 55.64]65.83 [61.95, 69.52]52.83 [48.83, 56.80]51.50 [47.50, 55.48]
8UMR66.42 [64.50, 68.28]60.92 [58.95, 62.85]47.25 [45.26, 49.25]63.42 [61.47, 65.32]45.67 [43.68, 47.66]47.00 [45.01, 49.00]
8BEFMark81.33 [78.02, 84.25]74.17 [70.52, 77.51]57.83 [53.84, 61.72]76.50 [72.95, 79.72]59.33 [55.36, 63.19]60.83 [56.87, 64.66]
16PredMPAC2.33 [ 1.39,  3.88]1.67 [ 0.91,  3.04]1.00 [ 0.46,  2.16]2.33 [ 1.39,  3.88]1.00 [ 0.46,  2.16]0.50 [ 0.17,  1.46]
16PredBiMark14.33 [11.76, 17.36]12.00 [ 9.64, 14.84]5.33 [ 3.80,  7.43]12.83 [10.39, 15.75]5.83 [ 4.22,  8.00]4.67 [ 3.25,  6.66]
16UMR22.50 [20.87, 24.21]16.25 [14.83, 17.78]8.33 [ 7.29,  9.51]19.25 [17.72, 20.88]9.00 [ 7.92, 10.21]7.33 [ 6.36,  8.45]
16BEFMark37.83 [34.04, 41.78]29.67 [26.15, 33.44]14.17 [11.60, 17.19]29.33 [25.83, 33.10]14.33 [11.76, 17.36]17.17 [14.36, 20.39]
32PredMPAC0.17 [ 0.03,  0.94]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.17 [ 0.03,  0.94]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]
32PredBiMark0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]0.00 [ 0.00,  0.64]
32UMR0.58 [ 0.35,  0.98]0.42 [ 0.23,  0.77]0.42 [ 0.23,  0.77]0.42 [ 0.23,  0.77]0.17 [ 0.06,  0.43]0.17 [ 0.06,  0.43]
32BEFMark5.17 [ 3.66,  7.24]1.83 [ 1.03,  3.25]1.67 [ 0.91,  3.04]1.33 [ 0.68,  2.61]0.50 [ 0.17,  1.46]0.50 [ 0.17,  1.46]

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the rebuttal

We thank the Area Chair for coordinating the review process and all three reviewers for their thoughtful feedback. Our BEFMark embeds multi-bit watermarks in discrete diffusion language models by prioritizing weakly supported message bits during denoising. On both LLaDA and Dream, BEFMark recovered complete messages more often than the baselines after text editing and achieved higher bit accuracy at comparable log-perplexity. In response to the reviewers’ feedback, we broadened our evaluation of BEFMark, and the results provide further empirical support for our conclusions. Below, we summarize our clarifications and the main findings from our additional experiments.

1. BEFMark’s bitwise evidence feedback mechanism. Reviewer wNKF asked how BEFMark differs from Gloaguen et al. and UMR. Gloaguen et al. optimize the expected green-token ratio for zero-bit detection. UMR uses regret-based remasking to regenerate tokens that violate local watermark constraints. Our BEFMark framework uses votes from tokens already generated to track how much evidence has accumulated for each bit of the target message. It uses this feedback to adapt watermark embedding as generation proceeds, favoring tokens that reinforce weakly supported bits when filling the remaining masks.

At Reviewer vKkf’s suggestion, we examined how BEFMark handles vote-count normalization in Eq. (6), the tractable surrogate for the expected gain in watermark score in Eq. (5). Under the frozen-count approximation, we treat expected vote-count increments as fixed within each update and retain only the term for gains in vote margin. To test this choice, we added a count correction while keeping the same factorized completion model and margin weighting. The correction yielded no consistent gain in bit accuracy and reduced exact-message recovery in five of eight settings. It also increased generation time by 0.35 seconds on LLaDA with 16-bit messages and a 500-token limit. These results support our choice to guide embedding with the simpler margin-only update in Eq. (6).

2. Watermark decoding and robustness to text editing. At the request of all three reviewers, we expanded our text-editing experiments on LLaDA and Dream. In addition to the six word-deletion and synonym-substitution settings in the main paper, we tested two stronger LLM-based attacks, DIPPER paraphrasing and back-translation. Across all eight editing settings, BEFMark achieved higher bit accuracy and exact-message recovery (EMR) than the baselines on both models. For example, after back-translation on LLaDA, 8-bit EMR was 42.67% for BEFMark and 25.50% for UMR (random), the strongest baseline in this setting. We also report 95% Wilson confidence intervals for all EMR estimates.

3. Baseline settings and an expanded UMR comparison. At the request of Reviewers wNKF and fj9V, we provided more detail on baseline settings, including watermark strength, layer count, and sampling configuration. We used each baseline’s official codebase and kept regret-based remasking enabled for UMR. We also added an experiment with UMR (random), in which we sampled a watermark message independently for each example. On both LLaDA and Dream, BEFMark achieved higher bit accuracy and EMR than all evaluated baselines for 8- and 16-bit messages.

4. Recovery–quality trade-off. We examined the recovery–quality trade-off at the request of Reviewers vKkf and fj9V. Using the baselines' recommended configurations as a starting point, we found that BEFMark achieved higher bit accuracy than every baseline in all 18 settings in Table 1 of the main paper and lower log-perplexity in 10. In the remaining eight settings, BEFMark already led the most accurate baseline by 3.47–15.12 percentage points at the original embedding strengths, while its log-perplexity was 0.019–0.240 above the lowest baseline value. We therefore ran additional experiments to compare recovery at similar levels of text quality. BEFMark maintained higher bit accuracy than every baseline and also achieved lower log-perplexity in all eight settings.

5. Unwatermarked text and wrong-key decoding. At the suggestion of Reviewers vKkf and wNKF, we examined how BEFMark behaves on unwatermarked text. We introduced an auxiliary hypothesis test for watermark presence based on the imbalance between votes for 0 and 1 at each message bit. On LLaDA, the test distinguished watermarked from unwatermarked texts with an AUROC of approximately 0.986 for both 8- and 16-bit messages. This test extends the scope of BEFMark from recovering multi-bit messages to detecting whether a text carries its watermark. At Reviewer wNKF’s request, we also evaluated wrong-key decoding on LLaDA. With random wrong keys, only 0.017% of decoding attempts recovered the embedded 16-bit message.

We hope this summary and our detailed responses are helpful in your final evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.