acceptodds
Under review as a conference paper at ICLR 2027

Robust and Scalable Multi-bit LLM Watermarking via Coded Payload Spreading

Abstract

Multi-bit watermarking for large language models enables content source tracing by embedding user-identifiable messages into generated text. Existing methods face a fundamental trade-off among extraction accuracy, text quality, and payload capacity. We propose WeaveMark, a robust and scalable multi-bit LLM watermarking scheme based on coded payload spreading. WeaveMark shifts this trade-off frontier by improving payload capacity through multi-bit-per-token spreading (weaving), improving extraction accuracy through soft-decision error-correcting codes, and preserving text quality through unbiased multilayer reweighting. It further introduces dedicated zero-bit layers for reliable watermark presence detection. Extensive experiments demonstrate substantial gains in extraction performance, especially for long messages and edited text, without degrading text quality. WeaveMark achieves an 89.8% match rate for 32-bit messages at 200 tokens, compared with 20.8% for BiMark. Under 10% substitution attacks on 16-bit messages at 200 tokens, it maintains 86.0% versus 30.7%.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM watermarking, multi-bit watermarking, error-correcting codes, unbiased watermark, robustness

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_qNYGOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work aims to improve the exact message decoding rate while preserving text quality in multi-bit text watermarking for LLMs. The proposed method, WeaveMark, builds on a previous method, BiMark, and introduces three additional components: payload spreading, layer shuffling, and soft-decision ECC decoding. Ablation studies show that these components substantially improve performance over the BiMark baseline. Experimental results further demonstrate that WeaveMark generally outperforms several existing multi-bit watermarking methods across different evaluation settings.

Critical strengths
  1. This work addresses a timely and important topic in multi-bit text watermarking for LLMs.

  2. The proposed WeaveMark improves upon the previous BiMark method through more balanced spreading of encoded bits, layer shuffling to distribute observations more evenly across bit positions, and soft-decision ECC decoding to correct bit errors, where the layer shuffling is well motivated. Experimental results show that WeaveMark substantially improves the exact message recovery rate while maintaining comparable text quality.

Critical weaknesses
  1. One key contribution claimed in the manuscript is payload spreading, where each token carries information from multiple payload bits. The authors state around Line 200 that each bit is assigned to approximately κT/n\kappa T/n token positions instead of T/nT/n in BiMark. However, because the fixed layer budget of each token is divided among κ\kappa selected bits, the expected total number of layer-level observations received by each bit remains approximately unchanged. According to Table 2, multi-bit embedding alone improves the match rate only modestly over BiMark (less than 4%), whereas much larger gains are obtained after adding layer shuffling and especially soft ECC. In particular, BiMark equipped with the same soft ECC already achieves performance close to WeaveMark. A stronger comparison that equips BiMark with the same layer shuffling and soft ECC decoding would be needed to clearly establish the contribution of the proposed method.

  2. Soft ECC decoding appears to be the most important component contributing to the improvement in match rate, while similar ECC-based decoding has been used in prior methods. The novelty and specific contribution of this component therefore need to be clarified more clearly.

Questions for authors
  1. How does the performance of BiMark with layer shuffling and soft ECC compare to the proposed WeaveMark in the main experimental settings (e.g., in Figure 5 and Figure 6)?

  2. The expected total number of layer-level observations per bit appears to remain approximately unchanged after payload spreading. Moreover, ECC expands the payload into a longer codeword, which further reduces the average number of layer-level observations available to each encoded bit under a fixed token and layer budget. Could the authors clarify why distributing the same observation budget across more token positions improves decoding reliability?

5 comments
AuthorsOpenReview ↗

BiMark with shuffling and soft ECC

We thank you for your careful review and constructive feedback. We conducted additional experiments and analyses to address your concerns regarding (1) the contribution of payload spreading under the same ECC, (2) why performance improves under a fixed observation budget, and (3) the novelty of soft ECC.

Unless otherwise stated, the experimental settings and labels A–I follow the General Response.


Reviewer Comment

How does the performance of BiMark with layer shuffling and soft ECC compare to the proposed WeaveMark in the main experimental settings (e.g., in Figure 5 and Figure 6)?

Response

We conducted controlled experiments to examine whether the additional gains from payload spreading persist when the same ECC and soft decoder are used.

In BiMark (κ=1\kappa=1), all layers are assigned to the same bit, so layer shuffling, which permutes the bit-to-layer assignments, is an identity operation. Therefore, the requested BiMark + shuffling + soft ECC is equivalent to BiMark + soft ECC.

We compared BiMark + soft ECC and WeaveMark using the same ECC and soft decoder under the main conditions in Fig. 5 (clean) and Fig. 6 (substitution). (16-bit: RM(32,16); 24-bit: 2×Golay(24,12); 32-bit: 2×RM(32,16))

Method16-5016-10016-20024-5024-10024-20032-5032-10032-200
BiMark + Soft ECC49.688.795.29.459.091.40.627.482.9
WeaveMark76.493.397.139.981.494.316.362.290.5
  • Column headings: message bits – generated tokens. For example, 16-50 denotes a 16-bit message and 50 generated tokens.
MethodSub. 10% - 16Sub. 10% - 24Sub. 20% - 16Sub. 20% - 24Ins. 10% - 32Del. 10% - 32
BiMark + Soft ECC76.537.528.23.746.737.6
WeaveMark86.561.344.511.971.467.2
  • Column headings: attack type and ratio – message bits. Sub.: substitution, Ins.: insertion, Del.: deletion.
  • Evaluated at 200 generated tokens.

Even with the same ECC, WeaveMark achieves higher recovery rates than BiMark under all compared conditions (Tables 21–22).

Multi-bit embedding without shuffling

To distinguish the contributions of multi-bit embedding and shuffling, we compared C, F, and I using the same soft ECC.

LabelMulti-bitShufflingECCClean (100)Clean (200)Subst. 10%Ins. 10%Del. 10%
C (BiMark+Soft ECC)✗✗Soft27.482.910.846.737.6
F✓✗Soft33.772.114.939.637.2
I (WeaveMark)✓✓Soft62.290.533.671.467.2

F has a lower match rate than C under some conditions. This is because assigning layers to multiple bits without shuffling can repeatedly assign less reliable layers to certain bit positions (Fig. 3). In contrast, I performs best under all conditions. Thus, multi-bit embedding and layer shuffling form an effective design when used together, which also explains the modest gain from multi-bit embedding alone in the original Table 2.

We will add the following, more detailed ablation table covering all components of WeaveMark, together with a discussion of the effects of the individual components, to a new Appendix H of the revised manuscript (Table 20).

LabelMulti-bitShufflingECCCleanSubst. 10%Ins. 10%Del. 10%
A (BiMark)✗✗None20.60.54.12.3
B✗✗Hard63.44.119.916.1
C✗✗Soft82.910.846.737.6
D✓✗None24.52.17.46.9
E✓✗Hard50.15.819.019.0
F✓✗Soft72.114.939.637.2
G✓✓None52.64.920.317.7
H✓✓Hard79.413.045.142.1
I (WeaveMark)✓✓Soft90.533.671.467.2
AuthorsOpenReview ↗

Why Multi-Bit Embedding Helps Under a Fixed Observation Budget

Reviewer Comment

The expected total number of layer-level observations per bit appears to remain approximately unchanged after payload spreading. Moreover, ECC expands the payload into a longer codeword, which further reduces the average number of layer-level observations available to each encoded bit under a fixed token and layer budget. Could the authors clarify why distributing the same observation budget across more token positions improves decoding reliability?

Response

As the reviewer points out, the average number of layer-level observations per bit does not increase. For the effective token count TeffT_{\mathrm{eff}}, codeword length nn, total number of layers ℓ\ell, and number of selected bits per token κ\kappa, the expected observation counts under uniform allocation are as follows (when ℓ\ell is a multiple of κ\kappa).

=\frac{\kappa T_{\mathrm{eff}}}{n}, \qquad \mathbb{E}[\text{layer votes/bit}] =\frac{\ell T_{\mathrm{eff}}}{n}.$$ That is, rather than increasing the total observation budget, we **distribute the observations for each bit across more token positions**. Because entropy and watermark signal strength vary across tokens, this spreading can alleviate the concentration of weak observations on particular bits. From the perspective of ECC, this produces an effect similar to interleaving an imbalanced channel, improving the probability of successful ECC decoding ([Figure 2](https://anonymous.4open.science/r/WeaveMark_Rebuttal_Figures-0FAB/Figure2.pdf), [Figure 3](https://anonymous.4open.science/r/WeaveMark_Rebuttal_Figures-0FAB/Figure3.pdf)). To further support this explanation, we defined the evidence $e_p$ for each codeword bit as (correct votes − incorrect votes) and measured its distribution as a function of $\kappa$. |Metric|$\kappa=1$|$\kappa=2$|$\kappa=5$|$\kappa=10$| |---|---:|---:|---:|---:| |Mean $e_p$|7.78|7.86|7.97|7.95| |Std. $e_p$|7.09|6.26|5.45|5.11| |Weak bits (%)|19.6|11.7|10.9|8.3| |Unobserved bits|4.14|0.40|0|0| |Match rate (%)|84.0|88.7|92.5|91.1| - Weak bits: the proportion of bits with $e_p \le 0$. - Unobserved bits: the number of bits that are never observed in a text. - 200 generated tokens, the same key/message, and a shared set of 213 prompts. As $\kappa$ increases from $1\to10$, the mean evidence remains nearly unchanged, while the standard deviation decreases from 7.09 → 5.11, reducing the variance by approximately half. Weak bits decrease from 19.6% → 8.3%, and unobserved bits decrease from more than 4 per text → 0. In particular, expanding the codeword through ECC can increase the number of unobserved or unreliable bits. Payload spreading reduces this imbalance, providing more reliable decoding inputs even with the same observation budget and ECC. We will add this analysis to Appendix H of the revised manuscript ([Table 23](https://anonymous.4open.science/r/WeaveMark_Rebuttal_Figures-0FAB/Table23.pdf)).
AuthorsOpenReview ↗

Novelty of soft-decision ECC

Reviewer Comment

Soft ECC decoding appears to be the most important component contributing to the improvement in match rate, while similar ECC-based decoding has been used in prior methods. The novelty and specific contribution of this component therefore need to be clarified more clearly.

Response

ECC, soft-decision decoding, and splitting messages into short code blocks are existing techniques, and we do not claim novelty for these techniques themselves. The core contribution of WeaveMark is, under fixed token and layer budgets, an embedding structure that balances the reliability of observations across the ECC codeword, optimizing the input channel to maximize ECC's error-correction capability.

  • (i) Multi-bit embedding: distributes the same layer observation budget across more token positions, reducing unobserved and weak bits.
  • (ii) Layer shuffling: reduces the tendency for particular bit positions to receive disproportionately high- or low-reliability layers.
  • (iii) Integration with soft ECC: uses the reliability information in the evenly accumulated vote margins to improve error correction.

Components (i) and (ii) alleviate the scarcity and imbalance of observations caused by ECC codeword expansion, while (iii) uses these improved decoding inputs.

Under the conditions of the component ablation in Table 2 of the main paper, ECC accounts for most of the improvement in match rate. However, the additional experiments reveal a substantial performance gap between soft ECC alone and WeaveMark on short texts and under editing attacks.

  • Components beyond ECC also contribute substantial performance gains: Even with the same soft ECC, a large gap remains between BiMark with ECC alone (scheme C) and WeaveMark (I). On clean text with 24-bit messages and 50 tokens, the match rate improves from 9.4% → 39.9%, a gain of 30.5%p. Under 10% deletion with 32-bit messages and 200 tokens, it improves from 37.6% → 67.2%, a gain of 29.6%p.
  • ECC is also more effective when combined with the other components: Soft decoding has a greater effect when used together with payload spreading and shuffling. Under 10% substitution, switching BiMark from hard to soft ECC reduces the decoding failure rate by 7.1% (B→C), whereas the reduction is 23.7% when (i) and (ii) are applied (H→I).

Thus, WeaveMark's main novelty lies primarily in the synergistic design of its constituent techniques, rather than in each technique individually, allowing them to compensate for one another's limitations and reinforce their respective strengths. In the revised discussion of ECC, we will explicitly cover related work, including RS coding in RSBH, BCH coding in BREW, and soft-information decoding in the concurrent work CORE-BREW, and clarify this design contribution and its experimental support.

Reviewer_GYEFOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes WEAVEMARK, a multi-bit LLM watermarking scheme based on coded payload spreading. WEAVEMARK improves payload capacity by spreading multiple bits per token across watermark-embedding operations, improves extraction accuracy with soft-decision error-correcting codes, and preserves text quality through unbiased multilayer reweighting.

Critical strengths
  1. Each technical component, such as spreading bits and soft-decision error-correcting codes, has clear motivation, and its effectiveness has been experimentally evaluated.
  2. There are many experimental details.
Critical weaknesses
  1. The problem statement could be more detailed and comprehensive for broader readers understanding the task setting and its challenges.
  2. Technical details of soft-decision ECC can be more clear to address the concerns of potential cost may introduced by the component.
Questions for authors
  1. What does the phrase "SCALABLE" in the paper's title mean?
  2. In Section 6, Zero-Bit Detection, it says, "WEAVEMARK reserves a few dedicated zero-bit layers," and in the pure zero-bit setting, "Here, WEAVEMARK uses all layers for zero-bit detection." How many dedicated zero-bit layers is WEAVEMARK designed to use by default and in different scenarios?
  3. It says, "BiMark and MPAC address this by treating the most-voted partition as the target and then computing a z-score. However, this max-over-partitions operation inflates the statistic even for non-watermarked text." Although this inflation exists, does it affect the separation between detection-score distributions such that a calibrated threshold on non-watermarked text can still work?
  4. Is Bit acc calculated based on the ECC bit string or the target message?
6 comments
AuthorsOpenReview ↗

Problem Statement

We thank you for your careful review and constructive feedback. We will clarify the problem formulation and the implementation cost of soft-decision ECC, and address your four questions.

Unless otherwise stated, the experimental settings and labels A–I follow the General Response.


Reviewer Comment

The problem statement could be more detailed and comprehensive for broader readers understanding the task setting and its challenges.

Response

Thank you for the opportunity to clarify our problem setting. Given a secret key and a message, the generator embeds the message during token generation. An authorized detector uses the resulting text and the key to recover the message and detect watermark presence without knowing the message in advance. Under a fixed token and layer budget, longer messages and editing attacks make reliable exact recovery increasingly difficult.

Our goal is to develop a multi-bit system that simultaneously achieves the following four objectives for a given generation length and message length:

i) extraction accuracy: exact recovery of the entire message
ii) text quality: preservation of the original token distribution in expectation
iii) payload capacity (scalability): support for embedding longer messages
iv) zero-bit detection: detection of watermark presence even when the message is unknown.

Existing methods address only some of these aspects and have not improved the trade-off as a whole. WeaveMark, in contrast, considers all of them and combines its components to push the trade-off frontier outward. We will revise Section 3 to state this problem setting and its constraints more clearly.


Reviewer Comment

What does the phrase "SCALABLE" in the paper's title mean?

Response

The term scalable in the title refers to payload scalability: the ability to maintain high recovery rates within a limited number of generated tokens as the message length increases. To evaluate this, we compared the number of tokens required to reach a 90% match rate for different message lengths.

Method16-bit24-bit32-bit48-bit64-bit
WeaveMark75125200350525
BiMark300525>1000 (88.2)>1000 (70.8)>1000 (49.2)
MPAC350>1000 (74.5)>1000 (66.0)>1000 (37.0)>1000 (11.5)
RSBH300>1000 (84.0)>1000 (77.6)>1000 (70.3)>1000 (56.2)
XMark>1000 (85.8)>1000 (77.1)>1000 (60.0)>1000 (37.3)>1000 (18.1)
StealthInk>1000 (83.7)>1000 (68.1)>1000 (43.3)>1000 (9.7)>1000 (2.1)
  • ">1000 (x)" means that the target is not reached within 1,000 tokens; the value in parentheses is the match rate (%) at 1,000 tokens.
  • RSBH, XMark, and MPAC use δ=2\delta=2.

WeaveMark achieves a match rate of at least 90% at 525 tokens even for 64-bit messages, whereas the comparison methods do not reach 90% within 1,000 tokens for messages of 32 bits or longer.

The same trend is observed with other models. In the Mistral-7B experiment in Appendix B.1 (Table 7), WeaveMark achieves 89.3% for 64-bit messages at 500 tokens, substantially outperforming the strongest baseline (16.7%). Furthermore, concatenating short ECC blocks allows decoding cost to grow linearly with message length. We will clarify the meaning of scalable in the Introduction of the revised manuscript.

AuthorsOpenReview ↗

Soft-Decision ECC and Computational Cost

Reviewer Comment

Technical details of soft-decision ECC can be more clear to address the concerns of potential cost may introduced by the component.

Response

WeaveMark applies short constituent codes block by block and uses the vote margin s[p]=v1[p]−v0[p]s[p]=v_1[p]-v_0[p] instead of hard majority decisions during decoding. For each block, it selects the codeword that maximizes the following correlation.

c^=arg⁡max⁡c∈C∑ps[p] (2c[p]−1)\hat{c}=\arg\max_{c\in\mathcal{C}}\sum_{p} s[p]\,(2c[p]-1)

The encoding and decoding procedures are as follows (see also Algorithm 3 of the revised manuscript).

Algorithm: Soft-decision ECC

Constituent code C: length n_b, dimension k_b, generator G
  24-bit: 2 × Golay(24,12), n = 48
  32-bit: 2 × RM(32,16),   n = 64

Encoding E(m):
  Input: Message m, B = k/k_b blocks
  1. Split m into B blocks of k_b bits.
  2. For each block b:
       c^(b) = m^(b) G (mod 2)
  3. Return c = concat(c^(1), ..., c^(B))

Decoding D(s):
  Input: Soft-vote vector s, s[p] = v1[p] - v0[p]
  1. Split s into B blocks of length n_b.
  2. For each block b:
       ĉ^(b) = argmax_{w ∈ C}
                Σ_j s^(b)[j] (2w[j] - 1)
       Recover m̂^(b) from ĉ^(b).
  3. Return m̂ = concat(m̂^(1), ..., m̂^(B))

Tied and unobserved positions have s[p]=0s[p]=0 and therefore do not contribute to the correlation. Hard ECC uses hard majority decisions in place of s[p]s[p], with the same codebook and decoding procedure. Because short code blocks are processed independently, the total decoding cost grows linearly with message length.

The measured computational costs are as follows.

Time (μs)BiMarkWeaveMark
Generation / token2,7922,812
Detection / token6578
ECC encoding / message–7
ECC decoding / text (hard / soft)–244 / 240
  • 32-bit message
  • Generation: NVIDIA RTX A5000, batch 50, time normalized per token
  • Detection: AMD EPYC 7763

Soft ECC decoding takes 240 μs per text, comparable to hard decoding (244 μs), and accounts for only about 1.5% of the total extraction time for a 200-token text. The generation-time overhead of the full WeaveMark configuration relative to BiMark is approximately 0.7%, from 2,792→2,812 μs per token. Thus, soft ECC itself adds little computational burden, and the increase in generation cost for the overall system is also limited. We will add an analysis distinguishing generation, detection, and ECC decoding costs to Section 4.3 and Appendix H of the revised manuscript (Table 24).


Reviewer Comment

Is Bit acc calculated based on the ECC bit string or the target message?

Response

Bit acc is measured before ECC decoding: it is the accuracy of the codeword bits when ECC is used, and of the message bits otherwise. This metric compares embedding and extraction accuracy without the error-correction effect of ECC. Because RSBH embeds symbols, its recovered symbols are converted to bits for this calculation. Adding ECC lowers bit accuracy due to the redundant bits, but increases match rate through error correction. We will expand the explanation where the metric is first introduced in the revised manuscript.

AuthorsOpenReview ↗

Number of Zero-Bit Layers

Reviewer Comment

In Section 6, Zero-Bit Detection, it says, "WEAVEMARK reserves a few dedicated zero-bit layers," and in the pure zero-bit setting, "Here, WEAVEMARK uses all layers for zero-bit detection." How many dedicated zero-bit layers is WEAVEMARK designed to use by default and in different scenarios?

Response

With a total of ℓ=10\ell=10 layers, the number of zero-bit layers ℓz\ell_z is set according to the use case.

  • When evaluating message extraction only: ℓz=0\ell_z=0, κ=10\kappa=10
  • When jointly performing message extraction and presence detection (default): ℓz=2\ell_z=2, κ=8\kappa=8
  • Pure zero-bit detection: ℓz=10\ell_z=10

To examine the basis for this default, we measured detection power and message recovery as a function of ℓz\ell_z.

ℓz\ell_z / κ\kappaZero-bit TPR (100 / 200 tokens)Match rate (100 / 200 tokens)
0 / 10–93.6 / 96.8
1 / 992.9 / 97.291.1 / 96.3
2 / 898.4 / 99.184.4 / 95.9
3 / 798.7 / 98.869.1 / 91.0
10 / 098.3 / 98.7–
  • 16-bit message, RM(32,16), soft ECC
  • TPR (%) at FPR=1%

Increasing ℓz\ell_z from 1→21\to2 substantially improves TPR (+5.5%p) with a moderate MR loss (−6.7%p) at 100 tokens. In contrast, increasing ℓz\ell_z from 2→32\to3 yields only a marginal TPR gain (+0.3%p) but a much larger MR loss (−15.3%p).

These results indicate that detection performance largely saturates at ℓz=2\ell_z=2, while further increasing ℓz\ell_z substantially degrades message recovery. We therefore selected ℓz=2\ell_z=2 as the default, considering both detection power and message recovery. We will expand the discussion of how to choose the number of zero-bit layers in Section 6 (Table 5).

Reviewer_LMoFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

WEAVEMARK extends multi-layer unbiased watermarking: allocating multiple codeword bits on different watermark layers of each token, balancing the layer reliability obtained by scrambling the allocation of each bit, and decoding short error correction code blocks using soft voting difference. The method also reserves a message independent layer for detecting the presence of watermarks. The paper evaluates the complete message recovery, editing robustness, text quality, and layer allocation between zero bit detection and message recovery.

Critical strengths
  1. Before ECC was added, diffusion allocation and layer shuffling also improved the recovery rate. Therefore, the redistribution of observation values for each bit is indeed a well founded source of improvement.
  2. The experiment covers a wide range and useful statistical verification has also been conducted. The appendix examines other model families, messages up to 64 bits, insertion and deletion, mixed editing, and rewriting. Appendix G provides the sample size, repeated experiments using new prompts, new messages, and new keys, as well as validation by leaving a false positive rate for text grouping. The independent detection layer also provides a solution to the problem of maximizing the partition of unknown messages. key issue
Critical weaknesses
  1. Table 2 shows that the addition of soft ECC has explained a significant improvement compared to the original BiMark, but the attack table and longer message table are still mainly compared to BiMark without this component. Therefore, it is difficult to determine how much robustness gain comes from diffusion allocation and layer shuffling, and how much comes from stronger decoding and redundancy.
  2. The translation BLEU of Table 3 decreased from 25.24 to 23.15 without watermark, and the abstract ROUGE-2 decreased from 19.17 to 18.22. The results are similar to BiMark and overall better than the biased baseline, but this is more limited than 'relatively ordinary generation with no quality loss'. The expected unbiasedness of masks cannot replace the testing of differences in these limited sample tasks.
Questions for authors
  1. When the error correction code, soft decoder, context window, and total number of layers are consistent, how does BiMark+soft ECC perform in terms of 32-bit recovery rate under replacement and insertion/deletion attacks, as well as on longer messages? Selecting a small number of representative points from existing evaluation settings can help determine the remaining gain.
  2. Can you provide paired uncertainty estimates for downstream task metrics, especially the BLEU differences with watermark free models? Is there a difference between theoretical unbiasedness and actual performance unbiasedness?
4 comments
AuthorsOpenReview ↗

Source of the Robustness Gain

We thank you for your careful review and positive assessment. We conducted additional experiments to address your concerns regarding (1) separating the contributions of ECC and payload spreading and (2) statistically evaluating text quality.

Unless otherwise stated, the experimental settings and labels A–I follow the General Response.


Reviewer Comment

Table 2 shows that the addition of soft ECC has explained a significant improvement compared to the original BiMark, but the attack table and longer message table are still mainly compared to BiMark without this component. Therefore, it is difficult to determine how much robustness gain comes from diffusion allocation and layer shuffling, and how much comes from stronger decoding and redundancy.

When the error correction code, soft decoder, context window, and total number of layers are consistent, how does BiMark+soft ECC perform in terms of 32-bit recovery rate under replacement and insertion/deletion attacks, as well as on longer messages? Selecting a small number of representative points from existing evaluation settings can help determine the remaining gain.

Response

We agree that a direct comparison between the original BiMark and WeaveMark alone makes it difficult to separate the contributions of ECC and payload spreading. To examine this, we evaluated 9 configurations (A–I), covering combinations of multi-bit embedding, layer shuffling, and ECC, under the same conditions.

LabelMulti-bitShufflingECCCleanSubst. 10%Ins. 10%Del. 10%
A (BiMark)✗✗None20.60.54.12.3
B✗✗Hard63.44.119.916.1
C✗✗Soft82.910.846.737.6
D✓✗None24.52.17.46.9
E✓✗Hard50.15.819.019.0
F✓✗Soft72.114.939.637.2
G✓✓None52.64.920.317.7
H✓✓Hard79.413.045.142.1
I (WeaveMark)✓✓Soft90.533.671.467.2

Comparing C and I, which use the same soft ECC, WeaveMark provides additional gains of +7.6%p on clean text, +22.8%p under substitution, +24.7%p under insertion, and +29.6%p under deletion.

Improvements are observed from A→G even without ECC, while G→I demonstrates the additional benefit of ECC. Thus, the robustness gains arise from the complementary effects of payload spreading and ECC, rather than ECC alone.

We will add the above ablation study and a discussion of the effects of the individual components to a new Appendix H of the revised manuscript (Table 20).

Longer Messages

We extended the comparison using the same ECC and soft decoder to 48-bit and 64-bit messages (Table 21).

Method48-20048-40048-60064-20064-40064-600
C (BiMark + Soft ECC)39.085.190.98.766.184.5
I (WeaveMark)74.091.994.143.685.391.2
  • Column headings: message bits – generated tokens. For example, 48-200 denotes a 48-bit message and 200 generated tokens.
  • 48-bit: 3×RM(32,16), 64-bit: 4×RM(32,16)

The additional gains of WeaveMark persist for longer messages. In particular, for 64-bit messages, the number of tokens required to achieve an 80% match rate decreases from 525 → 350 tokens (33% reduction), demonstrating improved recovery efficiency even with the same ECC.

AuthorsOpenReview ↗

Text Quality and Statistical Uncertainty

Reviewer Comment

The translation BLEU of Table 3 decreased from 25.24 to 23.15 without watermark, and the abstract ROUGE-2 decreased from 19.17 to 18.22. The results are similar to BiMark and overall better than the biased baseline, but this is more limited than 'relatively ordinary generation with no quality loss'. The expected unbiasedness of masks cannot replace the testing of differences in these limited sample tasks.

Can you provide paired uncertainty estimates for downstream task metrics, especially the BLEU differences with watermark free models? Is there a difference between theoretical unbiasedness and actual performance unbiasedness?

Response

Your comments on text quality prompted us to re-examine the evaluation settings. We identified an issue in which the seq2seq models' default beam settings had been applied, resulting in generation with beam sampling rather than the intended top-K sampling. We corrected the evaluation settings and re-evaluated all methods under the same top-K sampling conditions.

The corrected downstream task results are as follows. (MPAC, RSBH, XMark: δ=2.0\delta=2.0)

MethodSumm. BERTScoreR-1R-2R-LTrans. BERTScoreBLEU
No watermark24.5737.2113.9724.2093.5919.13
WeaveMark24.5437.2313.9024.1793.6218.88
BiMark24.3637.0813.6724.0593.6218.88
StealthInk23.6736.6713.4923.6693.5218.92
XMark21.7635.1411.9322.4193.0316.29
MPAC21.3334.7911.5822.1492.9616.58
RSBH21.1034.7211.5522.0693.0016.18

For BLEU and ROUGE-2, the metrics highlighted by the reviewer, we performed paired bootstrap analysis against unwatermarked generation at the source-text level.

  • BLEU: −0.26 (95% CI: [−0.76, +0.22])
  • ROUGE-2: −0.08 (95% CI: [−0.40, +0.25])

We additionally conducted 10 repeated runs with different keys, messages, and sampling seeds. The mean differences across the 10 runs, each measured against its corresponding unwatermarked baseline, are as follows.

  • BLEU: +0.03 (95% CI: [−0.13, +0.19])
  • ROUGE-2: +0.01 (95% CI: [−0.09, +0.11])

Theoretical unbiasedness means that the token distribution is preserved in expectation; it does not guarantee identical downstream scores for individual generated outputs. Nevertheless, under the corrected sampling settings, the average quality differences for WeaveMark were very small, and no statistically significant decline was observed in either BLEU or ROUGE-2. On the remaining quality metrics, WeaveMark also achieved scores at the same level as unwatermarked generation.

We will replace Table 3 with the corrected quality evaluation results (revised Table 3). We will also report the paired confidence intervals (Table 16) and clearly distinguished theoretical unbiasedness from actual downstream quality. Without your comments, we would not have discovered this configuration error. We are deeply grateful for your review, which has materially improved the accuracy of our paper.

AuthorsOpenReview ↗

Closing Remarks

We thank the reviewer for the careful review and concrete suggestions. Your comments helped us strengthen the experimental evidence and clarify our claims. Specifically:

  • Separating the contributions of ECC and payload spreading: We evaluated nine configurations under controlled conditions. Even with the same soft ECC, WeaveMark improves the 32-bit match rate by 22.8%p under substitution, 24.7%p under insertion, and 29.6%p under deletion at a 10% attack rate.
  • Verifying scalability with the same ECC: We extended the comparison to 48-bit and 64-bit messages. For 64-bit messages, WeaveMark reduces the tokens required to achieve an 80% match rate from 525 to 350—a 33% reduction.
  • Re-evaluating text quality with statistical uncertainty: We corrected the sampling settings and re-evaluated all methods. Paired bootstrap analysis and 10 repeated runs showed no statistically significant decline in BLEU or ROUGE-2 under the evaluated conditions. We also clarified the distinction between theoretical unbiasedness and empirical downstream quality.

We will incorporate these additional results and clarifications into the revised manuscript.

We would like to emphasize that WeaveMark is not limited to a few favorable experimental settings; it consistently achieves top-tier payload capacity, extraction accuracy, and text quality across a very broad range of experimental conditions.

Achieving these three objectives together is essential for practical multi-bit watermarking, and the additional experiments demonstrate that WeaveMark’s gains persist even against a baseline equipped with the same soft ECC.

We respectfully ask you to give full consideration to this contribution to both performance and practical utility, and to reconsider your assessment of the paper. We sincerely thank you for the time and effort devoted to providing such a careful and constructive review.

Thank you for the author's reply and explanation, which has eased some of my doubts. I will keep my positive score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

[Update] We have uploaded a revised manuscript that incorporates all changes promised in our responses, including the new Appendix H; the corresponding figures and tables are linked in each response.

We sincerely thank the reviewers for their careful evaluation and constructive feedback.

We conducted additional experiments to address questions raised by multiple reviewers. Unless otherwise stated, the message recovery and ablation experiments use the following settings.

Experimental setup

  • Model / Dataset: LLaMA-3-8B / C4 RealNewsLike
  • Generation: top-K=50K=50, temperature =1.0=1.0
  • Watermark: δ=1.0\delta=1.0, total layers ℓ=10\ell=10, context window h=2h=2
  • Main ablation: 32-bit message, 200 generated tokens, no dedicated zero-bit layers
  • ECC: 2 × RM(32,16), producing a 64-bit codeword from a 32-bit message
  • Decoding: Hard ECC uses hard majority decisions; Soft ECC uses vote margins v1[p]−v0[p]v_1[p]-v_0[p]. Both use the same ECC codebook and correlation-decoding framework.
  • Attacks: Substitution, insertion, and deletion, following Section 5.2 and Appendix C.2
  • Metric: Match rate (%), measuring exact recovery of the original message

Experiments using different message lengths, generation lengths, datasets, or zero-bit layer configurations are specified separately in the corresponding responses.

Configuration labels

LabelMulti-bitShufflingECC
A (BiMark)✗✗✗
B✗✗Hard
C✗✗Soft
D✓✗✗
E✓✗Hard
F✓✗Soft
G✓✓✗
H✓✓Hard
I (WeaveMark)✓✓Soft

Multi-bit ✓ denotes κ=10\kappa=10, while ✗ denotes κ=1\kappa=1. When κ=1\kappa=1, all layers use the same bit, so layer shuffling is an identity operation. Configurations without ECC directly embed the original message bits.

We sincerely thank you for the time and effort devoted to providing such a careful and constructive review.

AuthorsOpenReview ↗

Summary of Discussion and Revisions

We sincerely thank the AC and all reviewers for their careful evaluation and constructive feedback. The concern shared across reviewers was the source of WeaveMark's gain; the remaining comments requested support for key claims and clarification of design details. We summarize them into three issues:

  • Source of the gain and novelty (LMoF, qNYG): Is the gain mainly due to soft ECC, a known technique, or do spreading and shuffling contribute beyond it?
  • Key claims (LMoF, GYEF): Are the "no quality loss" and "scalable" claims sufficiently supported?
  • Design details and cost (GYEF): What does soft ECC cost, how many zero-bit layers are used, and does max-z still hurt detection after calibration?

1. Responses to the main issues

Source of the gain and novelty

  • We do not claim ECC or soft decoding as new. Our contribution is an embedding structure that balances observation reliability across the ECC codeword so that soft ECC can fully exploit it.
  • With the same soft ECC, WeaveMark outperforms BiMark + soft ECC by +22.8 / +24.7 / +29.6%p under 10% substitution / insertion / deletion (32-bit, 200 tokens).
  • The reason is observation balance: spreading keeps the mean per-bit evidence unchanged but reduces weak bits (19.6% → 8.3%) and unobserved bits (4.14 → 0 per text).
  • The components reinforce each other: hard → soft ECC cuts the decoding failure rate by 23.7% with spreading and shuffling, versus 7.1% for BiMark.

Key claims

  • After correcting a beam-search setting in the downstream evaluation, paired bootstrap shows no significant decline in BLEU or ROUGE-2.
  • "Scalable" means payload scalability: WeaveMark reaches 90% for 64-bit messages within 525 tokens; no baseline does so within 1,000 tokens at 32 bits or more.

Design details and cost

  • Soft decoding takes ~1.5% of extraction time; generation overhead over BiMark is ~0.7%.
  • Detection saturates at the default ℓz=2\ell_z=2; more layers only reduce the match rate.
  • Calibrated max-z trails the oracle by 24–31%p in TPR at 25 tokens; dedicated zero-bit layers avoid this.

2. Additional evidence

ConcernAdded experimentKey result
Source of the gain9-configuration component ablationFull design is best in all conditions
Source of the gainBiMark + same soft ECC, 16- to 64-bitGains persist on clean and attacked text
Source of the gainPer-bit evidence over κ\kappaFewer weak and unobserved bits
Text qualityCorrected re-evaluation, paired bootstrap, 10 runsNo significant quality decline
ScalabilityTokens to reach a 90% match rate, 16- to 64-bitOnly WeaveMark reaches 90% at 32 bits or more
Soft ECC costGeneration, detection, and ECC runtimeNegligible overhead
Zero-bit detectionℓz\ell_z sweep; calibrated max-z vs. oracle-zℓz=2\ell_z=2 suffices; max-z gap on short texts

Corresponding changes are in the revision (Secs. 1 and 3, Tables 3 and 5, App. G.2, G.3, H) and linked in each response.

3. Requested reconsideration

In summary, WeaveMark simultaneously achieves:

  1. text quality at the level of unbiased watermarking,
  2. higher extraction accuracy than recent methods,
  3. payload scalability up to 64 bits,
  4. robustness to insertion, deletion, and diverse other attacks, and
  5. zero-bit and multi-bit detection within a single framework.

These gains do not come from selecting favorable settings; they hold across matched comparisons with recent methods, 12- to 64-bit payloads, multiple model families, clean generation, position-preserving edits, insertion and deletion, copy-paste mixing, and regeneration attacks.

Moreover, WeaveMark is not a specialized scheme that maximizes multi-bit recovery alone; its zero-bit detection is on par with the strongest dedicated schemes. Whereas prior methods satisfy only some of accuracy, quality, capacity, and detection, WeaveMark provides all of them in one coherent system.

This work therefore goes beyond a benchmark improvement: through new component interactions, WeaveMark reaches an operating point that prior methods have not attained, and its anonymized implementation provides a practical basis for real-world provenance verification and watermark-presence detection.

If any concern remains, we would be glad to clarify it before the discussion closes. We respectfully ask the AC and reviewers to consider this work not as incremental engineering optimized for a specific condition, but as a practically significant contribution that introduces a new system-level algorithmic design and achieves the highest match rate across the evaluated practical regimes while preserving text quality.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.