- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work aims to improve the exact message decoding rate while preserving text quality in multi-bit text watermarking for LLMs. The proposed method, WeaveMark, builds on a previous method, BiMark, and introduces three additional components: payload spreading, layer shuffling, and soft-decision ECC decoding. Ablation studies show that these components substantially improve performance over the BiMark baseline. Experimental results further demonstrate that WeaveMark generally outperforms several existing multi-bit watermarking methods across different evaluation settings.
Critical strengths
-
This work addresses a timely and important topic in multi-bit text watermarking for LLMs.
-
The proposed WeaveMark improves upon the previous BiMark method through more balanced spreading of encoded bits, layer shuffling to distribute observations more evenly across bit positions, and soft-decision ECC decoding to correct bit errors, where the layer shuffling is well motivated. Experimental results show that WeaveMark substantially improves the exact message recovery rate while maintaining comparable text quality.
Critical weaknesses
-
One key contribution claimed in the manuscript is payload spreading, where each token carries information from multiple payload bits. The authors state around Line 200 that each bit is assigned to approximately token positions instead of in BiMark. However, because the fixed layer budget of each token is divided among selected bits, the expected total number of layer-level observations received by each bit remains approximately unchanged. According to Table 2, multi-bit embedding alone improves the match rate only modestly over BiMark (less than 4%), whereas much larger gains are obtained after adding layer shuffling and especially soft ECC. In particular, BiMark equipped with the same soft ECC already achieves performance close to WeaveMark. A stronger comparison that equips BiMark with the same layer shuffling and soft ECC decoding would be needed to clearly establish the contribution of the proposed method.
-
Soft ECC decoding appears to be the most important component contributing to the improvement in match rate, while similar ECC-based decoding has been used in prior methods. The novelty and specific contribution of this component therefore need to be clarified more clearly.
Questions for authors
-
How does the performance of BiMark with layer shuffling and soft ECC compare to the proposed WeaveMark in the main experimental settings (e.g., in Figure 5 and Figure 6)?
-
The expected total number of layer-level observations per bit appears to remain approximately unchanged after payload spreading. Moreover, ECC expands the payload into a longer codeword, which further reduces the average number of layer-level observations available to each encoded bit under a fixed token and layer budget. Could the authors clarify why distributing the same observation budget across more token positions improves decoding reliability?
5 comments
Why Multi-Bit Embedding Helps Under a Fixed Observation Budget
Reviewer Comment
The expected total number of layer-level observations per bit appears to remain approximately unchanged after payload spreading. Moreover, ECC expands the payload into a longer codeword, which further reduces the average number of layer-level observations available to each encoded bit under a fixed token and layer budget. Could the authors clarify why distributing the same observation budget across more token positions improves decoding reliability?
Response
As the reviewer points out, the average number of layer-level observations per bit does not increase. For the effective token count , codeword length , total number of layers , and number of selected bits per token , the expected observation counts under uniform allocation are as follows (when is a multiple of ).
=\frac{\kappa T_{\mathrm{eff}}}{n}, \qquad \mathbb{E}[\text{layer votes/bit}] =\frac{\ell T_{\mathrm{eff}}}{n}.$$ That is, rather than increasing the total observation budget, we **distribute the observations for each bit across more token positions**. Because entropy and watermark signal strength vary across tokens, this spreading can alleviate the concentration of weak observations on particular bits. From the perspective of ECC, this produces an effect similar to interleaving an imbalanced channel, improving the probability of successful ECC decoding ([Figure 2](https://anonymous.4open.science/r/WeaveMark_Rebuttal_Figures-0FAB/Figure2.pdf), [Figure 3](https://anonymous.4open.science/r/WeaveMark_Rebuttal_Figures-0FAB/Figure3.pdf)). To further support this explanation, we defined the evidence $e_p$ for each codeword bit as (correct votes − incorrect votes) and measured its distribution as a function of $\kappa$. |Metric|$\kappa=1$|$\kappa=2$|$\kappa=5$|$\kappa=10$| |---|---:|---:|---:|---:| |Mean $e_p$|7.78|7.86|7.97|7.95| |Std. $e_p$|7.09|6.26|5.45|5.11| |Weak bits (%)|19.6|11.7|10.9|8.3| |Unobserved bits|4.14|0.40|0|0| |Match rate (%)|84.0|88.7|92.5|91.1| - Weak bits: the proportion of bits with $e_p \le 0$. - Unobserved bits: the number of bits that are never observed in a text. - 200 generated tokens, the same key/message, and a shared set of 213 prompts. As $\kappa$ increases from $1\to10$, the mean evidence remains nearly unchanged, while the standard deviation decreases from 7.09 → 5.11, reducing the variance by approximately half. Weak bits decrease from 19.6% → 8.3%, and unobserved bits decrease from more than 4 per text → 0. In particular, expanding the codeword through ECC can increase the number of unobserved or unreliable bits. Payload spreading reduces this imbalance, providing more reliable decoding inputs even with the same observation budget and ECC. We will add this analysis to Appendix H of the revised manuscript ([Table 23](https://anonymous.4open.science/r/WeaveMark_Rebuttal_Figures-0FAB/Table23.pdf)).Novelty of soft-decision ECC
Reviewer Comment
Soft ECC decoding appears to be the most important component contributing to the improvement in match rate, while similar ECC-based decoding has been used in prior methods. The novelty and specific contribution of this component therefore need to be clarified more clearly.
Response
ECC, soft-decision decoding, and splitting messages into short code blocks are existing techniques, and we do not claim novelty for these techniques themselves. The core contribution of WeaveMark is, under fixed token and layer budgets, an embedding structure that balances the reliability of observations across the ECC codeword, optimizing the input channel to maximize ECC's error-correction capability.
- (i) Multi-bit embedding: distributes the same layer observation budget across more token positions, reducing unobserved and weak bits.
- (ii) Layer shuffling: reduces the tendency for particular bit positions to receive disproportionately high- or low-reliability layers.
- (iii) Integration with soft ECC: uses the reliability information in the evenly accumulated vote margins to improve error correction.
Components (i) and (ii) alleviate the scarcity and imbalance of observations caused by ECC codeword expansion, while (iii) uses these improved decoding inputs.
Under the conditions of the component ablation in Table 2 of the main paper, ECC accounts for most of the improvement in match rate. However, the additional experiments reveal a substantial performance gap between soft ECC alone and WeaveMark on short texts and under editing attacks.
- Components beyond ECC also contribute substantial performance gains: Even with the same soft ECC, a large gap remains between BiMark with ECC alone (scheme C) and WeaveMark (I). On clean text with 24-bit messages and 50 tokens, the match rate improves from 9.4% → 39.9%, a gain of 30.5%p. Under 10% deletion with 32-bit messages and 200 tokens, it improves from 37.6% → 67.2%, a gain of 29.6%p.
- ECC is also more effective when combined with the other components: Soft decoding has a greater effect when used together with payload spreading and shuffling. Under 10% substitution, switching BiMark from hard to soft ECC reduces the decoding failure rate by 7.1% (B→C), whereas the reduction is 23.7% when (i) and (ii) are applied (H→I).
Thus, WeaveMark's main novelty lies primarily in the synergistic design of its constituent techniques, rather than in each technique individually, allowing them to compensate for one another's limitations and reinforce their respective strengths. In the revised discussion of ECC, we will explicitly cover related work, including RS coding in RSBH, BCH coding in BREW, and soft-information decoding in the concurrent work CORE-BREW, and clarify this design contribution and its experimental support.
BiMark with shuffling and soft ECC
We thank you for your careful review and constructive feedback. We conducted additional experiments and analyses to address your concerns regarding (1) the contribution of payload spreading under the same ECC, (2) why performance improves under a fixed observation budget, and (3) the novelty of soft ECC.
Unless otherwise stated, the experimental settings and labels A–I follow the General Response.
Reviewer Comment
Response
We conducted controlled experiments to examine whether the additional gains from payload spreading persist when the same ECC and soft decoder are used.
In BiMark (κ=1), all layers are assigned to the same bit, so layer shuffling, which permutes the bit-to-layer assignments, is an identity operation. Therefore, the requested BiMark + shuffling + soft ECC is equivalent to BiMark + soft ECC.
We compared BiMark + soft ECC and WeaveMark using the same ECC and soft decoder under the main conditions in Fig. 5 (clean) and Fig. 6 (substitution). (16-bit: RM(32,16); 24-bit: 2×Golay(24,12); 32-bit: 2×RM(32,16))
16-50denotes a 16-bit message and 50 generated tokens.Even with the same ECC, WeaveMark achieves higher recovery rates than BiMark under all compared conditions (Tables 21–22).
Multi-bit embedding without shuffling
To distinguish the contributions of multi-bit embedding and shuffling, we compared C, F, and I using the same soft ECC.
F has a lower match rate than C under some conditions. This is because assigning layers to multiple bits without shuffling can repeatedly assign less reliable layers to certain bit positions (Fig. 3). In contrast, I performs best under all conditions. Thus, multi-bit embedding and layer shuffling form an effective design when used together, which also explains the modest gain from multi-bit embedding alone in the original Table 2.
We will add the following, more detailed ablation table covering all components of WeaveMark, together with a discussion of the effects of the individual components, to a new Appendix H of the revised manuscript (Table 20).