acceptodds
Under review as a conference paper at ICLR 2027

UNICOMARK: UNBIASED AND ROBUST MULTI-BIT LLM WATERMARKING VIA CO-DESIGN OF ERROR- RESILIENT RECOVERY AND SAMPLING

Abstract

Multi-bit watermarking for large language models (LLMs) aims to embed decodable source identifiers into generated text, but exact and robust recovery remains difficult while preserving text quality in normal and adversarial settings. We propose UniCoMark, which co-designs two tightly coupled components: an embedding–extraction process for reliable watermark message recovery and a matched sampling process for text generation. UniCoMark combines a fixed-length linear code with soft-information decoding and a sampling rule tailored to the recovery objective. To ensure high text quality, we introduce an unbiasedness constraint into the sampling design, enabling the sampling process to be formulated as an optimal transport problem. We further establish a recovery theorem for UniCoMark, giving a message-accuracy bound that characterizes robustness under textual attacks. Extensive experiments validate the effectiveness of UniCoMark. In the normal setting, with 256 tokens generated by base LLMs and a 32-bit payload, UniCoMark achieves 94.9% message accuracy, significantly outperforming the strongest baseline at 60.2%, while also attaining lower perplexity (10.10 vs. 11.22). Under adversarial conditions, UniCoMark remains highly robust: for a 512-token sequence and a 32-bit payload, it achieves 88.0% message accuracy under 15% token-level mixed edits and 87.7% under 30% sentence-level mixed edits, exceeding the best baseline by 64.3 and 52.1 percentage points, respectively.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM watermarking, Multi-bit watermarking

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pQ44OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes UniCoMark, which first encodes the payload with a linear error-correcting code and distributes codeword bits across token positions through seed-dependent routing. During generation, an optimal-transport sampler minimizes the Hamming mismatch between token labels and watermark labels while preserving the host token distribution. During detection, routed observations are aggregated into soft reliability scores and decoded with ordered statistics decoding. Experiments on two LLMs show strong exact message recovery under clean and text-editing settings while maintaining competitive text quality.

Critical strengths
  1. The proposed method UniCoMark is technically sound. The use of error-correcting coding (ECC), repeated bit routing, soft decoding, and optimal-transport sampling provides a clear and technically coherent framework for multi-bit watermarking.
  2. The experimental results are strong across multiple models and datasets. UniCoMark achieves high exact message recovery under both clean and text-editing settings.
Critical weaknesses
  1. The contribution of ECC to the strong exact-message recovery remains unclear. UniCoMark uses a codeword length of N_c=2r, but the manuscript does not report a no-ECC setting. Such an ablation would help separate the gains from error-correcting redundancy, repeated routing, and the proposed sampling strategy.

  2. The PPL evaluation uses Qwen3-14B as the only oracle model for both Llama and Gemma generations. The manuscript should clarify whether this cross-model PPL reliably reflects the quality degradation of the corresponding host models.

Questions for authors
  1. Could the authors evaluate PPL using a larger model from the same model family? For example, Llama-3.1-70B could be used to evaluate generations from Llama-3.1-8B.

  2. Could the authors clarify the difference between UniCoMark and prior work that also uses optimal transport, such as ArcMark [1]?

  3. Could the authors compare UniCoMark with more recent multi-bit watermarking methods, such as XMark [2]?

  4. Could the authors quantify the contribution of the ECC for UniCoMark? Additionally, applying similar ECC to the existing method will also help readers understand its role.

Reference: [1] ArcMark: Distortion-Free Multi-Byte LLM Watermark via Optimal Transport. [2] XMark: Reliable Multi-Bit Watermarking for LLM-Generated Texts.

4 comments
AuthorsOpenReview ↗

Response to Reviewer pQ44 (1/2)

We thank the reviewer for recognizing the technical coherence and empirical recovery performance of UniCoMark.

W1/Q4. ECC contribution and results on no-ECC setting.

ECC introduces redundant constraints across codeword coordinates, allowing the decoder to recover the message jointly from noisy or incomplete observations. Our soft decoder uses the reliability of each coordinate to exploit this redundancy.

Table 6 of the submitted paper compares four combinations of sampling and recovery. Here, we further report the exact-message recovery results under no-ECC, explicitly distinguishing the effects of routing and error correction.

SamplingRoutingRecoveryExact-message recovery
BiMarkConcentratedNo ECC42.58%
BiMarkUniformNo ECC66.60%
OTConcentratedNo ECC54.69%
OTUniformNo ECC75.59%
BiMarkUniformECC + corresponding decoder87.50%
OTUniformECC + corresponding decoder97.27%

In the comparison above, the first, fourth, fifth, and sixth rows match the configurations SB+EBS_B+E_B, SU+EBS_U+E_B, SB+EUS_B+E_U, and SU+EUS_U+E_U in the submitted Table 6. The two additional controls isolate routing without ECC.

The ablations show that routing, sampling, and coded recovery each improve message recovery. ECC improves recovery with both our sampler and the existing BiMark sampler. With uniform routing fixed, adding ECC and its corresponding decoder raises message recovery from 66.60% to 87.50% (+20.90 percentage points) for BiMark and from 75.59% to 97.27% (+21.68 percentage points) for OT.

W2/Q1. PPL evaluation with a larger same-family model

Choice of the original evaluator. Using a separate, larger PPL evaluator follows prior watermarking work. BiMark [1, Sec. 5.1] evaluates Llama3-8B generations with Gemma-9B, explicitly selecting an oracle with more parameters. Our use of Qwen3-14B-Base as a common evaluator for Llama and Gemma follows this practice.

Same-family validation. We additionally evaluate Gemma-2-9B generations on C4 RealNewsLike using Gemma-2-27B Base, providing the requested larger same-family check. The table reports median PPL; lower is better.

MethodQwen3-14B-BaseGemma-2-27B Base
Plain7.0696.593
UniCoMark, s=0.67.4096.652
UniCoMark, s=0.88.0847.331
UniCoMark, s=1.07.9227.249
MPAC, δ=17.6927.027
MPAC, δ=29.2158.441
BiMark8.5077.779
SegMark, δ=17.4366.542
SegMark, δ=29.1348.273

With the same-family evaluator, UniCoMark at s=0.6s=0.6 remains close to Plain: 6.652 versus 6.593, approximately a 0.9% increase. All three UniCoMark settings retain lower PPL than BiMark and the δ=2\delta=2 MPAC/SegMark settings under both evaluators. These comparisons therefore remain consistent under the larger same-family oracle. We report each oracle's scores separately because their absolute PPL scales differ.

References

[1] Xiaoyan Feng, He Zhang, Yanjun Zhang, Leo Yu Zhang, and Shirui Pan. BiMark: Unbiased Multilayer Watermarking for Large Language Models. ICML, 2025.

AuthorsOpenReview ↗

Response to Reviewer pQ44 (2/2)

Q2. Differences between UniCoMark and ArcMark

We already cited ArcMark in the paper. Although both methods use OT sampling, their starting points differ: ArcMark emphasizes distortion-free channel capacity and rate [3], while UniCoMark targets robust finite-length message recovery under textual edits. These objectives motivate different choices in codeword construction, evidence aggregation, and decoding. As already described in Sec. 3 and Appendices B–C, UniCoMark co-designs embedding–extraction and sampling by deriving its OT cost from a recovery-margin objective. This codeword design supports the following advantages in robustness and decoding scalability.

Robustness objective. ArcMark scores context-dependent random-linear-code symbols for each candidate message [3]. In contrast, UniCoMark targets finite-length exact-message recovery under textual edits, using a fixed codeword as the shared target of embedding and extraction. Local-context routing maps surviving valid observations to its coordinates, where repeated evidence is aggregated into soft reliabilities for joint ECC recovery. Edits change the available evidence while the codeword constraints remain fixed, enabling structured recovery from partially preserved text. Our analysis connects recovery to useful observations AA, noisy observations BB, and sampling mismatch pp, supported by token- and sentence-level mixed-edit and paraphrasing experiments.

Sampling objective. ArcMark minimizes angular distance on a circle under the token-marginal constraint [3, Eq. (5)]. UniCoMark instead derives its Hamming transport cost from a one-step recovery-margin loss under the uniform competing-target assumption (Appendix C), subject to the host-marginal constraint. Our sampling objective is therefore directly aligned with the binary evidence used in codeword recovery, with the resulting flip rate explicitly incorporated in our recovery theorem.

Decoding scalability. ArcMark's published Algorithm 2 exhaustively scores all 2r2^r candidate messages for an rr-bit payload [3]. UniCoMark instead aggregates observations into a fixed-length reliability vector and applies structured decoding, such as OSD. OSD-2 tests only 1+r+(r2)1+r+\binom{r}{2} candidate codewords (Appendix B, Eq. (21)). At 32 bits, this is 4,294,967,296 versus 529 candidates—an approximately 8.1-million-fold reduction in candidate count. This replaces exponential candidate growth with quadratic growth. ArcMark reports evaluations up to 32 bits [3, Sec. 5.1], whereas our payload-scaling experiments extend to 256 bits (Fig. 4(a)). Our supplementary 16-bit and 24-bit comparisons show close agreement between OSD-2 and exact ML near the recovery boundary.

Q3. Comparison with XMark

We added XMark [4] as a baseline at δ=2\delta=2 in the revised paper. Tables 2 and 3 (Sec. 5.1) compare message recovery across payload sizes and text lengths. For example, on ASAP-essay with Gemma-2-9B, UniCoMark at s=0.6s=0.6 recovers 81.6% of 32-bit messages from 256 tokens, compared with 15.62% for XMark, with PPL values of 10.26 and 9.98, respectively (Table 2).

MethodMessage accuracy ↑PPL ↓
XMark, δ=2\delta=215.62%9.98
UniCoMark, s=0.6s=0.681.6%10.26
UniCoMark, s=1.0s=1.098.8%11.27

Tables 16–19 (Appendix I.3) report message accuracy and bit accuracy for each model and dataset across all tested payload sizes and text lengths. Table 20 (Appendix I.4) reports PPL at 64, 128, 256, and 512 tokens.

For summarization and translation, Table 4 (Sec. 5.1) reports task quality and Table 22 (Appendix I.6) reports 4-bit message recovery. Figure 5 and Table 7 (Sec. 5.2) show recovery under mixed edits and paraphrasing, respectively. Tables 23 and 24 (Appendix I.7) give separate results for insertions, deletions, and replacements at the token and sentence levels.

References

[3] Atefeh Gilani, Sajani Vithana, Carol Xuan Long, Oliver Kosut, Lalitha Sankar, and Flavio P. Calmon. ArcMark: Distortion-Free Multi-Byte LLM Watermark via Optimal Transport. arXiv preprint arXiv:2602.07235v2, 2026.

[4] Jiahao Xu, Rui Hu, Olivera Kotevska, and Zikai Zhang. XMark: Reliable Multi-Bit Watermarking for LLM-Generated Texts. ACL, pp. 14747–14763, 2026. Official implementation: JiiahaoXU/XMark.

I appreciate the authors' responses, which have answered my questions. I have increased my score to 3.

Thank you for reconsidering our paper. Your suggestions to add XMark as a baseline and clarify the ECC ablation helped us strengthen the comparison and show more clearly how much ECC contributes to message recovery.

Reviewer_ptJVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

UniCoMark addresses exact multi-bit message recovery from LLM-generated text under local edits. It combines a fixed linear error-correcting code, prefix-dependent routing and masking, soft-information aggregation, and ordered statistics decoding. An optimal-transport sampler minimizes a local Hamming cost aligned with the decoding metric while preserving the host token marginal in expectation. Experiments across two language models and two datasets show substantial recovery improvements, supported by component ablations and edit-attack evaluations. Strong paraphrasing remains a demonstrated limitation.

Critical strengths

Substantial and consistent improvements in exact-message recovery. At 256 tokens and a 32-bit payload on Llama-3.1-8B/C4, UniCoMark achieves 94.9% accuracy versus 60.2% for SegMark and 35.0% for BiMark. The broader payload and length sweeps, together with local-edit experiments, support a meaningful improvement on the practically relevant exact-recovery objective.

A coherent design supported by informative ablations. Preserving soft evidence for codeword recovery and matching the sampling cost to the decoding metric provides a useful design principle. Table 6 shows that both sampling and recovery contribute independently, with their combination performing best. The paper also experimentally establishes its limitations under strong paraphrasing.

Critical weaknesses

The quality-preservation claim exceeds the joint empirical evidence. The strongest recovery results use (s=1), where PPL increases and translation BLEU decreases relative to unwatermarked generation. Lower strengths achieve comparable downstream quality, but Table 4 does not report message recovery on those same tasks. Consequently, the evidence establishes a favorable recovery–quality trade-off more clearly than simultaneous high recovery and unchanged quality.

The recovery theorem has a limited connection to the implemented system. Appendix E assumes exactly balanced routing counts, conditionally independent noise contributions, and exact metric decoding, whereas the implementation uses keyed routing and OSD-2. The all-success comparison in Table 8 and the correlation analysis in Table 9 provide useful supporting evidence, but do not establish a quantitative recovery guarantee for the practical pipeline. This limits the strength of the theoretical contribution.

Questions for authors

Clarification of the central empirical claim: What are the exact-message recovery rates for the same summarization and translation outputs evaluated in Table 4 at (s=0.6,0.8,1.0)? Please also specify sample sizes and uncertainty estimates. These results would clarify whether quality and recovery are simultaneously maintained.

Clarification of the theoretical scope: For representative experimental settings, does Theorem 1 yield a non-vacuous bound? How closely does OSD-2 match exact decoding near the recovery-failure boundary, beyond the all-success settings in Table 8?

Clarification of novelty: ArcMark also combines linear coding, optimal transport, and soft-distance message recovery. Beyond differing evaluation objectives, what precise methodological advance distinguishes UniCoMark, particularly regarding edit synchronization, the sampling objective, and decoding scalability?

2 comments
AuthorsOpenReview ↗

Response to Reviewer ptJV (1/2)

We thank the reviewer for recognizing the consistent recovery improvements and the component ablations.

W1/Q1. Downstream quality, message recovery and experimental setup

We follow the task-specialized BART/mBART evaluation pipelines used in BiMark [1, Sec. 5.2, Table 4] and MCMark [2, Appendices C–D, Tables 6–7], primarily to assess downstream task performance and verify that unbiased sampling preserves generation quality (Table 4). Since these task-specialized models typically produce short, low-entropy outputs, they provide limited watermark evidence for recovering longer messages. Therefore, we report message recovery results for 4-bit messages on both downstream tasks.

Experimental setup, sample sizes, and uncertainty. Our experiments use BART-large-CNN on CNN/DailyMail for summarization, evaluated with ROUGE-1 and BERTScore, and mBART-large-50 on WMT16 En→Cs for translation, evaluated with BLEU. Each downstream task uses 512 samples. Values reported with “±” in Table 4 represent mean ± standard deviation.

MethodSummarization recoveryTranslation recovery
UniCoMark, s=1.083.43%70.86%
UniCoMark, s=0.876.57%55.43%
UniCoMark, s=0.656.00%40.57%
MPAC, δ=134.29%28.57%
MPAC, δ=270.29%48.00%
BiMark, default configuration73.71%57.14%
SegMark, δ=121.29%15.57%
SegMark, δ=250.38%38.42%

Our downstream evaluation supports the quality-preserving behavior of unbiased sampling, while maintaining reliable message recovery. As shown in Table 4, at s=0.8s=0.8, UniCoMark achieves ROUGE-1/BERTScore scores of 38.8/87.5, closely matching the unwatermarked baseline of 38.9/87.6. Similarly, the BLEU score on the translation corpus is 15.4651, compared to 15.5255 without watermarking. Importantly, at s=1s=1, UniCoMark recovers 83% of messages in summarization and 71% in translation, demonstrating its ability to preserve generation quality without substantially compromising message recovery.

W2/Q2. Non-vacuous recovery bounds and OSD-2 vs exact decoding

1. Theorem 1 produces non-vacuous bounds in representative experimental settings.

We evaluate the recovery lower bound from Theorem 1 (Sec. 4.2, Eq. (6)) using the measured p,A,Bp,A,B for each text, with s=1s=1, a 32-bit payload, a 64-bit codeword (dmin⁡=10d_{\min}=10), a 512-token generation budget, and 5% token-level mixed edits. For each model, the table below compares the mean lower bound with exact-message recovery on the same texts from ASAP Essay and C4 RealNewsLike.

ModelMean evaluated recovery lower boundEmpirical recovery
Llama-3.1-8B21.92%95.80%
Gemma-2-9B19.31%98.73%

At the 5% edit rate, the mean lower bound is about 20% for both models, while both recover more than 95% of messages. Although conservative at this edit rate, the bound provides insight into the factors that make recovery easier or harder. As discussed in Appendix E.3, its effective length LeffL_{\mathrm{eff}} depends on how many tokens still carry the watermark (AA), how many contribute only noise (BB), and how often bits are flipped in the surviving watermark signal (pp). Across the conditions in Table 9, larger LeffL_{\mathrm{eff}} was associated with higher exact-message recovery (Spearman ρ=0.976\rho=0.976).

Two steps in the proof help explain why the numerical lower bound is conservative.

First, Lemma 4 uses Hoeffding’s inequality for each competing codeword. It bounds the probability that the competitor scores at least as well as the true codeword. A Chernoff bound tailored to the flip probabilities of preserved and noisy bits could sharpen this estimate.

Second, Theorem 1 applies a union bound over all competing codewords. Two competitors can differ from the true codeword at some of the same coordinates, so the same flipped bits can favor both. The union bound counts these overlapping events separately.

Combining sharper pairwise estimates with an analysis of how often these errors overlap would require a more detailed treatment of the codeword structure. We leave this tighter finite-length analysis of message recovery to future work.

References

[1] Xiaoyan Feng, He Zhang, Yanjun Zhang, Leo Yu Zhang, and Shirui Pan. BiMark: Unbiased Multilayer Watermarking for Large Language Models. ICML, 2025.

[2] Ruibo Chen, Yihan Wu, Junfeng Guo, and Heng Huang. Improved Unbiased Watermark for Large Language Models. ACL, pp. 20587–20601, 2025.

AuthorsOpenReview ↗

Response to Reviewer ptJV (2/2)

2. OSD-2 versus exact decoding near the recovery-failure boundary.

Table 8 of the submitted paper reports 100% recovery with both OSD-2 and exact ML. To compare them near the recovery boundary, we re-decode 16-bit, 64-token records from Llama-3.1-8B and Gemma-2-9B on ASAP Essay and C4 RealNewsLike. The results below cover near-total failure through high recovery.

StrengthOSD-2 recoveryExact ML recoveryOSD reaches the ML-optimal objective
s=1.087.55%87.55%99.95%
s=0.872.66%72.56%99.85%
s=0.634.67%34.67%99.80%
s=0.46.84%6.88%99.56%
s=0.20.20%0.20%99.61%

The recovery gap is at most 0.10%, and OSD-2 reaches an ML-optimal objective in at least 99.56% of cases at every strength.

We additionally test 24-bit messages to examine a higher payload while keeping exact ML tractable (32-bit decoding is computationally impractical for exact recovery). We generate new outputs with Llama-3.1-8B on C4 RealNewsLike at s=1s=1, using BCH(48,24), and decode the first 64 tokens with identical soft counts. We observe a similar conclusion.

PayloadOSD-2 recoveryExact ML recoveryOSD reaches the ML-optimal objective
24 bits77.54%78.71%99.02%

Q3. Methodological distinction from ArcMark

ArcMark focuses on distortion-free channel capacity and rate analysis [3], whereas UniCoMark prioritizes robust finite-length message recovery under textual edits. These distinct objectives motivate different design choices in codeword construction, evidence aggregation, and decoding. The details of our construction and theoretical analysis are provided in Secs. 3–4 and Appendices B–C.

Edit synchronization. Both methods regenerate shared randomness from surviving local contexts. ArcMark scores context-dependent random-linear-code symbols for each candidate message [3, Sec. III and Algorithm 2]. In contrast, UniCoMark aggregates repeated binary observations into a fixed-length reliability vector for an explicit codeword. The codeword constraints remain fixed as edits change the available evidence, enabling structured ECC recovery from surviving observations with unequal reliability. Our A/BA/B analysis further quantifies how insertions, deletions, and replacements reduce useful evidence or introduce noise.

Sampling objective. ArcMark minimizes angular distance on a circle under the token-marginal constraint [3, Eq. (5)]. UniCoMark instead derives its Hamming transport cost from a one-step recovery-margin loss under the uniform competing-target assumption (Appendix C), subject to the host-marginal constraint. Our sampling objective is therefore directly aligned with the binary evidence used in codeword recovery, with the resulting flip rate explicitly incorporated in our recovery theorem.

Decoding scalability. ArcMark relies on exhaustive search over all 2r2^r possible messages [3, Algorithm 2], which quickly becomes intractable as the payload grows. UniCoMark instead summarizes token observations into a fixed-length reliability vector and applies structured decoding (OSD-2), reducing the search space from exponential to quadratic (1+r+(r2)1+r+\binom{r}{2} for order 2, Appendix B, Eq. (21)). For a 32-bit payload, this cuts the candidate list from 4.3 billion down to just 529. ArcMark reports payload evaluations up to 32 bits [3, Sec. 5.1], while our payload-scaling experiments already extend to 256 bits (Fig. 4(a)).

References

[3] Atefeh Gilani, Sajani Vithana, Carol Xuan Long, Oliver Kosut, Lalitha Sankar, and Flavio P. Calmon. ArcMark: Distortion-Free Multi-Byte LLM Watermark via Optimal Transport. arXiv preprint arXiv:2602.07235v2, 2026.

Reviewer_vEkpOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes UniCoMark, a multi-bit watermarking scheme for large language models, which co-designs two tightly coupled components: an embedding–extraction process for reliable watermark message recovery and a matched sampling process for text generation. It formulates the sampling process as an optimal transport problem for a fixed-length linear code with soft-information decoding and a sampling rule tailored to the recovery objective.

Critical strengths
  1. The presentation of the paper is good, and the writing is easy to follow.
  2. The core idea of co-designing an embedding–extraction process for reliable watermark message recovery and a matched sampling process for text generation is reasonable. Formulating the sampling process with unbiased constraints as an optimal transport problem sounds novel.
Critical weaknesses
  1. The claim of co-design of the embedding and extraction processes is not sufficiently supported. For every multibit watermarking method, these two parts should be designed adaptively for each other.
  2. The computational overhead should be reported.
Questions for authors
  1. Does watermark detection of UniCoMark require accessing the output distribution of LLMs?
4 comments
AuthorsOpenReview ↗

Response to Reviewer vEkp

We thank the reviewer for recognizing the technical soundness of UniCoMark and the motivation for matching sampling to message recovery.

W1. What distinguishes UniCoMark's co-design from matching embedding and extraction?

We agree that multi-bit watermarking requires embedding and extraction to be matched. In SegMark, for example, the extractor counts the green-list tokens favored by the sampler. Methods of this kind specify how the two procedures work together, but they do not derive the sampling objective and the robustness bound from a common recovery criterion.

UniCoMark’s co-design contribution is this mathematical connection. We formulate recovery of the full message using a single additive decoding metric (Sec. 3.1, Eq. (1)). For a candidate message mm, the decoder sums the Hamming mismatches between each observed token label xix_i and its predicted label yi(m)y_i(m).

D(u_{n:L};m)=\sum_{i=n}^{L}d_H\\!\left(x_i,y_i(m)\right). \tag{1}

First, the metric gives the sampling objective. Each token contributes a Hamming mismatch to the score of a candidate message. Minimizing the expected contribution for the embedded message while preserving the host marginal gives the per-token optimal transport problem (Sec. 3.3, Eq. (3)).

\pi^\star &= \arg\min_{\pi}\sum_{y,x}\pi(y,x)d_H(y,x) \\\\ \text{s.t.}\quad &\sum_x\pi(y,x)=P_{Y_i\mid m}(y), \\\\ &\sum_y\pi(y,x)=P_{X_i}(x). \end{aligned} \tag{3}$$ **Second, the same metric gives the robustness analysis.** After text edits, we use the difference in scores between the true message and each competing message to bound the probability of recovery failure. The expected mismatch minimized by the sampler determines the average bit-flip rate $p$ over the $A$ positions that retain the watermark relation. Each token label contains $k$ bits (Sec. 4.1, Eq. (5)). $$p_i=\frac{1}{k}\\,\mathbb E_{\pi_i^\star}\\!\left[d_H(Y_i,X_i)\right],\qquad p=\frac{1}{A}\sum_{i=1}^{A}p_i. \tag{5}$$ Under its routing and noise assumptions, Theorem 1 shows how $p$, together with the numbers of informative positions ($A$) and noisy positions ($B$) after editing, governs exact-message recovery (Sec. 4.2, Eq. (6)). For an $r$-bit message and a code with relative minimum distance $\Delta_{\min}$, the effective length is $L_{\mathrm{eff}}=k(1-2p)^2 A^2/(A+B)$ (Appendix E.3, Eq. (38)), and the error probability satisfies $$\Pr\\!\left(\widehat m_{\mathrm{exact}}\ne m\right)\le (2^r-1)\exp\\!\left(-\tfrac12\Delta_{\min}L_{\mathrm{eff}}\right). \tag{6}$$ ### W2. Generation overhead Table 10 in Appendix I.1 of the submitted paper reports generation speed (tokens/s) with and without watermarking at batch sizes 1, 4, and 8. The table below gives the wall-clock time to generate 128 tokens per prompt for an entire batch, with and without UniCoMark. It includes timings for Qwen at batch 128 and Llama at batch 224. We use the same NVIDIA RTX 5080 setup, 200 Sinkhorn iterations, top-k=64, and temperature 1.0. | Model | Batch | Generation time (no watermark) | Generation time (UniCoMark) | Slowdown | |---|---:|------:|------:|---:| | Qwen3-4B | 1 | 2.2146 s | 2.3789 s | 6.91% | | Qwen3-4B | 8 | 2.8592 s | 3.0273 s | 5.55% | | Qwen3-4B | 128 | 8.3566 s | 8.6361 s | 3.24% | | Llama-3.1-8B, 4-bit weights | 1 | 1.8224 s | 1.9918 s | 8.50% | | Llama-3.1-8B, 4-bit weights | 8 | 6.5370 s | 6.7365 s | 2.96% | | Llama-3.1-8B, 4-bit weights | 224 | 18.9483 s | 19.3838 s | 2.25% | Timings include CUDA synchronization and exclude model loading and warmup. Slowdown is the percentage decrease in tokens generated per second. UniCoMark’s OSD-2 decoder runs entirely on CPU and takes 4.49 ms on average to decode a 32-bit message from a 256-token text using a single thread on an AMD Ryzen 9 9900X. ### Q1. Detection without the host LLM distribution **No, UniCoMark detection does not require the LLM's output distribution.** This avoids additional LLM inference overhead and enables detection even when the original prompt is unavailable.
AuthorsOpenReview ↗

Follow-up on Your Feedback and Further Clarifications

Thank you for reviewing our paper. We noticed that your score changed from 3 to 2, and we would appreciate it if you could clarify which concerns led to this change. Could you also let us know which aspects of our work or rebuttal would benefit from further explanation? We would be happy to provide additional clarification.

Area_Chair_ADRGOpenReview ↗

Request for Clarification on Score Change

Dear Reviewer vEkp,

Please explain the change in your score from 3 to 2 and respond to the authors’ follow-up, clarifying any unresolved concerns. With the discussion deadline approaching (Nov 18, AoE), please do so promptly to allow the authors time to respond.

Best,

AC

Thank you to the authors for their clarifications. Most of my concerns have been addressed. This is a very well-presented paper. However, I find that the new knowledge it contributes to the field is relatively limited. I also feel that my initial rating was somewhat high relative to the overall scoring at this conference. Therefore, I have adjusted my overall rating accordingly.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Summary of revisions to UniCoMark

We thank the reviewers for their feedback. We have highlighted the revisions in red in the manuscript.

  • Routing and ECC ablation (Sec. 5.1, Table 6). We expanded the ablation to six configurations. For both BiMark and OT sampling, we compare concentrated and uniform routing without ECC, then measure the contribution of ECC with uniform routing held fixed.

  • XMark baseline (Secs. 5.1–5.2). We added XMark (δ=2\delta=2) to the evaluations of message recovery, text quality, and robustness under editing and paraphrasing (Tables 2–4, Figure 5, and Table 7).

  • Downstream recovery and text quality (Sec. 5.1 and Appendices I.5–I.6). We clarified the downstream quality evaluation in Table 4 and added 4-bit message recovery results for summarization and translation (Table 22). We also evaluated the same Gemma-2-9B outputs with two PPL evaluators, Qwen3-14B and Gemma-2-27B (Table 21).

  • Decoding, recovery bounds, and runtime (Sec. 5.3). We compared OSD-2 with exact ML for 16-bit messages (Appendix B, Table 9) and evaluated the recovery bound from Theorem 1 against observed recovery after editing (Appendix E.3, Table 11). Appendix I.1 reports generation and Sinkhorn timings (Table 13) and CPU decoding time.

  • Relation to ArcMark (Sec. 6). We clarified that ArcMark focuses on channel capacity, while UniCoMark focuses on recovering the full message after textual edits. We also compared how their decoders scale with payload length.

AuthorsOpenReview ↗

Second-round manuscript revisions

We updated the downstream quality and recovery results (Sec. 5.1 and Appendix I.6) and clarified the scope of Theorem 1 (Sec. 4.2). The revisions are highlighted in blue.

Dear Reviewers,

As the discussion deadline (Nov 18, AoE) approaches, please review the authors' rebuttal if you have not done so, and post a follow-up indicating whether your main concerns have been addressed or whether any issues remain.

Best,

AC

AuthorsOpenReview ↗

Request for AC's Consideration

Dear Area Chair,

We believe we have sufficiently addressed the reviewers' concerns during the rebuttal. Reviewer pQ44 increased their score from 2 to 3. However, Reviewer vEkp lowered their score from 3 to 2 without providing further comments or responding to our clarification.

We would greatly appreciate your consideration of this situation in your assessment of our submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.