acceptodds
Under review as a conference paper at ICLR 2027

TIDEmark: Joint Token Watermarking for Diffusion Language Models via Multi-Marginal Optimal Transport

Abstract

Watermarks for autoregressive language models typically operate one token at a time. Diffusion language models (DLMs) instead generate text through successive steps that update multiple token positions in parallel. Many existing methods adapt side-information generation to DLMs while leaving the other watermarking components similar to their autoregressive counterparts. In this work, we exploit parallel token generation to watermark multiple tokens jointly, aiming to strengthen detection without changing their individual distributions. To construct a watermark over several token distributions at once, we leverage multi-marginal optimal transport and introduce TIDEmark (**T**ransport-based **I**nteraction for **D**istortion-free **E**mbedding of water**mark**s). TIDEmark coordinates token choices using token-pair watermark scores, and we prove an ideal distortion-free guarantee for the recorded token marginals. We make joint watermarking computationally tractable through a tree-structured interaction graph, combining belief propagation (BP) with the Sinkhorn algorithm for entropy-regularized transport. For fixed candidate-set sizes and a fixed number of side-information states, the computational complexity per solver pass scales linearly rather than exponentially with the number of jointly watermarked tokens, matching token-level optimal transport’s dependence on token count. Experiments on summarization, question answering, and code generation show that TIDEmark can strengthen detection. TIDEmark results suggest that when the watermark is applied can affect the balance between generation quality and watermark detection.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
text watermarking, diffusion language models, joint watermarking, multi-marginal optimal transport, distortion-free watermarking

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_FDnmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TIDEmark, which jointly watermarks tokens in diffusion language models through multi-marginal optimal transport, using tree structure to reduce computational cost. Experiments demonstrate competitive detection–quality trade-offs in selected settings.

Critical strengths
  1. Joint watermarking is well motivated by the parallel generation capabilities of diffusion language models.
  2. Combining tree structure with Sinkhorn and belief propagation avoids exponential enumeration of token combinations.
  3. The evaluation covers two models and multiple tasks, reporting both detection performance and generation quality.
Critical weaknesses
  1. Differences in temperature and decoding settings across methods, together with the lack of controlled ablations, make it difficult to attribute the observed gains specifically to the joint watermarking mechanism.
  2. Detection thresholds are selected and evaluated on the same clean cohort, without independent validation of false-positive rates. Consequently, the generalizability of the reported TPR at fixed FPR remains unclear.
  3. Robustness is evaluated only for the Chain configuration on a single model under random edits, without matched baseline comparisons. Resistance to more realistic attacks, such as meaning-preserving paraphrasing, remains insufficiently established.
Questions for authors

See above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer FDnm (1/2)

Thank you for your suggestions. We have added temperature sweeps and a resampling control, tested whether detection thresholds transfer across datasets, and expanded robustness comparisons to additional baselines, topologies, models and rewriting attacks. Below, we explain what these experiments show and their limitations.

W1

Differences in temperature and decoding settings across methods, together with the lack of controlled ablations, make it difficult to attribute the observed gains specifically to the joint watermarking mechanism.

The resampling control shows that quality loss occurs even without watermark rewards or coupling fitting. On LCC with LLaDA-Instruct, we redraw each token independently from the saved predictions after its block is complete. Code similarity decreases by 2.94 points under this control, compared with 2.69 points for block-ready TIDEmark. Thus, the observed loss cannot be attributed entirely to the joint watermarking mechanism.

We also compared KGW, Gumbel, HW and TIDEmark at temperatures of 0.3, 0.4, 0.5 and 0.6 under token-commit and block-ready insertion, giving 32 configurations. Other settings remain fixed within each method's sweep. Temperature changes the distributions supplied to fitting for HW/TIDEmark, but changes sampling for KGW/Gumbel. Higher temperatures increased detection rates, but also increased cross-entropy across all four methods. These sweeps show how detection and quality change with temperature; they do not make the methods' decoding procedures identical.

Manuscript changes: Appendices C.3 and C.6, Table 7 and Figure 6‾\underline{\text{Appendices C.3 and C.6, Table 7 and Figure 6}}.

Deferred-resampling control on LCC (Table 7)

ConfigurationCE ↓\downarrowCode similarity (%) ↑\uparrow
Native generation0.70458.0
Independent deferred resampling1.27555.0
Block-ready TIDEmark with chain1.28055.3

CE is cross-entropy in nats per token; lower values are better.

W2

Detection thresholds are selected and evaluated on the same clean cohort, without independent validation of false-positive rates. Consequently, the generalizability of the reported TPR at fixed FPR remains unclear.

We added a test with separate calibration and evaluation datasets. It shows that a cutoff targeting 5% false positives on one dataset does not necessarily retain that rate on another.

For block-ready TIDEmark with chain and block-index states on LLaDA-Instruct, we select the cutoff using one dataset's clean texts, then apply it unchanged to another dataset's clean and watermarked texts. Across these transfers, false-positive rates range from 1.5% to 10.1%. The largest increases occur on LCC. For example, the Finance QA cutoff gives 10.1% false positives on LCC, compared with 4.8% using LCC's own cutoff.

These results provide independent cross-dataset evaluation, but do not establish a universal 5% operating point. Entries using the same dataset for calibration and evaluation, and the combined-dataset row, reuse calibration texts and are not independent validation.

Manuscript changes: Appendix C.4, Table 8‾\underline{\text{Appendix C.4, Table 8}}.

Detection-threshold transfer (Table 8)

Entries are false-positive rate / true-positive rate (%).

Cutoff calibrated on clean text fromHumanEvalLCCFinance QAMulti-News
HumanEval4.9 / 26.29.0 / 62.82.5 / 72.54.0 / 61.0
LCC2.4 / 15.94.8 / 52.11.5 / 62.51.5 / 52.5
Finance QA5.5 / 28.010.1 / 64.95.0 / 75.05.0 / 63.5
Multi-News5.5 / 28.09.6 / 64.94.0 / 75.05.0 / 63.5
All four datasets pooled4.3 / 26.29.0 / 62.82.5 / 72.54.0 / 61.0
AuthorsOpenReview ↗

Response to Reviewer FDnm (2/2)

W3

Robustness is evaluated only for the Chain configuration on a single model under random edits, without matched baseline comparisons. Resistance to more realistic attacks, such as meaning-preserving paraphrasing, remains insufficiently established.

We expanded the experiments beyond one topology, one model and random edits. The new comparisons include LR-DWM, DMark and dgMARK on Finance QA and Multi-News, all three TIDEmark topologies on Finance QA, and a comparison with DMark on a second model, LLaDA2.1-mini.

We also added BERT word substitution, English-German-English back-translation and Qwen rewriting. The severity sweeps cover random deletion, insertion and substitution at 10%, 20% and 30%, and BERT word substitution at 10%, 20%, 30% and 50%.

These additions let us compare robustness across methods, topologies and models, including attacks beyond random edits. Tables 11 and 12 report detection rates and the corresponding generation-quality costs, respectively. Each detector's cutoff is selected on its unedited clean texts and kept fixed under attack. Quality changes are reported separately for each tested configuration, including those using post-fit tilting.

Manuscript changes: Appendix C.10, Tables 11-12 and 16-17, Figure 11‾\underline{\text{Appendix C.10, Tables 11-12 and 16-17, Figure 11}}.

Detection under text attacks on LLaDA-Instruct (Table 11)

Detection rates (%) at cutoffs targeting 5% FPR, held fixed under attack. Deletion, insertion, substitution and BERT substitution use 20% editing.

Co denotes token-commit; Ch denotes block-ready chain. All TIDEmark configurations use previous-block MinHash. For token-commit and block-ready chain, +tilt uses strengths of 0.25 and 0.5, respectively. Rows marked †\dagger use stronger tilting, with strengths of 0.5 and 1.0, respectively.

Finance QA

MethodDeletionInsertionSubstitutionBERT substitutionBack-translationRewriting
LR-DWM21.031.025.026.523.52.5
DMark91.590.582.586.071.015.5
dgMARK21.030.533.032.530.010.5
TIDEmark (Co)55.057.055.061.536.517.0
TIDEmark (Co+tilt)58.562.561.065.547.019.5
TIDEmark (Co+tilt)†^\dagger61.059.564.569.547.024.0
TIDEmark (Ch)22.521.524.531.016.04.5
TIDEmark (Ch+tilt)38.041.539.051.022.59.5
TIDEmark (Ch+tilt)†^\dagger63.061.558.571.035.510.5

Multi-News

MethodDeletionInsertionSubstitutionBERT substitutionBack-translationRewriting
LR-DWM20.529.522.028.520.51.0
DMark82.583.079.581.070.011.0
dgMARK30.035.546.548.026.014.5
TIDEmark (Co)33.034.038.537.520.08.0
TIDEmark (Co+tilt)47.048.058.056.527.013.0
TIDEmark (Co+tilt)†^\dagger55.065.067.564.027.510.0
TIDEmark (Ch)28.026.531.034.015.59.5
TIDEmark (Ch+tilt)57.056.559.557.524.58.0
TIDEmark (Ch+tilt)†^\dagger75.578.583.578.534.59.5

Detection under text attacks across topologies: Finance QA (Table 16)

Detection rates (%) at cutoffs targeting 5% FPR. All three configurations use block-index states without tilting.

Attackchainbalanced treemiddle-star
Unedited74.049.541.5
Deletion 20%33.013.09.5
Insertion 20%33.59.07.0
Substitution 20%55.029.530.0
BERT word substitution 20%54.032.522.0
Back-translation30.58.59.5
Rewriting12.07.07.0

Thank you again for your thoughtful comments and helpful feedback. We have responded to the points you raised and updated the manuscript. We would be happy to know if there is anything left we can further clarify.

Reviewer_TXCkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a TIDEMark, a watermarking scheme for diffusion language models. It extends the HeavyWater scheme (designed for auto-regressive language models) to the diffusion setting. In particular, the authors find that with diffusion language models, because multiple tokens are decoded at once, the watermarking scheme can use the joint distribution of multiple tokens to increase its strength. They therefore suggest solving a multi-marginal optimal transport problem that maximizes the watermark score across the joint token distribution of a diffusion block. To tackle computational complexity, they restrict the dependency to specific graph structures. Their results suggest that for block-ready generation TIDEMark achieves a better detectability-quality trade-off than prior works.

Critical strengths
  • Diffusion language model (DLM) watermarking is an important challenge to tackle given the recent regulatory requirements on LLM watermarking. In particular, most prior works are not distortion-free watermarking schemes, despite distortion-free schemes being the one chosen by the frontier-lab in practice to watermark their model (e.g. SynthID and TextGrain). The proposed method, based on HeavyWater and using OT for watermarking, is novel with multiple new components to make OT-based watermarks work in the diffusion setting (using MOT instead of OT, and using graph dependency to lower computational complexity).
  • The experimental evaluation is mostly complete: TIDEMark is evaluated on 2 different DLMs against several baselines, with multiple prompt datasets, and with multiple quality metrics. The robustness evaluation however lacks comparison with baselines.
Critical weaknesses
  • The paper is very hard to understand. Some concepts like "block-ready" and "token-commit" generations are not well explained. The intuition behind the choice of the different graph structure (chain, middle star, or balanced tree) is lacking. For the detection, I am still uncertain how the the block structure is recovered at detection (i.e. in equation 7 how can Ek\mathcal{E}_{k} be recovered). Lastly, tree sampling is very sparsely described in the main part of the paper even though it appears to be a key component of the method.
  • The scheme itself has two significant limitations. First, according to Table 6, despite the graph dependency, it still appears to induce significant overhead compare to unwatermarked generation. The main benefit of DLMs is to be fast when generating fast, therefore in this setting watermark overhead is important. The second limitation, more technical, is that the number of "seeds" (i.e. the cardinality of S\mathcal{S}) appears to be very limited in practice. I suspect that this small cardinality increases the issues linked with repeated context across generations [1] (e.g. quality degradation despite distortion-freeness or higher FPR than expected).

[1] Analyzing and Evaluating Unbiased Language Model Watermark, Wu et al., ICLR 2026

Questions for authors
  • Could the authors clarify the unclear concepts listed in the weakness section?
  • There seems to be significant difference between the LLaDA-instruct and LLada 2.1 results. Could the authors explain why this is the case?
5 comments

Thank you for the feedback. Shared updates refer to the general comment.

W1 / Q1: Method clarity.

Thank you for identifying these clarity gaps.

W1.a. Block-ready versus token-commit. Section 3‾\underline{\text{Section 3}} now defines both timings before the joint construction. Block-ready records each position's distribution at its last commit, then watermarks the completed block before generating the next block. Token-commit intervenes during denoising before selected tokens are written, so later updates can condition on watermarked tokens. Shared update 1 explains the guarantee boundaries.

W1.b. Graph intuition. Sections 4.2-4.3 and Figure 4‾\underline{\text{Sections 4.2-4.3 and Figure 4}} now explain these graph choices. Chain uses local neighbor pairs, Middle-star connects positions through the middle token as a hub, and Balanced tree recursively links interval midpoints. All have b−1b-1 edges but reward different token pairs. Trees make BP tractable, while shallower trees permit shorter message-passing depth under a level-parallel schedule.

W1.c. Recovering EkE_k at detection. Section 4.4‾\underline{\text{Section 4.4}} now specifies the public construction. For block-ready detection, the specified completion origin and block length partition the observed tokens. The public topology rule constructs EkE_k from positions, while keyed seeding reproduces the state. Commit-time detection instead scores a public predecessor chain, including prompt/block boundary edges. It neither reconstructs private update graphs nor requires the commit history (Section 4.5, Appendix B.6‾\underline{\text{Section 4.5, Appendix B.6}}). Edited text is scored without original alignment or boundary search. Source-aware recovery analyses are offline diagnostics.

W1.d. Tree sampling. Section 4.3 and Appendix B.4 (Algorithm 5)‾\underline{\text{Section 4.3 and Appendix B.4 (Algorithm 5)}} detail tree sampling. BP computes the root marginal and child-given-parent probabilities for the selected state. Sampling draws the root, then recursively draws each child conditional on its sampled parent, rather than selecting tokens greedily. Selected base configurations sample the fitted law directly, without a post-fit temperature transformation. Shared update 1 covers finite-fitting accuracy.

W2: Practical cost and state cardinality.

W2.a. Generation overhead. We appreciate the reviewer's emphasis on generation overhead and have added synchronized compute measurements in Appendix C.12 and Figure 13‾\underline{\text{Appendix C.12 and Figure 13}}. The measurements cover 50 LCC requests per arm and include generation, fitting and sampling. Token-commit and block-ready Chain take 1.268×1.268\times and 1.085×1.085\times a common pooled native-clean reference. Model loading, warmup, logging, I/O and external scoring are excluded. Block-ready fits once per completed block, whereas token-commit fits during successive updates.

W2.b. State cardinality and repeated context. We thank the reviewer for raising the state-cardinality question and recommending Wu et al. (ICLR 2026). Appendix C.5‾\underline{\text{Appendix C.5}} now discusses the broader reuse issue and examines how state-space size affects generation quality and detection. The state count is distinct from secret-key size and the generation random seed. State/key reuse is not unique to TIDEmark. Increasing state cardinality can reduce accidental collisions between different seeding inputs, but does not prevent reuse for the same input. Table 9‾\underline{\text{Table 9}} varies it from 128 to 2048 on Finance QA. Detection does not improve monotonically, while ROUGE-L varies over a narrower range. Shared update 2 separately reports calibration evidence.

Q2: Differences between the LLaDA models. The reviewer's observation about the differing LLaDA results prompted a clearer discussion of model-specific generation behavior, supported by temperature sweeps (Section 6, Appendices C.1, C.6, Figure 7‾\underline{\text{Section 6, Appendices C.1, C.6, Figure 7}}). The denoisers supply different distributions, while planners determine which positions are coupled and when. LLaDA-Instruct uses low-confidence unmasking with at most two positions per update. Selected LLaDA2.1 Commit/Chain configurations use its native revision-capable adaptive sampler, unmasking/editing thresholds 0.7/0.5 and at most 16 post-refinement steps. The refreshed base operating points prioritize quality preservation (shared update 3). The recipes changed beyond temperature, so their improvements are not attributed solely to temperature.

Robustness-baseline remark under Critical Strengths. Following the reviewer's suggestion, we expanded the robustness evaluation to include competing watermarking methods, with the new comparisons reported in Appendix C.10, Tables 11 and 17‾\underline{\text{Appendix C.10, Tables 11 and 17}}. The LLaDA-Instruct comparisons include LR-DWM, DMark and dgMARK alongside both TIDEmark timings. LLaDA2.1 is compared with DMark (shared update 4). These report protocols, quality costs and frozen unedited-clean cutoffs.

AuthorsOpenReview ↗

Reviewer TXCk: Appendix 1

Generation compute time

Source: Appendix C.12, Figure 13‾\underline{\text{Appendix C.12, Figure 13}}. LLaDA-Instruct on LCC, 50 requests per arm, synchronized generation including fitting and sampling on NVIDIA RTX 6000 Ada GPUs. All methods use the same request-local KV-cache procedure. Model loading, warmup, logging, I/O and external scoring are excluded. Ratios use the common pooled native-clean mean of 56.40 seconds.

MethodCommit-time (s)RatioBlock-ready (s)Ratio
KGW57.491.01954.040.958
Gumbel55.150.97858.501.037
HW57.981.02858.931.045
TIDEmark71.501.26861.181.085

Block-ready TIDEmark fits once per completed block rather than during successive denoising updates. These are generation-compute measurements, not isolated watermark-call timings. Ratios below one do not establish generation speedups and may reflect run-to-run timing variation relative to the pooled native-clean reference.

State cardinality

Source: Appendix C.5, Table 9‾\underline{\text{Appendix C.5, Table 9}}. Finance QA, LLaDA-Instruct, block-index-derived states. Solver input temperatures are 0.6 for commit-time and 0.3 for block-ready Chain. Other settings are fixed within each timing configuration. TPR is at a 5% FPR target. CE is lower-is-better, ROUGE-L and AUC are higher-is-better.

TimingStatesTPR (%)AUCCE (nats/token)ROUGE-L
Commit-time12891.00.9701.76017.5
Commit-time25690.50.9771.83717.2
Commit-time51288.00.9711.78517.5
Commit-time102494.00.9841.85017.3
Commit-time204889.00.9701.73917.2
Chain12864.00.9081.62917.0
Chain25675.00.9071.68416.9
Chain51269.00.9101.74617.1
Chain102467.00.8801.69416.9
Chain204858.00.8811.74217.3

Increasing state cardinality does not monotonically improve detection or quality. State count is not secret-key size. It can reduce accidental collisions between different seeding inputs, but does not prevent reuse for the same input.

I thank the authors for their reply. However, my concerns remain.

  • I have read the authors reply and modifications regarding lack of clarity. However, I find their writing very poor (both in the rebuttal and in their reply) such that I still can't understand clearly their answer. Clarity was the main criticism I had of the paper, and the authors rebuttal doesn't address it at all.
  • This is a minor point: the authors should have highlighted their change to their draft in a different color so that it is easier to see which content has been updated.
  • It seems that the overhead is significant and should be clearly mentioned as a limitation
  • My other concerns have been addressed. Because of the lack of clarity, I will maintain my score
Reviewer_cRstOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TIDEmark jointly watermarks groups of tokens in diffusion language models using multi-marginal optimal transport. Its ideal coupling preserves each fixed input token distribution after averaging over a shared watermark state. A tree of token-pair interactions makes fitting and sampling tractable. Experiments compare detection, generation quality, and runtime across code generation, question answering, and summarization, using both block-ready and commit-time watermarking.

Critical strengths

Jointly choosing tokens is a useful way to exploit parallel generation. The tree construction provides a concrete method for doing this without enumerating all token combinations.

The experiments report both detection and task quality, including functional correctness on HumanEval. The paper acknowledges quality losses and distinguishes its marginal guarantee from preservation of the full text distribution.

Critical weaknesses
  1. The block-ready marginal constraints can force a severe loss of token consistency, regardless of the watermark score or solver. Section 3 records each position's prediction at its last commitment, then Eq. (2) constrains resampling of the completed block to preserve these historical predictions.

Consider a native process that samples T1=BT_1=B uniformly from KK tokens, then copies BB into T2T_2 with probability 1−η1-\eta, assigning each other token probability η/(K−1)\eta/(K-1). It records p1(a)=1/Kp_1(a)=1/K and p2(B)=1−ηp_2(B)=1-\eta, with p2(a)=η/(K−1)p_2(a)=\eta/(K-1) otherwise. For every fixed record R=(p1,p2)R=(p_1,p_2), any ideal output that preserves these marginals satisfies P(X1=X2∣R)≤∑amin⁡{p1(a),p2(a)}=1/K+ηP(X_1=X_2\mid R)\leq\sum_a\min\{p_1(a),p_2(a)\}=1/K+\eta, for 0<η<1−1/K0<\eta<1-1/K. With K=100K=100 and η=0.01\eta=0.01, native consistency is 99%99\%, but every feasible output coupling has consistency at most 2%2\%. All target probabilities are positive. The peak of p2p_2 reveals the original first token, yet the constraint forces that token to be uniform again.

This construction does not refute Eq. (4) or estimate benchmark losses. It shows that even the best feasible coupling cannot avoid the loss in this example. Changing the score, regularization, or tree cannot help. A matched zero-reward resampling control would separate this structural source of quality loss from watermark embedding. Such a control is not explicitly reported.

  1. The analytical null does not match shared-MinHash seeding, and the experimental calibration is insufficiently specified. Eq. (16) assumes independent uniform block states. Appendix C.2 sets sk=g(arg⁡min⁡x∈Bk−1h(x))s_k=g(\arg\min_{x\in B_{k-1}}h(x)), so blocks whose predecessors select the same token have identical states. The formula is correct under its stated null, which excludes this configuration.

Fix a clean text of eight identical two-token blocks (a,a)(a,a) and a score table with f(a,a,1)>f(a,a,0)f(a,a,1)>f(a,a,0). Suppose U=g(a)U=g(a) is uniform over two states and independent of this table. Blocks 2–8 all use UU, so at least seven blocks receive the high score with probability 1/21/2. Eq. (16) assigns a tail probability at most P(Binomial(8,1/2)≥7)=9/256<0.05P(\mathrm{Binomial}(8,1/2)\geq7)=9/256<0.05. Applying it with this seeding would therefore reject this fixed clean text with probability at least 50%50\% over UU, despite a nominal 5%5\% level.

Section 5.1 calls the experimental pp-values empirical without defining their calculation, so this example does not invalidate the tables. Sections 5.1 and C.1 disclose that thresholds are selected and evaluated on the same clean set. These results describe discrimination on those cohorts, not validated false-positive rates on new text. Each experiment's seeding and calibration must be specified. Eq. (16) cannot justify shared-MinHash calibration.

  1. The fixed-input guarantee does not establish marginal preservation during adaptive commit-time generation. The page 5 proposition fixes the predictions before drawing a uniform state. Appendix B.6 reuses the state while later predictions depend on earlier watermarked tokens. Conditional on a later prediction, that state need not remain uniform.

Consider two updates in one block, each coupling a binary token XiX_i with a deterministic token dd. Reuse a uniform S∈{0,1}S\in\{0,1\}, with pair reward f(x,d,s)=1{x=s}f(x,d,s)=\mathbf{1}\{x=s\} and ϵ=0.5\epsilon=0.5. For p1(1)=0.5p_1(1)=0.5, exact fitting gives P(X1=1∣S=1)=a=(1+e−2)−1≈0.8808P(X_1=1\mid S=1)=a=(1+e^{-2})^{-1}\approx0.8808 and P(X1=1∣S=0)=1−aP(X_1=1\mid S=0)=1-a. Let the next recorded probability be p2(1)=0.5p_2(1)=0.5 if X1=1X_1=1, and 0.10.1 otherwise. Observing p2(1)=0.5p_2(1)=0.5 then implies P(S=1∣p2(1)=0.5)=aP(S=1\mid p_2(1)=0.5)=a. The second exact fit still averages to 0.50.5 under a uniform state, but the actual probability is P(X2=1∣p2(1)=0.5)=a2+(1−a)2≈0.7900P(X_2=1\mid p_2(1)=0.5)=a^2+(1-a)^2\approx0.7900.

This discrepancy concerns a single token and remains under exact fitting. The fixed-input theorem is correct, but the paper should explicitly state that it does not give this stronger commit-time guarantee. Extending it requires addressing the state information carried by earlier tokens.

Questions for authors
  1. Does the clean reference include the same block-ready recording and resampling steps? If not, a matched zero-reward control is needed to assess how much quality loss comes from resampling alone.
  2. What pp-value definition, reference sample count, and treatment of zero tail counts produced Table 4's mean −log⁡10p=5.103-\log_{10}p=5.103? Please specify the seeding and calibration rules for the main and robustness results, including any use of Eq. (16).
  3. Is any guarantee beyond the fixed-input proposition claimed for commit-time generation? If so, what additional assumption or implementation detail rules out the counterexample in Weakness 3?
  4. For block-ready TIDEmark, where are the output temperatures 0.10.1 and 0.30.3 applied relative to fitting and Algorithm 5? Which marginals remain preserved, and what errors were measured after the complete fitting and sampling procedure?
  5. What is the score normalization in Appendix C.1? Adding a state-only offset c(s)c(s) leaves the optimal coupling unchanged because its state marginal is fixed, but can change Eq. (16)'s ranking. For one block, sufficiently separated offsets can make this pp-value depend only on the state, even when the coupling strongly links the tokens to that state. Does normalization exclude this failure? This question concerns the analytical detector, not all empirical score tests.
5 comments
AuthorsOpenReview ↗

Response to Reviewer cRst (1/2)

We appreciate your detailed questions and examples. In response to your comments, we added the zero-reward resampling control, clarified what the block-ready marginal-preservation proposition guarantees and why it does not extend to adaptive token-commit generation, and expanded the descriptions of seeding, calibration, and numerical accuracy.

W1 / Q1

The block-ready marginal constraints can force a severe loss of token consistency, regardless of the watermark score or solver.

Does the clean reference include the same block-ready recording and resampling steps?

The zero-reward control you requested supports your concern: quality loss occurs even without watermark rewards. On LCC, independent resampling reduced code similarity by 2.94 percentage points, compared with 2.69 for block-ready TIDEmark. The native reference does not include deferred resampling. The control resamples recorded predictions independently, without rewards or coupling fitting, and later blocks condition on the resampled text.

Preserving each token's recorded probabilities does not necessarily preserve consistency between tokens, because the predictions come from different generation stages. Our marginal-preservation proposition guarantees the recorded probabilities after averaging over watermark states, not the original token dependencies. We clarified this scope and added the control (Section 3.1, Appendix C.3, Table 7‾\underline{\text{Section 3.1, Appendix C.3, Table 7}}).

Deferred-resampling control on LCC (Table 7)

LLaDA-Instruct on LCC. The two resampling configurations use input temperature 0.6 and renormalized top-20 support. Native generation is the reference without deferred resampling. Lower CE and higher code similarity are better.

ConfigurationCE (nats/token)Code similarity (%)
Native generation0.70458.0
Independent deferred resampling1.27555.0
Block-ready TIDEmark with chain1.28055.3

W2

The analytical null does not match shared-MinHash seeding, and the experimental calibration is insufficiently specified.

For practical robustness tests, we use MinHash to derive states from the preceding block's content, inspired by the use of local context in LR-DWM [1] and DMark [2]. However, different blocks can select the same token and reuse the same state. This falls outside Eq. (16)'s independent-uniform-state assumptions. Similar seed reuse can occur in other context-based methods and is not specific to transport coupling. We therefore evaluate this practical variant empirically and choose cutoffs from clean scores, rather than relying on Eq. (16). Developing new seeding strategies is a valuable direction, while our focus here is on transport coupling.

We added a comparison on Finance QA using block-ready chain at δ=1\delta=1. Under 20% substitution, the block-index configuration gave 95.5% TPR versus 58.5% for MinHash, with unedited ROUGE-L losses of 0.87 versus 0.41 points (Appendix C.10.6, Table 18‾\underline{\text{Appendix C.10.6, Table 18}}).

Block-ready chain seeding comparison (Table 18)

LLaDA-Instruct, δ=1\delta=1. Detection entries are TPR (%) at cutoffs targeting 5% FPR on each configuration's unedited clean reference, held fixed under attack. Quality columns are unedited changes: Δ\DeltaROUGE-L is clean minus watermarked, and Δ\DeltaCE is watermarked minus clean.

DatasetSeedingUneditedDeletion 20%Substitution 20%RewritingΔ\DeltaROUGE-LΔ\DeltaCE
Finance QAMinHash94.063.058.510.5+0.41+0.584
Finance QABlock index95.574.595.520.5+0.87+0.698
Multi-NewsMinHash88.575.583.59.5+1.94+0.619
Multi-NewsBlock index98.070.097.518.0+2.36+0.835

W3

The fixed-input guarantee does not establish marginal preservation during adaptive commit-time generation.

The proposition remains valid for fixed recorded predictions: the block-ready coupling preserves each token's marginal after averaging over the prescribed states. Token-commit is an extension intended to address token consistency by allowing later predictions to condition on watermarked tokens.

It does not inherit that guarantee. Earlier watermarked tokens depend on the reused state, so later predictions can carry information about it. The theorem's uniform-state averaging therefore does not establish marginal preservation during adaptive generation, even with exact fitting. We make this distinction explicit (Section 3.1, Appendix B.6‾\underline{\text{Section 3.1, Appendix B.6}}).

References

[1] ORaban, Ofek, Gal Chechik, and Ethan Fetaya. "LR-DWM: Efficient Watermarking for Diffusion Language Models."

[2] Wu, Linyu, Linhao Zhong, Wenjie Qu, Yuexin Li, Yue Liu, Shengfang Zhai, Chunhua Shen, and Jiaheng Zhang. "Dmark: Order-agnostic watermarking for diffusion large language models."

AuthorsOpenReview ↗

Response to Reviewer cRst (2/2)

Q2

What pp-value definition, reference sample count, and treatment of zero tail counts produced Table 4's mean −log⁡10p=5.103-\log_{10}p = 5.103? Please specify the seeding and calibration rules for the main and robustness results, including any use of Eq. (16).

For the value you asked about, we built a reference score distribution over possible watermark-state assignments, keeping the text fixed. Repeated MinHash selections of the same token shared a state. We use this distribution's mean and variance to standardize the observed score. The p-value is the probability of an equal or higher score under the same state model. The calculation does not assume a Gaussian distribution.

In the main experiments, the key and block number determine the watermark state. In the MinHash robustness experiments, it comes from the preceding block's content. Detection cutoffs are selected from matched clean examples. We do not claim that Eq. (16) guarantees calibration for MinHash. The value from your original Table 4 is retained in Table 13 (Appendices C.1 and C.10.3, Table 13‾\underline{\text{Appendices C.1 and C.10.3, Table 13}}).

Q3

Is any guarantee beyond the fixed-input proposition claimed for commit-time generation? If so, what additional assumption or implementation detail rules out the counterexample in Weakness 3?

No. We do not claim a stronger marginal-preservation guarantee for token-commit generation. We introduced it to address the consistency problem that can arise when a completed block is resampled from historical predictions. Instead, watermarking during denoising allows later predictions to use the watermarked tokens. The supporting zero-reward control is included under W1 / Q1.

Our implementation does not rule out your counterexample because later predictions can depend on the reused watermark state through earlier watermarked tokens, as explained in W3 (Section 3.1, Appendix B.6‾\underline{\text{Section 3.1, Appendix B.6}}).

Q4

For block-ready TIDEmark, where are the output temperatures 0.1 and 0.3 applied relative to fitting and Algorithm 5? Which marginals remain preserved, and what errors were measured after the complete fitting and sampling procedure?

We now tune the solver input temperature while keeping the watermarked sampling temperature fixed at 1. Changing the input temperature changes the probabilities supplied to the solver. This is an operating-point choice, analogous to choosing a temperature for ordinary LM generation, but here the adjustment happens before fitting. We then sample directly from the fitted distribution.

The preservation statement concerns these chosen target probabilities, after input-temperature and support adjustment, averaged over watermark states. We measure the largest difference between a target token probability and the final sampler's probability after averaging over states. Table 4 reports the measured mean and maximum errors, including nonconverged fits, together with convergence rates.

Manuscript changes: Section 4.3, Appendix C.1, Table 4‾\underline{\text{Section 4.3, Appendix C.1, Table 4}}.

Numerical accuracy of selected untilted block-ready chain configurations (Table 4)

LLaDA-Instruct. Means and maxima include all fits, including those that did not converge within the 300-iteration budget.

DatasetMean iterationsMean sampling errorMax sampling errorConverged (%)
HumanEval54.628.69×10−68.69\times10^{-6}6.76×10−56.76\times10^{-5}99.55
LCC82.949.97×10−69.97\times10^{-6}2.87×10−42.87\times10^{-4}96.35
Finance QA64.148.41×10−68.41\times10^{-6}1.66×10−41.66\times10^{-4}98.96
Multi-News62.638.37×10−68.37\times10^{-6}1.88×10−41.88\times10^{-4}98.54

Q5

Adding a state-only offset c(s)c(s) leaves the optimal coupling unchanged because its state marginal is fixed, but can change Eq. (16)'s ranking. For one block, sufficiently separated offsets can make this pp-value depend only on the state, even when the coupling strongly links the tokens to that state. Does normalization exclude this failure? This question concerns the analytical detector, not all empirical score tests.

You are right that normalization does not generally exclude this example. We standardize scores across states separately for each token pair, then additionally range-normalize each candidate-pair/state table for fitting. Detection uses the underlying standardized scores.

These operations do not guarantee that the analytical ranking reflects only the relationship between tokens and the state. At fixed reward scale, a state-only bonus can change that ranking without changing the optimal coupling. We therefore do not claim that normalization guarantees detection of every token–state dependence. Detection performance for our chosen score construction is evaluated empirically.

The revised manuscript distinguishes the normalization stages and analytical versus empirical calibration (Section 4.4, Appendices B.5 and C.1‾\underline{\text{Section 4.4, Appendices B.5 and C.1}}).

Thank you again for your thoughtful comments and helpful feedback. We have responded to the points you raised and updated the manuscript. We would be happy to know if there is anything left we can further clarify.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Shared updates in the revised manuscript

Thank you for the detailed reviews. We have clarified the method, expanded the experiments and made the practical limitations explicit. This comment summarizes the manuscript changes. Individual replies explain the results and answer each reviewer's questions.

1. Clearer method explanations and guarantee scope.

We rewrote the explanations of insertion timing, graph choices, fitting, sampling and detection, and added a four-token worked example. We clarified that the ideal coupling preserves fixed recorded token distributions after averaging over watermark states, and not native token dependencies or the full text distribution. This is a fixed-input guarantee for base block-ready TIDEmark: the recorded token distributions are held fixed, and preservation is obtained after averaging over watermark states. We also report achieved fitting and sampling errors. Revised passages are highlighted in blue. Sections 3-4, Appendices B.2, B.4, C.1, Table 4‾\underline{\text{Sections 3-4, Appendices B.2, B.4, C.1, Table 4}}.

2. Temperature, resampling and quality tradeoffs.

We added the following controls and comparisons:

  • Temperature sweeps: across methods, insertion timings and models, specifying where temperature acts in each method.
  • Unwatermarked block-ready resampling control: independently resample tokens from saved predictions after block completion, to examine the quality effect of resampling itself without watermark rewards or coupling fitting.
  • Model comparisons: updated evaluations including LLaDA2.1.
  • Post-fit tilting: examine detection-quality tradeoffs separately from the base coupling and its guarantee.
  • Adaptive sampling: compare EB-sampler with the default low-confidence token-selection procedure.
  • Token-commit graph scope: compare fitting over the selected interval with fitting over the whole live block.
  • State-set size: vary the number of watermark states and examine detection and generation quality.

Appendices C.1-C.3, C.5-C.9, Tables 7 and 9, Figures 6-7‾\underline{\text{Appendices C.1-C.3, C.5-C.9, Tables 7 and 9, Figures 6-7}}.

3. Expanded robustness evaluation.

We expanded the attack experiments in the following directions:

  • Baseline comparisons: LR-DWM, DMark and dgMARK alongside token-commit and block-ready TIDEmark on Finance QA and Multi-News.
  • Graph topologies: block-ready chain, balanced tree and middle-star on Finance QA, without tilting.
  • Second model: TIDEmark and DMark on LLaDA2.1-mini, including base and tilted TIDEmark configurations.
  • Attack types and severity: random deletion, insertion and substitution at 10%, 20% and 30%, and BERT word substitution at 10%, 20%, 30% and 50%. Additional attacks include English-German-English back-translation and Qwen rewriting.
  • Seeding comparison: previous-block MinHash versus block-index states for tilted block-ready chain.
  • Reconstruction diagnostics: state agreement and recovery of the original token-pair connections after editing.

The new method, topology and second-model comparisons keep each detector's unedited-clean threshold fixed under attack. We also report corresponding generation-quality costs. Appendix C.10, Tables 11-18, Figure 11‾\underline{\text{Appendix C.10, Tables 11-18, Figure 11}}.

4. Runtime and release-delay limitations.

We now explicitly discuss both computational overhead and the wait before a completed block can be released. Runtime measurements distinguish block-ready from token-commit insertion and compare block-ready TIDEmark with HW and Gumbel. Appendices C.12-C.13, Figure 13‾\underline{\text{Appendices C.12-C.13, Figure 13}}.

5. Independent calibration checks.

We added a test that selects a detection cutoff on one dataset and applies it unchanged to another. We distinguish these separate-dataset evaluations from tests that reuse calibration texts, and clarify analytical versus empirical calibration. Appendices B.5, C.1, C.4, Table 8‾\underline{\text{Appendices B.5, C.1, C.4, Table 8}}.

AuthorsOpenReview ↗

Summary of Discussions and Revisions

Dear Area Chair and Reviewers,

With the discussion period nearing its end, we thank the reviewers for their constructive feedback. We have uploaded a revised manuscript incorporating the additional results and clarifications from our responses, with changes highlighted in blue. Below, we summarize the main concerns and how we addressed them.

TIDEmark introduces joint token watermarking for diffusion language models using multi-marginal optimal transport. Its tree-structured formulation makes the joint computation tractable, and our experiments examine detection alongside generation quality and task performance.

We are pleased that reviewer TXCk raised their score after our follow-up and now recommends acceptance. Reviewer cRst also raised their score to 3, noting that the revisions sufficiently addressed their concerns. We appreciate both reviewers’ suggestions for further improvements.

Main concerns and revisions

  • Clarity and limitations (TXCk): We rewrote the method explanations, added a worked example, and explicitly discussed computational overhead and block-release delay as limitations.
  • Theoretical guarantee (cRst): We clarified that, for fixed recorded token distributions, the ideal block-ready coupling preserves each token distribution when averaged over the uniform watermark state. We also discussed the advantages and limitations of the proposed watermarking timings.
  • Robustness (FDnm, TXCk): We substantially expanded the evaluation across watermark baselines, graph topologies, models, attack types and severities, including back-translation and rewriting. We report detection alongside generation-quality costs.
  • Controls and comparisons (cRst, FDnm): We added an independent resampling control, temperature sweeps across methods and watermarking timings, and comparisons of model configurations, samplers and watermark settings.
  • Calibration (cRst, FDnm): We added cross-dataset threshold-transfer experiments and clarified the analytical and empirical calibration procedures.

At the time of writing, FDnm has not yet responded to our latest reply. We would be grateful if you could consider our responses and the revised manuscript alongside the reviews when making your assessment.

We hope our responses and revisions address the reviewers’ concerns. We remain available for any further questions during the discussion period.

Thank you for your time and consideration.

Sincerely,
The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.