- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TANGO, a watermarking method for masked diffusion language models that uses a secret vocabulary coloring and modular checksum relations between token pairs. The method guides token selection using already-unmasked neighbors, requires no left-to-right generation order, and does not need to reconstruct the generation order during detection.
Critical strengths
- The method naturally accommodates out-of-order generation, and both its generation and detection mechanisms are clearly described.
- The evaluation covers two masked diffusion language models and includes editing and forgery attacks, with useful calibration diagnostics and ablation studies in the appendix.
- The authors discuss the method's limitations candidly, including failures under deletion, paraphrasing, and pair-based attacks, which helps clarify its applicability.
Critical weaknesses
-
Missing comparison with the closest baseline. Section 5 identifies LR-DWM [1] as the closest work but omits it from the experiments. Both methods use already-unmasked neighbors and logit biasing, so a direct comparison is needed to establish gains in detection, editing robustness, or forgery resistance.
-
Limited applicability of Theorem 1. Assumption U replaces tap-dependent favored colors with independent uniform draws, removing the pairwise coupling and reducing the detection margin to zero (Appendix A.1). The authors should clarify which frequency-hiding conclusions remain valid under the actual tap dependence and distinguish these guarantees from empirical observations.
-
Unclear scope of resistance to forgery. Pair-chaining forgeries succeed in 74% of cases with 200 collected texts, and the language-model-based attack reaches 55% at the stronger bias (Tables 9–10). Since all pairs share one coloring, the O(V²) pair-counting argument may not capture the difficulty of joint color inference.
Questions for authors
-
Comparison with prior work: Under matched decoding settings and comparable text quality, how does TANGO compare experimentally with LR-DWM [1] in detectability, robustness to editing, and resistance to forgery? Please also discuss its relationship to [2][3] in design, security objectives, and applicability.
-
Theorem 1: Which conclusions remain valid when the actual dependence of the favored color on the tap's color is retained? Please clarify the implications for token-frequency shifts in the implemented algorithm and identify which conclusions still rely on empirical evidence.
-
Scope of resistance to forgery: Considering approaches that jointly recover hidden classes [3], could the shared color structure reduce the learning difficulty? Does the O(V²) argument apply specifically to the evaluated pair-counting attack, or does it also constrain attacks that exploit shared structure? Please clarify the scope of the security conclusions accordingly.
[1] Raban, O., et al. [LR-DWM: Efficient Watermarking for Diffusion Language Models][1]. Findings of the Association for Computational Linguistics: ACL 2026, 43510–43517, 2026.
[2] Shen, H., Huang, B., and Wan, X. [Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attacks][2]. Advances in Neural Information Processing Systems, 38, 2025.
[3] Zhang, Z., et al. [Large Language Model Watermark Stealing With Mixed Integer Programming][3]. arXiv preprint arXiv:2405.19677, 2024.
6 comments
W3 and Q3. "Considering approaches that jointly recover hidden classes [3], could the shared color structure reduce the learning difficulty? Does the O(V²) argument apply specifically to the evaluated pair-counting attack [...]? Please clarify the scope of the security conclusions accordingly."
Yes, the O(V²) argument applies to the pair-counting attack (, Page 8). Following your comment, we added a joint-recovery attack. The attack recovers the shared coloring from all observed token pairs at once, without the key. We ran three variants, one of them based on the mixed-integer program of the paper that you mentioned, and we report the most successful one. In the evaluated setting, the attacker uses the recovered coloring to generate text that imitates the watermark. A forgery passes if the detector accepts it at 1% FPR.
This attack is successful when the attacker can collect a large amount of watermarked text under a single key, so rotating keys limits its effectiveness. With as few as eight keys, at most 7% of its forgeries pass.
TANGO's detector checks a text against every key. Here, we assume that the attacker knows which key produced each text, which is the best case for the attacker (, Page 41):
| Number of keys | 1 | 2 | 4 | 8 |
|---|---|---|---|---|
| Forgeries that pass, 38,400 watermarked tokens | 62% | 27% | 9% | 5% |
| Forgeries that pass, 115,200 watermarked tokens | 92% | 75% | 42% | 7% |
An attacker who cannot tell which key produced each text does worse. With four keys, at most 5% of its forgeries pass.
With 32 colors instead of 3, TANGO roughly halves this attack's forgery success rate under a single key at the same text quality and is more robust to deletion (, Page 9, and , Page 45).
Regarding the high success rates cited for pair chaining (Table 9), we note that they were achieved only with token chains whose median perplexity is above 1000. The 55% success of the language-model attack (Table 10) is also achieved only at a perplexity 61% above that of unwatermarked text. Within a reasonable perplexity of 1.2× that of unwatermarked text, at most 6% of this attack's forgeries pass (, Page 41).
(Page 10) now states the scope more clearly.
Dear Reviewer BjG4,
As the discussion deadline approaches, we wanted to follow up to ensure our responses have adequately addressed your concerns. If there are any additional questions or points for discussion, we would be happy to address them promptly.
We look forward to your feedback and sincerely appreciate the time and effort you put into reviewing our work.
We thank the reviewer for the helpful suggestions. We are glad that you find that the method "naturally accommodates out-of-order generation" and that its generation and detection are "clearly described". We respond to the weaknesses (W) and the questions (Q) below.
W1 and Q1. "Under matched decoding settings and comparable text quality, how does TANGO compare experimentally with LR-DWM in detectability, robustness to editing, and resistance to forgery?"
We ran this comparison, and it is now in the main text (Table 2 and Sec. 4.4, Page 9). Under matched decoding settings and at comparable text quality, TANGO detects more texts than LR-DWM, both unedited and after synonym substitution and insertion.
We implemented LR-DWM from its paper. Both methods run on LLaDA with the same decoding settings and the same 200 prompts, and both detectors are set to a 1% FPR on human text. The table shows the perplexity of the generated text (12.9 without a watermark) and the TPR on unedited text, after replacing 30% of the tokens with synonyms, and after inserting 20% new tokens. δ and β are the watermark strengths. Each pair of rows has similar perplexity.
As the table shows, TANGO is more robust than LR-DWM to both edits. On resistance to forgery, an attacker who counts token pairs forges the two methods at similar rates (Table 28, Page 44), and forgeries that preserve the quality of the text pass the detector in at most 6% of attempts.
Q1. "Please also discuss its relationship to [2][3] in design, security objectives, and applicability."
The revision now discusses SEEK and the mixed-integer programming attack (Sec. 5, Page 10, and Appendix G, Pages 45 and 46). SEEK has the same security objectives as TANGO, which are to resist both scrubbing and spoofing. SEEK is designed for left-to-right generation. It keys each token on the tokens before it, which a masked diffusion model may not have generated yet. TANGO keys each token on one already generated neighbor, so it applies to masked diffusion models. The mixed-integer programming attack recovers the hidden token classes of a watermark jointly. We ran an attack of this kind against TANGO and report it under W3 below.
W2 and Q2. "Which conclusions remain valid when the actual dependence of the favored color on the tap's color is retained? Please clarify the implications for token-frequency shifts in the implemented algorithm and identify which conclusions still rely on empirical evidence."
The main conclusion is unchanged. An attacker cannot forge TANGO's watermark from token frequencies. For the idealized sampler, Theorem 1 concludes that the watermark does not change expected token frequencies and that a token's frequency does not reveal its color. Our measurements show that both conclusions remain approximately valid for the implemented algorithm, even when accounting for the dependence. An attacker who guesses each token's color from its frequency does barely better than chance against TANGO (Table 8, Page 32):
We retitled Theorem 1 "Frequency cancellation under an idealized sampler" and state its scope in Sec. 3.4 (Page 6). Table 24 (Page 43) lists which results are proved and which are measured.