acceptodds
Withdrawn from ICLR 2027

Fragile Watermark in Masked Diffusion Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

With the rapid development of masked diffusion models, ensuring the integrity and security of their generated content has become increasingly important. Consequently, fragile watermarking has attracted considerable attention as a promising solution. It is designed to determine whether model outputs have been altered by third parties, thereby safeguarding their integrity. In contrast to robust watermarking, which is intended to remain detectable after common transformations such as translation and editing, fragile watermarking is invalidated by any unauthorized modification. It therefore functions similarly to peer-to-peer hashing, although the detector does not need to store a hash value. However, because masked diffusion models generate sequences with non-Markovian dependencies, autoregression-based fragile watermarking algorithms are difficult to apply, leaving this problem largely unexplored. To address this gap, we propose Dandelion: a block-dependent watermarking algorithm that verifies sequence integrity through the Markov states of individual blocks. We enable the minimum possible entropy increase at the watermark-embedding positions, thereby preserving the model’s generative capability. Experiments show that Dandelion achieves a 100% detection rate on unperturbed watermarked text. Under 13 types of perturbations, including substitution, truncation, and deletion, it achieves a watermark invalidation rate exceeding 99%. Our codes are available at: https://anonymous.4open.science/r/Dandelion-DLLM_Fragile_Watermark-32EF

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Diffusion Language Models, Fragile Watermark, AI Safety

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_HXhzOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors of the paper introduce Dandelion, a fragile watermark for blockwise masked diffusion language generation.

Dandelion works as follows: after completing each block, it selects reconstructible carrier positions outside designated exact-content regions, computes a keyed tag from the remaining payload and preceding chain state, and substitutes high-probability carrier tokens that encode the required bits. The next chain state commits both the tag and realized carriers, making earlier modifications affect later checks. Detection requires the text, tokenizer, key, and shared configuration, but not the generation model or original output. The authors of the paper prove conditional completeness and tamper-sensitivity results and evaluate six MDLM backbones on HC3 prompts, including perturbation detection, unwatermarked controls, quality measures, and runtime.

Critical strengths

(1) The construction provides a nice approach to text-only verification of blockwise generated content: Separating payload tokens from carriers avoids direct circularity in tag construction and the process of committing realized carrier identities into the subsequent state makes earlier carrier changes propagate through the chain. Deterministic carrier reconstruction and serialization checks address practical obstacles to detecting from received text without storing the original generation. I believe that these are useful design choices for fragile watermarking.

(2) The paper evaluates more than intact-output detection: It reports perturbation results across six MDLM backbones, varies block sizes and output lengths, and separately evaluates naturally unwatermarked controls. The difference between tamper detection and accidental acceptance of unwatermarked text seems valuable. The appendix also identifies the final-block boundary and tokenization-equivalent edits, helping explain what the detector actually authenticates.

Critical weaknesses

(1) The protection against consequential localized edits is too weak to fully support the integrity motivation: Corollary 6 and Appendix G acknowledge that changing one final-block carrier can pass verification with probability 1/21/2. A final-block payload modification with four reconstructed carriers has only a 2−4=1/162^{-4}=1/16 false-acceptance bound. Since carrier locations are publicly reconstructible, choosing a vulnerable location does not require knowing the key. A final block can contain a fact, negation, or conclusion, so this is consequential even if the aggregate TNR over sampled perturbations exceeds 99%. I appreciate that Appendix G explicitly limits the method to a lightweight tamper-detection signal and does not claim general cryptographic authentication. Nevertheless, the aggregate metric provides limited evidence of protection for the localized edits that most challenge the paper’s integrity use case.

(2) The efficiency and embedding-availability evaluation does not cover important operating conditions. Page 8 states that the sampler reveals one token per denoising step, and Appendix E measures the reported 5.42% overhead under those same settings. Appendix B adds one model-scoring call per carrier, so those calls can represent a substantially larger fraction of generation time under a genuinely parallel reveal schedule. The construction is structurally compatible with such schedules, but low overhead there is not tested. In addition, embedding fails if any nonempty block lacks an eligible carrier outside the excluded code/math regions. A complete block inside a sufficiently long fenced-code region therefore causes failure. The paper does not report embedding success, abort, repair, and retry rates across all attempted outputs in such settings. The 100% intact TPR is conditional on successful embedding and self-verification, and the reported successful-output metrics do not resolve availability.

Questions for authors

(0) The authors discuss some related works on watermarking of discrete diffusion. But there's one work that has pretty good empirical evaluation and that does not have hyperparameter tuning. Can the authors compare to [1]?

(1) Perturbation evaluation: Please specify how many tokens each perturbation changes and how locations are sampled. Can the existing results be stratified into earlier-block edits, final-block payload edits, and final-block single-carrier edits? This would show how the aggregate TNR relates to the boundary probabilities (as acknowledged by the authors).

(2) Parallel-generation efficiency: What latency, quality, and detection results are obtained when multiple tokens are revealed per denoising step? Please include carrier-rescoring and canonical-repair costs in those measurements. This is necessary to understand efficiency in the parallel-generation setting motivating the paper.

(3) Embedding availability: Across all attempted outputs, what are the success, abort, repair, and retry rates, including outputs with substantial code or mathematics? How are these failures included in reported quality and latency? Appendix C’s unwatermarked-control statistics do not imply successful embedding on such outputs.

[1] https://arxiv.org/abs/2511.02083

Reviewer_iWXkOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a fragile watermarking scheme dedicated to large language models based on (masked) diffusion (as opposed to autoregressive models). The embedding is performed by fixing apriori a set of "safe" positions in the text block: a set of legal positions is fixed beforehand (do they depend on a secret, in does not look like it does), and filtered after the text block has been denoised depending on a "blacklist" of tokens. Once a set of positions has been selected, a bit value (which is completely determined by the current state of the process) is embedded by select the most probable token that agrees with this bit value (akin to a hard-KGW, which is never optimal). Once a block is embedded, the state of the process is updated and the embeddding process continues. The main idea is that the reliance of the embedded bit on the previous state enforces the fragility of the mark: a modification destroys not only the signal in its block at detection but all following blocks. The approach is validated empirically on a set of of 5 LLaDa variants, as well as LaVida. However, as I will explain, the experimental design is deeply flawed and does not allow to verify the claims of the paper.

Critical strengths

Without any desire to be needlessly harsh, I was not able to find any substantive strengths, except for the relative novelty of the application of fragile watermarking to masked diffusion.

Critical weaknesses

In its current state, I believe the paper is very far from the rigor and quality expected of an A* conference like ICLR. Both the content and the form of the paper is deeply flawed and I don't believe I will be able to raise my score up to an acceptance, notwithstanding a deep overhaul of the paper. I will now argue my case; if the authors find my assessment unfair, I am of course ready to discuss all of these points. I will divide my points between form and content:

Form

Unclear, ad-hoc terminology: The Related Work section is quite well-written. I was thus quite surprise by Section 3 which contains many undefined, non-standard term -- which to be honest looks a lot like LLM rambling to me. The authors never define what a "strict-heavy" block is (to the best of my knowledge, a strict-heavy block is a term only used within the power-lifting community which is somewhat far from our discipline), neither what a "strict-region" is. I also wonder what a "structure-aware ablation" is when refering to score selection.

Meaningless or outright incorrect terms: Notwithstanding the bizarre terminology, the paper also contains blatant mistakes. Some examples. The author refer to the log-likelihood of a token as its "normalized score" (line 242). The use of the term score is misleading here since it usually refers to the gradient of the log-likelihood in the machine learning community. Pedantism aside, I don't see why it would be a "normalized score". A "high-entropy" key (line 251) does not mean anything, I guess that the author refer to key length, but given that the author never precisely define what is a "high entropy", even my guess does not help me understand the design of the algorithm. Line 284 "where GK is domain-separated from FK and gK" makes no sense to me: the domain of gKg_K is {0,1}\{0,1\} (it outputs a bit value), the output domain of FKF_K is never clearly defined (and I honestly cannot guess it from the paper). As such, what does it mean for GKG_K it to be "domain-separated"?

Content

Flawed and incompletely specified experimental design:
- FPR and TPR: In the main paper, the authors only report the true positive and negative rates of their detector, which is insufficient by istelf: any detector can reach 100% TPR/TNR provided I set my false alarm rate high enough. To be fair, the authors propose a study of the false positive rate of their method on unwatermarked text (both natural and generated) in Appendix C. The test is performed on a thousand samples, and shows, if we are charitable that a FPR of around 10−410^{-4} can be reached (in truth, the standard confidence interval would only guarantee an order of magnitude less, but that does not matter here). The results themselves, however, suspicious: how come, over the total 7000 samples, no re-tokenization error happen? I know from experience that they are far more common than one in a thousand. Also, given the time alloted to describing the fail states of the algorithm in the main paper, I am suprised to see that none of them seem to happen empirically (there is always a carrier position available). In any case, the results themselves are not replicable: the authors never state how the control set prompts were selected from HC3, and if it is distinct from the one used in the main paper experiments, the parameters of the detector (block size? was the text size limited for natural text? What about generated text?). Furthermore, the authors only test for the unwatermarked test, not the "watermarked with the incorrect key case". Also, I would appreciate the authors explain to me what "Both failure columns contain counts and are included in N ; because the detector fails closed, these samples are not counted as false acceptances." mean on line 855-856, which looks like AI rambling to me. - Missing perturbation specification: The authors make a long list of the perturbations used to test the fragility of their watermark. Yet they never actually explain nor define what any of these perturbation actually do, even worse, even if I guess what one does like "character insertion", they never state the parameters of these perturbation. Taking the "character insertion" example: how many characters are inserted? Are there rules to where they can be inserted? Is their a list of allowed characters or not? Etc...Without these explanations, the experiments are useless since we don't know what they actually measure.

Claims never proven: The paper contributions rest on three main claims: a precedence claim ("first fragile watermarking for masked diffusion"), a theoretical claim ("incompatibility of methods from the auto-regressive litterature" ) and finally a performance claim. I see no reason to refute the first. However the second is never mentionned in the paper again, except in the related work where the authors defer to works from Tsur et al...which do not deal with this problem at all (indeed they are paper about classical, robust watermarking)! The only theoretical analysis (in Appendix A), also never refers to it. As such, one of the main claim of the paper is never verified nor discussed and links back to irrelevant litterature. For the third claim, I refer to the previous weakness.

Non-rigorous algorithm specification: After reading the paper, I would be hard pressed to reimplement the proposed approach: FKF_K and GKG_K are never formally defined or exemplified. Even in formal algorithm specification on p15, some procedures are only handwaved. Candidates is meant to "preserves the original leading whitespace and lexical signature", but it is never explained how this is achieved (much less what the lexical signature actually is). Eligibility is more intuitive, and is actually somewhat explained from line 796-802, yet what counts as "stable leading word boundary" and how "code or mathematical regions" are recognized is never explained. The latter point might seem less critical:the Eligibility procedure could have been left for the practionner to define. However since the authors spend quite some time on it in the main paper, and since its design directly impacts the capacity of the scheme, it should be precisely defined for replicability reasons.

What is the purpose of your proofs?: There is an appendix dedicated to "Detailed Proof", which are never refered to anywhere else. What is their purpose then? Theorem 5 would have been interesting to refer to in the paper since it claims an explicit bound on the FPR! It would also have been interested to compare this bound to the one obtained in Appendix C with the actual parameters used at detection. In any case, the theorem itself does not seem to be meaningful since AdvPRFAdv_{PRF} is never defined in the paper to the best of my knowledge. Line 662-666 also don't make any sense to me, hence I cannot understand what M(x)M(x) is; again a formal definition would have clarified a lot of this verbose handwaving.

Questions for authors

In its current state, the paper should be rejected due to its lack of scientific rigor: 1) experiments are incorrectly designed, non-replicable due to many missing elements and parameters, 2) the algorithm itself is not completely specified with gaps in the definitions of its procedures, 3) contains many unclear, non defined terms, undefined notation and functions and 4) contains claims which are never proven.

I won't give false hopes: raising my score under these circumstances will require many modifications and improvements which I doubt are doable during the rebuttal period.

Here are mandatory actions and questions to answer in order to improve my score:

  • Please, make sure all terms, functions, procedure and anything related to the watermarking method possess 1) a correct and formal definition and 2) a clear explanation (not just LLM verbiage)
  • Please, provide a complete specification on how the perturbations were applied to the texts and what these perturbations actually do.
  • Double-check the FPR experiments, complete them with a "wrong-key" experiments, provide a complete specification of the experiment itself.
  • Prove the second claim theoretically and/or empirically or remove it outright from the paper
  • Correct the proofs, notably correct Theorem 5 and demonstrate empirically that the bound matches the observed behaviour (I understand that it might be quite loose since it is a union bound, that's okay)
  • Make captions self-sufficient from Tables and Figures self-sufficient for understanding the figure.
  • Why use a hard-decision in Eq 14? In understand the intuituion but is wrong: the seminal work of Kirchenbauer et al. on the Red-Green list watermarking scheme demonstrates that it is always sub-optimal to choose a token deterministically with respect to text quality compared simply sampling from the distribution itself.
Reviewer_SFeyOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a fragile watermarking method, Dandelion, for masked diffusion language models. It links text blocks through keyed authentication states and embeds watermark bits by replacing selected tokens. The method verifies text without the original output or generation model. Experiments evaluate tamper detection, output quality, and computational overhead across several models.

Critical strengths
  1. The paper addresses an interesting problem: checking whether generated text has been modified.
  2. The blockwise design supports verification without the original text or generation model. It also considers tokenization consistency and avoids using code and mathematical expressions as watermark carriers.
Critical weaknesses
  1. The security evaluation is insufficient. Synthetic random perturbations do not establish protection against deliberate tampering. The paper acknowledges that, under its idealized model, replacing a single final-block carrier while preserving sequence length and carrier layout can pass verification with probability 50%. Missing baselines also make the benefit over simpler methods unclear.

  2. The quality and efficiency claims lack sufficient evidence. MPPL shows a macro-average relative increase of 33.75%, while the CLIP and human evaluations are insufficiently described. Runtime is measured with only one token revealed per step, so the reported low overhead may not hold under faster parallel sampling.

Questions for authors
  1. Given the substantial MPPL increase, what evidence supports preserved output quality? Which CLIP model was used, and how were long texts handled? How was correctness assessed by human evaluators? Please also provide paired watermarked and unwatermarked outputs for the same prompts, including cases where watermarking reduces quality.

  2. Which components are necessary for the reported performance? In particular, how does Dandelion compare with independent block authentication under the same carrier budget and generation settings?

  3. How well does it detect deliberate edits to final-block carriers or to important facts and negations? Why does Table 1 report 100% CC detection despite the acknowledged final-block weakness? The paper argues that existing autoregressive fragile watermarking methods are unsuitable for MDLMs, but provides no direct experimental comparison. Can the authors demonstrate this limitation.

  4. Beyond the aggregate control-set results, what are the false-acceptance rates across text lengths, carrier counts, and keys? Please report uncertainty and, where applicable, TPR at 1% FPR. For an example of this metric, see Watermarking Diffusion Language Models.

  5. What is the overhead when multiple tokens are revealed per step?

  6. For the tokenizers actually used, which different byte strings produce identical token sequences? Could these accepted differences affect displayed meaning or downstream use?

  7. I am confused by several citations. Wen et al. (2023) study image watermarking, while Peng et al. (2025) and Hao et al. (2020) also concern image watermarking. They are also not related to fragile watermarking. How do these works support the claims about green-list token biasing, token distributions, generated-text detection and the paper's center objective?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.