- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DenMark introduces semantic watermarking for diffusion language models. Its central idea is to evaluate local denoising decisions through temporary semantic completions: the method samples candidate token updates, fills the remaining masks within the current semantic block, and scores the resulting embeddings against keyed directions. Only the selected local update is committed; the temporary completion is discarded. Detection scans possible unit sizes using length-stratified empirical calibration and a multiple-testing correction.
The paper evaluates the method on four DLM backbones and three datasets under paraphrasing, token editing, and length-changing attacks. Additional experiments examine the encoder, decoding choices, blockwise semantic baselines, generation quality, and runtime. The results show strong detection robustness, although the generation overhead is substantial.
Critical strengths
A meaningful methodological contribution. DenMark uses temporary semantic completions to guide local denoising updates without committing the full completion. This directly addresses the difficulty of evaluating partially masked text and offers a substantive adaptation of semantic watermarking to DLM generation, rather than simply applying an existing detector to a new backbone. Strong empirical support. Evaluation across four backbones, three datasets, and diverse attacks supports the method’s robustness. The encoder ablation, blockwise semantic comparisons, and calibration checks address important alternative explanations and failure modes, making the evidence sufficiently broad to support acceptance.
Critical weaknesses
The benefit of local steering is not isolated from computational expenditure. The blockwise comparisons do not establish matched total compute: equal candidate counts do not equalize repeated selections and rollout costs. Given the approximately twelvefold slowdown over clean generation, this leaves uncertainty about both the source of the gains and their practical value. Missing artifacts limit independent verification. The submission omits the exact trained encoder and raw experimental outputs, although the encoder ablation shows that fine-tuning materially affects performance. The available code and training recipe are useful, but do not fully resolve uncertainty about reproducing the reported gains.
Questions for authors
Comparison evidence: Does DenMark retain its advantage over the blockwise semantic baselines under a matched compute or wall-clock budget at comparable output quality? This would help distinguish the benefit of local steering from additional computation. Reproducibility clarification: Are the exact trained encoder and per-example detection scores available for verification? If the checkpoint cannot be provided, is there evidence that the main results are stable across independently retrained encoders?