- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Latent Chain Watermark (LCW), a watermarking framework designed to achieve both robustness and tamper evidence for blockwise diffusion language models. LCW assigns a fixed latent chain to each block to determine position-specific green lists, while using the finalized content of each block to select the latent chain for the subsequent block. This design avoids dependence on provisional tokens during parallel generation while introducing content-dependent relationships across blocks. To address the conflicting requirements of robustness and tamper sensitivity, the authors develop two detection procedures. Robust watermark detection searches over candidate block boundaries, latent chains, and within-block alignments to recover watermark evidence from modified text. Tamper detection instead measures the score differences between flexible and constrained detection procedures to identify structural inconsistencies caused by modifications. Beam search and dynamic programming are employed to improve detection efficiency. Experiments on DreamReasoner and LLaDA evaluate watermark detectability, robustness against text editing, and tamper detection. The authors also extend the framework to autoregressive generation and tree-based speculative decoding.
Critical strengths
1.Meaningful research problem. The paper addresses the challenge of achieving both robust watermark detection and tamper evidence in diffusion language models. These two objectives are inherently conflicting, making the problem relevant and technically challenging. 2.Interesting latent-chain design. The proposed approach uses fixed latent chains within blocks and content-dependent transitions across blocks, allowing watermark embedding to remain compatible with parallel generation while introducing sensitivity to post-generation modifications. 3.Well-structured detection framework. The separation between robust watermark detection and tamper detection is intuitive. The combination of beam search and dynamic programming also provides a practical approach to handling the otherwise expensive search over candidate watermark structures. 4.Relatively comprehensive evaluation. The experiments cover multiple models, text lengths, editing attacks, and generation paradigms. The ablation studies further help illustrate the contributions of the proposed components.
Critical weaknesses
- Limited robustness and tamper detection under paraphrasing attacks. Paraphrasing is a realistic and important threat to LLM watermarking, as attackers can substantially modify the wording and structure of watermarked text while preserving its meaning. However, LCW shows considerably weaker robustness under paraphrasing than under random token-level edits. As reported in Appendix C.4, LCW achieves watermark survival rates of only 43.6% and 66.3% under DIPPER paraphrasing at text lengths of 128 and 256, respectively, substantially below Unigram (88.1% and 99.0%) and GSJ (48.5% and 84.2%). Furthermore, the evaluation considers only one DIPPER configuration and does not report tamper detection performance after paraphrasing. Given that robustness and tamper evidence are central objectives of the paper, these results raise concerns about LCW's effectiveness against practical text rewriting attacks.
- Limited effectiveness against targeted and localized tampering. LCW relies on within-block alignment and cross-block latent-chain transitions to identify tampering, making it less effective against modifications that do not significantly disrupt these structures. The average tamper detection rate for single-token substitutions is only 30.6%, and Appendix C.6 reports a further decrease to 15.8% for substitutions in the final block at a 5% FPR. This limitation is particularly concerning because an adversary could deliberately modify the final block without knowing the secret key, thereby avoiding subsequent chain-transition evidence. The current evaluation does not sufficiently investigate such targeted modification strategies.
- Limited analysis of the reliability and discriminative power of tamper evidence. LCW identifies tampering by measuring differences between watermark scores obtained under different structural constraints. However, such differences may also arise from the stochastic nature of soft watermarking and the maximization over multiple candidate configurations, even for unmodified texts. Although the authors calibrate detection thresholds on unmodified watermarked texts, the extent to which the proposed alignment-based and chain-based scores reliably distinguish genuine tampering from normal statistical variations remains insufficiently analyzed. Furthermore, the effectiveness of these signals may depend on watermark strength, introducing a potential trade-off between tamper sensitivity and text quality. Additional analysis of score distributions and sensitivity to watermark strength would strengthen the justification of the proposed mechanism.
- Concerns about the evaluation protocol and baseline comparison. The survival tamper detection rate (STR) is evaluated only on edited texts that remain watermark-positive. While this conditional metric is meaningful, it does not directly reflect the overall probability of successfully detecting tampering. This distinction becomes particularly important under attacks with low watermark survival rates, such as paraphrasing. Reporting the joint success rate of watermark detection and tamper detection would provide a more complete assessment. Additionally, the tamper detection baselines employ different mechanisms: LCW uses its dedicated detector, CW uses its native edit detector, while Unigram and GSJ rely on watermark score decreases. The construction of these baseline scores and the information available during detection should be clarified to ensure a fair comparison.
Questions for authors
- Robustness against paraphrasing. LCW performs substantially worse than Unigram and GSJ under DIPPER paraphrasing. What are the main reasons for this performance gap? Could the authors evaluate additional paraphrasing settings with different lexical and structural modification strengths, and report the corresponding tamper detection performance?
- Targeted tampering and positional sensitivity. Given the low tamper detection rate for substitutions in the final block, how would LCW perform against an adversary who deliberately selects modification positions? Could the authors provide a position-wise analysis of tamper detection performance and discuss possible ways to mitigate this limitation?
- Reliability of tamper evidence. How well do the alignment-based and chain-based tamper scores distinguish modified texts from unmodified watermarked texts? Could the authors provide the score distributions for both cases and analyze how tamper detection performance varies with watermark strength?
- End-to-end tamper detection. Could the authors report the joint success rate of watermark detection and tamper detection under different editing attacks, particularly paraphrasing? This would help clarify the practical effectiveness of LCW beyond the conditional STR metric.
- Fairness of baseline comparisons. How are the tamper detection scores for Unigram and GSJ constructed? Do these baselines require access to the original watermarked text or its watermark score? Are all methods evaluated under comparable information availability and threshold calibration protocols?
5 comments
Q3 Reliability of Tamper Evidence How well do the alignment-based and chain-based tamper scores distinguishmodified texts from unmodified watermarked texts? Could the authors provide the score distributions for bothcases and analyze how tamper detection performance varies with watermark strength?
Thank you for the comment. We acknowledge that soft watermarking and the maximization over candidate configurations may introduce score variations even for unmodified watermarked texts. However, LCW measures tamper evidence through differences between watermark scores under different structural constraints, rather than relying on absolute watermark scores.
For unmodified watermarked texts, the original block alignments and chain transitions remain consistent with the embedding process, so enforcing these structures typically causes little watermark score loss. In contrast, text modifications can disrupt these structures. Specifically, measures the score loss caused by enforcing block boundaries and within-block alignments, primarily capturing insertion and deletion, while measures the score loss caused by enforcing chain transitions, providing evidence for substitution. Together, these complementary scores distinguish structural disruptions caused by tampering from normal score variations.
To further substantiate this mechanism, we analyze the score distributions of unmodified and tampered texts under the default configuration (, , ).
| Score | Text | Median | Q25–Q75 | P95 |
|---|---|---|---|---|
| Clean | 0.000 | 0.000–0.000 | 0.367 | |
| Deletion | 3.019 | 1.937–3.561 | 4.925 | |
| Insertion | 2.503 | 1.902–3.524 | 5.046 | |
| Substitution | 0.000 | 0.000–0.000 | 0.367 | |
| Clean | 0.000 | 0.000–0.000 | 1.225 | |
| Deletion | 1.249 | 0.625–1.874 | 2.603 | |
| Insertion | 1.402 | 1.001–2.003 | 3.204 | |
| Substitution | 2.450 | 1.429–4.083 | 5.920 |
The results show that alignment-based scores clearly distinguish deletion and insertion from unmodified texts, while chain-based scores provide stronger evidence for substitution. This is consistent with the complementary roles demonstrated in Table 5 of Appendix C.1, further validating the effectiveness of LCW's complementary detection design.
We further evaluate the sensitivity of tamper detection to watermark strength by varying and . Each component is independently calibrated to a tamper FPR of at most 5%. Results on DreamReasoner at under five token edits as follows.
| log(PPL) ↓ | Align STR@5 (Del/Ins) | Chain STR@5 (Sub) | Combined STR@5 (Sub) | ||
|---|---|---|---|---|---|
| 0.250 | 2 | 2.059 | 0.987 / 0.980 | 0.766 | 0.747 |
| 0.250 | 4 | 2.745 | 1.000 / 1.000 | 0.861 | 0.861 |
| 0.500 | 2 | 1.988 | 0.961 / 0.899 | 0.828 | 0.767 |
| 0.500 | 4 | 2.342 | 0.984 / 1.000 | 0.882 | 0.890 |
These results demonstrate that both and remain effective across different watermark strengths.
Q4 End-to-End Tamper Detection What is the joint success rate of watermark detection and tamper detection under different editing attacks, particularly paraphrasing?
Thank you for the suggestion. We report watermark survival rate (WSR) and survival tamper detection rate (STR) separately to evaluate two distinct security objectives: watermark robustness and tamper evidence. WSR measures the fraction of edited texts that remain watermark-positive, while STR measures the fraction of these watermark-positive texts that are correctly identified as tampered.
For the same set of attacked texts, the end-to-end tamper detection rate (E2E-TDR) can be directly computed as . To make the overall effectiveness more explicit, we additionally report the joint detection rates under different editing attacks, including paraphrasing.
| Attack | Setting | STR@5 | E2E-TDR@5 | STR@1 | E2E-TDR@1 |
|---|---|---|---|---|---|
| Deletion | 0.997 | 0.983 | 0.997 | 0.980 | |
| Insertion | 0.997 | 0.993 | 0.987 | 0.980 | |
| Substitution | 0.863 | 0.863 | 0.676 | 0.676 | |
| DIPPER | 20/0 | 1.000 | 0.663 | 1.000 | 0.475 |
| DIPPER | 60/60 | 1.000 | 0.248 | 1.000 | 0.089 |
| DIPPER | 80/80 | 1.000 | 0.185 | 1.000 | 0.064 |
Results on DreamReasoner (). Token-level editing follows the evaluation protocol in Figure 3 with . DIPPER settings denote lexical/order diversity.
The results demonstrate strong end-to-end tamper detection under token-level edits. For paraphrasing, LCW achieves 1.000 STR across all evaluated configurations, indicating that all watermark-positive paraphrased texts are identified as tampered. Under stronger DIPPER settings (60/60 and 80/80), extensive paraphrasing substantially changes the original token sequence, disrupting watermark evidence and reducing watermark survival. Consequently, the lower E2E-TDR primarily reflects watermark removal rather than a failure to detect tampering in texts where the watermark survives.
Q5 Fairness of baseline comparisons How are the tamper detection scores of Unigram and GSJ constructed, and are all methods compared under equivalent information availability and threshold calibration?
Thank you for the question. We would like to clarify that watermark embedding and detection are inherently coupled, and different watermarking mechanisms require corresponding detection procedures. Therefore, we adopt detection methods suited to each watermarking mechanism rather than applying a single tamper detector to all methods. Specifically, LCW uses its proposed alignment- and chain-based tamper detector, while CW uses its native edit detector, as its watermark structure does not involve blocks or latent chains. Unigram and GSJ are primarily designed for watermark detectability and robustness and do not provide native tamper-evidence detectors. For comparison, we use changes in their original watermark scores as surrogate tamper signals. For a consistent evaluation, tamper detection thresholds are calibrated at the same target FPR using unmodified watermark-positive texts.
Q1 Paraphrasing Robustness Why is LCW less robust than Unigram and GSJ under paraphrasing, and how does its tamper detection perform under different paraphrasing strengths?
Thank you for the comment. We acknowledge that LCW exhibits lower robustness than Unigram and GSJ under paraphrasing. This difference stems from their distinct design objectives. Unigram and GSJ primarily focus on watermark detectability and robustness, aiming to preserve watermark signals under text modifications. However, this robustness also makes tampering difficult to detect, as modifications may leave the watermark signals largely unchanged. As demonstrated in Figure 3, both methods exhibit limited tamper-detection capability.
In contrast, LCW explicitly introduces content-dependent cross-block transitions to provide tamper evidence while maintaining watermark robustness. Importantly, LCW does not simply trade substantial robustness for tamper evidence under localized edits. For example, Table 3 shows that introducing chain transitions improves substitution STR@5 from 7.0% to 55.8%, while substitution WSR@5 decreases only slightly from 99.0% to 97.7%. This demonstrates that LCW can substantially improve tamper detection while maintaining strong robustness in the targeted editing setting.
To further investigate paraphrasing robustness and tamper evidence, we additionally evaluate different DIPPER configurations with varying lexical and order diversity on DreamReasoner (NX=256). We report the two metrics already used in our paper: watermark survival rate (WSR) and survival tamper detection rate (STR). Following another reviewer's suggestion, we also include the end-to-end tamper detection rate (E2E-TDR), calculated as E2E-TDR=WSR×STR.
Notably, LCW achieves 100% STR across all evaluated paraphrasing configurations, meaning that every watermark-positive paraphrased text is correctly identified as tampered in these experiments. Under stronger DIPPER settings (60/60 and 80/80), extensive rewriting substantially disrupts the original watermark evidence, resulting in lower WSR and consequently lower E2E-TDR. GSJ and Unigram also exhibit reduced watermark survival under these settings.
Q2 Targeted tampering and positional sensitivity How does the position of an edit affect tamper detection, particularly for substitutions in the final block, and can this limitation be mitigated?
Thank you for the comment. We have discussed the difficulty of detecting substitutions in Section 4.2 and the final-block limitation in Appendix C.6. First, substitution is inherently more challenging to detect than insertion or deletion, as it preserves token positions and therefore introduces limited block-alignment evidence. This is not unique to LCW, and the evaluated baselines also exhibit limited substitution tamper-detection capability. Nevertheless, as shown in Figure 3, LCW substantially outperforms these baselines in substitution tamper detection.
Second, we further investigate the impact of modification positions. We conduct a position-wise evaluation on 256-token texts consisting of eight blocks. At 5% FPR, when applying three token deletions to the first, seventh, and final blocks, the STRs are 100.0%, 56.5%, and 42.1%, respectively. For insertion, the corresponding STRs are 100.0%, 65.6%, and 36.8%. For substitution, the corresponding STRs are 25.8%, 19.06% and 15.8%, respectively.
Furthermore, a simple practical mitigation is to append one or two non-essential, naturally generated watermarked blocks after the semantically important content. This allows modifications to the final content block to disrupt subsequent latent-chain transitions, providing additional tamper evidence. These trailing blocks can be discarded after watermark and tamper verification without affecting the substantive content presented to users.