- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper develops an information-theoretic framework for what survives when a watermarked document is rewritten by an operator that does not hold the original key. It models each downstream transformation as a key-blind operation with three budgets—erased, transmitted, and fresh—showing that a new key can enter only through fresh content, while the inherited key can be retained only through transmitted content and is lost through erasure. It further characterizes how task constraints protect key information via fibers and the “shadow” partition, with exact floors for statistic-preserving tasks and finite-key loss bounds. Experiments on language-model rewriting and factual-report tasks illustrate the separation between task success and inherited-watermark survival.
Critical strengths
- The three-budget decomposition (erased, transmitted, fresh) and the key-flow theorem yield sharp caps and explicit gaps for both fresh-key injection and inherited-key retention under key-blind rewrites. This framework unifies copying, paraphrasing, translation, and re-watermarking under one set of identities, and its equality conditions clarify exactly when the caps are attained. It is conceptually simple, technically rigorous, and directly applicable to provenance analysis.
- By modeling task fidelity through fibers and the shadow partition, the paper proves that every faithful rewrite protects at least the key information in the task’s shadow
Critical weaknesses
- The central theory is not connected to real semantic rewriting. The exact floors and attainment results require a completely specified task relation with computable fibers and shadow, but natural-language fidelity for paraphrase, translation, or summarization is never formalized.
- The re-watermarking study fails to reject zero difference—all intervals include zero, and no sign test rejects—which is not evidence of equivalence, yet it is presented as supporting a prediction about removal.
Questions for authors
See weaknesses.
3 comments
Response to Reviewer Wnkj (2/2)
Response to Reviewer Wnkj (2/2)
W2: "The re-watermarking study fails to reject zero difference—all intervals include zero, and no sign test rejects—which is not evidence of equivalence, yet it is presented as supporting a prediction about removal."
The paper states in three places that non-rejection is not equivalence: "This does not establish equivalence" (§6.4, p.8), "does not establish equivalence between the two arms" (G.1, p.28), and "nor show that the arms are equivalent" (H.2, p.33). The revised PDF will replace our phrase "Table 2 tests this prediction" with the test that matches the prediction.
The prediction is equality in distribution: every realization of the same key-averaged channel induces the same joint law of (A.6), so, under ideal randomness, a distortion-free second watermark cannot change the distribution of any inherited-key statistic. The matching analysis is an equivalence test. It is post hoc: our pre-specified family was the sign test, and the analysis will be labelled post hoc in [Rev: App. K].
In plain terms: if a second watermark removed the first, Llama copies would fall from 189/200 detected to about 2/200; with the second watermark they are at 193/200. Qwen paraphrases would fall from 25/200 to about 2/200; they are at 28/200.
Detection rates (inherited key at 1%, saved Table 2 pairs, 200 documents per cell; paired 90% document-bootstrap CI of race − plain, in percentage points):
| cell | plain → race | race − plain [90% CI] | within ±5 | within ±10 |
|---|---|---|---|---|
| Qwen copy | 191 → 191 | 0.0 [0.0, 0.0] | yes | yes |
| Qwen paraphrase | 25 → 28 | +1.5 [−2.5, +5.5] | no | yes |
| Qwen summary | 10 → 7 | −1.5 [−4.5, +1.5] | yes | yes |
| Llama copy | 189 → 193 | +2.0 [0.0, +4.5] | yes | yes |
| Llama paraphrase | 9 → 10 | +0.5 [−3.0, +4.0] | yes | yes |
| Llama summary | 14 → 8 | −3.0 [−6.5, +0.5] | no | yes |
| pooled | 438 → 437 of 1200 | −0.1 [−1.3, +1.3] | yes | yes |
The pooled CI treats pairs as independent, although the three operations share documents; the document-clustered interval is [−1.33, +1.08] points (App. K). Rates are equivalent within ±10 points in 6/6 cells and within ±5 points in 4/6; in the three near-null cells below, the ±10 margin exceeds what complete removal would change, so the informative cells are Llama copy and Qwen paraphrase. The two ±5 exceptions are Qwen paraphrase (upper bound +5.5) and Llama summary (lower bound −6.5; McNemar p = 0.26).
What each cell can detect. Not every cell can show removal:
- Qwen copy is close to automatic: race and plain scores are exactly tied in 194/200 documents, because copying leaves the second key little fresh budget. Llama copy has 139/200 ties but 61 untied pairs.
- The informative cells are those where removal would be visible:
- Llama copy (61 untied pairs). Removal would mean a drop of about 93.5 points toward the 1% null; the CI [0.0, +4.5] contains no drop.
- Qwen paraphrase. Removal would mean a drop of about 11.5 points; the CI is [−2.5, +5.5].
- In the other paraphrase and summary cells, the rewrite itself has already removed nearly all token-level evidence of the inherited key (plain-arm detection 4.5–7%, null 1%), so neither metric has much left to compare; the prediction is tested where that evidence remains, in the two cells above.
Mean scores. A post-hoc margin, the larger of 1 nat and 10% of the plain mean, gives score equivalence in 6/6 cells (paired-t TOST, largest p 0.045, Llama copy; Holm-adjusted, all pass); a flat 1 nat alone passes 5/6 (Llama copy, mean 109 nats, fails) and a flat 10% alone 2/6 (App. K). This margin is informative only where the signal exceeds it. In Qwen summary, Llama summary and Llama paraphrase, the plain-arm signal above the null mean (0.38, 0.45 and 0.83 nats) is smaller than the 1-nat margin, so those score tests cannot exclude complete erasure. For this reason we lead with detection rates.
The §6.4 sentence will read: "Table 2 compares the two arms; a post-hoc equivalence analysis (App. K) finds detection rates equivalent within ±10 points in all six cells and within ±5 points in four, and pooled detections of 438 vs 437 of 1200. These margins are informative where complete removal would exceed them, and there removal is excluded: Llama copy changes by +2.0 points [0.0, +4.5] against the 93.5-point drop removal would cause, and Qwen paraphrase by +1.5 [−2.5, +5.5] against an 11.5-point drop."
This point produced App. K; the prediction is now tested by the analysis it calls for. Together with (1/2), this shows that the theory reaches real semantic rewriting through certified lower bounds on retention, and that the re-watermarking prediction now rests on a post-hoc equivalence analysis that excludes removal wherever it would be visible. We are glad to clarify anything before the discussion ends; if these changes resolve your concerns, we would appreciate it if you would reconsider your rating.
Thank you very much for your response.
After careful consideration, I have decided to maintain my score. I remain open to the discussions and decisions of other reviewers.
Response to Reviewer Wnkj (1/2)
Response to Reviewer Wnkj (1/2)
Thank you for the review and for describing the framework as "conceptually simple, technically rigorous, and directly applicable to provenance analysis". In short: (W1) retention after real paraphrase is now measured, ≥ 1.27 nats certified while the token detector flags 8/64; (W2) a post-hoc equivalence analysis excludes removal wherever it would be visible. W1 is here, W2 in (2/2).
The paper deliberately posits no formal relation for paraphrase, translation or summarization (§7.1): fixing one is a modelling choice, just as a rate–distortion function needs a chosen distortion measure. We disagree that the theory is therefore not connected to real semantic rewriting; it reaches it in four ways.
1. The floors need only a necessary condition of fidelity. If every correct paraphrase meets a checkable requirement R (e.g. keep the named entities), it is R-faithful, so by Theorem 2(iii) its retention is at least I(K1;τR(Y)), whatever else "paraphrase" means, and this is positive whenever the key shapes that content (E1b); a fuller formalization only raises this floor.
2. Measuring retention needs no formal relation. The paper's inequality (§3, p.4) holds for any test applied to the rewrite and the key: Rret≥d(TPR∥FPR). A task relation is needed for the floors (what must survive), not to measure what did survive. We applied this to real paraphrases and summaries (E1, Qwen3-4B, n=64 each). A verifier holding only the key, A's prompt and A's model regenerates A's text under the true key and 15 decoy keys, and ranks the candidates by similarity to the rewrite:
If the provider keeps its logged output, paraphrase identification is 64/64; French translation, available only in this logged setting and scored with an English embedder (exploratory), still gives 30–34/64 against 1/64 token detections. We attribute the gap between the settings to regeneration fidelity, which a direct test supports (General response §1b): where regeneration is bit-exact (29 documents, a non-randomized split) they coincide, and elsewhere identification falls with how early it diverges.
This is what Remark 2 implies for semantic rewriting. For a deterministic-key watermark, Rret=MA (Remark 2), so whatever content a faithful paraphrase keeps also carries the key, even when token-level evidence is largely lost. Word overlap does about as well as the embedding (better on summaries): the evidence is the content the rewrite keeps, which is what MA counts.
3. The paper does contain a formalized natural-language task. The factual-report relation (strict JSON, four exact fields, free narrative wording; §6.3, H.3) is a statistic-preserving relation. Its fibers and shadow are computable: the shadow is the field tuple, and the floor is exactly I(K1;fields).
4. Relations with editing tolerance are easy to write, and their floors are strictly positive whenever the key shapes the required content. We will add a worked example [Rev: App. J.3]. The required entities come from a catalogue whose names are more than 2t edits apart; each may be misspelled by up to t edits, and the wording is free. The shadow is then the entity tuple, and the minimum retention is exactly I(K1;entities). Even a number reported only within ±k keeps a positive floor (ln20≈3.0 nats for a uniformly key-chosen attendance, k=5; Prop. J.4), and imperfect execution costs only a Fano term.
We will add this discussion to the revised §6.6 and §7.1 [Rev].
(continued in 2/2)