- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CohortMark, a watermark for a cohort of texts generated jointly by a language model. The method proposed ensures every proper subset of the documents follows the original distribution and contains no detectable watermark, while watemark signal can be detected when the full cohort is presen. The paper tested the proposed method on real dataset C4,ELI5 and models Qwen,Llama. Results shows that the TPR is close to 1 under cohort level detection and falls to ~1% if member text is changed, indicating that the signal only exists when the cohort is examined as a whole.
Critical strengths
The paper proposed a different setting of watermark where detection is only available when a full cohort of text is present, individual and subset of text does not show detectable signal.
Critical weaknesses
One issue is the practical applicability of this scheme. Detection requires "original token IDs in member order, source-model replay with the same vocabulary" and other source data (Sections 3.3), verification can be difficult for anyone other than the text provider, since model replay might not be feasible, baseline methods KGW and Synthid compared in experiments does not require model replay or any model parameters.
The paper didn't evaluate roubustness to editing or rewriting. This might be more significant to this work since replacing a member voids the detetection.
Questions for authors
Could the author elaborate on the motivation of such design, in what situation would it be undesirable to detecte watermark in individual part of the text?
In experiment, KGW and SynthID does not rely on model replay, but still achieves comparable TPR, can we design other experiments to demonstrate the advantage of the proposed method?