- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Existing language model watermarks only determine whether a text was generated by a model. This paper proposes Selective Watermarking, which determines which task of a single model produced a text. It requires a bounded false-attribution rate, ensures that a verifier holding one task’s key cannot gather evidence from another task’s outputs, and abstains when evidence is insufficient. The authors implement this in SAFIZ, which assigns each task an independent key, embeds a cryptographic watermark during generation and post-hoc synonym substitution in disjoint segments of a document, and applies homoglyph substitution across the whole document. At detection time, SAFIZ combines per-channel p-values with Fisher’s method and either attributes the text to a task or abstains based on a Bonferroni-corrected threshold. The authors prove a bound on the false-attribution rate under two assumptions, pseudorandom function security and channel independence.
Critical strengths
-
The paper is clearly written and well organized.
-
The observation that instruction tuning leaves more documents unable to carry a cryptographic watermark is useful.
Critical weaknesses
-
The guarantee follows mainly from existing statistical methods and cryptographic assumptions, and the technical contribution beyond combining existing components is unclear.
-
The paper’s setting requires the provider to log a registry for every document, so the provider could instead store the outputs and attribute texts by retrieval, yet the paper does not compare against this approach.
-
The full system is no more accurate than homoglyph substitution alone, so the experiments do not show the benefit of combining multiple channels.
-
The paper uses code as its motivating scenario, yet homoglyph substitution generally cannot be applied to code, and no code task is evaluated.
Questions for authors
Please address the concerns raised in the Weaknesses section.