- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposed a benchmark for evaluating whether different watermark methods can survive prompt based generative model editing. The authors evalauted eight watermark methods on five editors across four generative models. Editing separates Tree-Ring from other watermark methods.
Critical strengths
- The paper investigated an important and timely problem: will the watermark survive generateive model editing. It did extensive experiments on many watermark methods, generative backbones.
- The proposed evaluation, like clean score separation, d′, is easy to compute.
Critical weaknesses
- Even if the authors introduced watermark methods robust under image editing, like VINE and JIGMARK in the related work, they did not test the benchmark on them. I wonder if these methods can better survive image editing under the proposed benchmark.
- Watermarks are known to be easily erased under generative models [1]. Even for tree-ring, it is possible that the current generative models are not strong enough, and they may easily become much more capable in a short time. Therefore, the necessity of the benchmark is undermined.
[1] Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models https://arxiv.org/abs/2311.04378
Questions for authors
- Can you provide results, like d', for some of the editing-robust watermarks methods, like VINE and JIGMARK?
- I find the writing very hard to follow. For example, the author can add reference of the methods mentioned in table 1 right after the method names.