acceptodds
Withdrawn from ICLR 2027

Latent Watermarks under Generative Editing: A Benchmark and Analysis of Detection Survival

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Ordinary prompt-based editing can cause latent watermark detection to fail without explicitly targeting the watermark. We benchmark eight watermark methods against five editors across four generative backbones, four editing strengths, and five semantic categories, with edit-validity and threshold checks. Separating editing from seven subsequent distortions reveals that editing alone primarily distinguishes Tree-Ring, while added distortions expose a broader spectrum of detection survival. Sequential edits reveal a second hidden difference: score separation can decline while detection rates remain near their ceiling. Across methods, standardized clean score separation () organizes composite-survival tiers, whereas spatial overlap adds little to predicting edit-only survival beyond clean detectability. Embedding-strength interventions in two methods link higher clean separation to higher post-edit separation. In HSTR, the margin contrast is positive, while the angular layout contrast at matched clean separation remains unresolved. Together, outcome decomposition and continuous separation expose differences hidden by aggregate TPR. Method tiers are stable under threshold recalibration at the main operating points and alternative composite weights. Clean is thus a useful empirical diagnostic within this benchmark, with mixed transfer to unseen methods. Code will be released upon acceptance.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Image watermarking, Diffusion models, Latent watermarks, Text-guided image editing, Robustness evaluation, Signal detection theory, Content provenance

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xn9wOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposed a benchmark for evaluating whether different watermark methods can survive prompt based generative model editing. The authors evalauted eight watermark methods on five editors across four generative models. Editing separates Tree-Ring from other watermark methods.

Critical strengths
  1. The paper investigated an important and timely problem: will the watermark survive generateive model editing. It did extensive experiments on many watermark methods, generative backbones.
  2. The proposed evaluation, like clean score separation, d′, is easy to compute.
Critical weaknesses
  1. Even if the authors introduced watermark methods robust under image editing, like VINE and JIGMARK in the related work, they did not test the benchmark on them. I wonder if these methods can better survive image editing under the proposed benchmark.
  2. Watermarks are known to be easily erased under generative models [1]. Even for tree-ring, it is possible that the current generative models are not strong enough, and they may easily become much more capable in a short time. Therefore, the necessity of the benchmark is undermined.

[1] Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models https://arxiv.org/abs/2311.04378

Questions for authors
  1. Can you provide results, like d', for some of the editing-robust watermarks methods, like VINE and JIGMARK?
  2. I find the writing very hard to follow. For example, the author can add reference of the methods mentioned in table 1 right after the method names.
Reviewer_VWuQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper benchmarks eight latent watermarks against five prompt-based editors and asks what explains their survival differences. It concludes that clean score separation d′ organizes survival, while spatial overlap does not.

Critical strengths

Edit-only and post-distortion survival are reported separately, with threshold recalibration checks. Negative results are reported honestly.

Critical weaknesses
  1. The main finding is nearly trivial: methods with a larger clean margin keep detecting after edits. The paper does not say which design choices give a larger margin at the same image quality. The claim rests on only 4 to 6 methods, and held-out results are mixed (Table 3).

  2. The benchmark has little signal. Apart from Tree-Ring, every method has edit-only TPR ≥ 0.986, and four methods stay at ceiling across the whole strength sweep. The remaining spread comes from added distortions. The main text does not show survival conditioned on successful edits.

  3. The spatial conclusion is weakly supported. The saliency map is not shown to locate the watermark, and masking high-saliency pixels is no more harmful than random masking. With most methods at ceiling, "overlap adds little" may only mean the test cannot detect an effect.

Questions for authors
  1. Which design choices raise clean d′ at matched image quality?
  2. Does the method ordering hold when restricted to successful edits at the strongest settings?
  3. How is the saliency map validated as the location of the watermark?
Reviewer_8cMPOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a systematic benchmark evaluating the robustness of latent image watermarks against generative image editing. The authors evaluate eight watermarking methods across five image editors, four editing strengths, and five semantic editing categories. They distinguish between watermark survival immediately after editing and survival after seven additional image distortions.

Critical strengths
  1. Comprehensive empirical evaluation: The paper evaluates a diverse set of eight watermarking methods across multiple editing architectures, strengths, and semantic transformations. This provides a useful comparison of existing watermarking techniques under a common evaluation framework.

  2. Separation of editing and distortion effects: Distinguishing edit-only watermark survival from robustness against subsequent distortions is valuable. The experiments demonstrate that high detection rates immediately after editing do not necessarily imply robustness to additional image transformations.

  3. Analysis beyond binary detection: The use of continuous detection score separation ((d')) is particularly useful when TPR approaches 100%. The sequential editing experiments effectively demonstrate how watermark detectability can deteriorate even when binary detection rates remain unchanged.

  4. Investigation of factors influencing robustness: The paper goes beyond benchmarking by investigating spatial sensitivity, embedding strength, and watermark layout. The controlled embedding-strength experiments provide supporting evidence for the relationship between clean detection separation and post-edit robustness.

Critical weaknesses
  1. Limited methodological novelty: The paper is primarily an empirical benchmarking and analysis study rather than introducing a new watermarking method or robustness technique. Similar benchmarking efforts, including WAVES and W-Bench, have already investigated watermark robustness under image transformations and editing. The authors should more clearly establish the incremental insights provided by this benchmark.

  2. Limited generalizability of the (d') findings: A central conclusion is that clean detection score separation predicts post-edit robustness. However, the cross-method analysis involves only a small number of watermarking methods, and the authors acknowledge mixed generalization to held-out methods. Furthermore, the relationship is largely established using a composite metric dominated by subsequent distortions rather than generative editing itself. This limits the strength of the broader predictive claim.

  3. Detection saturation limits the analysis: Most evaluated watermarking methods achieve nearly 100% TPR under editing alone. Consequently, the benchmark provides limited differentiation among these methods for its primary evaluation scenario. Although additional distortions reveal differences, the authors could investigate more challenging generative editing scenarios to better characterize the robustness of stronger watermarking methods.

  4. Editing severity is not consistently controlled: Different editing methods and configurations can produce substantially different degrees of semantic and perceptual change. Although LPIPS and other metrics are reported, the experiments do not consistently match editing severity or semantic transformation quality across editors. This makes direct comparisons between editing methods less conclusive.

  5. Insufficient consideration of watermark quality trade-offs: The authors demonstrate that increasing embedding strength improves detection separation and post-edit robustness. However, increasing watermark strength may also degrade image fidelity. A more systematic analysis of the robustness–imperceptibility trade-off would help determine whether the observed gains are practically meaningful.

  6. Limited coverage of modern watermarking approaches: The benchmark focuses on initial-noise and inversion-based latent watermarks, excluding editing-aware learned watermarking methods such as Robust-Wide, JIGMARK, and VINE. Including representative editing-aware methods would strengthen the evaluation, particularly given that robustness to generative editing is the central research question.

Questions for authors

Check weakness for answering the questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.