acceptodds
Withdrawn from ICLR 2027

On the Intrinsic Limited Robustness of Latent-Based Watermarking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing latent-based watermarking methods for diffusion models have overestimated their robustness to image distortions, including geometric transformations such as rotation, scaling, and translation (RST). Moreover, this paradigm of watermarking approaches may suffer from inherent limitations arising from the domain in which the watermark is embedded. In this paper, we provide the first theoretical analysis explaining why these methods lack invariance to perturbations. By relaxing the invariant relation, we derive a maximum perturbation bound that characterizes the relationship between pixel-space perturbations and their corresponding effects in latent space. In addition, we present the first analytical formulation that captures all components of practical detection mechanisms. Finally, we conduct experiments to validate the theoretical findings and the limitations of latent-based watermarking methods. Our theoretical and empirical results indicate that, under the current design paradigm, latent-based watermarking methods intrinsically exhibit limited robustness. We conclude by providing the analytical tool and design guidelines that future research could follow.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AIGCs watermark, Latent-based watermark, Security

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_CmCLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the intrinsic robustness limitations of latent-based watermarking for diffusion models. By connecting DDIM processes with denoisers, the authors analyze invertibility, perturbation invariance, and Lipschitz bounds. They further establish a framework relating image perturbations, watermark detection thresholds, false positive rates, and latent distribution shifts. Experiments on representative watermarking methods are conducted to support the theoretical findings and highlight potential limitations of the current latent-based watermarking paradigm.

Critical strengths
  1. Valuable research problem and novel analytical perspective.
    The paper addresses an important problem by investigating the robustness limitations of latent-based watermarking from the perspective of diffusion generation and inversion processes. This analytical perspective is interesting and potentially valuable for future watermarking research.
  2. The experiments examine how different perturbations, watermark patterns, and detection thresholds affect watermark recovery. These observations provide useful insights and inspiration for improving watermark robustness.
Critical weaknesses
  1. Strong theoretical assumptions and insufficiently justified proofs.
    The core theoretical analysis relies on several strong assumptions, and some derivations are not sufficiently convincing. For example, Theorem 3.2 argues that distinct latent representations remain different after Fourier–Mellin transformations. However, distinct inputs may produce identical representations under invariant transformations. Even under the authors' assumptions, this conclusion requires stronger justification.
  2. Insufficient quantitative validation of theoretical findings.
    There is a gap between the theoretical analysis and experimental validation. The experiments primarily rely on qualitative comparisons of distributions and figures, without sufficient quantitative analysis to establish the proposed relationships or validate the predictive value of the theoretical bounds.
  3. Overly broad conclusions regarding latent-based watermarking.
    The authors strongly question the viability of latent-based watermarking and advocate post-hoc watermarking as an alternative. However, the analysis does not establish that all latent-based methods are inherently incapable of achieving high robustness. Furthermore, systematic comparisons with post-hoc watermarking methods under unified experimental settings are missing.
Questions for authors

The paper suggests fundamental limitations of latent-based watermarking and recommends post-hoc watermarking as an alternative. Could the authors clarify what theoretical evidence supports this broader conclusion? Could the authors provide comparisons with post-hoc methods under the same attack settings and FPR constraints to support this recommendation?

Reviewer_eWS1OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

In this paper, the authors introduce a theoretical framework to assess the robustness of latent-based watermarking methods by leveraging the robustness of the inverse diffusion, making the assessment agnostic to the watermarking scheme. The theoretical framework uses the Lipschitz constant of the transformations to estimate the robustness of the inverse diffusion by measuring its invariance to transformations. The authors provide an extensive empirical evaluation of the inverse diffusion against a wide set of transformations.

Critical strengths

This paper could have been excellent if not for the critical flaws I will outline in the next sections. The core idea of studying the robustness independently of the watermarking scheme is excellent, and I fully endorse this direction, especially when it is performed through a theoretical study. I also am quite convinced that the characterization of the Lipschitz constant of network is the correct quantity to study in this case (though this is nothing new! the adversarial machine learning literature far predates the modern watermarking community on that matter).

Critical weaknesses

As I said in the Strength section, I appreciate that a theoretical study was proposed. Sadly, I read the theorems and "proofs" extensively and I claim that they completely lack mathematical rigor, both in how the theorems are stated and in what arguments are used. I direct the authors to The Question section which contains an extensive list of these flaws. To summarize: some theorems are trivial (3.1), others are easily proven false (3.2), others are undetermined (3.3); some contains erroneous reasoning (3.4) and I could go on.

The value of this paper resides wholly in its mathematical analysis yet it is completely flawed.

Questions for authors

This paper should be rejected because (1) the theoretical framework contains mistakes in the proofs and hypotheses, (2) the discussion and analysis of the results rely directly on this framework and lead to flawed interpretations, and (3) the paper is imprecise.

My score can improve if the authors can justify or clarify the following points:

  • Theorem 3.1 demonstrates that an exact inverse exists under the assumption that the MMSE is exact, which is expected.
  • Theorem 3.2 does not hold for the counterexample fθ=Idf_{\theta} = Id.
  • Theorem 3.3 is unclear. Either it says that if the output of the function fθf_{\theta} is not RST-invariant, it is not invariant to all the transformations, which is trivial because RST transformations ⊂\subset all transformations, or it says that not a single transformation is invariant. If it is the second case, the demonstration does not match the theorem. It assumes that if the function fθf_\theta follows the construction in Theorem 3.1, which states that it is a bijection (under a strong, unrealistic hypothesis), then two different inputs yield two different outputs. It does not mention the RST-invariance stated in the Theorem.
  • Moreover, in the case of watermarking, the invariance does not need to be perfect in the output but must rely on a threshold in the output space, so Theorem 3.3 says nothing about the robustness of the inverse diffusion in this context.
  • Theorem 3.4 gives an upper bound on the Lipschitz constant, which is then used to show that the inverse process is an expansive mapping. This statement cannot be derived from an upper bound.
  • In Theorem 3.5, the authors relaxed their invariance hypothesis by using r, but the argument of exact inversion is used in the proof.
  • Findings 3.6 and 3.7 are derived from this theoretical framework.
  • I have a concern about using an upper-bound estimate of the Lipschitz constant to study the intrinsic robustness of a watermarking method. This is especially relevant when studying the effect of adversarial Machine Learning attacks in a white-box or black-box setup. In the paper, geometric transformations such as rotations lead to the very high upper bound shown in Figure 3 while being perfectly decoded. The Lipschitz formulation provides insufficient information to conclude robustness, except that, in the worst case, these transformations may cause high distortion in the output space.
  • Moreover, the authors conclude that the latent-based methods are useless compared to post-hoc methods. I argue that this take depends on a threat model that is not detailed in this work. Additionally, the authors' arguments rely on high distortion due to the Lipschitz constants. Pixel Seal, a modern post-hoc method cited as a reference and solution by the authors, relies on ConvNeXt and encountered the same problem with the Lipschitz constant. Additionally, the authors cite the methods presented as solutions without justifying the guidelines.

Questions that cannot improve the score:

  • How was the Lipschitz constant approximation computed in the experimental setup?
  • How is the crop actually performed in this paper? Is is crop then rescale (the correct way!) or simply a masking operation (cropped pixels are set to black)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.