acceptodds
Under review as a conference paper at ICLR 2027

Latent Diffusion Model Watermarks can be Removed by VAR Regeneration

Abstract

Diffusion regeneration has emerged as a universal removal attack against invisible image watermarking, but it is not able to remove latent-domain watermarks, while the underlying reason remains unaddressed. In this work, we reveal that the reason lies in the natural preservation of generation–inversion symmetry in diffusion regeneration. Motivated by this, we propose visual autoregressive regeneration (VARR), a simple yet powerful universal watermark removal attack that breaks this symmetry. Specifically, during the VAR model’s multi-scale quantization, it keeps low-scale semantic tokens while replacing high-scale ones with their predictions using VAR's autoregressive transformer. Subsequently, the partially modified token sets are used to reconstruct the image with the watermark removed. Across eight representative latent-domain watermarking methods, VARR achieves an average removal success rate of 92.8% while maintaining high visual fidelity with an average FID of 28.27, significantly lower than state-of-the-art regeneration methods (FID across 40 to 75). Furthermore, we show that the VAR domain is more effective for watermark removal compared with the pixel and frequency domains, empirically supporting the effectiveness of VARR. Our findings expose a fundamental vulnerability of latent-domain watermarks and establish autoregressive regeneration as a new paradigm for evaluating watermark robustness.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Watermark Removal, Model Watermarking, Autoregressive Models

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_41wUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Motivation and claims:

  • Regeneration attacks work well for removing watermarks but have inconsistent effectiveness for latent-domain watermarks.

  • Paper theoretically explains why diffusion regeneration fails against latent-domain watermarks.

  • Paper proposes VARR, VAR-based regeneration that breaks the generation-inversion symmetry.

  • Why diffusion regeneration (add a bit of noise, denoise) can not remove latent-domain watermarks:

    • Some upper bounds on zT reconstruction error after attack are given, as well as detector score.
    • Conclusion is that magnitude of the injected noise is not important but the reconstruction error of attacker’s VAE and denoiser.

Approach:

  • Threat model is no-box (no API queries), goal is to remove watermark

  • Attack consists of encoding the watermarked image into a scale pyramid, then dropping the last m scales (at highest resolutions) and re-generating them using the VAR backbone. Final image is the VAR decoder output.

Results:

  • Wide selection of attacked watermarks, including PRC, GS, TreeRing, RingID, Stable Signature, AquaLoRA. Evaluation could be extended with TrustMark and PixelSeal as baselines as well.

  • Attack baselines: simple perturbations, UnMarker, VAE regeneration, diffusion regeneration, next-frame prediction attack and CtrlRegen.

  • 5k images are watermarked for every method for each (of the two) datasets.

  • Only last 2 (out of 10) scales are re-generated. Note that the last m=2 scales are roughly 70% of total tokens during generation.

  • Evaluation metrics: removal success and PSNR, SSIM and FID (and CLIP for alignment).

  • The methods shows strong removal results, on par with UnMarker, except for AquaLoRA and TreeRing, where it is better, and for HSTR and Gaussian shading where UnMarker is better. However, it performs better at image fidelity than UnMarker, with high PSNR, and better FID. However, CtrlRegen has better PSNR and SSIM. Speed-wise, VARR is 200x as fast as UnMarker.

  • The method reports ~93% removal success across the tested latent-domain watermarks, but only ~60% for Gaussian Shading.

Critical strengths

S1: mostly thorough and reasonable evaluation, reasonable ablations (method is very simple so there isn’t much to ablate)

S2: simple training-free approach that should be easy to implement, no per-image optimization or detector access

S3: effective removal performance across tested watermarks, while being 200x faster and having slightly better image preservation than UnMarker

Critical weaknesses

W1: not clear how significant or practically consequential the theoretical analysis is. There is no empirical analysis to investigate theoretical claims in Section 3. Section 3 establishes that the detection score difference between the original watermarked image and the attacked image is upper-bounded (Eq. 3). Eq. 4 then concludes that the watermark remains detectable if the upper bound on detection score change is below the original image’s detection margin \gamma. However, it is never established whether the upper bound tends to be smaller than the margin so it’s not clear whether this theoretical treatment explains the robustness to diffusion regeneration.

W2: missing control and ablations. a) m=0 needs to be included as a control on how much using a different autoencoder and tokenizer already disrupts the watermark (quantization error in tokenization could have a substantial effect on watermark detectability) and b) m=1 is missing from ablations.

Other important weaknesses:

W3: BitMark is not included in the experiments. It is a watermarking method specifically designed for bit-wise generators (including infinity, VAR’s successor). It is very robust and works on the same generator family as VARR. I believe before this paper is accepted, BitMark needs to be included in the experiments to understand how the attack works against watermarks on other scale-autoregressive generators. It would also be interesting to see how stable the coarser scales in BitMark are when the finer scales are attacked by VARR.

W4: attack success metric has issues

a) it is unclear how the removal success thresholds have been calibrated.

b) it is unclear how separable attacked images are from unwatermarked images. It could be that even though the chosen thresholds are defeated, other thresholds still allow to separate attacked watermarked from unwatermarked (attacked or clean) images. For this reason, TPR at a fixed FPR should be reported.

W5: Limited novelty. While simple, the presented approach is a straightforward adaptation of an existing scale-autoregressive generative model to use in a regeneration attack.

Questions for authors

Q1: i wonder how much sense it makes to regenerate earlier scales while keeping the other scales fixed (in ablations). How bad are the image artifacts there? Could you include examples in appendix?

Remarks:

  • Citations should be formatted properly. Use \citep
  • By latent-domain watermarks, the authors assume watermarks relying on full inversion (treering-style)
  • RAVEN (Erasing Invisible Watermarks via Novel View Synthesis) at least be cited. It’s similar to NFPA.
  • Infinity and BitMark should be cited.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 4

We thank the reviewer for the helpful suggestions.

For W1: Due to the use of Lipschitz continuity in our theoretical derivation, it is difficult to empirically evaluate the tightness of the derived upper bound. We emphasize that our theoretical analysis primarily aims to provide an insight into why diffusion regeneration fails against latent-domain watermarks. Previous studies generally attributed this robustness to the semantic nature of latent-domain watermarks, arguing that diffusion regeneration mainly removes noise-like information. In contrast, our analysis provides a more fundamental explanation by identifying generation–inversion symmetry as the underlying mechanism responsible for this robustness.

For W2: We have added the corresponding experiments in the revised ablation study section. Overall, the parameter (m) controls the trade-off between watermark removal success rate and visual fidelity. Increasing (m) improves the watermark removal success rate but leads to greater degradation in visual fidelity.

Table1: Ablation study on the number of regenerated scales (m) for watermark removal and visual fidelity.

(a) m = 0

MethodWSR ↑PSNR ↑SSIM ↑FID ↓CLIP ↑
Tree-Ring25.27%24.48950.741710.07840.3199
ROBIN36.25%23.94110.72409.37710.3224
GaussMarker5.22%23.60450.75009.79400.3170
HSTR0.20%23.25280.727434.05030.3134
Gaussian Shading0.00%23.13440.722010.00970.3142
PRC32.05%24.42200.68579.00510.3149

(b) m = 1

MethodWSR ↑PSNR ↑SSIM ↑FID ↓CLIP ↑
Tree-Ring99.00%22.41450.665410.18800.3154
ROBIN99.40%21.63760.673914.75240.3175
GaussMarker59.58%21.42920.608611.38250.3152
HSTR28.40%21.10360.640739.15080.3139
Gaussian Shading12.00%20.95670.632511.09650.3141
PRC99.96%22.26100.49829.47400.3151

For W3: We emphasize in the title that the scope of our proposed VARR attack is limited to diffusion model watermarks, whereas BitMark is specifically designed for bit-wise generators and thus falls outside the scope of this paper. We leave the evaluation of VARR's effectiveness against watermarks in autoregressive generative models for future work.

For W4: (a) The detection thresholds used to determine watermark removal success are calibrated at a fixed false-positive rate (FPR), assuming that the decoded bits from unwatermarked images follow a binomial distribution, consistent with the settings adopted in the original watermarking papers. To further improve the clarity and reproducibility of our experimental results, we additionally report the watermark detection configurations and unattacked watermark detection rates in the revised manuscript.

Table2: Watermark Detection Configurations.

MethodScoreDecision thresholdUnattacked detection rate
DwtDctBit accuracy≥0.75\geq 0.75 (pbin=3.8665×10−5p_{\mathrm{bin}}=3.8665\times10^{-5})100%
DwtDctSvdBit accuracy≥0.75\geq 0.75 (pbin=3.5002×10−3p_{\mathrm{bin}}=3.5002\times10^{-3})100%
StegaStampBit accuracym^=m\hat{m}=m (dH=0, BA=1.0d_H=0,\ \mathrm{BA}=1.0)99.98%
Stable SignatureBit accuracy≥41/48\geq 41/48 (p≤10−6p \leq 10^{-6})99.70%
AquaLoRABit accuracy≥41/48\geq 41/48 (p≤10−6p \leq 10^{-6})97.50%
Tree-Ringℓ1\ell_1 distanceτTR\tau_{\mathrm{TR}} calibrated at FPR=10−2\mathrm{FPR}=10^{-2}95.06%
RingIDℓ1\ell_1 distanceτRI\tau_{\mathrm{RI}} calibrated at FPR=10−2\mathrm{FPR}=10^{-2}89.92%
GaussMarkerBit accuracyτGM\tau_{\mathrm{GM}} calibrated at FPR=10−2\mathrm{FPR}=10^{-2}99.56%
HSTR / HSQRDistance / pp-valueThreshold = 50, p≤10−2p \leq 10^{-2}100%
ROBINℓ1\ell_1 distanceτROBIN\tau_{\mathrm{ROBIN}} calibrated at FPR=10−2\mathrm{FPR}=10^{-2}99.98%
Gaussian ShadingBit accuracy≥0.6484\geq 0.6484 (p≤10−6p \leq 10^{-6})100%
PRCBit accuracyTarget FPR=10−2\mathrm{FPR}=10^{-2}100%
AuthorsOpenReview ↗

Response to Reviewer 4

For W4: (b) In the revised manuscript, we have added a Separability Analysis of Attacked and Unwatermarked Images section to evaluate the separability between VARR-attacked images and unwatermarked images using AUROC. The results show that the AUROC values for all watermarking methods except Gaussian Shading are close to 0.5, indicating that VARR-attacked images are nearly indistinguishable from unwatermarked images. These findings further support that VARR effectively removes watermark evidence and that simply adjusting the detection threshold is unlikely to substantially improve the separability between attacked and unwatermarked images.

Table3: Separability analysis of attacked and unwatermarked images.

MethodUnwatermarked Bit Acc (%)Attacked Bit Acc (%)TPR (%)AUROC
Tree-RingN/AN/A1.400.518
ROBINN/AN/A1.200.509
HSTRN/AN/A12.400.604
GaussMarker49.84±3.1849.84 \pm 3.1851.62±3.5451.62 \pm 3.542.600.531
PRC50.17±2.2650.17 \pm 2.2650.31±2.4150.31 \pm 2.411.000.504
Gaussian Shading49.93±3.1549.93 \pm 3.1560.42±4.6560.42 \pm 4.6542.600.742

For W5: We acknowledge that VARR directly applies an existing autoregressive model to regeneration attacks. However, this does not necessarily imply a lack of novelty. Prior work [1] also directly applied diffusion and VAE models to regeneration attacks without modifying anything, yet was accepted at NeurIPS. Furthermore, we emphasize that our contributions lie in the theoretical analysis of why diffusion regeneration attacks fail against latent-domain watermarks, the simultaneous improvements in watermark removal success rate, visual fidelity, and computational efficiency, and the insights into why VARR is effective, including the advantages of the VAR domain for watermark removal and VAR's strong image generation prior for preserving visual fidelity. Together, these contributions establish VARR as a state-of-the-art regeneration attack and reveal a significant threat to existing diffusion model watermarks.

[1] Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. Invisible image watermarks are provably removable using generative ai. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 8643–8672. Curran Associates, Inc., 2024. doi: 10.52202/079017-0276.

For Q1: We would like to clarify that regenerating earlier scales while keeping subsequent scales fixed is not a reasonable setting, as VAR follows a coarse-to-fine autoregressive process, where each scale is generated based on the preceding scales. Modifying earlier scales without updating subsequent scales would break the dependencies between scales and potentially introduce significant visual artifacts.

For Remarks: We believe that focusing on watermarks relying on full inversion is reasonable, as existing latent-domain watermarking methods that embed watermarks into the initial Gaussian noise rely on inversion for watermark detection. Additionally, we have added the suggested references, including RAVEN, Infinity, and BitMark in the revised manuscript.

We welcome any further questions from the reviewer and would be pleased to provide additional clarifications in further discussion.

Reviewer_n1GjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes visual autoregressive regeneration (VARR), a watermark-agnostic removal attack that requires a watermarked image and a publicly available pretrained model, without detector queries, watermark keys, clean counterparts, or per-image optimization.

Critical strengths

VARR is a simple detector-free attack that uses a public VAR model to regenerate images, without access to the detector, keys, clean images, or per-image optimization, and runs much faster than the optimization-based UnMarker.

Critical weaknesses
  1. The removal-rate comparison uses inconsistent averaging scopes, and the detector operating points are insufficiently documented. Sec. 5.2.1 contrasts VARR’s 92.8% average over eight latent-domain methods with UnMarker’s 90.0% average over all 13 methods. Recomputing Table 1 over the same eight latent-domain methods gives approximately 92.85% for VARR and 95.69% for UnMarker. The text should therefore distinguish the averaging scopes and qualify the comparison. Although the paper defines WSR and reports the sample sizes, it does not report numeric detector thresholds, target false-positive rates, or unattacked detection rates. These omissions limit the interpretation and reproducibility of the reported removal rates.
  2. The evidence for an overall removal–fidelity advantage remains incomplete. FID measures similarity between image distributions and does not directly establish preservation of individual input images. Table 2 reports lower FID for VARR, but its average PSNR/SSIM is 19.45/0.5272, compared with 21.67/0.6709 for CtrlRegen. These metrics reflect different aspects of image quality, so the current results do not establish a uniformly better trade-off. Comparisons at matched levels of paired-image fidelity would help determine when VARR is preferable to alternative regeneration attacks.
Questions for authors
  1. Detection calibration and comparable aggregation. What numeric detection thresholds and target false-positive rates were used for each watermarking method, and what were the detection rates before attack? Please clarify how images that were already undetected before attack are handled when computing WSR. Please also report comparison averages over identical subsets of watermarking methods and clarify the aggregation across the two datasets, particularly for the eight-method comparison between VARR and UnMarker.
  2. Input preservation and the removal–fidelity trade-off. How does VARR compare with CtrlRegen at matched levels of paired-image fidelity? Please clarify the reference set used for FID and discuss how the reported FID improvements relate to preservation of individual input images. A comparison of removal rates at comparable PSNR/SSIM or perceptual-distortion levels would help substantiate the claimed trade-off.
1 comment
AuthorsOpenReview ↗

Response to Reviewer 3

We thank the reviewer for the helpful suggestions.

For W1: We have revised the corresponding text to ensure a fair comparison. Specifically, across the 13 watermarking methods evaluated in this paper, VARR achieves an average watermark removal success rate of 94.67%, compared with 90.0% for UnMarker, demonstrating the advantage of our method over UnMarker. Additionally, in the revised manuscript, we have provided the specific numerical detection thresholds, target false-positive rates (FPRs), and unattacked watermark detection rates to improve the clarity and reproducibility of our experimental results.

Table1: Watermark Detection Configurations.

MethodScoreDecision thresholdUnattacked detection rate
DwtDctBit accuracy≥0.75\geq 0.75 (pbin=3.8665×10−5p_{\mathrm{bin}}=3.8665\times10^{-5})100%
DwtDctSvdBit accuracy≥0.75\geq 0.75 (pbin=3.5002×10−3p_{\mathrm{bin}}=3.5002\times10^{-3})100%
StegaStampBit accuracym^=m\hat{m}=m (dH=0, BA=1.0d_H=0,\ \mathrm{BA}=1.0)99.98%
Stable SignatureBit accuracy≥41/48\geq 41/48 (p≤10−6p \leq 10^{-6})99.70%
AquaLoRABit accuracy≥41/48\geq 41/48 (p≤10−6p \leq 10^{-6})97.50%
Tree-Ringℓ1\ell_1 distanceτTR\tau_{\mathrm{TR}} calibrated at FPR=10−2\mathrm{FPR}=10^{-2}95.06%
RingIDℓ1\ell_1 distanceτRI\tau_{\mathrm{RI}} calibrated at FPR=10−2\mathrm{FPR}=10^{-2}89.92%
GaussMarkerBit accuracyτGM\tau_{\mathrm{GM}} calibrated at FPR=10−2\mathrm{FPR}=10^{-2}99.56%
HSTR / HSQRDistance / pp-valueThreshold = 50, p≤10−2p \leq 10^{-2}100%
ROBINℓ1\ell_1 distanceτROBIN\tau_{\mathrm{ROBIN}} calibrated at FPR=10−2\mathrm{FPR}=10^{-2}99.98%
Gaussian ShadingBit accuracy≥0.6484\geq 0.6484 (p≤10−6p \leq 10^{-6})100%
PRCBit accuracyTarget FPR=10−2\mathrm{FPR}=10^{-2}100%

For W2: We emphasize that FID and CLIP are more suitable for evaluating fidelity in regeneration-based attacks. PSNR and SSIM primarily measure pixel-level differences, whereas regeneration-based attacks inherently modify every pixel value, often resulting in relatively high PSNR and SSIM scores. Therefore, these metrics provide limited insight into the fidelity of regenerated images, particularly when comparing VARR and CtrlRegen. Nevertheless, since previous studies have reported PSNR and SSIM, we also include these metrics for completeness and consistency. We believe that VARR's superior FID performance, together with its highest average watermark removal success rate, provides strong evidence of its favorable trade-off between watermark removal effectiveness and visual fidelity.

For Q1: We have provided the watermark detection configurations in Appendix D.2 (Watermark Detection Configurations) and W1. In Section 5.1 (Watermarking Methods and Datasets), we further clarify that images undetected before attack are excluded from metric computation, and results are aggregated by pooling both datasets, with the denominator being the total number of detected watermarked images before attacks.

For Q2: We have clarified in the Experimental Setup section of the revised manuscript that the reference set used for FID calculation consists of 5,000 real images from the MS COCO 2017 dataset. The remaining concerns have already been addressed in our response to W2.

We welcome any further questions from the reviewer and would be pleased to provide additional clarifications in further discussion.

Reviewer_CzHnOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates failure scenarios of existing watermark removal attacks, especially diffusion regeneration attacks, and theoretically explains the underlying mechanism as the destruction–reconstruction symmetry of diffusion regeneration: after adding noise and then denoising, the diffusion model may approximately recover the original latent representation, and thus the watermark information is preserved along with it. Accordingly, the authors seek a watermark removal method that can both destroy hidden watermarks and preserve the semantic content and quality of the original image as much as possible. The key idea is to preserve coarse-grained semantic information and regenerate fine-grained information.

The authors propose Visual AutoRegressive Regeneration (VARR), which preserves low-scale tokens containing image semantic information and uses an autoregressive model to regenerate high-scale tokens, thereby destroying watermark information while maintaining the semantic and visual quality of the original image as much as possible. Experimental results show that the proposed method achieves an average watermark removal success rate of 92.8% across eight latent-space watermarking methods, with an FID of 28.27.

Critical strengths

The theoretical analysis is somewhat novel and insightful. The paper not only focuses on improving the effectiveness of watermark removal attacks, but also theoretically analyzes why existing diffusion regeneration attacks fail against latent-space watermarks. By invoking the destruction–reconstruction symmetry, the authors explain the mechanism by which watermark information is preserved, and establish connections among reconstruction error, latent representation deviation, and watermark detectability. We find this theoretical analysis quite illuminating, as it provides a valuable perspective for understanding the robustness of latent-space watermarks and for designing new watermark removal methods.

The proposed method is simple yet effective, with a well-motivated theoretical rationale. Building on the preceding theoretical analysis, the authors propose VARR. The method exploits the multi-scale representation properties of visual autoregressive models by preserving low-scale semantic information and regenerating high-scale tokens, thereby destroying watermark information while preserving image semantics and visual quality as much as possible. We consider the design idea to be relatively novel, and there is a clear logical connection between the theoretical analysis and the concrete solution. Although the overall pipeline is relatively simple, it achieves competitive experimental results, demonstrating the practical value of the method.

The experimental evaluation is relatively thorough and demonstrates good overall performance. The paper conducts comprehensive experiments against several representative watermarking methods and existing attacks, and further validates the effectiveness of the proposed method through ablation studies, cross-representation-space comparisons, and computational efficiency analysis. The results show that VARR achieves an average removal success rate of 92.8% across eight latent-space watermarking methods, while also obtaining favorable FID. Compared with attack methods that require per-image optimization, VARR also has clear computational efficiency advantages. Overall, the experimental results support the main conclusions of the paper and demonstrate a competitive trade-off among watermark removal effectiveness, image quality, and computational efficiency.

Critical weaknesses
  1. The paper demonstrates that the method can remove watermarks when low-scale information does not contain much watermark evidence. However, does it sufficiently prove that this condition actually holds for such low-scale information in realistic settings? The authors are advised to provide further theoretical analysis or more direct experimental evidence to verify this key assumption. In addition, it is necessary to further clarify whether the success of the method arises only from the fact that high-scale tokens happen to carry watermark information more easily. If so, would other simpler high-scale removal methods also be effective? Please justify the necessity and advantages of the proposed method over alternative approaches. We would like to see improvements in this regard, either theoretically or experimentally.

  2. VARR does not outperform all existing methods in terms of average removal success rate on latent-space watermarks. Its main advantage is that, while achieving a relatively high removal success rate, it significantly reduces computational overhead compared with attack methods that require per-image optimization, and it achieves better FID. This can indeed be considered an innovative contribution. However, we note that the core operations of the current method consist of only three relatively simple steps. Is there further room for methodological improvement, for example through more refined operations, to improve the removal rate? We hope the authors can provide more discussion on this issue.

  3. Highly relevant prior work along the research direction and the attack/defense path considered in this paper has not been adequately incorporated or discussed. The robustness of latent-domain watermarks under reconstruction-based attacks has already been discussed in DiffuseTrace, which should be regarded as an important prior work for the research idea of this paper. Important existing watermarking methods such as DistriMark are not introduced. The paper should also clarify the limitations of the proposed removal attack with respect to the watermarks it targets. For example, for methods such as ALIEN, which adopt a time-dependent watermark embedding mechanism and whose watermark extraction process differs from inversion of the initial latent variables, is VARR still effective? We fully understand that the authors cannot comprehensively cover all existing work due to space limitations, but a more systematic review of related work and a few key additions would improve the rigor and readability of the paper.

Overall, we are currently inclined to support acceptance of this paper, but the above issues still deserve further clarification. We will reassess the impact of these issues on the core contribution and final recommendation in light of the explanations and evidence provided by the authors during the rebuttal phase.

Questions for authors

No additional questions beyond the concerns raised in the Critical Weaknesses section. We welcome further clarification and supporting evidence from the authors during the rebuttal phase.

1 comment
AuthorsOpenReview ↗

Response to Reviewer 2

We thank the reviewer for the helpful suggestions.

For W1: Regarding the limited watermark evidence in low-scale information, we conducted an additional experiment. Specifically, we first used a multi-scale feature extractor to obtain continuous feature representations of watermarked images. We then reconstructed the images using only the first eight scales, discarding the last two scales, which can be regarded as an ablation study of the VAR Regression module. We observed a significant decrease in the watermark detection rates across different watermarking methods, supporting Assumption 5.

Table 1: Ablation study on the VAR Regression module.

(a) Without VAR Regression Module

WatermarkWSR ↑PSNR ↑SSIM ↑FID ↓CLIP ↑
Tree-Ring100.00%17.88580.438282.55170.2788
Gaussian Shading69.80%16.31080.488988.22040.2808
PRC100.00%17.00610.470151.72650.2799
HSTR96.45%16.62130.405193.08150.2821

(b) With VAR Regression Module

WatermarkWSR ↑PSNR ↑SSIM ↑FID ↓CLIP ↑
Tree-Ring99.25%21.08500.591214.11710.3146
Gaussian Shading42.60%22.48720.545242.36990.3144
PRC100.00%19.87660.629913.75100.3347
HSTR80.26%18.71340.554834.24850.3273

We recognize that watermark evidence being primarily concentrated in high-scale features is a key reason for the effectiveness of our method. However, this property is shared by mainstream diffusion model watermarkings, highlighting the broad applicability of VARR. We also recognize that simpler high-scale removal methods, such as directly discarding the last two scales without regeneration, can remove watermarks, but at the cost of visual fidelity. By leveraging the strong image generation prior of VAR, VARR achieves a better trade-off between visual fidelity and watermark removal success rate.

For W2: We thank the reviewer for recognizing our contributions in terms of visual fidelity and efficiency. However, we would like to clarify that our method achieves the highest average watermark removal success rate among all compared methods. Although VARR does not achieve the best performance on certain watermarks (e.g., Gaussian Shading), it achieves the best overall average removal success rate across all evaluated watermarkings.

We recognize that the core idea of our method is simple, but we consider this as one add-value of the paper, as this simple attack is applicable to a series of watermarking methods. In addition, our contributions also include the theoretical analysis of why diffusion regeneration attacks fail against latent-domain watermarks, the simultaneous improvements in watermark removal success rate, visual fidelity, and computational efficiency, and the insights into why VARR is effective, including the advantages of the VAR domain for watermark removal and VAR’s strong image generation prior for preserving visual fidelity. Regarding potential methodological improvements, we have added a discussion to the Limitations section of the revised manuscript and leave further exploration for future work.

For W3: We thank the reviewer for pointing out these relevant works. We agree that they are directly related to our research and have added the corresponding references to the Related Work section of the revised manuscript. However, we emphasize that our method has been evaluated against 13 diffusion model watermarking and compared with 12 attack baselines, providing a more comprehensive evaluation than Diffusion Regeneration (4 watermarking methods and 8 attack baselines) and CtrlRegen (6 watermarking methods and 3 attack baselines). Regarding ALIEN, it is concurrent work with ours, and its code has not been publicly released. Therefore, we did not include it in our evaluation.

We welcome any further questions from the reviewer and would be pleased to provide additional clarifications in further discussion.

Reviewer_T7V7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper has three main contributions. First, it provides the first theoretical explanation of why latent-domain watermarks work well against diffusion regeneration attacks. Second, it proposes a method called VARR that preserve low-scale semantic tokens while regenerates high-scale tokens by using VAR. Third, authors conduct extensive experiments across 8 latent-domain watermarking methods and show their method achieve high removal success rate with high visual fidelity.

Critical strengths
  1. VARR makes the latent-domain watermark cheap to break. It removes them in a single path, without the need for training and the need to access the detector.
  2. The paper tests 13 watermarking methods against 12 baselines on two datasets with 5000 images each. This makes the main result credible.
Critical weaknesses
  1. The central assumption is insufficiently supported, which limits the scope of the claims. On line 265-266, the paper stated "Assuming that watermark evidence is primarily embedded in high-scale token maps". And the theorem 2 depends entirely on assumption 5, which literally states that the preserved low-scale tokens carry little watermark evidence (small ηm\eta_m). The paper cites Table 5 as empirical support, but Table 5 only shows that regenerating high-scale tokens removes more watermark than regenerating low-scale tokens. It doesn't show that the low-scale tokens carry little watermark evidence, and the ηm\eta_m is never measured. So VARR's guarantees apply only to watermarking methods whose evidence resides mainly in fine-scale tokens. But the paper claims VARR a "fundamental vulnerability" of latent-domain watermarks (line 26) and calls VARR "universal" (line 78). Watermarks that deliberately embed into coarse or low-scale structure are not evaluated. I suggest the authors (a) directly estimate ηm\eta_m, or the watermark detectability of images reconstructed from LmL_m alone, and (b) evaluate at least one coarse structure watermark or an adaptive watermark embedded in low-scale VAR tokens, or else narrow the claims accordingly.

  1. Right now in the table every attack sits at a single operation point, so best trade-off can't really be judged. I suggest authors to sweep each attack's strength (Diff strength, JPEG quality, VARR's m, etc) and plot removal rate against PSNR/SSIM/FID/CLIP.

  1. The cross-domain analysis(Table 3,6,7) is conducted only on Tree-Ring, yet it supports a general claim that the VAR domain is more effective for watermark removal than the pixel and frequency domains(section 5.3). Tree-Ring is the method on which VARR performs best (99.56%WSR), so it is not a fully informative test case. The author should repeat the analysis on additional latent-domain watermarks, particularly those where VARR is weaker (e.g., Gaussian Shading, HSTR) and those with a different embedding structure (e.g., PRC).

  1. In Table 3,6,7, only removal rates are reported but not the visual fidelity metrics. Since the same budget p% corresponds to different amounts of modification in the pixel, frequency and VAR domains, a domain may achieve a higher WSR simply by degrading the image more. I suggest that the authors report visual fidelity (e.g., PSNR, SSIM, FID) at each budget, so that the domains can be compared at matched image quality.
Questions for authors
  1. Table 4's FID at m=3-7 is better than the default m=2's FID of 28. I find it odd because more regeneration shouldn't improve FID.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 1

We thank the reviewer for the helpful suggestions.

For W1: Regarding (a), we conducted an additional experiment. Specifically, we first used a multi-scale feature extractor to obtain continuous feature representations of watermarked images. We then reconstructed the images using only the first eight scales, discarding the last two scales, which can be regarded as an ablation study of the VAR Regression module. We observed a significant decrease in the watermark detection rates across different watermarking methods, supporting Assumption 5.

Table 1: Ablation study on the VAR Regression module.

(a) Without VAR Regression Module

WatermarkWSR ↑PSNR ↑SSIM ↑FID ↓CLIP ↑
Tree-Ring100.00%17.88580.438282.55170.2788
Gaussian Shading69.80%16.31080.488988.22040.2808
PRC100.00%17.00610.470151.72650.2799
HSTR96.45%16.62130.405193.08150.2821

(b) With VAR Regression Module

WatermarkWSR ↑PSNR ↑SSIM ↑FID ↓CLIP ↑
Tree-Ring99.25%21.08500.591214.11710.3146
Gaussian Shading42.60%22.48720.545242.36990.3144
PRC100.00%19.87660.629913.75100.3347
HSTR80.26%18.71340.554834.24850.3273

Regarding (b), to the best of our knowledge, no existing watermarking methods explicitly embed watermarks into coarse-scale structures. We have evaluated representative diffusion model watermarkings and added this limitation to the Limitations section.

For W2: Among the compared attacks, VARR achieves the best FID and a competitive CLIP score, while also attaining the highest watermark removal success rate. Its superior performance in both aspects demonstrates a better trade-off than existing attacks under the evaluated settings. Although adjusting the parameters of other attacks may improve their performance on one metric, such improvements typically come at the expense of the other metric. Therefore, such parameter adjustments do not necessarily lead to a better overall trade-off than VARR. Nevertheless, we acknowledge that sweeping the attack strength of each baseline is necessary for a more comprehensive comparison. Given the substantial computational cost involved, we will incorporate these additional experiments into the manuscript before the final version is submitted. Regarding VARR's (m), we have already discussed its impact in the Parameter Ablation section and Table 4. We further include results for m=0m=0 and m=1m=1 in the revised manuscript.

For W3: We would like to clarify that we originally chose Tree-Ring because it is a frequency domain method, which obvsiously has a mismatch of the domain of our attack (VAR domain). However, VARR achieved better attacking performance compared to frequency domain attacks, justifying the effectiveness. To provide a more comprehensive evaluation, we have further extended the experiments to include Gaussian Shading, which embeds watermarks in the pixel domain of the initial Gaussian noise.

Table2: Cross-domain comparison of inpainting-based watermark removal against Gaussian Shading.

Domain2%5%7%10%20%40%
Pixel0.000.000.000.000.052.20
Frequency0.000.000.000.000.000.00
VAR0.000.000.000.504.2010.20

Table3: Cross-domain comparison of replacement-based watermark removal against Gaussian Shading.

Domain2%5%7%10%20%40%
Pixel0.000.000.000.201.202.40
Frequency0.000.000.000.000.000.00
VAR0.000.000.200.851.803.60

Table4: Cross-domain comparison of restoration-based watermark restoration against Gaussian Shading.

Domain2%5%7%10%20%40%
Pixel0.000.000.200.803.6014.50
Frequency0.000.200.802.409.8017.20
VAR1.204.8011.4017.6029.8044.20
AuthorsOpenReview ↗

Response to Reviewer 1

For W4: We acknowledge that modifying the same proportion of elements across different domains does not necessarily result in the same level of visual fidelity degradation. However, determining the modification ratio required to achieve same visual fidelity degradation would incur substantial computational overhead, as the corresponding ratio may vary across individual images. We emphasize that the primary purpose of this experiment is to provide insights into why VARR is effective, particularly the advantages of performing modifications in the VAR domain. Therefore, we believe that comparing different domains under the same modification ratio is a reasonable experimental setting for this purpose.

For Q1: Thank you for pointing this out. The difference in FID values arises from the different datasets and sample sizes used in the two experiments. Specifically, Table 2 reports the average FID over DiffusionDB and MS COCO 2017, with 5,000 images from each dataset (10,000 images in total), while Table 4 reports FID using only 1,000 images from MS COCO 2017. Therefore, the FID values in these two tables are not directly comparable. We have clarified this distinction in the revised manuscript.

We welcome any further questions from the reviewer and would be pleased to provide additional clarifications in further discussion.

Comments on the paper

3 comments

Dear Reviewers,

Thank you for your time and effort in reviewing this paper!

The authors have now posted their responses. I would like to kindly encourage you to review their rebuttal and engage in the discussion, particularly if any of your concerns remain unresolved.

Your feedback is greatly appreciated and will help us reach a fair and well-informed decision.

Best regards, Area Chair, ICLR 2027

Dear Area Chair and Reviewers,

We have uploaded a revised version of our manuscript last week. Compared with the original submission, we have made the following revisions to address the reviewers' comments and improve the clarity and completeness of the paper:

  1. Related Work: We have added references to DiffuseTrace, DistriMark, ALIEN, RAVEN, Infinity, and BitMark to provide more comprehensive coverage of related work.

  2. Section 5.1 (Experimental Settings): We have clarified how the results from the two datasets are aggregated and specified the reference dataset used for FID evaluation to improve the reproducibility of our experiments.

  3. Section 5.2.1 (Removal Effectiveness): We have corrected the average WSR reported for UnMarker to ensure a fair comparison with VARR.

  4. Section 5.4 (Separability Analysis of Attacked and Unwatermarked Images): We have added a separability analysis between VARR-attacked images and unwatermarked images to further evaluate the effectiveness of watermark removal.

  5. Ablation Study: We have extended the parameter ablation study to include m=0m=0 and m=1m=1, and added an ablation study of the VAR Regression module to better understand its contribution.

  6. Section 6 (Limitations) and Section 7 (Conclusion): We have added a dedicated Limitations section and expanded the discussion of future research directions in the Conclusion.

  7. Appendix D.2 (Watermark Detection Configurations): We have provided detailed watermark detection configurations to improve the reproducibility of our experimental results.

  8. Appendix D.3 (Cross-Domain Watermark Removal): We have extended the cross-domain analysis to include Gaussian Shading, providing additional evidence of the advantages of performing watermark removal in the VAR domain.

We sincerely thank the Area Chair and reviewers for their time and constructive feedback, which have helped us improve the manuscript. We welcome any further questions or comments and would be pleased to provide additional clarifications during the discussion.

Best regards,
The Authors

Dear Reviewers,

Thank you for your time and effort in reviewing our paper.

We have posted our rebuttal and would appreciate it if you could review our responses and share any remaining concerns. We would be happy to provide further clarification on any questions you may have.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.