- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper rethinks the verification process for initial-noise watermarks in diffusion models, breaking away from the traditional paradigm that relies on costly diffusion inversion. The authors discovered that, although the similarity between intermediate latent representations and the initial noise decays rapidly, the watermark information remains highly preserved throughout the generation trajectory and even within the final latent representation. Based on this insight, the proposed ZeroStep framework, which involves fine-tuning only a lightweight VAE Encoder LoRA and incorporates trajectory distillation and distortion-resilient training. Experimental results demonstrate that this method significantly improves extraction accuracy and robustness against distortions while reducing computational latency and exhibiting strong cross-model transferability.
Critical strengths
The proposed inversion-free watermark extraction scheme reduces verification latency to 19.0 ms and model parameters from 2.5B to 35.01M.
Critical weaknesses
-
While the trajectory distillation loss in Eq. (7) constrains clean image predictions to match sampled intermediate trajectory states , the distortion-robust loss in Eq. (8) rigidly anchors distorted image predictions to the initial watermarked noise . This design choice lacking theoretical justification.
-
The threat model is overly idealized, as it fails to consider critical security threats such as regeneration attacks (e.g., diffusion regeneration, vae regeneration) and adversarial perturbations.
-
The scope of validation is somewhat limited, as both the main text and appendix evaluate only four initial noise watermarking schemes (Tree-Ring, Gaussian Shading, RingID, and PRC).
-
The learned adapter is strictly coupled to a specific VAE architecture and latent dimensionality. As admitted in Appendix C.2, direct transfer fails across different VAE families.
Questions for authors
-
In Section 3, the authors present the paper's core motivation: watermark recoverability remains high near the generation endpoint despite a rapid drop in latent similarity to the initial noise. However, this motivating experiment is restricted solely to Tree-Ring. Can this analysis be generalized to other initial noise watermarking schemes?
-
While the trajectory distillation loss in Eq. (7) constrains clean image predictions to match sampled intermediate trajectory states , the distortion-robust loss in Eq. (8) rigidly anchors distorted image predictions to the initial watermarked noise . Could you provide some theoretical explanations?
-
The authors introduce into the weighting function to assign higher supervision weights to states farther from the initial noise. There is a lack of theoretical explanation for why states closer to the original watermarked noise should be down-weighted during trajectory distillation.
-
Figure 5 lacks essential experimental details in both the main text and appendix. Specifically, the metric "Correct evidence retained" is left undefined, and the exact setup (model backbone, watermark method, and sample size) used to generate the plot is not specified.
-
Comparing Frozen VAE with ZeroStep under image corruptions is unfair because ZeroStep benefits from data augmentation (Eq. 8), which is not the paper's core contribution. To evaluate the true impact of VAE adaptation, ZeroStep without data augmentation (the "No Noise" setting) should be compared against Frozen VAE.
4 comments
Rebuttal by Authors (2/3)
Re: Robustness and Security Evaluation (W2)
Additional attacks and security robustness. We further evaluate robustness under true crop and resize, as well as compound distortions, to assess generalization beyond the original training distribution. We also include three representative regeneration attacks: Regen-Diff, Rinse-2xDiff, and CtrlRegen. All methods are evaluated on the same attacked images using the original GS verifier.
Robustness under compound distortions and regeneration attacks on SD v2.1 with Gaussian Shading.
| Attack | DDIM-50 | DDIM-1 | FARI | ZeroStep |
|---|---|---|---|---|
| R.Crop + Resize | 0.9522 | 0.8479 | 0.9583 | 0.9997 |
| Resize + JPEG | 0.9818 | 0.9643 | 0.9921 | 0.9991 |
| G.Blur + M.Blur | 0.9813 | 0.9481 | 0.9935 | 0.9995 |
| Regen-Diff | 0.9486 | 0.8300 | 0.9547 | 0.9671 |
| Rinse-2xDiff | 0.9207 | 0.7974 | 0.9411 | 0.9584 |
| CtrlRegen | 0.8953 | 0.7251 | 0.9092 | 0.9204 |
ZeroStep achieves the highest bit accuracy across all evaluated attacks. However, the ability to resist attacks remains largely dependent on the underlying design of the base watermarking methods, as Zerostep serves as a plug-and-play approach to enhance their detection performance.
Re: Generalization to Additional Watermark Schemes (W3)
We thank the reviewer for the suggestion. To further evaluate the applicability of ZeroStep to different watermark mechanisms, we extend our experiments on SD v2.1 to two additional schemes: WIND and Spherical Watermark.
Top-1 identification accuracy of WIND (100,000 candidate keys across 2,048 groups).
| Method | NFE | Clean | Avg. | JPEG | R.Crop | R.Drop | Resize | G.Blur | M.Blur | G.Noise | S&P | Bright |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DDIM-50 | 50 | 0.9990 | 0.9640 | 0.9930 | 0.9170 | 0.9290 | 0.9910 | 0.9920 | 0.9910 | 0.9530 | 0.9470 | 0.9630 |
| DDIM-1 | 1 | 0.9840 | 0.9022 | 0.9490 | 0.8130 | 0.8370 | 0.9320 | 0.9360 | 0.9190 | 0.8920 | 0.8870 | 0.9548 |
| FARI | 1 | 1.0000 | 0.9873 | 0.9990 | 0.9700 | 0.9730 | 0.9990 | 0.9990 | 0.9950 | 0.9820 | 0.9780 | 0.9907 |
| ZeroStep | 0 | 1.0000 | 0.9927 | 1.0000 | 0.9820 | 0.9850 | 1.0000 | 1.0000 | 0.9980 | 0.9870 | 0.9850 | 0.9973 |
ZeroStep achieves the highest average identification accuracy of 0.9927, outperforming FARI while requiring zero denoiser NFE.
Bit accuracy of Spherical Watermark (512-bit payload).
| Method | NFE | Clean | Avg. | JPEG | R.Crop | R.Drop | Resize | G.Blur | M.Blur | G.Noise | S&P | Bright |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DDIM-50 | 50 | 0.9999 | 0.9629 | 0.9829 | 0.9487 | 0.9359 | 0.9987 | 0.9934 | 0.9918 | 0.9321 | 0.9237 | 0.9592 |
| DDIM-1 | 1 | 0.9914 | 0.9214 | 0.9638 | 0.8750 | 0.8642 | 0.9784 | 0.9741 | 0.9593 | 0.8682 | 0.8596 | 0.9500 |
| FARI | 1 | 0.9999 | 0.9785 | 0.9958 | 0.9572 | 0.9436 | 0.9987 | 0.9975 | 0.9954 | 0.9629 | 0.9588 | 0.9966 |
| ZeroStep | 0 | 0.9999 | 0.9903 | 0.9992 | 0.9791 | 0.9748 | 1.0000 | 0.9997 | 0.9982 | 0.9871 | 0.9818 | 0.9928 |
ZeroStep achieves an average bit accuracy of 0.9903, compared with 0.9785 for FARI, further demonstrating its effectiveness across different watermark encoding mechanisms without online diffusion inversion.
Rebuttal by Authors (3/3)
Re: Adapter Generalization and Frozen VAE Comparison (W4 & Q5)
We thank the reviewer for the helpful suggestions.
Cross-backbone adapter reuse. Although direct transfer across different latent dimensions remains challenging, ZeroStep supports bidirectional adapter transfer between SD v1.5 and SD v2.1, as well as direct reuse between SD v3.5 Medium and Large without retraining. We further demonstrate transfer to and from SDXL-Turbo, reducing the need for backbone-specific retraining and lowering deployment costs.
Cross-backbone adapter transfer between SD v1.5, SD v2.1, and SDXL-Turbo.
| Source → Target | Direct Transfer TR ↑ | Direct Transfer GS ↑ | Adapted Transfer TR ↑ | Adapted Transfer GS ↑ |
|---|---|---|---|---|
| SD v1.5 → SDXL-Turbo | 0.987 | 0.9647 | 0.999 | 0.9773 |
| SDXL-Turbo → SD v1.5 | 0.991 | 0.9942 | 0.999 | 0.9997 |
| SD v2.1 → SDXL-Turbo | 0.990 | 0.9655 | 0.999 | 0.9794 |
| SDXL-Turbo → SD v2.1 | 0.989 | 0.9945 | 0.999 | 0.9995 |
Controlled Frozen VAE comparison. To isolate encoder adaptation from distortion augmentation, we introduce ZeroStep (No Noise), which uses the adapted VAE encoder without distortion augmentation. All variants are evaluated on the same test images under identical verification protocols. Results on SD v2.1 are shown below.
| Method | Clean TPR ↑ | Avg. TPR ↑ | Clean Bit Acc. ↑ | Avg. Bit Acc. ↑ |
|---|---|---|---|---|
| Frozen VAE + Original Verifier | 0.866 | 0.235 | 0.9373 | 0.8692 |
| ZeroStep (No Noise) | 0.995 | 0.889 | 0.9945 | 0.9180 |
| ZeroStep (Ours) | 1.000 | 0.999 | 1.0000 | 0.9983 |
Encoder adaptation alone substantially improves verification over the Frozen VAE, while distortion augmentation provides further gains in robustness.
Re: Figure 5 Metric Definition (Q4)
Correct Evidence Retained (CER). We define CER as the fraction of latent elements whose predicted signs match those of the corresponding original watermarked latents. Specifically, we partition the predicted latent elements into four quartiles based on their absolute magnitudes and calculate the sign agreement rate within each quartile. CER is therefore the proportion of sign matched elements in each quartile. We have clarified this definition and the evaluation setting (SD v2.1, GS, and 1,000 test samples) in the revised manuscript.
We sincerely thank the reviewer for the score increase. Your expertise and insightful comments have substantially improved the quality of our paper.
Rebuttal by Authors (1/3)
We thank the reviewer for the insightful and detailed comments. We address each point below.
Re: Generality of Motivation and Trajectory Weighting (Q1 & Q3)
Generality of motivation. Our Tree-Ring analysis shows that latent similarity to the initial watermarked noise decreases along the generation trajectory, while watermark evidence remains recoverable. We extend this analysis to GS and PRC, measuring latent cosine similarity (Ct) and watermark recoverability (Rt) at intermediate and final stages using their original verifiers.
Watermark recoverability along the generation trajectory.
GS and PRC exhibit the same trend, confirming that this behavior is not specific to Tree-Ring. We have added the corresponding visualizations to Figure 8 of the revised manuscript.
Motivation for trajectory weighting. This observation also motivates our timestep weighting strategy. Our goal is to learn verifier compatible watermark representations rather than reconstruct the initial noise. For normalized latent vectors, 1−Ct reflects the deviation from the initial watermarked noise, as ∥zˉt−zˉTw∥22=2(1−Ct). States with high similarity retain watermark evidence but provide supervision largely redundant with initial noise regression. We therefore use 1−Ct to favor more distinct trajectory states, while Rt ensures that their watermark evidence remains recoverable. Accordingly, we define the timestep weight as wt=Rt(1−Ct), balancing supervision distinctiveness and watermark recoverability.
Re: Watermark Trajectory Distillation (W1 & Q2)
Rationale for Distortion Supervision. We thank the reviewer for raising this point. Eqs. (7) and (8) use different supervision targets for complementary objectives. For clean images, intermediate states ztw retain sufficient watermark evidence and provide suitable supervision. For distorted images, we use the initial watermarked noise zTw as a shared alignment target across different views of the same watermark instance. As a sample-specific reference determined by the watermark embedding process, it is independent of timestep sampling and provides consistent alignment across distorted views. This constraint serves as an auxiliary regularizer rather than requiring exact reconstruction of the initial noise. To validate this design, we compare different distortion supervision targets.
Ablation of distortion supervision targets.