acceptodds
Withdrawn from ICLR 2027

ResMark: Late-Stage Residual Insertion for Fidelity-Preserving Watermarking in Diffusion Denoisers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Text-to-image diffusion models need reliable watermarking for provenance and attribution. Denoiser-side watermarking couples the watermark to the iterative denoising process. A first-order sensitivity analysis suggests that early perturbations are transported through more subsequent solver transitions. As a result, the watermarked output can differ from its watermark-free counterpart under matched prompts and noise—a phenomenon we call paired-output drift. We formulate denoiser-side watermarking as late-stage residual insertion. Low-Noise Training (LNT) trains the watermark LoRA on low-noise timesteps. At inference, late-step activation enables it only over the final solver steps through a soft ramp. Because the first active step begins from a watermark-free trajectory, we propose a Teacher-Relative Insertion Loss (TRI), which trains the LoRA-enabled prediction to match the frozen base model prediction plus the desired watermark residual on the same watermark-free latent. Across SD1.5 U-Net and PixArt transformer denoisers, ResMark substantially improves paired-output fidelity over full-trajectory denoiser watermarking while retaining strong message recovery and robustness to conventional distortions and AI-regeneration attacks.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
diffusion models, text-to-image generation, diffusion denoisers, denoiser-side watermarking, generative watermarking, LoRA, image fidelity

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7F8zOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies multi-bit watermarks inserted by the denoiser of a text-to-image diffusion model. It argues that a watermark LoRA active at every sampling step, as in AquaLoRA, changes layout and content relative to the watermark-free image from the same prompt and seed. ResMark keeps AquaLoRA's message-conditioned LoRA and pretrained watermark encoder and decoder, and changes three things. It trains the LoRA only on low-noise timesteps (t≤160t \le 160). It switches the LoRA on only for the last 44 of 2525 solver steps. Its teacher-relative loss (TRI) makes the LoRA's clean-latent estimate match the frozen model's estimate plus the watermark residual, the encoder's output for the message.

Experiments on Stable Diffusion 1.51.5 and PixArt-α\alpha compare ResMark with Stable Signature, LaWa, StableGuard, MOLM and AquaLoRA. They measure detection, paired fidelity to the watermark-free image (PSNR, SSIM, LPIPS), Δ\DeltaFID and robustness to 10 distortion and regeneration attacks.

Critical strengths

The paper aims at an important problem of Diffusion Watermarking. The ablation on PixArt separates what each part of the method does.

Critical weaknesses
  1. The novelty is limited. Each part comes from earlier work. The LoRA, message mapper and watermark encoder and decoder are AquaLoRA's. TRI (Eq. 2) is the first term of SleeperMark's Eq. 6, as the paper notes. WaDiff (Min et al., ECCV 2024, not cited) already applies a per-user message only below a timestep threshold, in training and sampling, to keep outputs from the same noise consistent. The novelty claim at L125-127 does not hold.

  2. The trained LoRA adds little over adding the watermark residual directly to the final latent (App. D.3, PixArt). On clean images this shortcut reaches 96.92% bit accuracy and PSNR 31.89 dB, against 98.39% and 32.69 dB for ResMark (Table 10). ResMark's lead in bit accuracy is 7 to 10 points under SDEdit and Rinse-2 (Table 11). The gain is even worse under attacks.

  3. The only analysis does not explain why the method works. It shows that a change at an earlier sampling step passes through more later steps (App. B), which is true by construction. Appendix B shows this does not imply that early changes cause more drift (L770-771). Nothing explains why TRI works or why the watermark survives regeneration.

  4. Inversion-based baselines are missing. The paper puts LoRA deactivation and white-box removal out of scope. In the owner-controlled setting that remains, inversion-based methods apply directly. The paper cites Tree-Ring and Gaussian Shading but does not compare with them. Shallow Diffuse (NeurIPS 2025, not cited) reports this paper's paired-fidelity metrics with detection rates under regeneration, rinsing and averaging attacks, so the comparison is feasible but totally not conducted.

  5. The comparison is unfair. Many new watermarking methods are missed in the comparison of this work, including: [1-4]. I request the authors compare with these works.

[1] Xiaoxiao Hu, Jiaqi Jin, Sheng Li, Wanli Peng, Xinpeng Zhang, and Zhenxing Qian. Spherical watermark: Encryption-free, lossless watermarking for diffusion models. In The Fourteenth International Conference on Learning Representations, 2026.

[2] Jan Kociszewski, Hubert Jastrzebski, Tymoteusz Stepkowski, Filip Manijak, Krzysztof Rojek, Franziska Boenisch, and Adam Dziedzic. SERUM: Simple, efficient, robust, and unifying marking for diffusion-based image generation. In The Fourteenth International Conference on Learning Representations, 2026.

[3] Samuel Gunn, Xuandong Zhao, and Dawn Song. An undetectable watermark for generative image models. In International Conference on Learning Representations.

[4] Wenda Li, Huijie Zhang, and Qing Qu. Shallow diffuse: Robust and invisible watermarking through low-dim subspaces in diffusion models. In Advances in Neural Information Processing Systems 38, NeurIPS 2025, pp. 170876–170906. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2025. doi: 10.52202/085713-5147.

  1. Each attack is run at one mild setting, such as SDEdit at strength 0.1, and scored only by bit accuracy (Tables 2 and 6). At the paper's false-positive rate of about 1e-6, a 48-bit message needs 41 correct bits for detection. After Rinse-2, ResMark recovers 36.8 bits on average (76.64%). Unless the per-image distribution is strongly bimodal, most Rinse-2 images go undetected. The paper does not run the attacks it cites to motivate denoiser-side insertion (Zhao et al., 2024; Liu et al., 2025; L44).

  2. The paper doesn't evaluate the robustness under adversarial [9], new removal [10-11], or forgery attacks [5-8], which are evaluated by many existing works. I request the authors to evaluate with these attacks to show the robustness of the method.

[5] Pei Yang, Hai Ci, Yiren Song, and Mike Zheng Shou. Can simple averaging defeat modern watermarks? volume 37, pp. 56644–56673, 2024a.

[6] Anubhav Jain, Yuya Kobayashi, Naoki Murata, Yuhta Takida, Takashi Shibuya, Yuki Mitsufuji, Niv Cohen, Nasir Memon, and Julian Togelius. Forging and removing latent-noise diffusion watermarks using a single image. arXiv preprint arXiv:2504.20111, 2(7), 2025.

[7] Andreas M¨uller, Denis Lukovnikov, Jonas Thietke, Asja Fischer, and Erwin Quiring. Black-box forgery attacks on semantic watermarks for diffusion models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20937–20946. IEEE, 2025.

[8] Ziping Dong, Chao Shuai, Zhongjie Ba, Peng Cheng, Zhan Qin, Qinglong Wang, and Kui Ren. Wmcopier: forging invisible watermarks on arbitrary images. volume 38, pp. 138035–138065,2026.

[9] Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstein, et al. Waves: Benchmarking the robustness of image watermarks. In International Conference on Machine Learning, pp. 1456–1492. PMLR, 2024.

[10] Aistov, K., Abud, K., Serzhenko, I., Kovalev, E., Yakushev, A., Akimenkov, A., ... & Antsiferova, A. (2026). Exploring Weaknesses of Generative Image Watermarks against Latent Frequency Masking. arXiv preprint arXiv:2610.02010.

[11] Kassis, A., & Hengartner, U. (2025, May). Unmarker: A universal attack on defensive image watermarking. In 2025 IEEE Symposium on Security and Privacy (SP) (pp. 2602-2620). IEEE.

Questions for authors
  1. What is the TPR at your false-positive rate of about 1e-6 under each attack in Table 2? Please include SDEdit and rinsing at two or three increasing strengths.

  2. How does ResMark compare with Gaussian Shading, Tree-Ring and Shallow Diffuse on Stable Diffusion 1.5 under your protocol?

  3. How does ResMark stand under the averaging removal and forgery attacks that MOLM, a Table 1 baseline, evaluates? Given Appendix D.3, I expect forgery to succeed. Evidence otherwise would largely address my concern.

  4. What does ResMark's schedule do that WaDiff's timestep threshold does not, beyond exactly recovering the base model while the LoRA is off and using a narrower window? A comparison, or an argument for why that matters, would change my novelty assessment.

  5. Why are your LaWa PSNR and MOLM bit accuracy lower than in their own papers? LaWa's paper reports PSNR 33.52 dB at 48 bits, against your 29.05 dB. MOLM's paper reports about 98% bit accuracy, against your 89.10%. At 33.52 dB, ResMark would not be the best on PSNR.

Reviewer_NVc2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ResMark, a denoiser-side watermarking method that aims to preserve diffusion-model outputs under matched prompts and initial noise. Building on AquaLoRA’s message-conditioned LoRA framework, ResMark combines low-noise training, soft activation during the final sampling steps, and a teacher-relative insertion loss (TRI). TRI encourages the adapted model’s clean-latent prediction to match the frozen model’s prediction plus a target watermark residual, using the same watermark-free input. Experiments on SD1.5 and PixArt-α demonstrate substantial improvements in paired-output fidelity over full-trajectory AquaLoRA-style watermarking while maintaining reliable message recovery.

Critical strengths
  1. The paper is well structured and easy to follow. The motivation, method, and experimental results are presented in a logical sequence.

  2. The controlled ablation demonstrates that replacing PPFT with TRI under the same low-noise training and late-activation configuration substantially improves message recovery.

  3. Evaluation across two denoiser architectures. The paper evaluates ResMark on both the SD1.5 U-Net and the PixArt-α transformer denoiser. This provides useful initial evidence that the approach is applicable beyond a single architecture. The accompanying ablations on activation schedules and LoRA rank further help characterize the method’s behavior.

Critical weaknesses
  1. The paper places substantial emphasis on preserving the composition of the watermark-free output under the same prompt and random seed. However, sampling-based methods such as Gaussian Shading and Tree-Ring may produce a different composition while still generating natural, visually appealing, and prompt-consistent images. A deviation from a particular reference image does not, by itself, imply degraded generation quality. Meanwhile, ResMark introduces low-frequency color artifacts, as acknowledged in Section 4.5, which can directly affect perceived image quality despite high paired similarity. Although the paper distinguishes paired fidelity from distribution-level quality, the evaluation does not establish whether users actually prefer its composition-preserving outputs over alternative, potentially less artifact-prone generations. A blinded subjective comparison with sampling-based methods is needed to substantiate the practical benefit of this trade-off; the reported paired metrics and ΔFID do not resolve this concern.

  2. ResMark retains AquaLoRA’s overall watermark encoder–decoder and message-conditioned LoRA pipeline, including its pretrained watermark components and message-mapping mechanism. The primary changes are restricting training to low-noise timesteps, activating the LoRA during the final sampling steps, and modifying the training objective to accommodate watermark-free inputs. These changes yield meaningful improvements in paired-output fidelity, but the underlying framework remains largely unchanged. The training modifications are reasonable adaptations to the selected inference schedule, yet offer limited conceptual novelty, particularly given prior teacher-relative residual-matching approaches. The ablations demonstrate that the combination is effective, but I remain unconvinced that the methodological advance is sufficiently substantial.

  3. The paper lacks robustness comparisons with sampling-based methods such as Gaussian Shading and Tree-Ring, which are relevant alternatives under its owner-controlled generation setting. Moreover, its advantage over decoder-side methods is mixed: on SD1.5, LaWa achieves higher bit accuracy under all seven conventional distortions, while Stable Signature performs better under color jitter and cropping. Furthermore, although Table 1 reports paired-fidelity metrics, the paper lacks qualitative comparisons or subjective perceptual evaluations against Stable Signature and LaWa, leaving their relative visual quality and the perceptual impact of watermark artifacts insufficiently assessed. ResMark does outperform these methods under the evaluated regeneration attacks, but its gains over AquaLoRA in that setting are modest and lack uncertainty estimates. Consequently, the current evidence does not establish a clearly preferable balance among perceptual quality, conventional-distortion robustness, and regeneration robustness. Comparisons with sampling-based baselines, including detection TPR at a specified FPR and comparable generation-quality conditions, are needed.

Questions for authors
  1. Can you provide a blinded subjective comparison with Gaussian Shading, Tree-Ring, Stable Signature, and LaWa, evaluating image naturalness, prompt alignment, and artifact visibility separately from paired similarity? Please also quantify the prevalence and severity of the color artifacts acknowledged in Section 4.5.
  2. Could you clarify the principal methodological insight beyond modifying AquaLoRA’s training objective and activation schedule? In particular, how does the proposed combination compare with timestep-weighted teacher-relative residual training within the same message-conditioned LoRA framework, using matched watermark components, training data, and optimization budgets?
  3. Can you include robustness comparisons with sampling-based methods under comparable generation and detection conditions, accounting for differences in payload capacity? Please report post-attack TPR at a specified FPR and uncertainty estimates for the small improvements over AquaLoRA. Given the lower robustness than LaWa under conventional distortions and the remaining visual artifacts, under which operating conditions does ResMark provide a preferable perceptual-quality–robustness trade-off?
Reviewer_mfAfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Existing denoiser-side watermarking methods for text-to-image diffusion models, such as AquaLoRA, activate a watermark LoRA over the full sampling trajectory. As a result, early perturbations propagate and change the output compared with its watermark-free counterpart under the same prompt and noise. To address this paired-output drift, this paper formulates watermarking as late-stage residual insertion, named ResMark. Specifically, it trains the LoRA only on low-noise timesteps (LNT), activates it only over the final solver steps through a soft ramp, and proposes a Teacher-Relative Insertion Loss (TRI) that matches the LoRA-induced clean-latent shift to the target watermark residual on a watermark-free latent. Experiments on SD1.5 and PixArt-α show that ResMark greatly improves paired-output fidelity over full-trajectory watermarking while maintaining strong message recovery and robustness to distortions and AI-regeneration attacks. The paper thus effectively shows that watermarking can be confined to late denoising steps without losing detectability.

Critical strengths
  • S1: The paper addresses a clear and practical problem with a simple solution that is easy to adopt. It identifies paired-output drift, where a watermark that is active over the full sampling trajectory changes the content and layout of the output compared with its watermark-free counterpart. Because a zero modulation exactly recovers the frozen base model, keeping the LoRA off until the final solver steps keeps the two trajectories identical and directly removes the source of this drift.
  • S2: ResMark greatly improves paired-output fidelity without sacrificing watermark detection or robustness. Compared with full-trajectory denoiser-side watermarking, it brings paired fidelity to a level comparable to decoder-side methods, while maintaining strong message recovery and robustness to signal-processing distortions and AI-regeneration attacks. These gains hold on both a U-Net (SD1.5) and a transformer (PixArt-α) denoiser, which shows that the approach works across the two main types of denoiser architecture.
Critical weaknesses
  • W1: The significance of the problem and the need for a denoiser-side solution are not established. The paper does not show why paired-output fidelity is an important requirement, since users usually see only the watermarked image and the drifted outputs remain realistic and prompt-consistent. Even if the requirement is accepted, the owner-controlled setting assumed by the paper is exactly where decoder-side and post-hoc watermarks already preserve the matched output by design. In this setting, the late-step gate also removes the main reason for model-integrated watermarking, because it cannot be merged into released weights. The remaining advantage, robustness to regeneration, is shown only under mild attacks and with small margins over AquaLoRA. As a result, it is unclear both why the problem matters and why a denoiser-side solution is needed to solve it.
  • W2: The novelty is limited, and the source of the improvement is unclear. The method combines components from AquaLoRA and SleeperMark with timestep restriction, and its theoretical support is a first-order chain-rule expansion. The experiments do not show that the new parts, LNT and TRI, are what make the method work. Turning off a known watermark LoRA during early steps already gives most of the fidelity gain, and no comparison is made at matched detection or against a simply weakened full-trajectory watermark. Furthermore, the learned signal performs similarly to directly adding the fixed residual Δ(m) to the final latent. Taken together, it remains unclear whether ResMark offers a new insight about denoiser-side watermarking, or mainly a tuned schedule that delivers an existing latent watermark at a different time.
Questions for authors
  • Q1: In what concrete deployment scenario is paired-output fidelity required, and why is a denoiser-side watermark preferable there to decoder-side or post-hoc watermarking under the owner-controlled threat model? In particular, does ResMark keep a clear robustness advantage over decoder-side methods such as LaWa under stronger regeneration attacks, for example higher SDEdit strengths and the clean-noise regeneration attack of Liu et al. (2025)?

  • Q2: How does ResMark compare with simpler alternatives at matched detection performance, such as AquaLoRA with a reduced LoRA scale over the full trajectory, the AquaLoRA (PPFT) checkpoint with the same late gate, and TRI with full-trajectory activation? Fidelity–detection trade-off curves for these variants would show whether LNT and TRI improve the trade-off itself or only change the operating point.

  • Q3: What signal does the trained LoRA actually insert into the final latent? For example, how much of Δ(m) reaches the final latent after the ramped steps, and does the inserted residual change with image content or stay almost the same for a given message? If it is close to a fixed pattern, how does ResMark differ in principle from directly adding Δ(m), and how robust is it to averaging-based removal and forgery attacks?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.