acceptodds
Withdrawn from ICLR 2027

RepMark: Representation-Stable Radioactive Latent Watermarking for Diffusion Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Watermarks for image provenance do not necessarily survive downstream model training, limiting their use for tracing unauthorized training data. We introduce RepMark, a radioactive latent watermark designed to remain recoverable across representation changes and to propagate into diffusion models. RepMark maps a multibit owner message to a bounded displacement in shared writing coordinates, creating a consistent signal across protected images. A LoRA-adapted decoder writes the signal, while a private LoRA-adapted encoder and a proxy-VAE training path support message extraction after representation changes. The resulting interface supports post-processing of completed images, in-processing through compatible VAE replacement, and pre-processing for training-data tracing. In the last setting, the watermark is inherited through training, eliminating an additional online writing step at deployment and enabling black-box verification from novel model outputs. Across PixArt fine-tuning and LightningDiT training from scratch, RepMark achieves – per-image presence AUROC in the evaluated settings. Multi-image aggregation further recovers the registered owner message, including after protected training data are encoded by an external SD3.5 VAE, while maintaining image quality and limited separability under specified public-representation tests.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Watermark, Image Generation

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_WSQSOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RepMark, a representation-stable latent watermarking framework designed to trace the use of protected images in downstream diffusion models. The key idea is to improve radioactive watermark propagation through two mechanisms: (1) shared writing coordinates established by a frozen VAE, which allow the same message to induce a carrier-independent latent displacement across images, and (2) cross-representation verification, implemented using a private LoRA-adapted encoder and a proxy-VAE training path, to improve watermark recovery after representation changes. RepMark supports three deployment scenarios: post-processing, in-processing, and training-data protection.

The authors evaluate RepMark through fine-tuning and training-from-scratch experiments using PixArt and LightningDiT, with additional tests involving an unseen SD3.5 VAE. Results show strong watermark detection and message recovery under several settings, including near-perfect detection in some model-training experiments. The paper also studies the effects of the protected-data fraction, training budget, query aggregation, and image-processing attacks. Overall, the work presents a unified framework for latent watermarking and model-level tracing, while its effectiveness depends on the downstream training regime and the proportion of protected training data.

Critical strengths

A coherent design for radioactive watermark propagation. RepMark explicitly addresses both cross-sample writing consistency and cross-representation recovery. The use of shared VAE writing coordinates and proxy-representation training provides a technically motivated approach to improving watermark inheritance, rather than treating image-level detection as sufficient evidence of model-level propagation.

Broad and informative empirical evaluation. The experiments cover both fine-tuning and training from scratch, multiple resolutions and training budgets, varying protected-data fractions, and an unseen VAE representation. The results also distinguish watermark presence detection from message recovery, providing a more informative evaluation of model-level watermark propagation than a single detection metric.

Critical weaknesses

The practical assumptions behind black-box attribution need stronger validation. In the model-level evaluation, the authors train two separate models under matched settings: a protected model trained on watermarked data and a clean reference model trained on unwatermarked data, using the same architecture, dataset, and training protocol. The clean reference model is then used to calibrate the detection threshold for identifying watermark evidence in the protected model's outputs. While this provides a controlled evaluation, it remains unclear how the threshold can be reliably calibrated when an auditor has access only to a black-box third-party model and cannot obtain a suitably matched clean reference model or its training details. Such limitations may affect the false-positive rate and weaken the practical applicability of the proposed attribution method.

The empirical evidence for the method's comparative advantage is incomplete. RepMark performs strongly, especially at early training stages, but learned baselines such as TrustMark-Q and Watermark Anything also achieve near-perfect detection under some training-from-scratch settings. In addition, the image-quality comparison uses different reference conditions for RepMark and post-processing baselines, making direct comparisons difficult. More carefully matched comparisons would help establish whether the proposed design offers a consistent advantage in propagation, robustness, and visual quality over existing approaches.

Questions for authors

Black-box attribution and calibration: The evaluation uses separately trained clean-reference students with the same architecture, data, and training protocol as the protected students. Can the authors clarify whether this matched reference is required for threshold calibration? If the clean reference is unavailable, or if the target model uses a different training dataset or recipe, how does RepMark distinguish watermark inheritance from naturally occurring detector responses? Results under a more realistic reference-free or mismatched-reference setting would substantially affect my assessment of the practical attribution claim.

Comparative advantage and evaluation fairness: Since TrustMark-Q and Watermark Anything also achieve very high detection performance in some training-from-scratch settings, what specific conditions demonstrate a statistically reliable advantage of RepMark beyond early detection? Can the authors provide matched comparisons using the same protected-data fraction, training budget, and image-quality reference protocol, with variability across independent seeds? This would help determine whether the proposed design provides a robust advantage over existing methods rather than a setting-dependent improvement.

Reviewer_XfwZOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

RepMark proposes a radioactive watermarking system designed to leave recoverable evidence in diffusion models trained on protected images. It maps a multibit message to a content-independent displacement in a frozen VAE latent space, writes the signal through a LoRA-adapted decoder, and extracts it through a private LoRA-adapted encoder. Training includes a frozen proxy-VAE reconstruction path to encourage recovery after representation changes. The same interface supports watermarking existing images, inserting watermarks inside compatible generation pipelines, and tracing protected data through downstream model outputs.

The experiments demonstrate strong watermark-presence transfer under PixArt fine-tuning and LightningDiT training from scratch, including transfer through an SD3.5 VAE excluded from watermark training. These are meaningful results. My main reservations concern whether the evidence establishes reliable owner attribution, whether quality and concealment comparisons capture the complete protection pipeline, and whether the ablations isolate the proposed mechanisms.

Critical strengths

The held-out VAE experiment provides meaningful evidence of representation transfer.

Critical weaknesses
  1. Reliable owner attribution is less well established than watermark presence. The presence detector is deliberately independent of message values, so its near-perfect AUROC cannot by itself establish which owner’s data were used. Although the paper explicitly recognizes this distinction, the attribution evaluation mainly reports average bit accuracy. In Table 8, aggregating 64 PixArt outputs yields bit accuracies of 0.889, 0.803, and 0.915 at the three resolutions. Appendix C.1 reports that even voting over all 500 outputs reaches only 0.938 in the 1024-resolution setting.

Imperfect decoding could still support reliable attribution through a registered codebook or a calibrated error-tolerant matching rule. However, the paper does not evaluate a complete owner-identification procedure with multiple messages, competing owners, correct-owner acceptance, and false attribution. The evidence therefore supports persistent watermark presence and partial message recovery more strongly than reliable recovery of the registered owner’s identity. 2. The image-quality comparison does not measure the complete protection cost against a common reference. Tables 3 and 10 compare RepMark’s protected image with a reconstruction from the same LoRA decoder without message displacement. Other post-processing methods are compared with the original image. RepMark’s reference is useful for isolating the incremental cost of its latent displacement, but it excludes the reconstruction and decoder-adaptation changes introduced when protecting an existing image.

For the data-protection setting, the relevant quality measurement should also compare the final protected image directly with the original input. Without that measurement, the tables do not establish an end-to-end fidelity advantage. Methods additionally retain different native payloads and strengths. This does not invalidate their presence-AUROC comparison, but matched-distortion results would make the performance tradeoff considerably clearer. 3. The concealment interpretation needs to reflect the stronger public-space discrimination results. Neighborhood mixing near 0.5 does not establish that protected images are difficult to distinguish using other classifiers. Table 9 reports linear-probe AUROC of 1.000 in the original, public FLUX.2-dev VAE representation. Figure 11 reports spectral AUROC of 0.994 for original images versus final RepMark images, compared with 0.757 when the negative reference is the LoRA reconstruction.

The authors appropriately limit their claims to specified empirical tests. Nevertheless, these results show that concealment depends strongly on the representation, classifier, and reference distribution. The strong public-space discrimination results deserve comparable prominence to the favorable mixing statistics, particularly when discussing protection of existing images. 4. The main ablations do not cleanly isolate the mechanisms named in the paper. In Appendix C.5, the “without shared writing coordinates” control replaces the frozen writing encoder with the LoRA-adapted encoder. Both encoders still share their parameters across images, and the message-only writer still produces a fixed displacement for a fixed message. This experiment tests the benefit of freezing the writing encoder and separating it from the verification encoder; it does not remove cross-image coordinate sharing. Direct decoding accuracy and distortion also differ between the configurations.

The proxy-VAE ablation in Appendix C.6 is informative but uses a simplified model without image-quality constraints, at approximately 18 dB PSNR, with 40 student outputs per configuration. Its effect should also be measured in the final quality-constrained model. Together, the existing ablations support useful design choices, but provide weaker evidence for the broader causal explanation. 5. The model-level calibration does not yet establish the claimed black-box deployment setting. The problem formulation allows the downstream architecture, data, and training procedure to be unknown. The experiments calibrate detection using clean-reference students matched to those properties. Such controls are valuable for demonstrating inheritance, but their availability in a third-party audit is unclear.

The query-aggregation experiment further estimates thresholds from repeatedly sampled output sets of the available clean-reference students. The paper correctly discloses the overlap between query sets. The remaining question is whether a threshold calibrated on those references controls false attribution on independently trained, previously unseen clean models. This matters especially when moving from a controlled transfer experiment to an operational ownership decision.

Questions for authors

N/A

Reviewer_JusBOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces RepMark, a radioactive watermarking method intended to trace the use of protected images in diffusion-model training. An owner message is mapped to a content-independent displacement in a frozen VAE representation. A LoRA-adapted decoder writes the signal, while a private adapted encoder and proxy-VAE training support recovery after representation changes.

The evaluation covers PixArt fine-tuning, LightningDiT training from scratch, different protected-data fractions, and transfer through an external SD3.5 VAE. The paper reports strong watermark-presence detection and improved message recovery through query aggregation. The supplementary package provides an inference implementation, a checkpoint, and a direct embedding/extraction demo.

Critical strengths
  1. The method addresses a meaningful challenge: a watermark must recur coherently across protected images and remain recoverable after downstream training changes the representation. Fixed writing coordinates, message-dependent displacement, and private recovery form a coherent design. Including unwatermarked reconstructions among the detector-training negatives is also a useful precaution.

  2. The downstream experiments provide substantive evidence of signal inheritance. They cover both fine-tuning and training from scratch, independently trained student instances, and transfer through an SD3.5 VAE excluded from watermark training. The inference supplement adds useful implementation detail, and static inspection of the code and checkpoint structure did not reveal an obvious architectural inconsistency.

Critical weaknesses
  1. Reliable owner attribution, a central motivation of the paper, is not sufficiently validated. The presence score is independent of the registered message, so high presence AUROC does not establish that a student can be attributed to the correct owner. Table 8 reports aggregated BitAcc between 0.803 and 0.915 at 64 queries across the PixArt settings, but no owner-acceptance rule or false-attribution rate against clean students and students carrying other messages is established. Partial recovery could support attribution with an appropriate decision rule; average bit accuracy alone does not tell us whether that rule would be reliable. The released verifier also uses the expected message only to report BitAcc. I consider this a substantial gap in validating the intended tracing outcome.

  2. The reconstruction controls are insufficient to interpret the complete protection pipeline. Tables 3 and 10 compare RepMark with its own unwatermarked LoRA reconstruction, while the post-processing baselines are compared with original images. This disclosed choice isolates incremental watermark distortion, but leaves the total distortion comparison unresolved. Figure 11 also reports a spectral AUROC of 0.994 between original and RepMark images, limiting what can be concluded about concealment for the complete pipeline. More importantly, I could not find a downstream student trained only on unwatermarked LoRA reconstructions. Using these reconstructions as detector-training negatives is helpful, but does not replace this downstream control. The current experiments therefore leave uncertainty about how much inherited detection evidence is specifically attributable to the message-dependent watermark.

Although the reported inheritance results are valuable, these omissions leave the central attribution and complete-pipeline claims insufficiently supported for acceptance. I would raise my score if the authors establish calibrated owner attribution and resolve the reconstruction-control and common-reference comparisons.

Questions for authors
  1. What exact decision rule accepts or rejects a claimed owner from student outputs? Can you report correct-owner attribution and false-attribution rates on clean students and students trained with other registered messages?

  2. Are the model-level thresholds calibrated independently of the evaluated student instances? What evidence shows that these fixed thresholds control false positives on held-out clean students?

  3. Can you provide detection and attribution results for a student trained on unwatermarked LoRA reconstructions, together with image-quality and public-representation comparisons using original images as a common reference across methods?

Reviewer_iExXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RepMark, a latent-space radioactive image watermarking framework against diffusion models. The basic idea is to embed a carrier-independent latent displacement in a shared variational autoencoder (VAE) latent space and train a private verification pipeline that remains recoverable across representation changes. The framework supports three deployment modes: post-processing, in-processing, and pre-processing. Experimental results show that RepMark propagates into downstream diffusion models more reliably than the compared image watermarking approaches.

Critical strengths

RepMark aims to address a challenging and timely problem of detecting unauthorized image training dataset use.

It jointly tackles cross-representation stability, cross-sample consistency, and model-level radioactivity under a unified framework which can be applied to post-processing, in-processing, and pre-processing-based image watermarking schemes.

Critical weaknesses

RepMark is not compared with most relevant radioactive watermarking works, including SERUM (https://github.com/Hubizon/SERUM) and HMark (https://arxiv.org/html/2512.00094v1). The missing comparison with these highly related works raises critical concerns on how much advancement RepMark could really achieve over the state-of-the-art solutions.

The attacking model is relatively weak. RepMark should be evaluated against more advanced attacks, such as watermark-aware removal attacks, ownership collision attacks, latent purification and adversarial retraining, etc.

RepMark heavily relies on a private owner-controlled verification space. This raises another concern on the contributions of the watermarking scheme or detector. The experimental results only show the combined effect, but do not disentangle these factors clearly.

In the paper, it wrote "Unless stated otherwise, model-level results are means over three independent student-training seeds". Using only three seeds does not appear to be able to achieve a sound statistical significance of the results. Other than mean, the confidence interval should also be measured.

Questions for authors

How much advancement RepMark could achieve over state-of-the-art radioactive watermarking schemes, such as SERUM and HMark?

Would RepMark still outperform the compared methods if a public verifier were used instead of the private LoRA encoder?

Would RepMark be robust against an advance attacker that knows the algorithm details of RepMark and trains specifically to suppress attribution?

What is the capacity of watermark embedding? It appears that 16-bit owner messages are evaluated. But what would be the maximum length of messages RepMark could carry?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.