acceptodds
Under review as a conference paper at ICLR 2027

IdentityMark: Source-Side Identity Watermarking for Deepfake Provenance

Abstract

Face-swapping deepfakes can misuse publicly shared portraits to transfer a person’s likeness onto attacker-selected target images, spreading misleading information or damaging the victim’s reputation. Since the resulting deepfake may retain little or none of the source portrait’s pixels or shape, conventional watermarks become ineffective in this scenario. We introduce IdentityMark, a source-side iden- tity watermarking framework that traces such deepfakes to the protected sources, supporting deepfake verification when the target image is beyond the defender’s control. Rather than embedding provenance into image content, IdentityMark en- codes a binary message into the facial identity representation transferred during face swapping. A message-to-identity autoencoder produces a message-bearing identity embedding, while diffusion-based latent optimization moves the source identity em- bedding toward this target while preserving visual fidelity and the original identity. Given a suspicious face-swapped image, the message is decoded directly from its facial identity embedding and compared against the victim’s registered watermarks to verify its authenticity. IdentityMark requires no access to or training through downstream face-swap models, making it agnostic to the face-swapping method. In our evaluated face-swapping setting, IdentityMark achieves 71.66% bit accuracy, compared with approximately 50% for other watermarking methods, consistent with random guessing. These results demonstrate the feasibility of source-side provenance protection for tracing and verifying face-swapping deepfakes.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Source tracing in Deepfakes, Source image Watermark preservation, Deepfake provenance, Deepfake verification

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_7zKLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes IdentityMark, a source-side identity watermarking framework for face-swapping provenance. The method encodes a binary message into a facial identity representation, then optimizes a protected source portrait so that its machine-readable identity embedding moves toward a message-bearing pseudo-identity while the image remains visually close to the original. At test time, a suspicious face-swapped image is passed through a face identity encoder and a recovery head to decode the watermark. The paper evaluates bit accuracy on a single dataset, CelebA-HQ, using 29,000 training images and 500 test images at a single resolution of 256, across eight face-swapping methods and two cheapfake methods, and compares against four image watermarking baselines.

Critical strengths
  1. The paper identifies a meaningful problem setting: provenance when the protected image is the identity-providing source portrait rather than the target image. The distinction between target-side watermarking and source-side identity transfer is clearly motivated.

  2. The paper evaluates the proposed method on a relatively broad set of manipulation pipelines, including eight face-swapping methods and two cheapfake methods.

Critical weaknesses
  1. Incomplete novelty positioning against prior source-face protection work. The paper presents source-side identity watermarking as a new setting, but BiFPro: A Bidirectional Facial-data Protection Framework against DeepFake already includes a Face-Out setting for protecting the source face, where the protected face is used to replace other faces and its identity is transferred into fake images.

  2. Incomplete citation and weak baseline positioning. The paper uses LIDMark as a baseline but does not cite All in One: Unifying Deepfake Detection, Tampering Localization, and Source Tracing with a Robust Landmark-Identity Watermark. Moreover, most baselines are target-side watermarking methods, which the authors admit are designed for different settings. Thus, the experiments do not convincingly establish superiority in the source-side setting.

  3. Bit accuracy is insufficient for practical provenance. The paper reports only bit accuracy, with an average of 71.66% for a 32-bit watermark after face swapping. However, provenance requires reliable message-level identification: even one wrong bit may cause a database lookup failure or an incorrect identity match. The paper does not report exact message recovery, matching thresholds, false positives, false negatives, or retrieval accuracy over a realistic registered database.

  4. Unjustified payload design and poor visual quality. The paper does not justify why 32 bits is chosen as the main payload length, nor whether this capacity is sufficient for real-world provenance; the 64-bit and 128-bit variants are only briefly reported in an ablation. Meanwhile, Table 4 shows that IdentityMark uses a shorter payload than all baselines but has much worse visual quality: 31.96 dB PSNR for IdentityMark (32 b), versus 38.52 dB for SepMark-T (128 b), 37.71 dB for StableSignature (48 b), 41.80 dB for EditGuard (64 b), and 43.12 dB for LIDMark (152 b). A PSNR around 31.96 dB can introduce visible artifacts, which limits practical deployment. The paper does not explain why a shorter watermark causes larger distortion.

  5. Missing generalization evaluation. All experiments use only CelebA-HQ at a single 256 resolution, with 29,000 training images and 500 test images. The paper lacks generalization tests on other common face datasets such as LFW or FFHQ, and does not evaluate other resolutions such as 128 or 512. This weakens the real-world validity of the claimed provenance capability.

Questions for authors
  1. How is IdentityMark different from the Face-Out setting in BiFPro: A Bidirectional Facial-data Protection Framework against DeepFake? In both cases, the protected image is the source face whose identity is transferred into fake images.

  2. Why is LIDMark used as a baseline without citing All in One: Unifying Deepfake Detection, Tampering Localization, and Source Tracing with a Robust Landmark-Identity Watermark? In addition, why are most baselines target-side watermarking methods rather than source-side watermarking methods?

  3. Bit accuracy alone is insufficient to demonstrate provenance. Can the authors provide source-tracing or provenance accuracy, e.g., exact message recovery rate, database retrieval accuracy, false positive rate, and false negative rate?

  4. Why is 32 bits chosen as the main payload length, and is this capacity sufficient for real-world provenance? Also, why does IdentityMark have a much lower PSNR than baselines with longer payloads, such as SepMark-T (128 b), EditGuard (64 b), and LIDMark (152 b)? Does the reported PSNR of 31.96 dB introduce visible artifacts?

  5. Does the method generalize beyond CelebA-HQ at a single 256 resolution? Please provide additional generalization experiments on common face datasets such as LFW or FFHQ, and at other resolutions such as 128 and 512.

3 comments

Question 1

We thank the reviewer for pointing out BiFPro and acknowledge that source side Face Out protection has been studied previously. However, our work addresses a different objective. In BiFPro's Face Out setting, the watermark is intentionally destroyed after face swapping, and its absence is used for forgery detection. In contrast, IdentityMark aims to preserve and recover a registered watermark message after face swapping, enabling source tracing verification.

Moreover, the absence of a watermark alone does not necessarily distinguish a manipulated protected image from an image that was never watermarked. BiFPro's Face Out evaluation does not explicitly examine this distinction. Our results also show that conventional image watermarks generally fail to survive under faceswapping, highlighting the challenge of preserving recoverable information under this transformation.

Question 2

Thank you for pointing out the missing LIDMark citation, which we will add.

Regarding baseline comparisons, we acknowledge that these methods were primarily designed for different settings. However, to our knowledge, no existing method directly addresses recoverable source side watermarking under uncontrolled face swapping. Our comparisons are therefore intended to highlight the difficulty of preserving watermark information through identity transfer, rather than claim superiority in the baselines' original settings. Their near random recovery accuracy under the same source side protocol further illustrates this challenge.

Question 3

We agree that bit accuracy alone is insufficient to fully evaluate source attribution. We therefore conducted additional verification experiments using 32-bit and 128-bit watermarks across all 10 manipulation methods.

For more detail please refer to Our answer for Weakness 2 of Reviewer 1

Question 4

We acknowledge that selecting 32 bits as the main configuration was not sufficiently justified. Our initial evaluation focused on bit accuracy and robustness across different face swapping methods. However, we agree that longer payloads are more suitable for provenance verification. We have therefore extended our 128-bit evaluation to all 10 manipulation methods, including message-level verification and source retrieval experiments.

Regarding visual quality, the lower PSNR does not necessarily indicate visible artifacts. Unlike conventional watermarking methods that primarily introduce pixel level perturbations, our method intentionally modifies subtle facial features to shift the machine readable identity representation while preserving visual appearance. Similar trade-offs have been explored in DeepProtect (CVPR 2026), which reports 35.89 dB PSNR and 0.944 SSIM, and Machine Pareidolia (AAAI 2026), which reports 29.07 dB PSNR and 0.876 SSIM. However, we acknowledge that on FFHQ, few protected images may have artifacts because the Diffusion model we used is trained on CelebA.

We acknowledge that the reason for why a shorter watermark causes larger distortion is unclear. Our interpretation for this is that image distortion is not determined solely by payload length, but also by the identity optimization process.

Question 5

We appreciate the reviewer's suggestion. We conducted additional experiments on FFHQ using the pretrained Diffusion model trained on CelebAHQ. With 128-bit watermarks, IdentityMark achieves 97.66% clean bit accuracy, 92.91% under conventional attacks, PSNR 31.38 dB, and 65.41% across five face-swapping methods, demonstrating cross dataset generalization although with some degradation.

FFHQ Face-Swap Evaluation (128-bit)

MethodBA (%)TPR ≥82 bits (%)TPR ≥86 bits (%)
InSwapper65.6067.3545.95
InStyleSwapper64.9563.0041.65
HyperSwap64.2857.0033.85
SimSwap65.1764.4041.35
REFace67.0674.2058.20

Regarding resolution, the current 256×256 setting is constrained by the pretrained DiffusionCLIP backbone. Native resolution source protection requires a compatible diffusion model (or latent diffusion model), which may require training on face image, whereas recovery at different face-swapped output resolutions can be evaluated independently. We would like to be clarified and do more evaluation if needed.

Thank you for your rebuttal. After reviewing the authors’ response, I still find their arguments insufficient and unconvincing. The core limitations remain unresolved. Furthermore, the method exhibits poor performance and is not suitable for real-world deployment. I retain my score.

We would like to thank the reviewer for taking the time to reconsider our work and provide additional feedback. We respect your assessment and appreciate these suggestions, they will help us improve the method in future revisions.

Reviewer_HCQFOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a source-side identity watermarking method called IdentityMark for tracing face-swapping deepfakes. IdentityMark maps binary messages to pseudo-identity embeddings and realizes them in source portraits through diffusion-based optimization. The results on a public dataset are used to show their performance. But, there are some limitations, including novelty, presentation and reliability, left to be resolved.

Critical strengths
  1. The source-side setting is different from pixel-based watermarking: the signal is intended to travel through the identity representation preserved by face-swapping systems.
  2. The paper reports improved performance in Table 1.
Critical weaknesses
  1. Figure 1 is unclear. some of the notes are not clear in (a); the input of the ArcFace Psuedo encoding is unclear, both the some image and watermark bits?
  2. The novelty is limited by only working with a encoder-decoder for watermarking and face swapping, both are existing methods. Figure 2 is unclear, complicated and insufficiently organized, especially that some components are not explained and limited novel information are provided. The method seems more like an engineering work.
  3. Practical source attribution is not fully validated. Section 2.3 (Eq. 5) proposes matching the recovered message to a registered watermark, but Section 4.2 (Table 1) mainly reports bit accuracy (71.66% on average) rather than a specified matching threshold or source-level false-positive and false-negative rates. The collision analysis in Appendix H (Table 8) tests round-trip watermark uniqueness, not attribution errors on manipulated images. These missing results limit the evidence for reliable provenance verification.
  4. Identity preservation and cross-model generalization need stronger validation. Section 4.3 (Table 4 and Figure 3) reports visual fidelity, but the text also states that original-to-protected ArcFace cosine similarity is only 0.24. The visual results cannot support the superior performance of the method.
  5. The method works mainly with ArcFace-like identity encoders. Evaluations with independent/unseen recognition models would better support the generalization claim.
  6. Many experimental details and settings are missed.
Questions for authors

See weaknesses above.

1 comment

Weakness 1

We apologize for the unclear illustration. Figure 1 aims to describe a scenario where a user protects their source portrait before publishing it. An attacker may subsequently use this portrait as the identity source for face swapping with an arbitrary target image and an unknown face-swapping model. Regarding the inputs, given an original image and a watermark, we first transform the watermark into a pseudo face embedding, then we optimize the original image such that its face embedding will shift to pseudo embedding while keeping its appearance close to the original. We will revise the figure to make these roles and the information flow clearer.

Weakness 2

Figure 2: Our method first learns a mapping from binary watermark messages to pseudo face embeddings. Given an original portrait and a watermark, diffusion-based optimization then produces a visually similar protected image whose new identity (pseudo face embedding) carries the message. When this image is used as the source in an unknown face swapping pipeline, the watermark can subsequently be recovered from the manipulated output.

We agree that individual components, such as autoencoders, ArcFace, and diffusion optimization, have been explored in previous works. However, our contribution lies in using these components to enable a different functionality: preserving a message bearing watermark through uncontrolled manipulation, without accessing or training on downstream face swapping models. We believe this represents a meaningful contribution by addressing a distinct problem (non trivial) from prior works under a different threat model, rather than merely combining existing components. We will also simplify Figure 2 to emphasize the main idea and better explain the roles of each component.

Weakness 3

We agree that bit accuracy alone is insufficient to fully evaluate source attribution. We therefore conducted additional verification experiments using 32-bit and 128-bit watermarks across all 10 manipulation methods. For more detail please refer to Our answer for Weakness 2 of Reviewer 1

Weakness 4

The low ArcFace cosine similarity is an intended consequence of embedding watermark information into the identity representation. Similar behavior has been explored in DeepProtect (CVPR 2026), which reports an identity similarity measure of 0.201 while maintaining 35.89 dB PSNR and 0.944 SSIM on CelebA-HQ. Machine Pareidolia (AAAI 2026) also explores modifying machine-recognized identity while preserving visual appearance, reporting 29.07 dB PSNR and 0.876 SSIM.

In comparison, IdentityMark achieves 31.96 dB PSNR & 0.94 SSIM (32 bit) and 33.72 dB PSNR & 0.9588 SSIM (128 bit). We further evaluated original–protected image similarity using an independent AdaFace recognizer, obtaining a mean cosine similarity of 0.5462 (median 0.5496). Although identity similarity metrics are not directly comparable across models and evaluation protocols, these results suggest that low ArcFace similarity alone does not necessarily imply the loss of visual resemblance to the original person.

Weakness 5

Our evaluation already includes face swapping methods with different architectures and identity conditioning mechanisms. Not all methods use the same ArcFace encoder; for example, UniFace uses source image conditioning, while HyperSwap employs BlendFace-based identity features. Our method does not require access to or training on any of these downstream models.

We would appreciate clarification on whether the reviewer suggests evaluating additional face swapping models using different identity representations, or replacing ArcFace within our watermark encoding and recovery pipeline. The latter would require retraining the corresponding modules, as independently trained embedding spaces are not directly interchangeable.

Weakness 6

We appreciate the reviewer's concern regarding reproducibility of the whole experiment. The main components of our message to identity autoencoder, including the structured projection, residual correction, Gaussian sampling, and training objectives, are described in Section 3.2. We also provide the main training and optimization settings in Section 4.1 and Appendix B. We acknowledge that some architectural details could be further clarified and will improve their presentation. We would appreciate it if the reviewer could specify any additional implementation details they consider essential.

Reviewer_v3X7OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes IdentityMark, a method that utilizes the message-to-identity autoencoder to encode binary watermark information into an identity representation, which is then embedded into an image through a diffusion-model-based optimization process. The authors evaluate the watermark recovery performance across various face-swapping methods and standard image transformations.

Critical strengths
  1. This paper discusses the source tracking of deep fake face exchange, which is a practical related problem. The defender cannot control the downstream manipulation model or target image.
Critical weaknesses
  1. The concept of identity-centric proactive defense is not new. Representative prior works like 'IDGuard: Robust, General, Identity-Centric POI Proactive Defense Against Face Editing Abuse' and 'Arc2Face: A Foundation Model for ID-Consistent Human Faces',are not mentioned in the paper. Furthermore, the authors fail to discuss 'Robust Secure Swap: Responsible Face Swap With Persons of Interest Redaction and Provenance Traceability', which also addresses identity-based proactive provenance tracking. Although this paper employs a different model, and integrates ArcFace embeddings, autoencoder architectures, and diffusion-based latent space optimization. Beyond these established techniques, what does this method actually contribute that is new? Simply adopting a different implementation or optimization workflow is insufficient to substantiate the novelty claimed by the authors.

  2. The watermark recovery performance in Table 1 reported by the author cannot support the traceability function. The authors report that the bit accuracy of 99.55% is achieved on the watermarked source image, which is about 98% in the case of conventional image degradation. However, after face swapping, its performance is significantly reduced, with an average of 71.66%. In particular, the recovery accuracy of FaceDetector and diffswap were 56.33% and 60.31%, respectively, close to random guess. Although the accuracy is a little higher than other methods listed by the author, the average accuracy of about 70% is difficult to support the provenance (it means there are more than 9 errors in the total 32-bit watermark). In addition, the verification threshold, message level recovery rate, false acceptance rate and false rejection rate of the watermarking have not been discussed by the author.

Questions for authors
  1. In Section 4.3, the authors state that "Despite this visual similarity, the average identity cosine similarity between the original and protected images is only 0.24, indicating a substantial shift in the identity representation." I am confused by this saying. This method appears to intentionally modify ArcFace identity features, such that visual similarity no longer perfectly aligns with identity feature similarity. Given such a significant feature shift, one must ask: is the original identity effectively preserved? Can the authors demonstrate that the protected images retain the original identity, rather than having it replaced by other features associated with the watermark? Furthermore, could an independent face recognition model be used to evaluate the extent to which identity is preserved?
1 comment

Weakness 1

We would like to clarify that our work does not aim to prevent face swapping by disrupting the generated output, as in conventional proactive defense methods. We acknowledge the missing discussion of IDGuard and Robust Secure Swap. However, our paper already covers several related proactive defenses, including NullSwap, DeepProtect, SGD-Guard, ID-Eraser, and EA-APO (Appendix A.3).

Our contribution lies in addressing a different objective under a distinct threat model. Unlike Robust Secure Swap, our method does not require access to or control over downstream face swapping models. Moreover, prevention oriented defenses such as IDGuard may be bypassed by alternative manipulation strategies, including dedicated cheapfake methods. Rather than preventing manipulation, IdentityMark aims to preserve a recoverable watermark when a protected portrait is used as the identity providing source. The recovered message can then be matched against registered watermarks to provide evidence of which protected source may have been used, complementing conventional passive deepfake detection. Additionally, our method can also be used as conventional watermark.

Regarding Arc2Face, it addresses a different task, generating facial images conditioned on identity embeddings. It does not propose a watermarking framework that embeds a recoverable message into an existing source portrait while maintaining visual similarity.

We will revise the related work section to better acknowledge these studies and clarify the differences in objectives and threat models.

Weakness 2

We agree that bit accuracy alone is insufficient to evaluate provenance verification. We therefore conducted additional message level verification experiments using both 32-bit and 128-bit watermarks across all 10 manipulation methods. The theoretical matching thresholds are 26/32 and 27/32 bits for 32-bit watermarks, and 82/128 and 86/128 bits for 128-bit watermarks, corresponding to theoretical single-key FPR budgets of 0.1% and 0.01%, respectively. We agree that the 32 bit version is not as reliable as the 128 bit version.

Watermark Extraction Evaluation after Face Swapping

Method32-bit BA32-bit [email protected]%32-bit [email protected]%128-bit BA128-bit [email protected]%128-bit [email protected]%
InSwapper77.3845.1030.4069.7688.3275.60
InStyleSwapper77.0239.6626.0269.0985.4471.16
HyperSwap75.5932.5619.7467.7481.8660.88
SimSwap75.2329.7416.8268.0280.4457.42
UniFace74.7831.2819.8067.2276.0258.30
2D Cheapfake72.4912.116.3161.8239.1920.28
3D Cheapfake68.0630.6520.4166.2668.5849.76
FaceDancer56.336.622.9661.2631.7212.12
REFace79.3749.7235.0070.0586.8475.12
DiffSwap60.311.160.3356.958.132.08

To evaluate false acceptance, we further tested 5000 unprotected face-swapped images against the registered watermarks of their corresponding protected sources. The empirical FPRs were 0.6657% and 0.0605% at the two thresholds, respectively.

Furthermore, we evaluated source retrieval using 10 differently watermarked protected versions (128 bit) of each original image (different watermark for each person), tested on 50 persons from CelebAHQ. The recovered messages achieved 90.94% Top-1 and 95.88% Top-3 retrieval accuracy on average across five face-swapping methods.

Although performance varies across manipulation methods, five models achieve over 80% TPR at the 82-bit threshold. We acknowledge the remaining limitations, but believe these results demonstrate meaningful progress on the non trivial problem of recovering source-side watermarks after face swapping.

Question 1

We would like to clarify that the low ArcFace cosine similarity is an intended effect of embedding watermark information into the machine-readable identity representation while preserving visual appearance. Similar trade-offs have been explored in DeepProtect (CVPR 2026, Lee et al.; 35.89 dB PSNR, 0.944 SSIM) and Machine Pareidolia (AAAI 2026, Le et al.; 29.07 dB PSNR, 0.876 SSIM), suggesting that low identity similarity does not necessarily imply poor visual resemblance.

To further address this concern, we evaluated original protected image similarity using an independent AdaFace recognizer, obtaining an average cosine similarity of 0.5462 (median 0.5496), compared with 0.24 under ArcFace. While cosine similarity alone cannot fully establish identity preservation, our PSNR, SSIM, LPIPS, and qualitative results provide complementary evidence of visual fidelity.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.