- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes SeLo-Mark, a proactive facial integrity verification framework that preserves the designated facial region pixel-wise while embedding verification information only in the surrounding image. A 32-bit facial-content reference is derived from frozen CLIP ViT-L/14 features using PCA and binarization, concatenated with a 16-bit tracing field, and embedded through a localized watermark channel. At verification, a shared geometric estimator rectifies both the watermark-reading branch and the facial observation used to recompute the semantic reference, while the message reader is adapted to rectified inputs. Facial manipulation is detected from the Hamming distance between the recovered reference and the code recomputed from the received face. The evaluation covers four editing families and 22 processing conditions on CelebA-HQ, with additional FFHQ transfer, matched-quality comparisons, geometric controls, regeneration experiments, and recovery analyses. The method reports 99.23% AUC and 95.85% F1 on CelebA-HQ.
Critical strengths
-
The paper does substantially more than compare aggregate AUC values. Tables 5–7 isolate the effects of message correction, facial alignment, reader adaptation, geometric synchronization, and publication quality. The shared-valid-input analysis further shows that the performance advantage is not caused by method-specific missing observations. These controls make the empirical conclusions considerably more convincing.
-
The protected facial pixels are preserved by construction rather than merely encouraged through a reconstruction loss. The use of one geometric estimate for both watermark recovery and semantic re-encoding is also well motivated: geometric processing affects both the location of the carrier and the observation from which the facial reference is recomputed. Reader adaptation after rectification is directly supported by the reported crop-recovery improvements.
Critical weaknesses
-
The principal components have close precedents: content-derived facial references are already used by LampMark and RecoverMark, localized watermark transport follows MaskWM-style designs, and geometric synchronization is inherited from SyncSeal. The main contribution is an effective integration of these components, particularly the shared geometric frame and reader adaptation, rather than a fundamentally new learning mechanism. This makes the novelty somewhat limited for ICLR. The paper should also position itself against the very recent LocMark work on geometrically robust local watermarking for proactive deepfake forensics.
-
Section 3.1 explicitly assumes that editing procedures are not optimized against the verifier. This excludes the most security-relevant scenario for an integrity system. Moreover, Appendix B.2 reports that 0.079% of cross-identity code pairs already fall within the six-bit acceptance radius. An attacker able to select a target identity could therefore search for unusually close semantic codes. The paper does not evaluate targeted source-target selection, watermark removal, watermark replacement, or watermark forgery. Either these experiments are needed, or the claims should be narrowed to non-adaptive proactive facial-content consistency detection.
Questions for authors
-
Appendix B.2 reports that 0.079% of different-identity code pairs differ by at most six bits, while the operating threshold accepts distances up to six bits. What happens if the target identity for SimSwap or CSCS is deliberately selected to minimize the CLIP-PCA32 Hamming distance to the source? Can you report detection performance under such targeted source-target selection?
-
Section 3.1 states that the editing procedures are not optimized against the verifier. What security or integrity property is intended against an adversary who knows the system and attempts watermark removal, replacement, or forgery? If adversarial robustness is outside the scope, would you narrow the “facial integrity verification” claim accordingly?
-
Is the 32-bit semantic reference or 16-bit tracing field cryptographically authenticated or keyed? If an adversary can replace the external watermark with one corresponding to the manipulated face, what mechanism prevents acceptance?
-
How does SeLo-Mark differ technically from the recently published LocMark approach to local, geometrically robust proactive deepfake forensics? A direct comparison or a detailed conceptual distinction would help clarify the novelty.
-
The additional 20° shear experiment reduces Exact48 substantially relative to the main geometric suite. Is this primarily caused by coordinate-prediction error or by reader failure after rectification, and how broadly should the geometric-robustness claim therefore be interpreted?
6 comments
Response to Reviewer DhYV (2/6)
Q2 (W2). What is verified under watermark removal, replacement, or forgery?
In the original 48-bit system, SeLo-Mark verifies facial-content consistency by comparing the recovered 32-bit publication reference with the content code recomputed from the received face. It flags an inconsistency when more than six bits differ. This comparison does not authenticate the recovered reference, and errors in reference recovery can also affect the verification outcome. We examine these issues through reconstruction, regeneration, reader-targeted perturbations, and reference replacement.
(1) Verification under reconstruction and regeneration. Appendix E.3 of the original manuscript evaluates watermark recovery under regeneration. We further examine its impact on facial-content verification using VAE reconstruction and SD 1.5 regeneration with a denoising strength of 0.1. Both operations are applied to unedited publications and their SimSwap and CSCS counterparts from 128 source identities, with FPR measured on the processed publications without a preceding face swap. The original 48-bit system, receiver parameters, and six-bit threshold remain unchanged. Table R2(a) reports the resulting detection performance.
Table R2(a). Face-swap detection under reconstruction and regeneration (%).
| Processing condition | SimSwap AUC/F1 | CSCS AUC/F1 | FPR |
|---|---|---|---|
| No additional processing | 99.76 / 95.08 | 99.88 / 98.01 | 0.00 |
| VAE reconstruction | 96.87 / 89.60 | 98.47 / 94.66 | 7.81 |
| SD 1.5 regeneration (strength 0.1) | 87.20 / 83.04 | 93.68 / 85.81 | 32.03 |
As shown in Table R2(a), SeLo-Mark maintains strong face-swap detection under VAE reconstruction, achieving 96.87% AUC (89.60% F1) for SimSwap and 98.47% AUC (94.66% F1) for CSCS with a 7.81% FPR. Under SD 1.5 regeneration, SeLo-Mark still achieves 87.20% AUC (83.04% F1) for SimSwap and 93.68% AUC (85.81% F1) for CSCS, while FPR on unswapped publications rises to 32.03% (41 of 128 images). To further investigate these increased alerts on the same 128 unswapped regenerated images, we keep the regenerated images and facial observations unchanged while replacing the recovered references with the original publication references; this reduces FPR from 32.03% to 17.19% (22 of 128 images), indicating that both reference-reading errors and regeneration-induced facial drift contribute to the alerts. Overall, although diffusion regeneration increases false alarms at the fixed threshold, SeLo-Mark preserves strong face-swap detection under VAE reconstruction (96.87% and 98.47% AUC) and retains substantial discriminative ability under SD 1.5 regeneration (87.20% and 93.68% AUC).
Response to Reviewer DhYV (3/6)
(2) Verification under reader-targeted watermark disruption. We next consider an attack that directly targets watermark recovery, rather than relying on reconstruction or regeneration to disrupt the reference. Using 32 source identities and their unedited, SimSwap, and CSCS images (96 inputs per condition), we optimize perturbations for 20 steps to push the watermark reader's bit predictions toward the decoding boundary. The perturbations are restricted to the external watermark carrier, leaving all protected facial pixels unchanged relative to the corresponding pre-attack images. We test perturbation budgets of 4/255 and 8/255 on the [0, 1] RGB scale, with random-noise controls at the same budgets; measured against each pre-attack RGB image across the 96 inputs, the targeted perturbations yield mean PSNRs of 40.95 and 37.98 dB, respectively. All models and the original six-bit threshold remain unchanged. In Table R2(b), BA32 (bit accuracy over the 32 content-reference bits) and Exact48 (exact 48-bit payload recovery) are computed over all 96 readouts per condition, whereas detection metrics compare the 32 face-swapped images per editor with the 32 corresponding images without face swapping (defining FPR relative to facial-edit labels).
Table R2(b). Reader-targeted perturbations: watermark recovery and face-swap detection (%).
| Condition | BA32 | Exact48 | SimSwap AUC | CSCS AUC | FPR |
|---|---|---|---|---|---|
| No perturbation | 100.00 | 100.00 | 100.00 | 100.00 | 0.00 |
| Random noise, 4/255 | 100.00 | 100.00 | 99.95 | 100.00 | 0.00 |
| Targeted attack, 4/255 | 53.16 | 0.00 | 57.37 | 62.79 | 100.00 |
| Random noise, 8/255 | 100.00 | 100.00 | 100.00 | 100.00 | 0.00 |
| Targeted attack, 8/255 | 53.87 | 0.00 | 58.30 | 66.26 | 100.00 |
As shown in Table R2(b), matched random noise leaves watermark recovery and detection intact (100.00% Exact48 and 99.95%–100.00% AUC with 0.00% FPR), whereas white-box reader-targeted perturbations reduce BA32 to 53.16%–53.87% and Exact48 to 0.00%. Under both targeted budgets, all 64 face-swapped images are flagged (preventing false acceptance of manipulated faces), though all 32 unedited images also trigger inconsistency alerts because corrupted reference readouts cannot be distinguished from facial edits without reference authentication. Evaluating the same 96 attacked inputs per budget with the original publication references while keeping the attacked images and facial observations unchanged restores SimSwap/CSCS AUCs to 99.95%/100.00% at 4/255 and 100.00%/100.00% at 8/255, with FPR returning to 0.00%. This control confirms that disrupted reference recovery is the primary source of degradation under the tested carrier perturbations, whereas SeLo-Mark's facial-content comparison retains near-perfect discriminative ability (99.95%–100.00% AUC) when supplied with the original reference, highlighting the importance of authenticating recovered references as examined in Q3.
(3) Verification under reference replacement and forgery. Unlike the preceding attack, which disrupts the existing watermark reference, reference replacement attempts to re-embed a forged reference matching the manipulated face. We evaluate this threat using our keyed authentication extension: as shown in Table R3, in the controlled re-embedding experiment across 512 unauthorized evaluations, all 512 substitutions that modified the content reference while reusing the original authentication tag are rejected, preventing reference-replacement bypasses. Further details are provided in Q3.
Response to Reviewer DhYV (1/6)
Thank you for recognizing the exact preservation of protected facial pixels and the controlled analysis of our receiver. We appreciate your concerns about the scope and robustness of facial-content verification, and have conducted additional experiments to address them.
Q1 (W2). What happens when the target identity is selected to minimize code distance?
An attacker can select a target whose original content code is close to that of the source. We therefore examine how code-guided target selection affects face-swap detection with SimSwap and CSCS. For each of 128 source identities, we construct a pool of 100 candidate targets and compare random selection with minimum Hamming-distance selection among the first 10 or all 100 candidates, where source–target code distance is measured between the original photographs' 32-bit content codes before swapping. Both editors use the identical selected source–target pairs (hence sharing the same pre-swap mean distances) and generate 128 swapped images per setting. Detection metrics are evaluated against the corresponding unedited publications and averaged over two independently adapted receivers (256 swapped-image decisions per editor and setting), with all model parameters and the original six-bit threshold unchanged (FPR remains 0.78% across all settings). Table R1 summarizes the results.
Table R1. Source–target code distances and face-swap detection by target-selection strategy.
As shown in Table R1, relative to random target selection (16.19 bits), minimum-distance selection reduces the mean pre-swap source–target code distance to 11.46 bits at K=10 and 8.48 bits at K=100. From random selection to K=100, fixed-threshold TPR decreases from 91.41% to 68.75% for SimSwap and from 97.66% to 82.42% for CSCS. Nevertheless, even at K=100, SeLo-Mark maintains 97.56% AUC (81.10% F1) for SimSwap and 99.02% AUC (89.98% F1) for CSCS, demonstrating strong discriminative capability under code-guided target selection.
Importantly, code similarity before synthesis does not directly determine the verification outcome. The 0.079% statistic across the 2,996 test publications in Appendix B.2 of the original manuscript refers to the fraction of cross-identity photograph pairs whose original content codes differ by at most six bits. However, SeLo-Mark compares the recovered source reference with the content code recomputed from the synthesized face. At K=100, 11 of the 128 selected pairs have original code distances within six bits, yet four of these pairs under SimSwap and six under CSCS are still detected by both receivers after swapping. For SimSwap under K=100, the mean verification distances are 1.34 bits for unedited images and 8.02 bits for swapped images across the 128 source identities and two receivers (256 readouts each). Watermark recovery is not the source of missed detections: both receivers correctly recover all 48 payload bits in all 256 swapped-image readouts per editor (Exact48 = 100%). Among the 80 missed SimSwap decisions (out of 256 at K=100), 48 have verification distances of five or six bits, falling just inside the six-bit boundary. Overall, because SeLo-Mark verifies the recomputed code of the synthesized face rather than the pre-swap target photograph, pre-swap code proximity does not directly translate into evasion: even under K=100 code-guided selection, SeLo-Mark detects the majority of swaps at the fixed threshold and retains over 97.5% AUC across both editors.