acceptodds
Under review as a conference paper at ICLR 2027

SeLo-Mark: Robust Localized Watermarking with Content-Derived Semantic Codes for Facial Integrity Verification

Abstract

Image watermarking enables proactive facial forensics by embedding information before publication. Verifying facial content, however, requires more than recovering a payload: the embedded signal must represent the original face while remaining recoverable after image processing. Random payloads provide no such content reference, while geometric transformations disrupt the spatial correspondence required by localized watermarking. We propose SeLo-Mark, a content-derived localized watermarking framework for facial integrity verification. First, a compact semantic reference is derived from the protected face and embedded only in the surrounding unprotected region, leaving the protected facial pixels unchanged. Second, a shared geometric estimate is used to align both watermark recovery and facial-content encoding after geometric processing, so that the recovered reference can be compared with the received face in a common coordinate frame. The semantic reference is obtained by projecting frozen CLIP features with PCA and binarizing them into a 32-bit facial-content code, which is combined with an independent 16-bit tracing field. At reception, the payload is recovered by a reader adapted to rectified images, and facial consistency is assessed by comparing the recovered content code with one recomputed from the received face, without reconstructing the original face. Across four editor families and 22 processing conditions on 2,999 CelebA-HQ images, SeLo-Mark achieves 99.23% AUC and 95.85% F1. On unedited watermarked images, exact 48-bit recovery is increased from 88.38% with direct, unadapted decoding to 99.82% with the full receiver.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
image watermarking, localized watermarking, content-aware watermarking, geometric robustness, facial integrity verification

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_DhYVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SeLo-Mark, a proactive facial integrity verification framework that preserves the designated facial region pixel-wise while embedding verification information only in the surrounding image. A 32-bit facial-content reference is derived from frozen CLIP ViT-L/14 features using PCA and binarization, concatenated with a 16-bit tracing field, and embedded through a localized watermark channel. At verification, a shared geometric estimator rectifies both the watermark-reading branch and the facial observation used to recompute the semantic reference, while the message reader is adapted to rectified inputs. Facial manipulation is detected from the Hamming distance between the recovered reference and the code recomputed from the received face. The evaluation covers four editing families and 22 processing conditions on CelebA-HQ, with additional FFHQ transfer, matched-quality comparisons, geometric controls, regeneration experiments, and recovery analyses. The method reports 99.23% AUC and 95.85% F1 on CelebA-HQ.

Critical strengths
  1. The paper does substantially more than compare aggregate AUC values. Tables 5–7 isolate the effects of message correction, facial alignment, reader adaptation, geometric synchronization, and publication quality. The shared-valid-input analysis further shows that the performance advantage is not caused by method-specific missing observations. These controls make the empirical conclusions considerably more convincing.

  2. The protected facial pixels are preserved by construction rather than merely encouraged through a reconstruction loss. The use of one geometric estimate for both watermark recovery and semantic re-encoding is also well motivated: geometric processing affects both the location of the carrier and the observation from which the facial reference is recomputed. Reader adaptation after rectification is directly supported by the reported crop-recovery improvements.

Critical weaknesses
  1. The principal components have close precedents: content-derived facial references are already used by LampMark and RecoverMark, localized watermark transport follows MaskWM-style designs, and geometric synchronization is inherited from SyncSeal. The main contribution is an effective integration of these components, particularly the shared geometric frame and reader adaptation, rather than a fundamentally new learning mechanism. This makes the novelty somewhat limited for ICLR. The paper should also position itself against the very recent LocMark work on geometrically robust local watermarking for proactive deepfake forensics.

  2. Section 3.1 explicitly assumes that editing procedures are not optimized against the verifier. This excludes the most security-relevant scenario for an integrity system. Moreover, Appendix B.2 reports that 0.079% of cross-identity code pairs already fall within the six-bit acceptance radius. An attacker able to select a target identity could therefore search for unusually close semantic codes. The paper does not evaluate targeted source-target selection, watermark removal, watermark replacement, or watermark forgery. Either these experiments are needed, or the claims should be narrowed to non-adaptive proactive facial-content consistency detection.

Questions for authors
  1. Appendix B.2 reports that 0.079% of different-identity code pairs differ by at most six bits, while the operating threshold accepts distances up to six bits. What happens if the target identity for SimSwap or CSCS is deliberately selected to minimize the CLIP-PCA32 Hamming distance to the source? Can you report detection performance under such targeted source-target selection?

  2. Section 3.1 states that the editing procedures are not optimized against the verifier. What security or integrity property is intended against an adversary who knows the system and attempts watermark removal, replacement, or forgery? If adversarial robustness is outside the scope, would you narrow the “facial integrity verification” claim accordingly?

  3. Is the 32-bit semantic reference or 16-bit tracing field cryptographically authenticated or keyed? If an adversary can replace the external watermark with one corresponding to the manipulated face, what mechanism prevents acceptance?

  4. How does SeLo-Mark differ technically from the recently published LocMark approach to local, geometrically robust proactive deepfake forensics? A direct comparison or a detailed conceptual distinction would help clarify the novelty.

  5. The additional 20° shear experiment reduces Exact48 substantially relative to the main geometric suite. Is this primarily caused by coordinate-prediction error or by reader failure after rectification, and how broadly should the geometric-robustness claim therefore be interpreted?

6 comments
AuthorsOpenReview ↗

Response to Reviewer DhYV (1/6)

Thank you for recognizing the exact preservation of protected facial pixels and the controlled analysis of our receiver. We appreciate your concerns about the scope and robustness of facial-content verification, and have conducted additional experiments to address them.

Q1 (W2). What happens when the target identity is selected to minimize code distance?

An attacker can select a target whose original content code is close to that of the source. We therefore examine how code-guided target selection affects face-swap detection with SimSwap and CSCS. For each of 128 source identities, we construct a pool of 100 candidate targets and compare random selection with minimum Hamming-distance selection among the first 10 or all 100 candidates, where source–target code distance is measured between the original photographs' 32-bit content codes before swapping. Both editors use the identical selected source–target pairs (hence sharing the same pre-swap mean distances) and generate 128 swapped images per setting. Detection metrics are evaluated against the corresponding unedited publications and averaged over two independently adapted receivers (256 swapped-image decisions per editor and setting), with all model parameters and the original six-bit threshold unchanged (FPR remains 0.78% across all settings). Table R1 summarizes the results.

Table R1. Source–target code distances and face-swap detection by target-selection strategy.

EditorTarget selectionMean source–target distance (bits)AUC (%) ↑F1 (%) ↑TPR (%) ↑
SimSwapRandom target16.1999.5795.1291.41
SimSwapMinimum distance, K=1011.4698.6789.9882.42
SimSwapMinimum distance, K=1008.4897.5681.1068.75
CSCSRandom target16.1999.9498.4397.66
CSCSMinimum distance, K=1011.4699.0094.0489.45
CSCSMinimum distance, K=1008.4899.0289.9882.42

As shown in Table R1, relative to random target selection (16.19 bits), minimum-distance selection reduces the mean pre-swap source–target code distance to 11.46 bits at K=10 and 8.48 bits at K=100. From random selection to K=100, fixed-threshold TPR decreases from 91.41% to 68.75% for SimSwap and from 97.66% to 82.42% for CSCS. Nevertheless, even at K=100, SeLo-Mark maintains 97.56% AUC (81.10% F1) for SimSwap and 99.02% AUC (89.98% F1) for CSCS, demonstrating strong discriminative capability under code-guided target selection.

Importantly, code similarity before synthesis does not directly determine the verification outcome. The 0.079% statistic across the 2,996 test publications in Appendix B.2 of the original manuscript refers to the fraction of cross-identity photograph pairs whose original content codes differ by at most six bits. However, SeLo-Mark compares the recovered source reference with the content code recomputed from the synthesized face. At K=100, 11 of the 128 selected pairs have original code distances within six bits, yet four of these pairs under SimSwap and six under CSCS are still detected by both receivers after swapping. For SimSwap under K=100, the mean verification distances are 1.34 bits for unedited images and 8.02 bits for swapped images across the 128 source identities and two receivers (256 readouts each). Watermark recovery is not the source of missed detections: both receivers correctly recover all 48 payload bits in all 256 swapped-image readouts per editor (Exact48 = 100%). Among the 80 missed SimSwap decisions (out of 256 at K=100), 48 have verification distances of five or six bits, falling just inside the six-bit boundary. Overall, because SeLo-Mark verifies the recomputed code of the synthesized face rather than the pre-swap target photograph, pre-swap code proximity does not directly translate into evasion: even under K=100 code-guided selection, SeLo-Mark detects the majority of swaps at the fixed threshold and retains over 97.5% AUC across both editors.

AuthorsOpenReview ↗

Response to Reviewer DhYV (2/6)

Q2 (W2). What is verified under watermark removal, replacement, or forgery?

In the original 48-bit system, SeLo-Mark verifies facial-content consistency by comparing the recovered 32-bit publication reference with the content code recomputed from the received face. It flags an inconsistency when more than six bits differ. This comparison does not authenticate the recovered reference, and errors in reference recovery can also affect the verification outcome. We examine these issues through reconstruction, regeneration, reader-targeted perturbations, and reference replacement.

(1) Verification under reconstruction and regeneration. Appendix E.3 of the original manuscript evaluates watermark recovery under regeneration. We further examine its impact on facial-content verification using VAE reconstruction and SD 1.5 regeneration with a denoising strength of 0.1. Both operations are applied to unedited publications and their SimSwap and CSCS counterparts from 128 source identities, with FPR measured on the processed publications without a preceding face swap. The original 48-bit system, receiver parameters, and six-bit threshold remain unchanged. Table R2(a) reports the resulting detection performance.

Table R2(a). Face-swap detection under reconstruction and regeneration (%).

Processing conditionSimSwap AUC/F1CSCS AUC/F1FPR
No additional processing99.76 / 95.0899.88 / 98.010.00
VAE reconstruction96.87 / 89.6098.47 / 94.667.81
SD 1.5 regeneration (strength 0.1)87.20 / 83.0493.68 / 85.8132.03

As shown in Table R2(a), SeLo-Mark maintains strong face-swap detection under VAE reconstruction, achieving 96.87% AUC (89.60% F1) for SimSwap and 98.47% AUC (94.66% F1) for CSCS with a 7.81% FPR. Under SD 1.5 regeneration, SeLo-Mark still achieves 87.20% AUC (83.04% F1) for SimSwap and 93.68% AUC (85.81% F1) for CSCS, while FPR on unswapped publications rises to 32.03% (41 of 128 images). To further investigate these increased alerts on the same 128 unswapped regenerated images, we keep the regenerated images and facial observations unchanged while replacing the recovered references with the original publication references; this reduces FPR from 32.03% to 17.19% (22 of 128 images), indicating that both reference-reading errors and regeneration-induced facial drift contribute to the alerts. Overall, although diffusion regeneration increases false alarms at the fixed threshold, SeLo-Mark preserves strong face-swap detection under VAE reconstruction (96.87% and 98.47% AUC) and retains substantial discriminative ability under SD 1.5 regeneration (87.20% and 93.68% AUC).

AuthorsOpenReview ↗

Response to Reviewer DhYV (3/6)

(2) Verification under reader-targeted watermark disruption. We next consider an attack that directly targets watermark recovery, rather than relying on reconstruction or regeneration to disrupt the reference. Using 32 source identities and their unedited, SimSwap, and CSCS images (96 inputs per condition), we optimize perturbations for 20 steps to push the watermark reader's bit predictions toward the decoding boundary. The perturbations are restricted to the external watermark carrier, leaving all protected facial pixels unchanged relative to the corresponding pre-attack images. We test perturbation budgets of 4/255 and 8/255 on the [0, 1] RGB scale, with random-noise controls at the same budgets; measured against each pre-attack RGB image across the 96 inputs, the targeted perturbations yield mean PSNRs of 40.95 and 37.98 dB, respectively. All models and the original six-bit threshold remain unchanged. In Table R2(b), BA32 (bit accuracy over the 32 content-reference bits) and Exact48 (exact 48-bit payload recovery) are computed over all 96 readouts per condition, whereas detection metrics compare the 32 face-swapped images per editor with the 32 corresponding images without face swapping (defining FPR relative to facial-edit labels).

Table R2(b). Reader-targeted perturbations: watermark recovery and face-swap detection (%).

ConditionBA32Exact48SimSwap AUCCSCS AUCFPR
No perturbation100.00100.00100.00100.000.00
Random noise, 4/255100.00100.0099.95100.000.00
Targeted attack, 4/25553.160.0057.3762.79100.00
Random noise, 8/255100.00100.00100.00100.000.00
Targeted attack, 8/25553.870.0058.3066.26100.00

As shown in Table R2(b), matched random noise leaves watermark recovery and detection intact (100.00% Exact48 and 99.95%–100.00% AUC with 0.00% FPR), whereas white-box reader-targeted perturbations reduce BA32 to 53.16%–53.87% and Exact48 to 0.00%. Under both targeted budgets, all 64 face-swapped images are flagged (preventing false acceptance of manipulated faces), though all 32 unedited images also trigger inconsistency alerts because corrupted reference readouts cannot be distinguished from facial edits without reference authentication. Evaluating the same 96 attacked inputs per budget with the original publication references while keeping the attacked images and facial observations unchanged restores SimSwap/CSCS AUCs to 99.95%/100.00% at 4/255 and 100.00%/100.00% at 8/255, with FPR returning to 0.00%. This control confirms that disrupted reference recovery is the primary source of degradation under the tested carrier perturbations, whereas SeLo-Mark's facial-content comparison retains near-perfect discriminative ability (99.95%–100.00% AUC) when supplied with the original reference, highlighting the importance of authenticating recovered references as examined in Q3.

(3) Verification under reference replacement and forgery. Unlike the preceding attack, which disrupts the existing watermark reference, reference replacement attempts to re-embed a forged reference matching the manipulated face. We evaluate this threat using our keyed authentication extension: as shown in Table R3, in the controlled re-embedding experiment across 512 unauthorized evaluations, all 512 substitutions that modified the content reference while reusing the original authentication tag are rejected, preventing reference-replacement bypasses. Further details are provided in Q3.

Reviewer_A7PaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SeLo-Mark addresses a specific problem in proactive face forensics: recovering a watermark is insufficient if the watermark does not say anything about the published face, yet writing the watermark inside the face would violate the protected-region requirement. The method extracts a frozen CLIP feature from the face, projects it with PCA and binarizes it into a 32-bit content code. It adds a separate 16-bit tracing field and embeds all 48 bits only outside the protected region. At reception it uses one estimated geometric correction for both watermark reading and the face crop used to recompute the code. A Hamming-distance threshold between the recovered and observed 32-bit codes determines whether the face is inconsistent with the published reference.

The paper evaluates 2,999 identity-disjoint CelebA-HQ images under four editor families and 22 benign processing conditions, with an independent calibration set, then transfers the fixed mapping and threshold to 1,000 FFHQ images. The reported CelebA-HQ score is 99.23% AUC and 95.85% F1. The advantage is especially large under geometry and cropping: its AUC is 99.52% and 98.58% in those groups, versus 55.41% and 49.91% for the strong FractalForensics baseline before a shared geometry frontend is added. Exact recovery of all 48 bits reaches 99.82% across the 22 unedited processing conditions. Ablations test alternative content codes, recovery with the original reference, the two uses of geometric correction, and reader adaptation after rectification.

Critical strengths

The design respects its own protected-region constraint in a concrete way. Equation (4) composes the candidate watermark with the original image so that facial pixels are copied exactly, independent of how the learned encoder behaves. The 32 content bits give the verifier a reference to the source face, while the trace bits do not contaminate the consistency score. The distinction is important: a random payload could be recovered perfectly after a face swap and still say nothing about whether the face changed.

The controlled comparisons support the claimed mechanism. With the same watermark channel, CLIP-PCA32 gives 99.16% AUC and 95.71% F1, above a landmark-PCA code at 94.88% and 88.56%. Using the original embedded code instead of the recovered code barely changes end-to-end results (99.18% versus 99.23% AUC), suggesting that remaining decoding errors are not the main source of separation. On the 128-image receiver subset, correcting the message branch raises AUC from 93.31% to 98.79%; correcting the face branch as well raises it further to 98.92%. Training the reader on corrected crop views raises crop Exact48 from 80.96% to 99.32%. These numbers make a persuasive case that geometry matters to both recovery and comparison, though the message branch supplies most of the gain.

Critical weaknesses
  • Threat model. The edits come from SimSwap, CSCS, StarGAN and Stable Diffusion inpainting, followed by a fixed set of processing operations; the paper explicitly assumes that some face and carrier remain visible. An attacker who knows the 32-bit PCA code and the acceptance threshold could optimize an altered face to stay within six differing bits or corrupt the carrier/geometry estimate. The work does not show how difficult that is. Thus “facial integrity verification” is supported for the specified editor and transmission families, but the experiment should not be taken as a general tamper-resistance result.

  • Deployment threshold and transfer. On FFHQ, using the CelebA-HQ threshold without refitting yields 97.67% AUC but only 90.21% F1 overall; crop performance drops to 96.04% AUC, 89.37% F1 and 94.72% Exact48. This is still good performance, but it matters if the verifier must use one threshold after deployment. The main baselines also have different payload sizes and publication qualities, although the paper includes a smaller controlled 40-dB comparison. Finally, no demographic subgroup breakdown is given for false alarms or missed edits, even though the authors explicitly note this as open in their ethics statement.

Questions for authors

Can the authors test a limited adaptive attack that changes facial content while penalizing the 32-bit code distance or directly attacks the carrier/geometry predictor? The effort needed to cross the six-bit decision threshold would materially affect how strongly one should interpret the integrity claim.

Could the authors report false-positive and false-negative rates on FFHQ at the fixed CelebA-HQ threshold by processing group and, where annotations allow, by demographic subgroup? The paper already uses identity-disjoint train/calibration/test sets; the remaining issue is whether a single operating point behaves evenly across faces and conditions. I would also like to know how many observations are excluded as invalid under crop and whether treating those as verification failures changes the headline score.

4 comments
AuthorsOpenReview ↗

Response to Reviewer A7Pa (1/4)

Thank you for recognizing the exact preservation of protected facial pixels and the controlled comparisons in our evaluation. We agree that the fixed six-bit threshold warrants closer examination under adaptive manipulation and dataset transfer, and have conducted additional analyses to address these concerns.

Q1 (W1). How much adaptive optimization is needed to cross the six-bit threshold?

An attacker may optimize the received facial content to reduce its code distance from the publication reference while leaving the external watermark intact. We examine this possibility through a white-box attack on SimSwap and CSCS outputs from 128 source images. The attacker modifies only the facial region and uses gradients through the exact CLIP–PCA content mapping to move the edited face's continuous projections toward the original publication reference. We evaluate perturbation budgets of 1/255, 2/255, and 4/255, with a step size of 1/255, no random initialization, and up to 40 updates. Perturbation budgets are measured on the [0, 1] RGB scale. Random-noise controls use the same images, facial masks, and perturbation budgets. TPR is evaluated at the original six-bit threshold and averaged over two independently adapted receivers, whose parameters remain unchanged. Table R1(a) summarizes the results.

Table R1(a). Face-swap detection under white-box content-code optimization (TPR, %).

EditorL∞ budgetRandom noise1 step5 steps10 steps20 steps40 steps
SimSwap1/25591.4144.1427.3421.8817.1921.09
SimSwap2/25591.8044.146.646.644.693.52
SimSwap4/25591.4144.144.692.342.343.13
CSCS1/25596.8878.1360.1667.9764.4564.45
CSCS2/25596.8878.1336.3329.3019.9220.70
CSCS4/25596.8878.1317.978.597.034.69

As shown in Table R1(a), five white-box optimization steps at a perturbation budget of 4/255 yield TPRs of 4.69% for SimSwap and 17.97% for CSCS. Under matched random-noise perturbations, the corresponding TPRs are 91.41% and 96.88%, respectively. To examine this reduction on the same swaps, we measure verification distance as the Hamming distance between the recovered 32-bit publication reference and the content code recomputed from the received face. From the unperturbed swaps to the five-step attack outputs at 4/255, the mean verification distance decreases from 10.63 to 2.59 bits for SimSwap and from 13.14 to 4.26 bits for CSCS. The complete 48-bit payload is still recovered in every evaluation at this setting (Exact48 = 100%), indicating that the loss of detection arises from changes in the content code recomputed from the perturbed face, rather than errors in watermark recovery.

To further examine the effect of access to the exact fitted mapping, we independently fit a surrogate PCA using 1,000 FFHQ images and the same frozen CLIP encoder. In contrast, the original verifier uses PCA parameters fitted on 20,999 CelebA-HQ training observations. The attacker optimizes against the surrogate mapping without accessing the original PCA parameters or querying the verifier. Both the exact-PCA and surrogate-PCA attacks use the same face-swap images, a 4/255 perturbation budget, and five optimization steps. We also include the unperturbed swaps as a control. All three conditions are evaluated by the original verifier using two independently adapted receivers and the unchanged six-bit threshold (Table R1(b)).

Table R1(b). Face-swap detection under exact-PCA and surrogate-PCA optimization (TPR, %).

Attack settingSimSwapCSCS
No additional perturbation92.1996.88
Exact-PCA white-box attack4.6917.97
FFHQ surrogate-PCA transfer attack76.1789.06

As shown in Table R1(b), the independently fitted surrogate produces substantially weaker transfer than direct optimization against the verifier's PCA projection, retaining TPRs of 76.17% for SimSwap and 89.06% for CSCS (versus 4.69% and 17.97% under the exact-PCA attack). Measuring code distances on the same evaluated face-swap pairs explains this gap: using the surrogate PCA to compute both the source-face and swapped-face content codes, the mean Hamming distance decreases from 8.82 bits (before optimization) to 1.50 bits (after five steps) for SimSwap, and from 11.42 to 2.95 bits for CSCS. When the same perturbed images are evaluated by the original verifier, however, the mean verification distances remain at 8.32 and 10.13 bits, respectively. Thus, although white-box optimization can cross the fixed threshold when given the exact verifier PCA, SeLo-Mark retains strong detection performance (76.17% and 89.06% TPR) against the evaluated surrogate-PCA transfer attack when the exact fitted projection is unavailable.

AuthorsOpenReview ↗

Response to Reviewer A7Pa (2/4)

Q2 (W2). How does the fixed threshold perform on FFHQ and across subgroups, including when crop scores are unavailable?

To examine how the fixed operating point behaves across different faces and processing conditions, we evaluate false alarms and missed edits on FFHQ using the original CelebA-HQ-calibrated six-bit threshold. We also examine subgroup errors using the available CelebA-HQ annotations and assess the effect of unavailable crop scores.

(1) Fixed-threshold performance on FFHQ. We transfer the CelebA-HQ-trained models and six-bit threshold to FFHQ without further adaptation or recalibration. Table R2 reports AUC, F1, FPR, and FNR across the evaluated processing groups. AUC is computed across thresholds, whereas F1, FPR, and FNR use the original six-bit threshold calibrated on CelebA-HQ. Results are computed from valid scores using the evaluation weights in the original manuscript.

Table R2. Facial-edit detection across FFHQ processing groups (%).

Processing groupAUC ↑F1 ↑FPR ↓FNR ↓
No processing99.5091.190.1015.84
Regular processing97.6090.103.5315.05
Geometry98.4991.020.9615.44
Cropping96.0489.3711.829.53
All 22 conditions97.6790.213.7814.64

As shown in Table R2, across all 22 FFHQ processing conditions, SeLo-Mark achieves 97.67% AUC and, at the original six-bit threshold, 90.21% F1 without recalibration. Geometric processing yields 98.49% AUC with an FPR of 0.96%. Cropping presents a different error pattern: although AUC remains 96.04%, it produces the highest FPR (11.82%) and the lowest FNR (9.53%) among the processing groups. Examining verification distances on the same valid-score FFHQ observations explains these patterns: for unedited FFHQ images, the mean verification distance increases from 1.10 bits under no processing to 3.89 bits under cropping, consistent with the higher false-alarm rate. Meanwhile, among the missed edits (false negatives at the six-bit threshold) across all 22 FFHQ conditions, 85.42% have verification distances of four to six bits under the same evaluation weights, showing that most missed edits are concentrated near the six-bit acceptance boundary.

AuthorsOpenReview ↗

Response to Reviewer A7Pa (3/4)

(2) Attribute-based error analysis. To examine subgroup differences at the same operating threshold, we use the dataset-provided Male and Young labels from the identity-disjoint CelebA-HQ test set, as corresponding annotations are unavailable for the evaluated FFHQ subset. Edited observations are grouped according to the attributes of their source images. Male and Young each partition the same 2,999 source images. All groups use the same models, six-bit threshold, editing operations, and processing conditions. Table R3 reports F1, FPR, and FNR, with bracketed 95% confidence intervals for FPR and FNR obtained through identity-level bootstrap resampling.

Table R3. Facial-edit detection by source-image attribute on CelebA-HQ.

AttributeSource imagesF1 (%) ↑FPR (%) ↓FNR (%) ↓
Male=01,88396.174.84 [4.46, 5.21]3.18 [2.82, 3.61]
Male=11,11695.282.93 [2.56, 3.30]6.40 [5.48, 7.32]
Young=073694.503.04 [2.61, 3.45]7.74 [6.57, 9.04]
Young=12,26396.274.49 [4.13, 4.84]3.28 [2.95, 3.63]

As shown in Table R3, F1 ranges from 94.50% to 96.27% across the four groups, while FPR and FNR exhibit different trade-offs. For Male=1, FNR is higher than for Male=0 (6.40% versus 3.18%), whereas FPR is lower (2.93% versus 4.84%). Similarly, Young=0 has a higher FNR than Young=1 (7.74% versus 3.28%), accompanied by a lower FPR (3.04% versus 4.49%). These results show that SeLo-Mark maintains high F1 across the evaluated attribute groups, while the shared six-bit threshold produces different balances between false alarms and missed edits. Reporting both error types therefore provides a clearer picture of subgroup performance than F1 alone.

(3) Impact of unavailable scores under cropping. We also examine how unavailable verification scores affect the reported FFHQ performance. Across 1,000 FFHQ source images, two cropping conditions, and two independently adapted receivers, we obtain 4,000 planned crop evaluations for unedited inputs and 28,000 for edited inputs across seven editing operations. Among these planned evaluations, 24 unedited observations (0.60%) and 248 edited observations (0.89%) yield no valid score, including failures occurring before reception.

For a conservative sensitivity analysis, we count every unavailable score as an error: a false negative for an edited input and a false positive for an unedited input. We retain the original evaluation weights and six-bit threshold. Relative to the valid-score results in Table R2, counting all unavailable scores as errors across all 22 FFHQ conditions reduces overall F1 by 0.59 percentage points (from 90.21% to 89.62%), while incorporating the 24 unedited and 248 edited unavailable crop observations reduces crop-group F1 across the two cropping conditions by 0.61 percentage points (from 89.37% to 88.76%), indicating that the reported performance is not primarily driven by excluding unscoreable observations.

Overall, across all 22 FFHQ processing conditions, SeLo-Mark achieves 97.67% AUC and, at the original CelebA-HQ-calibrated six-bit threshold, 90.21% F1 (remaining at 89.62% when unavailable scores are counted as errors). Across the evaluated CelebA-HQ attribute groups, F1 ranges from 94.50% to 96.27%. These results support SeLo-Mark's cross-dataset detection performance at a fixed operating threshold, while the cropping and subgroup analyses clarify the remaining trade-offs between false alarms and missed edits.

AuthorsOpenReview ↗

Response to Reviewer A7Pa (4/4)

W2. Fairness of baseline comparisons under different payload sizes and publication qualities.

The main benchmark evaluates each method using its native payload and embedding configuration, resulting in differences in payload size and publication quality. To assess how these factors affect the comparison, we provide three complementary controls in the original manuscript and its appendix.

(1) Content-reference comparison with a fixed payload. Table 3 in the original manuscript compares different content representations while keeping the watermark channel, total 48-bit payload, protection masks, geometric predictor, and adapted reader fixed. Each representation uses an independently calibrated threshold. CLIP–PCA32 achieves 99.16% AUC and 95.71% F1, compared with 94.88% AUC and 88.56% F1 for Landmark–PCA32. This controlled comparison shows that the improvement from the content representation is obtained without changing the embedding capacity or watermark channel.

(2) Cross-method comparison with matched geometric processing and publication quality. Table 7 in the original manuscript compares SeLo-Mark and FractalForensics on shared valid inputs from the 128-image subset. At a matched publication PSNR of 40 dB, facial edits are regenerated and detection thresholds are independently recalibrated. SeLo-Mark achieves 99.08% AUC and 95.46% F1. FractalForensics achieves 94.86% AUC and 88.41% F1 with shared geometric correction alone, and 91.94% AUC and 85.55% F1 when this correction is combined with reader adaptation using the matched training budget. The adapted readers are reused at 40 dB rather than retrained at the matched publication strength. Thus, SeLo-Mark retains its detection advantage when geometric preprocessing and publication quality are controlled.

(3) Watermark recovery at matched publication quality. Table 21 in Appendix D.2 of the original manuscript compares crop bit accuracy at matched publication PSNR on the calibration set. The evaluation uses the native 32-bit payloads of WAM-32 and MaskWM-ED32 and the first 32 bits of SeLo-Mark's 48-bit payload, averaged over the two cropping conditions. At 40 dB, SeLo-Mark achieves 99.88% bit accuracy, compared with 85.62% for WAM-32 and 54.21% for MaskWM-ED32. These results, together with the full PSNR sweep in Table 21, demonstrate SeLo-Mark's crop-recovery advantage at matched publication quality: it achieves the highest bit accuracy at all six evaluated PSNR levels.

In summary, these comparisons address complementary aspects of evaluation fairness. The fixed-payload comparison supports the contribution of the CLIP–PCA32 content representation, while the cross-method experiments demonstrate SeLo-Mark's detection and crop-recovery advantages at matched publication quality. The detection comparison additionally controls geometric preprocessing, and the cross-method evaluations retain each method's native payload configuration.

Ethics. Subgroup errors and interpretation of verification results.

Although overall detection F1 remains high (94.50% to 96.27%) across the CelebA-HQ attribute groups in Q2 (Table R3), the shared six-bit threshold produces different error rates across the evaluated source-image attributes. For example, images labeled Young=0 have a higher false-negative rate than those labeled Young=1 (7.74% versus 3.28%), but a lower false-positive rate (3.04% versus 4.49%). The Male groups exhibit a similar trade-off. These differences indicate that the same operating threshold does not yield uniform false-alarm and missed-edit rates across the evaluated groups, underscoring the importance of reporting both error types alongside aggregate metrics.

The interpretation of verification outcomes is equally important. SeLo-Mark measures consistency between the recovered publication reference and the received facial content; it does not independently establish image authenticity. For a valid score, a distance of at most six bits indicates that the facial content is consistent with the recovered publication reference, while a distance exceeding six bits indicates inconsistency. If no valid facial observation or verification score is available, the result is unverifiable, rather than an automatic acceptance or rejection. In particular, as the adaptive evaluation in Q1 illustrates, a consistency result alone cannot rule out deliberate manipulation. Verification outcomes should therefore be presented as content-consistency assessments, with their error rates and score availability taken into account, rather than as proof of authenticity.

Reviewer_EuUoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SeLo-Mark checks the quality of a face while keeping a specific area of the face the same when the image is published. It gets a 32-bit binary content reference from frozen CLIP features and PCA. It puts this reference and a 16-bit tracing field outside the protected region.

When an image is received, the system can read watermarks and encode facial content. The system compares the recovered reference with a code recomputed from the received face. The paper reports an accuracy of 99.23% and an F1 score of 95.85% on the CelebA-HQ test. The system also works with FFHQ without changing the models or threshold.

Critical strengths

1.I find the spatial constraint useful and it is clearly defined. Equation 4 keeps the protected facial pixels the same when the image is published. The area around it shows where the face should be. The paper shows how to check tampering under these conditions. The transfer results on FFHQ also support the design.

2.The experiments do a good job of checking other possible reasons for the gains. Table 3 compares the content representations within the same watermark channel. Table 4 replaces recovered references with ground-truth references. Tables 7, 19 and 20 look at shared geometric correction, reader adaptation, and matched publication quality. These controls show more evidence than comparisons with baselines in their original settings.

Critical weaknesses

1.I couldn’t tell how much of the input data the reported detection results cover. The metrics use only available scores from valid facial observations (Section 4.1, Appendix A.1, and Eq. (10)). Publication success counts don’t show how many edited or processed inputs later fail to produce a score at reception. Table 27 compares methods on shared valid inputs. But the paper doesn’t report the overall fraction of excluded samples. The reported detection accuracy depends on whether a score is available. So I need to know the exclusion rate and how receiver failures are handled to assess system reliability.

2.The source of geometric generalization is still unclear to me. In Table 18, adaptation with watermarked RGB, watermark-free RGB, or shuffled residuals gives similar results. These variants share the same pretrained initialization, and all are evaluated on watermarked inputs. So this comparison doesn’t isolate the cues the receiver uses. A focused test on naturally tilted or off-center portraits would help assess this. The test should check whether the receiver distinguishes original portrait composition from later geometric processing. Table 23 also reports an Exact48 recovery rate of 72.50% under horizontal shear. This suggests that recovery depends strongly on the transformation. That dependence matters because geometric robustness accounts for a large share of the reported advantage.

Questions for authors

1.What proportion of inputs, edited or not, produces valid detection scores under each editing and processing condition?

2.What do you do if the person you're expecting doesn't show up? How much does the comparative advantage still apply when these cases are included in the evaluation?

3.What visual signals help the viewer to understand the changes made to the image without having to know about standard portrait composition? What happens if you use the existing receiver on portraits that are tilted or off-centre, without changing it first? Could you report how well the system detected and recovered?

3 comments
AuthorsOpenReview ↗

Response to Reviewer EuUo (1/3)

Thank you for recognizing the exact preservation of protected facial pixels and the careful design of our controlled comparisons. We appreciate your concerns regarding score availability, missing-face handling, and receiver behavior under natural portrait compositions and geometric variations, and have conducted additional analyses to address them.

Q1 (W1). What fraction of inputs produces a valid score under each condition?

We measure the fraction of all planned evaluations that yield valid detection scores, counting failures during watermark publication, edit generation, and reception as unavailable. Across 2,999 CelebA-HQ and 1,000 FFHQ source images (each evaluated in one unedited and seven edited versions across 22 processing conditions and two independently adapted receivers), Table R1 reports valid-score availability as CelebA-HQ / FFHQ for unedited inputs and each of the four editor families, grouping conditions with identical reported percentages for readability. Within each editor family, editing operations and receivers are aggregated using the evaluation weights in the original manuscript.

Table R1. Valid-score availability across processing conditions and editor families (%).

Processing conditionUneditedSimSwapCSCSStarGANSD inpainting
None; resize 0.75; mild blur; blur + sharpening99.90/99.5099.90/99.5099.90/99.5099.88/99.5098.76/98.77
JPEG Q80; Gaussian noise99.90/99.5099.90/99.5099.90/99.5099.90/99.5098.74/98.77
JPEG Q60; mild noise; contrast99.90/99.5099.90/99.5099.90/99.5099.90/99.5098.76/98.77
Resize 0.50; brightness; saturation99.90/99.5099.90/99.5099.90/99.5099.88/99.5098.74/98.77
Stronger blur99.90/99.5099.90/99.5099.90/99.5099.88/99.5098.73/98.80
Gaussian filter; median filter99.90/99.5099.90/99.5099.90/99.5099.88/99.5098.76/98.73
Salt-and-pepper noise99.90/99.5099.87/99.5099.87/99.5099.88/99.5098.73/98.83
Hue99.90/99.5099.90/99.5099.90/99.5099.90/99.5098.74/98.73
Rotation99.90/99.5099.90/99.5099.83/99.5099.88/99.5098.72/98.83
Perspective99.90/99.3099.90/99.3099.90/99.2099.88/99.1598.76/98.60
Horizontal flip99.90/99.5099.90/99.5099.90/99.5099.90/99.5098.77/98.83
Random crop 0.75; crop 0.75 + JPEG99.90/99.4099.90/99.4099.90/99.4099.88/99.4098.73/98.73

As shown in Table R1, valid-score availability remains consistently high across both datasets, including under geometric transformations and cropping: under FFHQ cropping, availability is 99.40% for unedited inputs, SimSwap, CSCS, and StarGAN, and 98.73% for SD inpainting, while the lowest entry across the entire table is 98.60% (SD inpainting under FFHQ perspective distortion). Averaged across all 22 conditions (using equal weights across the four editor families for edited inputs), valid scores are available for 99.90% of unedited and 99.61% of edited evaluations on CelebA-HQ, and for 99.48% and 99.30%, respectively, on FFHQ.

To separate upstream pipeline omissions from receiver failures, we also examine all raw evaluation requests that reach reception (1,050,104 of 1,055,648 planned on CelebA-HQ and 349,360 of 352,000 on FFHQ after publication and edit-generation filtering). Among inputs reaching the receiver, only 114 on CelebA-HQ (0.011%) and 142 on FFHQ (0.041%) fail to yield a valid score, showing that almost all unavailable cases stem from upstream publication or synthesis rather than reception. Overall, SeLo-Mark successfully scores 99.99% of received inputs on CelebA-HQ and 99.96% on FFHQ, confirming that our reported detection performance reflects nearly the entire planned benchmark rather than selective filtering (see Q2 for conservative error accounting).

AuthorsOpenReview ↗

Response to Reviewer EuUo (2/3)

Q2 (W1). How are missing faces handled, and does the comparative advantage remain?

When no valid facial observation is available, SeLo-Mark returns an unverifiable result rather than making a facial-content consistency decision. If a valid facial observation is available, following the face-selection rule in Appendix A.2 of the original manuscript, the verifier compares the content code of the highest-confidence detected face with the recovered publication reference at the original six-bit threshold, regardless of whether it is the expected person. Thus, the presence of a different face does not itself prevent verification; its content is evaluated against the published reference.

To assess whether excluding unavailable scores affects the reported performance, we recompute F1 over all planned evaluations by counting every unavailable score (across publication, edit generation, and reception) as an error: a false negative for edited inputs and a false positive for unedited inputs. Each method retains its original calibrated threshold and the evaluation weights in the original manuscript. Table R2 compares these all-planned-evaluation results with the valid-score F1 values reported in Tables 1 and 8 of the original manuscript.

Table R2. Effect of unavailable scores on facial-edit detection F1 (%).

DatasetMethodF1 on valid scoresF1 on all planned evaluations
CelebA-HQSeLo-Mark95.8595.60
CelebA-HQFractalForensics89.4588.97
FFHQSeLo-Mark90.2189.62

As shown in Table R2, counting all unavailable scores as errors reduces SeLo-Mark's overall F1 by only 0.25 percentage points on CelebA-HQ (from 95.85% to 95.60%) and 0.59 percentage points on FFHQ (from 90.21% to 89.62%). Applying the same conservative accounting to the two FFHQ cropping conditions (incorporating 24 unedited and 248 edited unavailable evaluations relative to the 89.37% valid-score crop F1 in Table 8 of the original manuscript) similarly reduces F1 by only 0.61 percentage points to 88.76%. Under the identical all-planned-evaluation protocol on CelebA-HQ, SeLo-Mark outperforms FractalForensics by 6.63 percentage points in F1 (95.60% versus 88.97%), demonstrating that SeLo-Mark's detection performance and comparative advantage remain solid under this conservative error accounting.

Q3 (W2). Can the original receiver handle naturally tilted or off-center portraits, and which visual cues does it use?

(1) Performance on natural portrait compositions. To examine whether SeLo-Mark relies on upright and centered portrait framing, we evaluate the frozen receivers on four groups of 32 CelebA-HQ test images (128 portraits in total), partitioned by natural facial tilt and horizontal face offset while preserving their original framing. All receiver models, geometric predictors, content mappings, and the six-bit threshold remain unchanged. In Table R3, Geometry aggregates rotation, perspective transformation, and horizontal flipping, and Crop aggregates random 0.75-side-length crops with and without JPEG compression; detection metrics (cross-threshold AUC and six-bit-threshold F1) average the four editor families and two independently adapted receivers, while recovery metrics on unedited publications report 32-bit content accuracy (BA32) and exact 48-bit payload recovery (Exact48) across 64 no-processing, 192 geometry, and 128 crop readouts per group. Across all 22 benchmark conditions on these 128 portraits, valid-score availability is 100.00% for unedited inputs and 99.60% across the four editor families.

Table R3. Facial-edit detection and watermark recovery by natural portrait composition (%).

Portrait composition (32 images/group)No processing AUC/F1Geometry AUC/F1Crop AUC/F1Crop BA32/Exact48
Upright, centered99.99/98.5899.91/98.2198.80/93.21100.00/100.00
Tilted, centered99.83/97.3298.89/95.7497.86/94.08100.00/100.00
Upright, off-center99.98/95.2799.93/97.2698.16/91.5399.98/99.22
Tilted, off-center99.99/97.7899.86/97.9299.13/94.1499.71/95.31

As shown in Table R3, across all four portrait-composition groups, SeLo-Mark achieves 98.89%–99.93% AUC (95.74%–98.21% F1) under geometric processing and 97.86%–99.13% AUC (91.53%–94.14% F1) under cropping. On unedited publications in every group, both BA32 and Exact48 reach 100.00% under no processing (64/64 readouts per group) and across the three geometric conditions (192/192 readouts per group), while under cropping BA32 remains at least 99.71% and Exact48 at least 95.31%. These results confirm that SeLo-Mark generalizes effectively to naturally tilted and off-center portraits without retraining or threshold recalibration.

AuthorsOpenReview ↗

Response to Reviewer EuUo (3/3)

(2) Effect of cropping on facial-content verification. In Table R3, crop F1 (91.53%–94.14%) is somewhat lower than F1 in the no-processing and geometry groups despite near-perfect watermark recovery. To isolate the cause on the same four portrait groups, we keep the received facial observations and six-bit threshold unchanged while substituting the original publication references for the recovered ones; across the four groups, this changes crop F1 by at most 0.63 percentage points in absolute value. Even with the original publication references, the mean verification distance (Hamming distance between the 32-bit reference and the recomputed face code) on unedited portraits shifts from 0.83–1.05 bits without processing (64 readouts per group) to 3.62–3.97 bits under cropping (128 readouts per group). This confirms that SeLo-Mark's crop reference recovery introduces minimal verification loss (within 0.63 F1 points of the oracle reference), and that the modest fixed-threshold shift arises from partial-face cropping while overall detection remains strong (97.86%–99.13% AUC and 91.53%–94.14% F1).

(3) Visual cues for geometric estimation. Table 18 in Appendix C.4 of the original manuscript shows comparable geometric accuracy when the coordinate predictor is adapted using watermarked RGB, watermark-free RGB, or shuffled watermark residuals. Because these variants share the same pretrained initialization and are evaluated on watermarked inputs, we conduct an inference-time intervention on the same 128 portraits across both adapted predictors (256 four-corner predictions per setting).

Relative to the ground-truth canvas corners, the mean corner error is 0.09 pixels on the original portraits and 1.11 pixels under the benchmark gray-padded rotation. Replacing the gray rotation padding with black or reflection padding while keeping all valid RGB pixels unchanged shifts the predicted corners by 89.24 and 91.89 pixels on average, whereas removing the watermark under the same gray-padded rotation changes the predicted corners by only 0.09 pixels on average. These controls indicate that under the benchmark rotation protocol, SeLo-Mark's coordinate predictor relies primarily on padding-boundary cues rather than watermark signal strength (achieving 0.09-pixel error on original portraits and 1.11-pixel error under matched gray-padded rotation, while shifting under altered padding). At the same time, when padding boundaries are absent under borderless 0.75 cropping, SeLo-Mark still maintains 99.71%–100.00% BA32 and 97.86%–99.13% AUC across all four portrait compositions (Table R3).

(4) Recovery and verification under additional shear. We also evaluate an unseen 20° horizontal shear not sampled during training or receiver adaptation. Table 23 in Appendix E.2 of the original manuscript reports 72.50% Exact48 across 2,996 unedited publications and two independently adapted receivers (5,992 readouts), yet our bit-level analysis of the same 5,992 readouts shows that 32-bit content accuracy (BA32) remains 99.09%. Evaluating SimSwap and CSCS face-swap detection under this 20° shear across both receivers at the original six-bit threshold (2,995 valid unedited and 2,995 valid swapped observations per editor–receiver pair), SeLo-Mark achieves 98.45% AUC and 93.71% F1, coming within 0.75 F1 percentage points of the 94.46% oracle-reference control. Overall, across natural portrait compositions, cropping, and the tested 20° shear, SeLo-Mark preserves high content-reference accuracy and strong facial-edit detection without receiver retraining or threshold recalibration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.