- Rating
- 1(−1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper argues that standard ArcFace face-crop evaluation overlooks identity cues in unchanged regions such as hair and ears. It introduces segmentation-based face, head, and ring probes with separately calibrated thresholds, using PHANTOM-150K full-head swaps as a control. At a 0.1% false-accept rate, the ring probe matches the original target in 29–97% of face-oval swaps, compared with 2.3% for the full-head teacher. The authors propose evaluating privacy-preserving swaps through both successful donor-identity transfer and the absence of remaining target identity.
Critical strengths
• The paper identifies an important evaluation gap. Passing a test on the central face does not tell us whether the surrounding hair and ears still match the original image.
• The measurements are carefully checked. The authors use separate thresholds for each region, statistical tests, identity-separated comparisons, and checks for changes in region boundaries.
• The reporting is open and detailed. The authors report rejected samples, provide information needed to reproduce results, explain baseline limitations, and correct their earlier bounding-box test.
Critical weaknesses
• The ring test may confuse copied pixels with identity, because Figure 4 shows that the surrounding region remains almost unchanged, so high similarity does not establish whether the probe recognizes the person or simply matches the original image.
• The paper never tests whether the ring can match separate images of the same person, leaving its usefulness under changes in lighting or hairstyle unproven.
• The gallery may contain the exact target photographs used to generate the swaps, allowing high accuracy through matching unchanged regions rather than identifying someone from another photograph. So, the retrieval results are difficult to interpret.
• The paper does not document who claims successful identity removal from face-crop similarity or discuss relevant anonymization research, making the scope of the problem it criticizes unclear.
• The contribution beyond existing methods needs clearer justification, because calibration, retrieval, region definitions, and the generation adapter come from prior work, leaving it unclear what new capability or insight their combination provides.
• The full-head comparison does not clearly establish an independent advantage, because BFS-Qwen shares the teacher’s adapter family and uses different evaluation pairs, galleries, and thresholds, so its lower leakage rate cannot be directly attributed to a better method.
• The proposed privacy rule does not assess all measured identity leakage, because Equation 17 considers only the ring region and ignores leakage within the central face, potentially accepting a result that still reveals the target’s identity.
• The evaluation is too limited to support broad benchmark claims, because the reported experiments use only 300 pairs, include several limited baselines, and omit FaceFusion/inswapper from most supporting analyses.
Questions for authors
• Does the gallery contain the exact target photographs used to generate the swaps, and what happens when those photographs are replaced with different images of the same people?
• How often does the ring test correctly match different photographs of the same person at a false-match rate of 0.1%, and how does a hairstyle change affect this?
• How do simple image-comparison methods, such as SSIM or perceptual hashing, perform, and what happens when the ring’s shape is preserved but its hair content is replaced?
• Can you cite a paper or system that claims identity removal based only on face-crop similarity, and explain how your work differs from DeepPrivacy, CIAGAN, FALCO, RiDDLE, and person re-identification research?
• Why is the independent full-head method by Jeong et al. (2025) not evaluated, even though it helps motivate your approach?
• Is BFS-Qwen-V4 evaluated using the same 300 pairs and thresholds as the other methods, and, if not, how should its 1.7% result be compared with theirs?
• How do the face-oval methods perform on image pairs rejected by the quality filter that was tuned using the teacher model?
• Why does Equation 17 consider only the ring region, and how would the conclusions change if identity leakage across all tested regions were considered?
• Can you explain how the Spearman p-value was calculated and include uncertainty in the ring threshold when reporting leakage rates?
• Why is FaceFusion/inswapper missing from Tables 3, 4, 7, 8, and 9, and can its results be added?
• Why could 994,293 image-pair records not be loaded, and could the missing examples differ in ways that affect the evaluation?