- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ZeroFace, a privacy-preserving framework for face verification that combines three components: Adaptive Matryoshka Compression (AMC) for reducing a 512-dimensional face embedding to a compact representation, IS-DOT for identity-specific protection through bounded DCPE perturbation and orthogonal transformation, and Secure Cosine Distance Computation (SCDC) for matching secret-shared templates using Beaver triples.
The method is evaluated on standard face-verification benchmarks including LFW, CFP-FP, AgeDB, CALFW, CPLFW, IJB-B, and IJB-C, as well as against FEM embedding-inversion and MAP2V score-based reconstruction attacks. The reported results show that the 128-dimensional representation retains recognition performance close to the underlying ArcFace model and substantially reduces reconstruction attack success compared with several template-protection baselines. Additional analyses consider soft-biometric attribute leakage, unlinkability, computational/storage cost, compression dimensionality, IS-DOT components, and AMC adaptation to other recognition backbones.
Critical strengths
-
The manuscript evaluates both embedding-inversion and score-based reconstruction using recent FEM and MAP2V attacks rather than relying only on reconstruction error or conventional template-security measures. Table 2 shows substantial reductions in attack success rates relative to MLP-Hash, PolyProtect, and SlerpFace under the evaluated settings. This is consequential because resistance to modern generative reconstruction is central to the paper's motivation.
-
Despite reducing the ArcFace embedding from 512 to 128 dimensions, the proposed representation remains close to the unprotected model across LFW, CFP-FP, AgeDB, CALFW, CPLFW, IJB-B, and IJB-C. The paper also includes meaningful analyses of dimensionality, attribute leakage, unlinkability, IS-DOT components, and adaptation to several other FR/PPFR backbones, giving a more complete picture than a single utility-versus-privacy table.
Critical weaknesses
-
AMC combines linear projections of nested prefixes using global learned scalar weights. Consequently, the complete mapping is itself an affine transformation. The paper compares AMC with PCA, an MLP, and Naive MRL in Table 5, but does not compare with the most important control: a single Linear(512,128) layer trained with exactly the same VGGFace2 data, ArcFace supervision, optimizer, and frozen recognition backbone. Without this comparison, it is unclear whether the reported benefit comes from the proposed Matryoshka structure or simply from supervised dimensionality reduction. This is central to the paper's claimed methodological contribution.
-
In Table 2, MLP-Hash, PolyProtect, and SlerpFace are attacked through their exposed protected representation or score, while the full ZeroFace setting assumes that a compromised party observes only one additive secret share. A single random share is intentionally insufficient to reconstruct the protected embedding, so the near-zero ASR is not directly comparable to attacks on complete protected templates. Furthermore, Section 3.4 states that basic SCDC can reconstruct the similarity score and that hiding the score requires the optional TEE. It is therefore unclear whether the 0% score-based ASR in Table 2 should be attributed to SCDC itself or to the SCDC+TEE decision-only configuration. The paper should separate representation robustness, secret-sharing confidentiality, and score suppression.
Questions for authors
-
Since AMC consists of linear branches combined using global learned scalar weights, the complete mapping can be collapsed into a single affine 512-to-128 transformation. What recognition and FEM-ASR results are obtained using a single Linear(512,128) layer trained with exactly the same VGGFace2 data, ArcFace supervision, optimizer, epochs, and frozen IR-50 backbone? This comparison would materially affect my assessment of AMC's novelty.
-
Can the authors separate Table 2 into results for: (i) AMC+IS-DOT with the complete protected embedding exposed, (ii) SCDC with only one additive share exposed, (iii) basic SCDC with the exact similarity score released, and (iv) SCDC+TEE with decision-only output? In particular, was the reported 0% score-based ASR for “Ours” obtained using the optional TEE?
-
Where are the identity-specific parameters stored, and how does a legitimate client obtain the parameters corresponding to claimed identity ? Under the client-compromise scenario in Figure 1, does the adversary obtain these parameters, and what protection remains if both the protected embedding and identity-specific key are compromised?
-
The irreversibility analysis reports 0% SAR using a numerical solver with only ten iterations. Have the authors evaluated stronger known-operator attacks such as pseudoinverse recovery, longer embedding optimization, or an inversion network explicitly trained against AMC?
-
Has identity overlap between the VGGFace2 dataset used for AMC training and the verification datasets been checked or removed? Because AMC is trained with identity supervision, confirmation of identity-disjoint evaluation would strengthen the generalization claim.
5 comments
Thank you. The added baseline and clarifications partly address my concerns.
However, several issues remain. First, AMC is exactly affine under the described structure, so please distinguish its training benefits from its final mapping. Second, the 0% score-based ASR applies to the single-share setting. With exact scores released, ASR reaches 37.1% on M.Face. Please clarify the attacker’s access in each setting and revise the security claims accordingly.
Third, the response does not establish protection when the target identity’s key and protected template are both compromised and the operators are known. Please evaluate this setting and stronger known-operator recovery attacks. Finally, please confirm identity-overlap checks for all evaluation datasets and state whether LFW filtering applied to the original experiments.
Effeteness of AMC
We agree that the final mapping from AMC is affine before L2 normalization. Inspired by MRL’s hierarchical representations, AMC uses a multi-granularity training parameterization. The effectiveness of this training parameterization can be observed from the previous Single Linear layer experiment. AgeDB achieves improved verification accuracy of 96.38% to 97.68% and M.Face achieves reduced FEM-ASR of 75.5% to 47.9%. We will distinguish the benefits of the training scheme.
Attacker’s Access
We thank reviewer’s detailed comments on attacker’s access in TI attack experiments. To distinguish the different protection settings, we explicit restrict the information that accessed by the attacker.
| Method / setting | Attacker access | Inversion-based ↓ E.Face | Inversion-based ↓ M.Face | Score-/decision-based ↓ E.Face | Score-/decision-based ↓ M.Face |
|---|---|---|---|---|---|
| Unprotected | Original embedding + exact scores | 92.6 | 99.3 | 87.3 | 98.1 |
| AMC+IS-DOT | Protected embedding + exact scores | 0.4 | 0.5 | 0.6 | 42.3 |
| SCDC (single additive share) | One additive share only | 0.0 | 0.4 | N/A | N/A |
| SCDC (exact score) | Exact scores only | N/A | N/A | 0.0 | 37.1 |
| SCDC+TEE (decision only) | Binary accept/reject decisions only | N/A | N/A | 0.0 | 12.8 |
N/A indicates that the evaluated attack is not applicable as its required information is unavailable.
-Unprotected: The attacker has access to both the original embedding and similarity scores. This allows both the inversion-based and score-based attacks.
-AMC+IS-DOT: The attacker has access to both the original embedding and similarity scores. This allows both the inversion-based and score-based attacks.
-SCDC (Single additive share): The attacker has access to only a single additive share. This allows the inversion-based attack using this share.
-SCDC (Exact score): The attacker has access to similarity scores. This allows the score-based attack.
-SCDC+TEE (Decision only): The attacker has access to only the accept/reject decision. This allows the score-based attack by replacing the score feedback as binary match in the MAP2V attack pipeline.
Overall, SCDC itself remains vulnerable to score-based attacks when attacker has access to exact similarity scores, with an ASR of 37.1% on M.Face.
Strong Known-operator Attack
We thank reviewer’s suggestion for evaluating on the strong known-operator attack. We implement a pseudoinverse attack (based on Moore–Penrose pseudoinverse) to evaluate AMC+IS-DOT under different attack settings regarding to knowledge of AMC weight and IS-DOT key on AgeDB evaluation dataset. We consider four combinations of attack exposure. In the unknown-weight setting, a randomly initialized AMC surrogate is used. We assume attacker has white-box access to structure/algorithms of AMC and IS-DOT.
| AMC weight | IS-DOT key | SAR |
|---|---|---|
| known | known | 98% |
| known | unknown | 0% |
| unknown | known | 0% |
| unknown | unknown | 0% |
These results indicate that exposing model structure, model weight and key at the same time can potentially break embedding protection. Keeping the IS-DOT key secret reduces the SAR to 0%, even when the AMC weights are known, demonstrating its contribution against this pseudoinverse recovery attack.
Overlap Identities
In our work, we used VGGFace2 training dataset and evaluation datasets provided by ArcFace paper [1]. All overlap identities between training dataset and evaluation datasets are removed. All our original experiments are conducted under this setting.
[1] Deng, Jiankang, et al. "Arcface: Additive angular margin loss for deep face recognition." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.
We thank again for reviewer’s careful reading of our paper. We would be glad for any further discussion.
Responses to Reviewer ad4V
Thanks for the constructive and valuable comments.
Clarification of Table 2, in our paper, we excluded TEE of SCDC in all the experiments.
W1&Q1 AMC:
We agree that AMC can be potentially approximated to an affine transformation. As the reviewer suggested, we include a baseline model using a Single Linear layer (Single Linear(512,128)) trained in the same way. AMC achieves higher verification accuracy than the Single Linear layer and demonstrates greater robustness to the FEM attack on M.Face, with an ASR of 47.9% compared with 75.5%. This demonstrates the benefits of training the model at multiple scales using AMC.
W2&Q2 Table 2:
TEE is not included in our two tested settings in Table 2. Therefore, the 0% score-based ASR for “Ours” obtained without using TEE. We would like to clarify the requirements for both TI attacks considered in our work. The inversion-based attack utilizes embeddings, while the score-based attack utilizes the cosine scores for image reconstruction. As the reviewer suggested, we report different settings as below. AMC+IS-DOT is already included in Table 2.
Since the score-based attack MAP2V relies on zeroth-order gradient estimation, it requires continuous cosine similarity scores to estimate gradients and iteratively optimize the StyleGAN latent codes. In the decision-only output setting, accessing only the binary output can greatly disrupt the attack's effectiveness.
Q3 IS-DOT Key:
The identity-specific parameters are generated during enrollment and stored as the local record on the client side, e.g., (ID1, IS-DOT key), (ID2, IS-DOT key). During 1:1 verification, the legitimate client retrieves the IS-DOT key from the local record for the claimed identity. In our experiments, the adversary's goal is to reconstruct a complete face image from the leaked protected embedding.
In the client-compromise scenario in Figure 1, we assume the adversary can submit images to the system and obtain corresponding protected embeddings. The adversary has no knowledge of the structure of AMC and IS-DOT. In the inversion-based attack, we assume the adversary has access to IS-DOT keys of identities in the training dataset (FFHQ). Although the adversary can train an FEM model, it cannot generalize well to another dataset (CelebA-HQ) due to unseen identities, see performance in Table 2.
Q4 Irreversibility:
In the irreversibility experiment, we randomly initialized ten solver runs, each with its own internal iterations. We will revise this description. The current inversion-based attack (FEM) presented in our work is a learned inversion network. We trained FEM models on compressed embeddings produced by AMC with different levels of compression, see results in Figure 5. AMC can reduce ASR from 99.1% (attacked based on the raw embedding) to 47.9% on 128-d compressed embedding. Thus, ZeroFace consists of AMC, IS-DOT and SCDC for enhanced protection.
Q5 Identity-disjoint Evaluation:
We thanks reviewer’s suggestion. We filtered out overlap identities between LFW and VGGFace2 when training AMC. In addition, we also trained the AMC on MS1M dataset using the same training configuration.