acceptodds
Under review as a conference paper at ICLR 2027

FaceFlow: Learning Intra-Class Variation via Flow Matching in the Identity Embedding Space for Synthetic Face Generation

Abstract

Generating synthetic data for face recognition (FR) training offers a practical alternative to authentic face datasets and supports privacy protection. Existing identity-conditioned generative models can synthesize photorealistic faces with high identity consistency, yet they typically exhibit limited intra-class diversity. Perturbation of the identity condition has been introduced to alleviate this limitation, but existing methods constrain only the perturbation magnitude and sample the perturbation direction uniformly at random in the identity embedding space, without modeling the intra-class variation of real faces. We propose FaceFlow, a flow-based model trained on same-identity pairs to learn an identity-conditioned flow whose endpoint follows the high-dimensional intra-class distribution. FaceFlow guides the perturbation of identity embeddings, which are then decoded by a pre-trained generator to construct synthetic face datasets. FR models trained on the resulting synthetic datasets outperform those trained on datasets constructed using state-of-the-art synthetic-data methods on standard benchmarks and surpass 94% average accuracy at a training scale of 0.5M synthetic images. These results show that FaceFlow enables the reliable generation of diverse intra-class samples while maintaining high identity consistency. The FaceFlow code and generated data will be publicly available at https://anonymous.4open.science/r/FaceFlow-63EF/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Face recognition, Face image synthesis, Riemannian flow matching

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_VL2kOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a synthetic face generation paradigm, FaceFlow. It promotes generation diversity by introducing learnable intra-class variation in the identity embedding space. The method replaces the direction-agnostic perturbation used in IDPerturb with an identity-conditioned direction learned via Riemannian flow matching. Experiments show improved identity consistency and downstream face recognition performance over random perturbation.

Critical strengths
  1. The paper identifies a reasonable limitation in synthetic face generation, regarding random angular perturbation. The analysis in Fig. 1 provides useful evidence of anisotropic real intra-class variation and reasonably motivates learning the perturbation direction.
  2. The proposed method is conceptually simple and can be applied without retraining the face generator.
Critical weaknesses
  1. This paper exhibits relatively limited novelty over prior identity-perturbation SOTAs. Angular embedding perturbation has been well established as an effective way to enrich generation diversity (e.g., in IDPerturb, SteerFace, and NegFaceDiff). SteerFace further introduced learnable perturbation magnitude. Therefore, FaceFlow neither introduces embedding perturbation itself nor the paradigm of learnable perturbation; its main methodological contribution is limited to learning the direction instead of sampling it randomly.
  2. Current experiments do not sufficiently isolate where the improvement comes from. The main performance comparison in Tab. 1 is built on top of UIFace, which is itself a strong SOTA method, rather than a neutral baseline, making it difficult to separate the gains inherited from UIFace from those introduced by FaceFlow. It also lacks decisive evidence whether the paper’s key modeling choices (learned directional prior and being identity-specific) are necessary; further see questions.
Questions for authors
  1. FaceFlow is built on a pretrained UIFace generator, a strong SOTA that already performs the second best in Tab.1, rather than the neutral baseline commonly used in this line of work (which UIFace itself is based on), i.e., IDiff-Face. Therefore, its FR performance in Tab. 1 already benefits from the established strength of UIFace and does not clearly show how much improvement is brought by FaceFlow alone. The authors should supplement a parallel, controlled comparison between IDiff-Face and an IDiff-Face-based FaceFlow to isolate the contribution.
  2. FaceFlow neither newly introduces embedding perturbation itself nor the paradigm of learnable perturbation. The methodological distinction from recent identity-perturbation SOTA methods (IDPerturb, SteerFace, and NegFaceDiff) needs to be made much clearer.
  3. The authors miss several relevant SOTA methods (e.g., MorphFace, NegFaceDiff and SteerFace) for comparison. I find the omission of SteerFace particularly puzzling, as FaceFlow is not only methodologically related to SteerFace in both developing learnable embedding perturbation, but also resembles it in several specific aspects of the experimental presentation, e.g., the trade-off visualization and SDD-FIQA-based evaluation. It is therefore quite surprising that SteerFace is not discussed at all.
  4. While adaptive perturbation directions are plausible, it is unclear whether Riemannian flow matching is necessary to obtain them. Once real intra-class variation is known to be anisotropic, simpler data-driven models such as PCA-based sampling, or GMM-based sampling to account for the multimodal structure claimed by the authors, could already provide non-random, adaptive directions. The authors should consider comparing flow matching with such methods as baselines for anisotropic sampling.
  5. It is also unclear whether the learned directional prior needs to be identity-specific. The paper models p(u∣A,θ)p(u\mid A,\theta), but Fig. 1 only shows that intra-class directions are structured and non-uniform; it does not show that this structure must vary with the reference identity. Could the authors compare against a shared learned prior such as p(u∣θ)p(u\mid\theta)? This would directly test whether conditioning on AA is necessary.
2 comments

For Question 1. To isolate FaceFlow’s contribution from the choice of generator, we conducted a controlled comparison using the IDiff-Face generator and the same 10,000 virtual identities used in our paper. IDPerturb achieves scores of 98.73, 91.73, 91.64, 92.65, and 88.07 on LFW, AgeDB-30, CFP-FP, CALFW, and CPLFW, respectively, with a mean of 92.56. Replacing its random perturbation directions with directions produced by the FaceFlow flow increases the scores to 98.97, 92.78, 93.64, 92.95, and 89.00, respectively, with a mean of 93.47. Thus, FaceFlow improves the mean by 0.91 percentage points even with the IDiff-Face generator, indicating that its gains are not attributable solely to UIFace.

For Question 2. FaceFlow’s contribution is to learn the identity-conditioned distribution of intra-class directions through flow matching on real same-identity pairs. IDPerturb samples random directions within a spherical cap around the reference identity. SteerFace uses random orthogonal perturbations as a training regularizer to suppress intra-class semantics in the learned identity representation, then removes the perturbation at inference. In contrast, FaceFlow keeps the generator frozen and uses the endpoint of a flow trained on real same-identity pairs as the perturbation direction. NegFaceDiff takes a different approach by modifying the diffusion sampling process with negative identity guidance to push generated samples away from other identities. Thus, the methods target different aspects of generation: SteerFace suppresses intra-class semantics during training, NegFaceDiff promotes separation from other identities, and FaceFlow models intra-class variation in the identity embedding space.

For Question 3. MorphFace and SteerFace are valuable works. We did not include MorphFace in our experimental comparison because we could not find publicly available code during our experimental period. SteerFace emerged as concurrent work during our experiments and was therefore not included in the original evaluation. NegFaceDiff is also relevant, as it introduces negative identity guidance during diffusion sampling to promote inter-class separation, whereas FaceFlow learns intra-class directions in the identity embedding space.

We recognize the similarity between SteerFace and our use of SDD-FIQA and trade-off visualizations. During our experiments, we found that the conventional intra-class diversity metric DintraD_{intra} can increase when face generation quality is poor, potentially overstating useful diversity. We therefore use SDD-FIQA as a complementary and more intuitive way to assess the quality of FaceFlow-generated faces alongside their intra-class diversity.

For Question 4. To compare direction samplers, we evaluated them under the same lb=0.5l_b = 0.5, 10k × 50 downstream face-recognition protocol. FaceFlow achieves scores of 99.15, 93.68, 94.49, 93.40, and 89.32 on LFW, AgeDB-30, CFP-FP, CALFW, and CPLFW, respectively, with a mean of 94.01. The isotropic Gaussian sampler achieves 91.08, 83.12, 77.67, 83.97, and 68.62, with a mean of 80.89. The rank-16 PCA sampler achieves 94.37, 76.80, 57.71, 84.72, and 62.20, with a mean of 75.16. FaceFlow achieves the highest mean among the three methods.

For Question 5. To test whether the directional prior benefits from identity conditioning, we trained a shared flow $p(u \mid \theta)$ that does not take the reference embedding A as input. Using the same 10,000 identities and UIFace decoder, with 50 images generated per identity, this shared flow achieves scores of 98.85, 91.25, 83.26, 91.60, and 82.70 on LFW, AgeDB-30, CFP-FP, CALFW, and CPLFW, respectively, with a mean of 89.53. Compared with the identity-conditioned FaceFlow result of 94.01, the largest drops are on CFP-FP and CPLFW. This suggests that conditioning the flow on the reference identity is particularly beneficial for intra-class variation.

Thank you for the questions and constructive feedback. We hope these clarifications and planned revisions address the concerns and support an improved final assessment.

Dear Reviewer,

Thank you again for the careful review and for the questions. We have posted our responses, together with the additional experimental results.

Could you please let us know whether these responses and results address your concerns? If any point remains open, we would be very grateful if you could tell us which further experiment you would still find necessary. We will prioritize that within the discussion period.

Thank you again for your time.

Best regards, The Authors

Reviewer_cdRPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a new method to generate synthetic face dataset for training face recognition models. The proposed method, FlowFace, is based on a flow-based model trained on same-identity pairs to learn an identity-conditioned flow whose endpoint follows the high-dimensional intra-class distribution. Models trained with the proposed dataset achieve remarkable recognition performance.

Critical strengths
  • Models trained with the proposed dataset achieve remarkable recognition performance.
  • The proposed method increases intra-class variation in the synthetic datasets.
Critical weaknesses
  • Authors need to expand their comparison with previous methods. For diffusion-based methods, MorphFace [1] is missing (and possibly SteerFace [2]). For GAN-based methods, Langevin-Disco [3] can also be used as baselines.
  • Most experiments are based and compared to IDPerturb. Authors need to expand their experiments (see my questions).
  • Authors have not verified if there is any leakage of identity / memorisation in synthetic dataset. See [4].
  • Some sentences are not very clear and hard to read. The writing and presentation can be improved.

[1] Data Synthesis with Diverse Styles for Face Recognition via 3DMM-Guided Diffusion, CVPR, 2024.

[2] SteerFace: Debiasing Synthetic Face Generation via Adaptive Residue Perturbation, ACM MM, 2026.

[3] Synthetic Face Datasets Generation via Latent Space Exploration from Brownian Identity Diffusion, ICML, 2025

[4] Unveiling Synthetic Faces: How Synthetic Datasets Can Expose Real Identities, arxiv:2410.24015, 2024

Questions for authors
  • The proposed method is mainly focused on intra-class variation. Can we also enhance inter-class variation?
  • FaceFlow represents a perturbed identity embedding by its angular displacement from the reference identity embedding (Boutros et al., 2026; Lin et al., 2025). Can we use other methods as reference identity and how it affects the performance?
  • Can we have control on different face attributes in FaceFlow data generation?
  • Can proposed method reduce bias in model trained with synthetic dataset?
4 comments

For Question 1. FaceFlow models intra-class variation conditioned on a reference identity. In future work, we plan to extend the framework to jointly model reference identities and improve inter-class variation and identity-space coverage.

For Question 2. To investigate whether FaceFlow can use reference identities obtained through an alternative sampling method and how this affects downstream performance, we conducted an additional experiment using 10,000 reference identities sampled following the synthetic identity-sampling protocol of IDiff-Face (Boutros et al., 2023). Each identity is sampled from a 512-dimensional standard Gaussian distribution and L2-normalized onto the unit hypersphere. We use the frozen UIFace generator to decode these embeddings into aligned faces. FaceFlow and IDPerturb use the same reference identities: FaceFlow learns intra-class variation through flow matching, whereas IDPerturb samples perturbation directions uniformly.

For downstream face recognition, we train ResNet-50 models with CosFace for 34 epochs under the same protocol for both methods. We evaluate on LFW, AgeDB-30, CFP-FP, CALFW, and CPLFW. FaceFlow / IDPerturb achieve verification accuracies of 99.08 / 97.30 on LFW, 91.58 / 85.87 on AgeDB-30, 91.79 / 77.21 on CFP-FP, 92.53 / 89.28 on CALFW, and 87.68 / 78.28 on CPLFW, with averages of 92.53 / 85.59, respectively. These results suggest that learning intra-class directions in the identity embedding space improves recognition while keeping the reference identities fixed, with the largest gains on the pose benchmarks CFP-FP and CPLFW.

For Question 3. The attribute-direction signal reported in the paper provides evidence that FaceFlow captures meaningful attribute-related intra-class variation. At an angular perturbation of 52.1°, the hat and eyeglasses occurrence rates are 38.9% and 61.2%, respectively, compared with 4.2% and 14.7% for the unperturbed anchors. Attribute-specific control is a valuable direction for future work. For example, in building access-control systems, a person may be misrecognized as a different identity when wearing a different style of glasses. We aim to improve recognition performance for such variations in specific domains. However, our experiments indicate that this requires paired data of sufficient scale and quality, which are difficult to obtain from current datasets. We therefore leave systematic, domain-specific attribute control and its impact on recognition performance to future work.

For Question 4. We agree that bias evaluation requires a clearly defined metric and would welcome your perspective on the most appropriate measure for this setting.

For Weakness 1. MorphFace and SteerFace are valuable works. We did not include MorphFace in our experimental comparison because we could not find publicly available code during our experimental period. SteerFace emerged as concurrent work during our experiments and was therefore not included in the original evaluation. We will include Langevin-Disco as a GAN-based baseline in the final version of the paper.

Regarding Writing and Presentation. We appreciate this comment and will revise the manuscript to improve sentence clarity and overall presentation in the final version. If there are specific passages that are particularly difficult to follow, please point them out, and we would be happy to revise them.

Thank you for the questions and constructive feedback. We hope these clarifications and planned revisions address the concerns and support an improved final assessment.

Dear Reviewer,

Thank you again for the careful review and for the questions. We have posted our responses, together with the additional experimental results.

Could you please let us know whether these responses and results address your concerns? If any point remains open, we would be very grateful if you could tell us which further experiment you would still find necessary. We will prioritize that within the discussion period.

Thank you again for your time.

Best regards, The Authors

Thanks authors for their responses. However, the paper has not been revised and most of my concerns are not addressed. Therefore, I maintain my rating.

Thank you for reading our response and for the follow-up. We will add the experimental data reported in our response to the camera-ready version, together with the corresponding updates to the text, tables, and discussion.

Thank you again for the careful review.

Best regards, The Authors

Reviewer_SpQxOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

FaceFlow increases intra-class diversity in synthetic face recognition data by replacing the random perturbation direction of IDPerturb with a direction learned via Riemannian flow matching on the unit tangent hypersphere at the reference identity embedding. The flow is trained on CASIA-WebFace same-identity pairs embedded by a frozen ElasticFace-Arc R100 encoder, and perturbed embeddings are decoded by a frozen UIFace generator. FR models trained on the resulting data reach 94.01% (0.5M) and 94.82% (1.0M) average accuracy, slightly above IDPerturb (93.62% / 94.48%).

Critical strengths
  • Simple, intuitive idea with a controlled comparison: a zero velocity field reduces exactly to IDPerturb, and both use identical embeddings, angles, and noise.

  • Clear gains in identity consistency and rejection rate at matched angles (Table 2).

Critical weaknesses
  • Dataset bias. The flow learns intra-class variation from CASIA-WebFace, embedded by an encoder trained on MS1MV2. Both are web-crawled celebrity datasets that are demographically skewed (about 84.5% and 76.3% Caucasian; Racial Faces in-the-Wild, Wang et al., ICCV 2019) and biased toward professional photography and make-up (FairFace, Kärkkäinen & Joo, WACV 2021; DigiFace-1M, Bae et al., WACV 2023). Synthetic FR data is known to inherit such bias (Bias and Diversity in Synthetic-based Face Recognition, Huber et al., WACV 2024). Since FaceFlow explicitly learns to reproduce real variation and applies it to every synthetic identity, it may propagate majority-group patterns to all identities. The paper provides no demographic analysis, no per-group evaluation, and no discussion of label noise in the retained low-similarity pairs.

  • Single encoder. ElasticFace is used for flow training, generation, and evaluation. Table 2’s identity-consistency metrics are therefore measured by the same encoder in whose space the flow was optimized. It is unknown whether the learned directions reflect facial variation in general or properties of this one encoder, and whether results hold with other encoders.

  • Lack of ablations. No component of the design is ablated, and no simpler alternative is compared: e.g., a Gaussian/PCA direction sampler, removing identity conditioning, linear interpolation with renormalization instead of Slerp, Euler instead of the exponential map, or the number of steps K. Figure 6 cannot separate the flow from a covariance-based sampler, since PCA energy is a second-order statistic. Given small downstream gains (+0.39 / +0.34) reported from single runs without variance, it is unclear that the proposed design is necessary.

Questions for authors
  • What is the demographic composition of the generated data, and how do trained FR models perform per group (e.g., on RFW)?

  • Are identity-consistency results preserved when evaluated with an independent encoder (e.g., ArcFace or AdaFace trained on different data)?

  • How does FaceFlow compare with a Gaussian/PCA direction sampler and with an unconditional flow?

2 comments

For Weakness 1 and Question 1

We used AI tools to assign each of the 10,000 FaceFlow identities to one of the four RFW demographic groups based on a single generated image, where possible. The resulting counts were 7,875 Caucasian (78.75%), 386 Asian (3.86%), 356 African (3.56%), and 261 Indian (2.61%); the remaining 1,122 identities (11.22%) could not be reliably assigned. We applied no demographic filtering or rebalancing. These are the UIFace-aligned virtual identities used throughout the paper. We have emailed the RFW authors to request access to the dataset and will report group-wise verification results once it becomes available.

For Weakness 2 and Question 2

We evaluated identity consistency on the 50-identity subset at $l_b = 0.5$ using an independent ArcFace R100 model trained on Glint360K. The relative ordering was preserved: the CintraC_{intra} rate was 0.979 for FaceFlow and 0.856 for the IDPerturb sampler.

For Weakness 3 and Question 3

We thank the reviewer for raising this concern. To provide a direct comparison, we evaluated FaceFlow and two alternative direction samplers under the same lb=0.5l_b = 0.5, 10k × 50 downstream face-recognition protocol. FaceFlow achieves scores of 99.15, 93.68, 94.49, 93.40, and 89.32 on LFW, AgeDB-30, CFP-FP, CALFW, and CPLFW, respectively, yielding a mean of 94.01. The isotropic Gaussian sampler obtains 91.08, 83.12, 77.67, 83.97, and 68.62, respectively, with a mean of 80.89. The rank-16 PCA direction sampler obtains 94.37, 76.80, 57.71, 84.72, and 62.20, respectively, with a mean of 75.16. These results show that FaceFlow achieves the highest mean score among the three methods across the five benchmarks.

We hope these clarifications address your concerns and will be taken into account in your final assessment.

Dear Reviewer,

Thank you again for the careful review and for the questions. We have posted our responses, together with the additional experimental results.

Could you please let us know whether these responses and results address your concerns? If any point remains open, we would be very grateful if you could tell us which further experiment you would still find necessary. We will prioritize that within the discussion period.

Thank you again for your time.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.