acceptodds
Withdrawn from ICLR 2027

From Short Clips to Real-Time Streams: Identity-Aware Supervision and Context-Efficient Generation for Video Face Swapping

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video face swapping transfers the reference identity to a target video while preserving its other attributes and motion. Despite recent progress in reference-guided flow-based methods, jointly achieving faithful identity transfer, high fidelity, and real-time generation over long video streams remains challenging. To address this challenge, we rethink the task-specific structure of reference-guided video face swapping. First, face swapping changes only identity while preserving most visual content, yet flow-matching supervision is weakly sensitive to these changes. We therefore develop Joint Rec-ID Loss, which balances explicit identity supervision and latent reconstruction according to the noise level. Second, the target video already provides non-edited attributes and temporal dynamics, enabling generation with fewer spatial tokens and shorter temporal contexts through Face-RoI Generation and Short-Context Rollout Training. By integrating these designs, we propose StreamingSwap, a reference-guided flow-matching framework for real-time streaming video face swapping. We also introduce HumanFaceVid, comprising 1,000 hours of cinematic training data and evaluation sets of 200 short and 200 long videos. Experiments demonstrate that our method achieves competitive identity transfer and visual quality at 20 FPS on a single NVIDIA RTX 4090 GPU, with stable generation over videos up to 10 minutes long.

Primary area:
generative models
Keywords:
video face swapping, video generation, diffusion model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ncmPOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes StreamingSwap, a real-time streaming video face swapping framework built on DreamID-V. It introduces three components: a Joint Rec-ID Loss that adds noise-dependent identity supervision, Face-RoI Generation that processes only face-region tokens, and Short-Context Rollout Training that adapts Self Forcing for chunk-wise streaming. The authors also build HumanFaceVid, a 1,000-hour dataset derived from OpenHumanVid. The method runs at about 20 FPS on a single RTX 4090 and handles videos up to 10 minutes.

Critical strengths
  1. The paper targets a practical problem, achieving real-time reference-guided face swapping (about 20 FPS) on a single consumer GPU.
  2. The appendix includes fairly extensive ablations on loss weighting, denoising steps, and window sizes.
Critical weaknesses
  1. The designs of this paper are not very impressive. Face-RoI Generation resembles existing mask-aware token selection methods (e.g., YOSE, RegionE, SpotEdit), and the rollout training is a direct adaptation of Self Forcing. These related works are not discussed.
  2. The results in experimental section are not very strong. Identity similarity (0.577) is far below InSwapper and DreamID-V (~0.73), and the attribute metrics seem to reward weaker swaps. All results are on the authors' own benchmark without a user study.
Questions for authors
  1. The identity loss is computed on the clean estimate at all noise levels. At high noise, the decoded frames are likely blurry and ArcFace embeddings may be unreliable. Have you examined whether the identity loss gives meaningful gradients in this regime? Also, why use a hard step function for λ_rec instead of a continuous schedule?
  2. Restricting tokens to the face region removes surrounding context such as hair, neck, and scene lighting. How do you handle boundary consistency (e.g., hairline, skin tone at the blending edge), and how sensitive are the results to the dilation size?
  3. How does Face-RoI Generation handle multiple faces, large head motion where the RoI changes size across frames, and occlusions crossing the RoI boundary?
  4. Training uses only five autoregressive windows, while inference runs for up to 10 minutes. Why is this enough to prevent error accumulation over hundreds of windows? Did you try longer rollouts during training?
  5. You replace the distribution-matching objective in Self Forcing with the Phase I reconstruction and identity losses. What motivates this choice, and how does it compare to using DMD?
  6. The latent MSE in Sec. 3.1 is computed between real videos and InSwapper outputs, which also contain synthesis artifacts. How do you separate the effect of identity change from InSwapper artifacts in this analysis?
2 comments

Thanks for your careful reading and constructive feedback. We have revised the manuscript to address your concerns, with changes and additions marked in blue. Below, we provide point-by-point responses, including references to the corresponding sections, figures, tables, and pages in the revised manuscript.

Response to W1: Relation to Prior Editing and Rollout Methods

We acknowledge that regional computation and training on generated histories build on prior work. We have added discussions of YOSE, RegionE, SpotEdit, and Self Forcing in Appendix A.3 (p. 16) and explicitly acknowledged their influence in §4.2–4.3 (p. 6–7).

Our contribution is the task-specific analysis and resulting adaptation: we establish that reference-guided face swapping can substantially reduce spatial and temporal context because the target video already supplies detailed appearance and motion cues. Face-RoI omits outside-region Transformer computation without an additional background-context mechanism, while short-context rollout retains within-window bidirectional attention and uses direct paired supervision without an additional DMD stage. Their practical benefits are supported by Table 4 and Fig. 8 (p. 10).

Response to W2: Identity Transfer and Human Perceptual Evaluation

We agree that attribute metrics alone can favor insufficient identity changes. Indeed, removing identity supervision improves attribute scores but reduces ID Sim. from 0.573 to 0.241, illustrating precisely this trade-off (Table 4, p. 10).

To evaluate whether our complete method achieves convincing identity transfer, we have added a user study comprising 145 participant–case evaluations. Our method receives the highest mean ratings among the three evaluated methods for identity resemblance, attribute preservation, and video quality: 4.05/4.21/4.15, compared with 3.97/2.63/2.84 for DreamID-V and 2.98/3.19/2.44 for InSwapper. Please see Table 3 and Fig. 7 (p. 9), with the protocol in Appendix D (p. 23).

The added real same-identity analysis also illustrates how appearance variation affects ArcFace similarity, explaining why higher embedding similarity alone does not establish better perceptual identity transfer (§5.1 and Fig. 5, p. 9).

Response to Q1: High-Noise Identity Supervision and the Weighting Schedule

The denoising-trajectory analysis shows that substantial identity information emerges within the early sampling steps: the decoded clean estimates need not remain uninformative simply because the current latent has high noise (Fig. 16, p. 24). Moreover, removing identity supervision from either noise interval weakens final identity transfer, supporting the usefulness of retaining it throughout training (Appendix F, Figs. 17–18, p. 25).

We use a step function as a minimal, interpretable parameterization for examining how reconstruction and identity supervision should be balanced across noise regimes. Comparisons with constant weighting and alternative configurations, together with the threshold sweep, support its effectiveness (Fig. 17, p. 25–26). Continuous schedules are compatible with our formulation; we do not claim that the step function is uniquely optimal.

Response to Q2: Boundary Consistency and RoI Dilation

Our computation region is an enlarged facial region, retaining nearby context rather than tightly cropping the face boundary. Selected tokens preserve their original positional encodings, and the predicted latents are scattered back and smoothly blended with target-video latents using a feathered mask (§4.2, Eq. 10, p. 6).

We have added a dilation-sensitivity experiment over 32–128 pixels, keeping the blending parameters fixed. Identity and attribute metrics vary modestly; increasing the radius from 64 to 128 pixels raises token coverage from 38.53% to 68.84% without consistent quality gains. This supports the default radius as a practical quality–computation trade-off. Please see Appendix H and Table 8 (p. 26–27).

Response to Q3: Multiple Faces, Head Motion, and Occlusion

The per-frame face mask specifies the selected person and tracks changes in the editing region, accommodating head motion and varying RoI size. Tokens retain their original spatiotemporal positions, while padding and attention masking accommodate different sequence lengths during batched training (§4.2, p. 6). In multi-person scenes, this mechanism selects the person to edit. For occlusions, the model uses target-video conditioning together with pretrained diffusion priors, while the enlarged RoI and feathered blending support integration with surrounding content. Qualitative examples illustrate multiple faces, challenging motion and occlusion handling in Fig. 4 (p. 8) and Appendix C, Figs. 11–13 (p. 18–21).

Response to Q4: Short Training Rollouts and Long-Duration Stability

Five-window training exposes the model to its generated histories. During inference, the context remains bounded, while the target video continuously supplies appearance and motion cues and the source image provides a persistent identity reference. This reduces dependence on generated history for maintaining long-term content.

We have added duration-resolved evaluation of five outputs lasting 3.5–19 minutes, reporting identity and attribute trajectories in 30-second intervals, together with changes relative to the first-minute baseline. The curves show no consistent progressive deterioration across the evaluated cases (Appendix I and Figs. 20–21, p. 27–28). These results support the chosen rollout length empirically; they do not guarantee stability for arbitrary durations. The current study does not include a controlled comparison with longer training rollouts.

Response to Q5: Direct Supervision Instead of DMD

Our Phase-I model already supports competitive four-step generation (Table 5, p. 24), and our paired data provide real-video reconstruction targets and source-identity supervision. We therefore adapt the model through generated-history rollouts using these existing objectives, without introducing an additional DMD stage.

This choice directly supervises the task requirements while addressing the training–inference history mismatch. Its benefits are demonstrated by the self-rollout ablation (Table 4 and Fig. 8, p. 10). We clarify this distinction from distribution-matching approaches in §4.3 (p. 7) and Appendix A.3 (p. 16).

Response to Q6: Identity Changes Versus Synthesis Artifacts

We selected qualitative cases without visible synthesis artifacts and used the same InSwapper pipeline across comparisons to control the synthesis procedure. The self-swap control provides a reference for pipeline-induced deviations, while the identity-preserving mild-blur example contextualizes the latent-error scale (§3.1 and Fig. 2(a), p. 4–5). The training ablation provides independent supporting evidence: removing explicit identity supervision substantially weakens identity transfer (Table 4, p. 10).

Reviewer_C1zgOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes STREAMINGSWAP, a reference-guided flow matching framework for real-time streaming video face swapping. The authors identify two task-specific properties of reference-guided face swapping: weak identity sensitivity of flow matching supervision and computational redundancy in full-frame long-context processing, and accordingly introduce three components: Joint Rec-ID Loss, Face-RoI Generation, and Short-Context Rollout Training. They also contribute a dataset HumanFaceVid and report 20 FPS inference on a single RTX 4090 GPU.

Critical strengths
  1. The paper targets a practically relevant scenario: real-time long-video face swapping, and achieves a notable speedup over the baseline DreamID-V on consumer-grade GPUs.
  2. The ablation study is relatively complete, verifying the individual contribution of each proposed component.
Critical weaknesses
  1. For the Joint Rec-ID Loss, adding explicit identity loss to complement reconstruction loss has been widely adopted in nearly all prior face swapping methods. The only modification here is a piecewise noise-dependent weighting schedule, which is a trivial hyperparameter tuning trick rather than a principled methodological innovation.
  2. For the Face-RoI Generation, restricting computation to the face region is an obvious engineering optimization that has been used in image/video editing for years. It does not require task-specific analysis to motivate.
  3. For the Short-Context Rollout Training, this is directly adapted from self-forcing with minimal modification for the face swapping setting. The paper does not propose any new training paradigm for streaming generation.
  4. No user study is provided. This is a perceptual claim that requires human evaluation to validate. A few qualitative example images are highly selective and cannot substitute for a systematic user study.
  5. The paper omits comparison with recent streaming-capable face swapping methods and only tests against a small set of baselines. Notably, it does not compare with GAN-based real-time methods on temporal stability and perceptual realism, which are critical for streaming scenarios.
  6. The paper claims stable generation over 10-minute videos but provides no quantitative metrics across video duration (e.g., identity drift curve, quality degradation over time). A single overall FVD score cannot reflect temporal stability.
Questions for authors

Please see Weaknesses.

3 comments

Thanks for your careful reading and constructive feedback. We have revised the manuscript to address your concerns, with changes and additions marked in blue. Below, we provide point-by-point responses, including references to the corresponding sections, figures, tables, and pages in the revised manuscript.

Response to W1: Motivation and Evidence for Noise-Dependent Rec-ID Supervision

Thank you for this comment. We agree that explicit identity loss itself is not new. Our contribution lies in identifying the supervision challenge in reference-guided face swapping and developing noise-dependent balancing based on the observed identity–fidelity trade-off.

  1. Why explicit identity supervision matters here.

    Its use is not uniform across diffusion-based face-swapping methods, as clarified in the expanded related-work discussion (Appendix A, p. 15). Face-recognition embeddings retain non-identity information, so identity optimization can also encourage source-specific appearance at the expense of target attributes. Unlike inpainting-based methods, which remove the original target face, reference-guided methods retain it as a condition, creating direct competition between source and target identities. Our analysis shows that latent reconstruction is relatively insensitive to identity changes (§3.1, p. 4–5); correspondingly, removing identity supervision reduces ID Sim. from 0.573 to 0.241 (Table 4, p. 10).

  2. Why balance the objectives by noise level.

    Stronger identity supervision alone can compromise target-attribute preservation. The denoising-trajectory analysis reveals different identity and attribute-preservation trends across noise levels, motivating noise level as a balancing variable (Fig. 16, p. 24). We therefore retain identity supervision throughout training while reducing reconstruction weight at low noise levels, giving identity refinement greater relative emphasis. We have clarified this rationale in §4.2 (p. 7).

    The schedule provides a simple and effective implementation of the insights from our task-specific analysis. Compared with constant weighting (L1), our schedule (L2) strengthens identity transfer with only small changes in target-attribute preservation and imaging quality. Conversely, weakening high-noise reconstruction supervision (L5/L6) compromises expression and lighting preservation despite higher identity similarity. These comparisons, together with the switching-threshold sweep, support the effectiveness of noise-dependent balancing (Appendix F, Figs. 17–18, p. 25–26). Our contribution encompasses both the analysis of identity-sensitive supervision in reference-guided flow models and the resulting supervision design, offering guidance for future work on balancing identity injection with target-attribute preservation.

Response to W2: Task-Specific Motivation and Validation of Face-RoI Generation

We agree that region-restricted computation is established in image/video editing. We have explicitly acknowledged this inspiration in §4.2 (p. 6) and expanded the discussion of related methods in Appendix A.3 (p. 16).

Our analysis does not seek to establish regional computation itself as novel, but to determine how much spatial context reference-guided face swapping requires. To our knowledge, prior diffusion-based video face-swapping methods have not adopted face-region token selection to eliminate outside-region Transformer computation without an additional background-context mechanism. Inpainting-based methods remove target-face information and use surrounding context to guide reconstruction, making such context removal nontrivial. Reference-guided swapping instead retains fine-grained target attributes within the face region, allowing us to investigate and validate this more restricted computation. Unlike EditCtrl and YOSE, which retain or approximate unedited-region context, our design directly processes an enlarged facial region. We clarify these distinctions in Appendix A.3 (p. 16).

The experiments validate this choice: Face-RoI increases throughput from 2.355 to 6.702 FPS while maintaining comparable identity and target-attribute scores (Table 4, p. 10). The qualitative ablation further shows that Face-RoI does not affect visual quality (Fig. 8, p. 10). Together, these results support substantial computational savings while retaining strong face-swapping quality.

Response to W3: Task-Specific Adaptation of Rollout Training

We acknowledge that training on generated histories follows Self Forcing, as explicitly stated in §4.3 (p. 7) and the expanded related-work discussion (Appendix A.3, p. 16). Our contribution is the task-specific analysis and adaptation, rather than a new general-purpose streaming-training paradigm.

Specifically, the target video continuously supplies appearance and motion cues, enabling short-history conditioning while retaining bidirectional attention within each window. Moreover, our Phase-I model already supports competitive four-step generation (Table 5, p. 24), and paired training data allow direct supervision with our reconstruction and identity losses. We therefore perform supervised rollout adaptation without an additional DMD stage. These distinctions are clarified in Appendix A.3 (p. 16).

The adaptation has measurable benefits: self-rollout improves target-attribute preservation and mitigates progressive visual degradation (Table 4 and Fig. 8, p. 10). The newly added full-duration analysis further shows no consistent progressive deterioration across evaluated sequences lasting up to 19 minutes (Appendix I, p. 27–28).

Response to W4: Added Human Perceptual Evaluation

We agree and have added a systematic user study covering 145 participant–case evaluations. Participants view the source image, target video, and anonymized outputs in randomized method order, independently rating identity resemblance, target-attribute preservation, and video quality on a five-point scale.

Our method obtains the highest mean ratings among the three evaluated methods: 4.05/4.21/4.15, compared with 3.97/2.63/2.84 for DreamID-V and 2.98/3.19/2.44 for InSwapper. These results provide human evidence supporting our perceptual-quality claims beyond selected qualitative examples. Please see the added Table 3 and Fig. 7 in §5.1 (p. 9), with the protocol and study interface in Appendix D and Fig. 15 (p. 23).

Response to W5: Long-Video Baselines and Streaming Quality Evaluation

Thank you for highlighting streaming-capable baselines. Our evaluation includes real-time GAN-based methods, InSwapper and BlendFace, and we have expanded the long-video comparison to include the recently released streaming method VividFace. Compared with VividFace, our method achieves lower temporal flicker (2.945 vs. 4.084) and FVD (136.23 vs. 245.24), alongside substantially higher throughput (21.174 vs. 0.631 FPS). Compared with InSwapper, our method achieves lower long-video temporal flicker (2.945 vs. 3.319). The newly added perceptual evaluations further address realism and streaming quality: Gemini assigns our method better average ranks for realism (1.16 vs. 2.52) and temporal stability (1.17 vs. 2.47), while human participants give higher video-quality ratings (4.15 vs. 2.44). These additions provide systematic evidence beyond frame-level comparisons; please see Tables 2–3 and Figs. 6–7 (p. 9), with evaluation protocols in Appendix D (p. 22–23).

Response to W6: Quantitative Evaluation of Long-Duration Stability

We agree that an overall FVD score cannot establish stability over time. Therefore, we added full-duration evaluations of five outputs from three target videos, lasting 3.5–19 minutes. Identity similarity and target-attribute metrics are sampled at 2 FPS and averaged over non-overlapping 30-second intervals. We report both absolute trajectories and changes relative to each case’s first-minute mean, without additional smoothing.

The curves show broadly stable identity similarity and bounded attribute errors, with no consistent progressive deterioration across the evaluated cases. These results provide duration-resolved evidence beyond aggregate FVD. Please see the added Appendix I (p. 27) and Figs. 20–21 (p. 28) for the protocol and trajectories.

Response to the Ethics Concern

We recognize the privacy and impersonation risks of face swapping, particularly in real-time settings. Our Ethics Statement discusses these risks, identifies the sources of demonstration materials, and emphasizes consent, respect for privacy and content ownership, and clear labeling of manipulated media (p. 11).

Reviewer_WxMXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

They propose StreamingSwap, a flow-matching framework for video face swapping. Its main components include Joint Rec-ID Loss, which dynamically balances identity supervision and latent reconstruction based on the noise level, and Face RoI Generation with Short-Context Rollout Training, which leverages the unedited regions of target videos while reducing the number of spatial tokens and the required temporal context. They also introduce a dedicated dataset for training, validation, and testing.

Critical strengths

The visual results are highly promising. The substantial improvement in throughput is particularly impressive, and the method outperforms the DreamID-V baseline across most evaluation metrics, with the exception of identity similarity (ID Sim).

Critical weaknesses

1.The two analyses and the corresponding methodological contributions primarily appear to improve throughput and efficiency. However, their benefits for the core objectives of face swapping, such as identity preservation, attribute preservation, and visual fidelity, are less clear.

2.The quantitative results show that the proposed method achieves substantially lower identity similarity than its DreamID-V baseline. This is an important weakness because identity preservation is one of the central goals of face swapping.

3.The reported improvements in attribute preservation and visual quality are not sufficiently motivated or explained by the proposed methodological components. The current analysis does not clearly establish why these components should lead to improvements in these metrics.

4.InSwapper appears to achieve stronger overall results than the proposed approach on several important metrics. This comparison should be discussed more carefully.

5.The method starts from a pretrained DreamID-V model, yet after fine-tuning, the identity similarity decreases noticeably. According to the initial motivation and analysis, the proposed components do not seem expected to significantly affect identity preservation. Therefore, it is unclear why ID similarity degrades after fine-tuning. The authors should provide an explanation or additional analysis for this behavior.

6.One of the listed contributions, the HumanFaceVid dataset, is not discussed in sufficient detail in the main text. Given that it is presented as a key contribution, more information about its construction, characteristics, scale, diversity, and differences from existing datasets would strengthen the paper.

Minor weakness

7.Please add citations for the compared methods in the quantitative results tables. Some of the methods appearing in the tables are not cited in the paper.

Questions for authors
  1. In Line 131, according to the equation [ v_t^* = x_1 - x_0, ] the target velocity appears to be independent of (t). Is this interpretation correct? If so, please clarify the role of (t) in the formulation and explain why the notation (v_t^*) is used.

  2. What are the results when other methods are trained or fine-tuned using HumanFaceVid? At minimum, comparing InSwapper and DreamID-V under the same HumanFaceVid training setting would help isolate the contribution of the proposed dataset from the contribution of the proposed method.

  3. The motivation for the proposed components appears to be primarily related to improving computational efficiency, without an obvious mechanism for affecting identity preservation or video quality. However, the results show that ID similarity decreases while FVD improves. How do the authors explain these changes, and which components of the proposed method are responsible for the degradation in ID similarity and the improvement in FVD?

3 comments

Thanks for your careful reading and constructive feedback. We have revised the manuscript to address your concerns, with changes and additions marked in blue. Below, we provide point-by-point responses, including references to the corresponding sections, figures, tables, and pages in the revised manuscript.

Response to W1, W3, and W5: Methodological Contributions to Face-Swapping Quality and Identity Transfer

Thank you for these related comments. We have expanded the explanations and evaluations in the revision to clarify how our components contribute to identity transfer, attribute preservation, and visual quality, alongside efficiency. The relevant revisions are marked in blue.

  1. Quality benefits beyond efficiency.

    Table 4 shows that removing identity supervision reduces ID Sim. from 0.573 to 0.241, demonstrating its direct contribution to identity transfer. Self-rollout training improves pose, expression, and lighting errors from 6.751/2.524/0.259 to 6.336/2.466/0.238, respectively, while improving gaze similarity from 0.875 to 0.881. Fig. 8 further illustrates reduced progressive visual degradation. At the complete-system level, compared with DreamID-V on long videos, our method reduces lighting error by 51.3%, FVD by 45.4%, and temporal flicker by 30.0% (Table 1). The newly added perceptual evaluations further support these quality improvements. Under Gemini evaluation, our method achieves the best average ranks for realism, temporal stability, and overall quality (1.16/1.17/1.22, respectively). In the user study, it receives the highest mean ratings among the three evaluated methods for identity resemblance, target-attribute preservation, and video quality (4.05/4.21/4.15, respectively). These results complement the objective metrics with model-based and human assessments. Please see §5.1, Tables 2–3, and Figs. 6–7 (p. 9) for the results and Appendix D (p. 22–23) for the evaluation protocols.

  2. Why these improvements arise.

    Identity loss directly supervises source identity, while reconstruction loss constrains deviations from the real-video target. We have clarified their potential conflict and the rationale for noise-dependent weighting in §4.2 (p. 7), with supporting schedule ablations in Appendix F (p. 24–26). Our data construction also provides a foundation for fidelity: frame-wise GAN-based swapping produces paired videos, whose training roles are reversed so that the original real videos provide reconstruction targets (§2.2, Appendix B). Unlike DreamID-V’s pipeline involving an additional video-synthesis stage, this construction retains direct frame-wise correspondence without regenerating temporal dynamics. The expanded dataset discussion in Appendix B (p. 16) explains how this supervision supports fine-grained reconstruction, following the rationale demonstrated in LivingSwap. Self-rollout addresses the mismatch between ground-truth training histories and generated inference histories (§4.3). The newly added long-duration analysis in Appendix I and Figs. 20–21 (p. 27–28) further examines streaming stability, showing no consistent progressive deterioration across the evaluated cases. Face-RoI primarily improves efficiency while maintaining comparable quality; short-context inference and four-step denoising provide additional acceleration (Tables 4–5).

  3. Identity similarity after fine-tuning.

    Table 1 does not isolate the effect of fine-tuning because the inference configurations differ. The no-CFG initialization comparison in Table 4 (p. 10) shows that Phase-I training increases ID Sim. from 0.445 to 0.573. We nevertheless acknowledge that our final ArcFace score is lower than DreamID-V’s reported configuration. To contextualize this difference, we have added the real same-identity analysis in §5.1, “Identity similarity against real-data,” and Fig. 5 (p. 9). It illustrates that pose, expression, and illumination affect embedding similarity, and genuine same-identity pairs can receive moderate scores. Thus, higher ID Sim. alone does not establish better identity transfer while preserving target attributes. Our distribution is closer to the real-pair reference, providing context rather than proof of superior identity transfer. The newly added user study in Table 3 and Fig. 7 (p. 9) directly complements this analysis: our method receives a slightly higher mean identity-resemblance rating than DreamID-V (4.05 vs. 3.97), alongside higher attribute-preservation (4.21 vs. 2.63) and video-quality (4.15 vs. 2.84) ratings. These results support effective identity transfer and improved target fidelity at over 20 FPS.

Response to W2 and W4: Comparison of Identity Similarity and Overall Face-Swapping Quality

Thank you for these comments. We acknowledge that our method obtains a lower ArcFace identity similarity than DreamID-V and InSwapper. We have expanded §5.1 with real-data analysis and perceptual evaluations to better interpret this difference and discuss the comparisons more carefully.

  1. Interpreting identity similarity.

    ArcFace similarity is informative, but it is also affected by non-identity attributes such as pose, expression, and illumination. Our newly added analysis of 67,588 real same-identity pairs illustrates that genuine pairs can only receive moderate similarity scores under appearance variation. Our output distribution is closer to this real-pair reference, with a mean of 0.5759, compared with 0.5907 for real pairs. This provides context for the lower score; it does not by itself establish superior identity transfer. Higher embedding similarity should therefore be interpreted jointly with target-attribute preservation and perceived identity resemblance. Please see the added “Identity similarity against real-data” analysis and Fig. 5 in §5.1 (p. 9).

    The newly added user study directly complements this analysis. Our method receives a mean identity-resemblance rating of 4.05, compared with 3.97 for DreamID-V and 2.98 for InSwapper; the corresponding first-place shares are 48.7%, 41.1%, and 10.1%. Thus, lower ArcFace similarity does not translate into lower average perceived identity resemblance in this study. Our method also receives the highest attribute-preservation and video-quality ratings. Separately, Gemini assigns our method the best average overall-quality rank (1.22, versus 2.35 for DreamID-V and 2.43 for InSwapper), providing complementary model-based evidence. These additions are reported in Tables 2–3 and Figs. 6–7 (p. 9), with evaluation protocols in Appendix D (p. 22–23).

  2. Comparison with InSwapper on other attributes.

    As a GAN-based method that processes frames independently, InSwapper achieves higher throughput and lower latency. However, frame-wise processing can introduce visual artifacts and temporal inconsistency, as illustrated in Fig. 4. Our method achieves better long-video target-attribute preservation and lower temporal flicker: pose, expression, and lighting errors decrease from 6.701/2.584/0.248 to 6.336/2.466/0.238, gaze similarity increases from 0.867 to 0.881, and temporal flicker decreases from 3.319 to 2.945. The long-video FVD scores are comparable (134.03 for InSwapper vs. 136.23 for ours); this small numerical difference alone does not establish a meaningful perceptual-quality advantage. Please see Table 1 and Fig. 4 (p. 8).

    The newly added perceptual evaluations provide clearer evidence of the quality difference. Our method receives substantially higher mean human ratings for video quality (4.15 vs. 2.44) and target-attribute preservation (4.21 vs. 3.19). Gemini likewise assigns our method better average ranks for realism (1.16 vs. 2.52) and temporal stability (1.17 vs. 2.47). These complementary evaluations favor our method’s perceptual quality despite the close FVD scores. Please see Tables 2–3 and Figs. 6–7 (p. 9), with protocols in Appendix D (p. 22–23). Thus, while InSwapper remains faster, our method combines throughput above 20 FPS with better target-attribute preservation and perceptual quality in the reported evaluations.

Response to W6 and W7: Dataset Documentation and Baseline Citations

Thank you for these suggestions. We have expanded the dataset documentation and added citations for the compared methods in the quantitative tables.

  1. HumanFaceVid documentation.

    The expanded analysis reports 15.5 million face-centered clips at 640 × 640 resolution, totaling 18,757.7 hours across 540,031 identity clusters. We present cross-video examples of the same identity to illustrate variations in pose, expression, and appearance, and analyze the distributions of clip durations and identity-cluster sizes. Notably, 83.7% of clips belong to clusters containing more than 100 clips, providing abundant cross-video samples for same-identity pairing. Please see the expanded “Dataset statistics” discussion in Appendix B (p. 16) and the added Fig. 10 (p. 17).

    Appendix B also details how we transform OpenHumanVid into face-swapping training pairs through face-track extraction and filtering, mask construction, cross-video identity pairing, GAN-based swapping, and facial-contour adjustment. Unlike the original general-purpose human-video collection, HumanFaceVid provides face-centered, task-specific paired samples, with synthetic swapped videos as inputs and original real videos as reconstruction targets. The construction pipeline is described in Appendix B (p. 16) and illustrated in Fig. 9 (p. 17). The dataset additionally includes 200 short-video and 200 long-video evaluation cases, as summarized in the main text (p. 2–3).

  2. Baseline citations.

    We have added the corresponding references alongside the compared method names in the quantitative comparison tables, making their attribution explicit. Please see the revised Tables 1–3 (p. 8–9).

Response to Q1: Time Dependence of the Target Velocity

Yes. For a fixed pair (x0,x1)(x_0,x_1), the target velocity vt⋆=dxt/dt=x1−x0v_t^\star=dx_t/dt=x_1-x_0 is constant along the linear interpolation path. This is consistent with the linear-path conditional flow-matching objective in [1] (Eq. 23, with σmin⁡=0\sigma_{\min}=0). The subscript tt denotes the velocity at time tt; it does not imply that its value must vary along each sampled path. The predictor vθ(xt,c,t)v_\theta(x_t,c,t) remains time-conditioned because tt specifies the noise level of xtx_t.

[1] Lipman Y, Chen R T Q, Ben-Hamu H, et al. Flow matching for generative modeling[J]. arXiv preprint arXiv:2210.02747, 2022.

Response to Q2: Separating Dataset and Method Contributions

Thank you for suggesting this informative comparison. Our method already fine-tunes the pretrained DreamID-V model on HumanFaceVid. However, we agree that retaining DreamID-V’s original training formulation on the same dataset would further separate the benefits of the data from those of our methodological changes.

Our existing ablations provide partial evidence by evaluating component contributions under the same HumanFaceVid training setting. Removing identity supervision reduces ID Sim. from 0.573 to 0.241, while removing Face-RoI reduces throughput from 6.702 to 2.355 FPS. These results show that access to the same training data alone does not recover the complete method’s identity-transfer performance or efficiency. Please see Table 4 (p. 10); the effects of training phase and denoising steps are further examined in Table 5 (p. 24).

LivingSwap provides complementary evidence for the applicability of this data-construction strategy to another backbone: it adapts VACE using frame-wise GAN-generated pairs and role-reversed supervision, achieving strong attribute preservation and visual realism. Our work follows and extends this strategy to HumanFaceVid, with its large-scale cinematic data; please see the expanded discussion in Appendix B (p. 16–17). Together, these findings support the usefulness of these data pair construction and supervision strategies across different model initializations.

For InSwapper, the publicly available release provides an ONNX inference model without the corresponding training implementation or a documented training recipe. We therefore cannot faithfully reproduce its fine-tuning on HumanFaceVid and evaluate the released model as provided.

Response to Q3: Identity Similarity and Video Quality

Please see our responses to W1/W3/W5 for component contributions and W2/W4 for the identity–quality comparison. Identity supervision strengthens identity transfer, while real-video reconstruction supervision and self-rollout support fidelity and stability, consistent with the improved FVD and added perceptual evaluations. The lower ID Sim. score does not imply worse perceived identity resemblance, as illustrated by the added real-data analysis and user study (§5.1, p. 9).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.