acceptodds
Withdrawn from ICLR 2027

FlowFaceSwap: Training-free Video Face Swapping via Structure-Preserved Identity Flow Editing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

High-fidelity video face swapping requires transferring a source identity seamlessly while strictly preserving the target's expressions, poses, and temporal consistency. Achieving satisfactory swapping performance typically necessitates resource-intensive, task-specific training, making these models expensive and inherently limiting their generalization to unseen scenarios. Recent training-free approaches have emerged as promising alternatives and achieved notable progress. However, they still struggle with temporal inconsistencies and identity leakage in face swapping. In this work, we reformulate the face swapping task as an identity-guided flow residual generation process, and propose a training-free framework FlowFaceSwap based upon a frozen rectified flow prior. Specifically, by optimizing compact identity tokens at test time, our method establishes a discriminative identity context. To prevent the overwriting of target dynamics, we introduce a structure-aware residual modulation mechanism driven by temporally tracked semantics and landmark-guided gates. This spatial control accurately localizes identity transfer while rigorously maintaining the target's subtle expressions and fine-grained motions. Extensive experiments on FaceForensics++ and CelebVHQ demonstrate that FlowFaceSwap generates temporally stable, highly realistic results. It significantly outperforms existing training-free baselines in both visual quality and identity retention, achieving an overall performance highly competitive with specialized, training-based models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Face Swapping, Flow Editing, Training-free

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_aYCXOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

FlowFaceSwap is a video face-swapping framework built on a frozen rectified-flow video generator. It optimizes source and target identity tokens at test time, then edits the target video using the difference between identity-conditioned velocity predictions. Semantic region weights and landmark-guided gates modulate the residual to preserve target expressions and motion. Experiments on FaceForensics++ and CelebVHQ compare the method with specialized face-swapping systems and methods without face-swap-specific training.

Critical strengths
  1. The paper is clearly presented. The identity-token optimization, velocity-difference editing, and structure-aware residual modulation are easy to follow. The main text and appendix provide reasonably detailed implementation descriptions and ablations.

  2. Face swapping without training a dedicated generator is a valuable direction. Leveraging a frozen video-generation prior reduces the need for task-specific generator training. However, the method still requires substantial test-time identity optimization, so this benefit should be distinguished from overall computational efficiency.

Critical weaknesses
  1. Visible artifacts in the supplementary videos undermine the central quality claims. Among the four examples, cases 02, 03, and 04 exhibit pronounced, unnatural twitching around the eyes, eyebrows, or mouth. These artifacts impair usability and weaken the claims of fine-grained motion preservation and temporal consistency (Abstract; Section 4.2). Source-identity resemblance also appears limited, and some results in Figure 3 do not convincingly retain distinctive source features. Presentation problems with the supplied target videos prevent reliable frame-by-frame assessment of expression fidelity, but the unnatural motion in the generated outputs is clearly visible.

  2. The structure-preservation mechanisms do not adequately address temporal instability. The method uses a video-generation backbone, temporal mask smoothing, and landmark-derived dynamics, but controls the residual primarily through regional weights and frame-dependent gates, without directly constraining motion continuity in the edited output (Sections 3.4–3.5). The observed twitching suggests that these mechanisms are insufficient to support the claimed fine-grained dynamic preservation. On CelebVHQ, Smoothness is also lower than FlowDirector’s (0.9719 versus 0.9916; Table 1). Direct evaluation of local motion is needed beyond aggregate smoothness and expression-feature distances.

  3. Regional suppression may sacrifice identity without achieving a convincing identity–expression balance. The eyes, eyebrows, and mouth encode both expression and identity; reducing edits in these regions does not itself disentangle the two (Sections 3.4–3.5). In Table 2, adding the structure gate reduces Top-1 retrieval from 0.94 to 0.92, suggesting a possible identity cost, although this small difference requires cautious interpretation. Meanwhile, visible twitching persists in the supplementary results. The current evidence therefore does not demonstrate a satisfactory balance between source-identity fidelity and natural target dynamics.

  4. The video comparisons do not sufficiently establish the claimed practical advantage. PhotoSwap and SwapAnything primarily target images, while FlowDirector is a general video-editing method rather than a native face-swapping system (Section 4.1; Table 1). Although video pipelines such as VFace are reported separately, synchronized qualitative comparisons would better establish the relative performance. FlowDirector’s publicly released examples appear less affected by the discontinuities seen here; these are not matched comparisons, but they motivate closer examination of why the proposed editing procedure introduces such artifacts despite using a video prior.

Questions for authors
  1. What causes the eye, eyebrow, and mouth twitching in supplementary cases 02–04, particularly given the use of a video-generation backbone? Could you provide correctly displayed, synchronized target videos for frame-by-frame comparison?

  2. Why do these artifacts persist after semantic modulation and landmark-guided protection? Could you provide matched video comparisons with and without the structure gate, together with mouth- and eye-motion trajectories relative to the target?

Reviewer_CZceOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FlowFaceSwap adapts inversion-free rectified-flow editing to video face swapping with a frozen Wan2.1-T2V-1.3B prior. It optimizes compact source and target identity tokens using masked flow reconstruction and an ArcFace identity loss. A difference of identity-conditioned velocities produces an editing residual. Semantic face-region weights and a landmark-derived gate that varies across frames and sampling steps attenuate this residual around the mouth and eyes. Experiments use 50 FF++ and 100 CelebVHQ source-target pairs with 61-frame clips, including component, sampling, guidance, and difficult-motion analyses.

Critical strengths
  1. Optimized source and target contexts define a clear identity intervention, while local controls address identity transfer overwriting target expression. The generator stays frozen, and the token-optimization and residual-editing implementation is supplied. On FF++, the method matches the adapted FlowDirector's Top-1/Top-5 retrieval (0.92/0.98), while improving FVD from 366.57 to 313.02 and expression error from 0.5154 to 0.4939. This supports the usefulness of the adaptation.

  2. The appendices document the identity-expression tradeoff through distinct optimization/evaluation recognizers, retrieval and failure-handling protocols, hard mouth/eye subsets, failure examples, and token/guidance/schedule ablations. On the two difficult-motion subsets, Table 5 achieves perfect retrieval with lower expression error than the other methods that also achieve perfect retrieval. These diagnostics help delimit the operating range and make the results more informative than successful still frames alone.

Critical weaknesses
  1. The central claim of precise target mouth/eye preservation is not yet established, and the media supplement contains a directly relevant failure. In example 03, the target mouth is closed or pursed at 0.00, 0.75, 3.00, and 3.75 s, while the swapped mouth remains visibly open. The open-mouth source reference makes this consistent with source-expression leakage, without establishing its population frequency or exact cause. Table 5 uses whole-face FECNet error on 13 pairs per hard subset, rather than output lip/blink trajectories. Table 2 improves expression error from 0.5098 without the gate to 0.4939 with it, while Top-1 decreases from 0.94 to 0.92. A static local gate or weaker edit matched for identity fidelity is needed to distinguish frame-adaptive preservation from simple attenuation. This directly affects the main contribution.

  2. The closest flow-editing baseline is not specified precisely enough to establish the claimed advantage over the prior method. Section 4.1 describes adapting FlowDirector's direct residual formulation, and Section 4.3 calls the result an unmodulated residual. The cited FlowDirector also includes direction-aware correction, motion-appearance decoupling, and differential averaging, beyond the direct ODE. It is unclear whether these controls are retained. If this row uses only a raw residual, it is a useful internal control but does not establish superiority over the complete prior method. The distinction matters because identity conditioning and local residual control are the paper's main advances beyond existing inversion-free editing. [Reference: FlowDirector, Sections 3.3-3.5, https://arxiv.org/html/2506.05046v2.]

Questions for authors
  1. Which exact FlowDirector implementation and controls produced Tables 1 and 5? Were its direction-aware correction, attention mask, motion-appearance correction, and differential averaging retained? Please identify an available full-FlowDirector comparison with the same optimized identity contexts, backbone, clip windows, and comparable compute, or label a raw-ODE implementation as an internal ablation.

  2. How does the dynamic gate compare with a static mouth/eye gate of matched average strength and with residual/guidance settings matched for identity fidelity? Can output-versus-target lip-aperture or blink trajectories, or another independent local motion measure, clarify the example 03 failure and show that the gain persists on the difficult-motion subsets? Such evidence would materially affect the assessment.

  3. Can you add a human perceptual study (identity similarity + temporal realism) against FlowDirector and CanonSwap, and report how detectable your outputs are to a state-of-the-art deepfake detector?

Reviewer_U4bzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes FlowFaceSwap, a training-free video face-swapping framework built on a frozen rectified-flow text-to-video model (Wan2.1-T2V-1.3B). It has three components:

Test-time identity token optimization. A learnable token embedding is optimized per identity, for both source and target, with the generator frozen. The objective is a face-masked flow-matching loss plus an ArcFace identity loss computed on the one-step clean estimate. Identity-guided flow residual editing. Following FlowEdit, the method edits the target video inversion-free by accumulating the difference between source-token-conditioned and target-token-conditioned velocities along the target’s noisy trajectory. Structure-aware residual modulation. A face-parsing-based semantic strength map weights the residual by facial region, and a landmark-guided, timestep-dependent gate suppresses it in the mouth and eye regions according to target motion.

Experiments on FF++ (50 pairs) and CelebV-HQ (100 pairs) show the method outperforms the training-free baselines (PhotoSwap, SwapAnything, an adapted FlowDirector) on FVD and identity retrieval, and is competitive with several training-based face swappers. Ablations cover the identity loss weight, number of tokens, sampling shift, guidance scales, region weights and gate floors.

Critical strengths
  • Proposed training-free video face swapping that avoid task-specific training of a dedicated swapper. It can be benefited from continuous foundation models' improvements. The results show a frozen video prior can produce temporally stable swaps competitive with several specialized trained systems.

  • Identity tokens are a sensible way to condition a video generator on identity. Optimizing compact identity tokens and using them as the two conditions of the velocity-difference edit is a clean, interpretable design. It turns a text-to-video model into a reference-specific identity editor without changing its weights, and the tokens can be cached and reused across videos.

  • Thorough component-level analysis. The appendix includes ablations on most hyperparameters, a hard-motion subset evaluation, trajectory visualizations and honest failure cases, which help in understanding the method’s behavior. Practical inference cost. Per-frame editing time (13.3 s/frame) and memory (21.8 GB) are lower than the inversion-based training-free baselines, and everything runs on a single 24 GB GPU.

Critical weaknesses
  • Limited novelty: largely an adaptation of existing flow-editing methods to the face domain. Inversion-free editing with source and target conditioned velocity differences is established for images (FlowEdit, FlowAlign, FIA-Edit) and has already been extended to video: FlowDirector, Wan-Edit and Pyramid-Edit (FiVE), FlowAnchor, and DynaEdit. FlowAnchor in particular already targets the same two problems as the proposed modulation, namely spatial localization of the editing signal and control of its magnitude in video latents. The other components are also established: per-identity token optimization follows Textual Inversion, and combining a diffusion/flow loss with an ArcFace identity loss is common in identity personalization (e.g., PuLID). The contribution is mainly the combination plus face-specific masks (face parsing and landmark gating). That is a useful engineering contribution but limited methodological novelty.

  • No experiments on other video generation models. The central claim is a training-free framework built on “a frozen rectified flow prior,” but all experiments use a single backbone (Wan2.1-T2V-1.3B). Several key hyperparameters are tuned on this backbone: guidance scales (10.5/3.5), sampling shift (5.0, specific to Wan’s sampler), λ_ID, region weights and gate floors. Many results are sensitive to them (Tables 6, 8, 9). Without results on other backbones, it is unclear whether the method generalizes or is a carefully tuned pipeline for one model. I understand that rerunning the full benchmark is expensive, but even small-scale or qualitative results would substantially strengthen the claim. e.g. LTX-Video (2B), and a larger one such as Wan2.2-TI2V-5B or Wan2.1-14B.

  • No ablation on the strength of the editing direction. The method is essentially a CFG-like identity direction ("source direction"- "target direction" velocity) integrated onto the target latent, so its magnitude is a controllable parameter. The paper never directly ablates it. The guidance-scale ablation (Table 9) is related but confounded: each scale applies CFG within its own branch against the null context, and the asymmetric setting (10.5 vs. 3.5) adds an extra term pushing away from the null condition, so it doesn’t isolate the strength of the identity direction. The identity-region weight in Table 4 acts as a strength multiplier but is tested at only two values (1.0 vs. 0.85), and lowering it to 0.85 drops Top-1 from 0.92 to 0.76. This suggests the method operates near a sensitive threshold.

  • The necessity of the semantic strength map is not demonstrated. In Table 2, removing the semantic map gives higher Top-1 retrieval (0.94 vs. 0.92) and lower pose error (4.49 vs. 4.53), equal Top-5, and nearly equal expression error (0.4952 vs. 0.4939, a 0.3% difference). Only FVD clearly favors the full model (313 vs. 329). The text nevertheless states that removing the map degrades video quality and increases expression error, without discussing the identity and pose results. The paper should justify the map’s necessity more convincingly.

  • “Training-free” framing and per-identity cost. Each identity, including every target identity, needs about 33 minutes of gradient-based token optimization. A new source–target pair therefore costs about an hour of optimization before about 13.5 minutes of editing per 61-frame clip. The model weights are indeed frozen, but this is test-time training, and it should be compared more transparently against encoder-based identity methods that need no per-identity optimization.

Questions for authors
  • Have you tried FlowFaceSwap on any other rectified-flow video model?

  • How well does the method handle (source, target )pairs with substantially different face shapes(or size)?

  • How many video frames used to generate an identity embeddings?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.