- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces PixReenact, a streaming portrait-animation system that transfers facial motion from a driving video to a reference identity. It directly conditions a causal video diffusion model on reference and driving pixels, avoiding keypoints or specialized motion features. To prevent the model from copying the driver’s identity, it combines Wan-Animate-generated cross-identity pseudo-labels with identity, motion, and reconstruction losses. For low-latency generation, PixReenact uses four-stage rolling denoising and separate teachers for stream initialization and steady-state generation. The resulting student continuously outputs four frames every 239 ms while maintaining identity over long sequences.
Critical strengths
-
The method directly uses visual latents from the reference image and driving video, without relying on keypoints or specialized motion representations. This potentially preserves richer information about facial expressions, head poses, and occlusions, especially when external motion extractors are unreliable.
-
The separation between cold-start and steady-state teachers is well motivated. The first emission must handle a potentially large transition from the static reference to the initial driving pose, whereas later emissions mainly follow incremental motion using generated history. Modeling these states separately is a reasonable design choice.
Critical weaknesses
-
In the supplementary material, Long Video 2 exhibits severe facial deformation, with the subject’s face becoming substantially distorted. This failure appears inconsistent with the paper’s claims regarding long-horizon identity and appearance stability. The current long-video evaluation mainly emphasizes identity similarity and may not adequately capture structural facial degradation.
-
Several central components—including four-level rolling denoising, causal attention, KV caching, DMD, and autoregressive self-rollout—are inherited from prior causal video diffusion work. The main contribution is therefore the adaptation and combination of these techniques for pixel-conditioned avatar reenactment, rather than a fundamentally new streaming-generation mechanism.
Questions for authors
-
Could the authors provide long-horizon measurements of facial structure, motion fidelity, and temporal quality in addition to identity similarity? It would also be useful to report how often similar failures occur across the 28 long-video examples.
-
The steady-state teacher is trained with clean target context at , while it is queried during student self-rollouts under potentially degraded generated histories. Although long self-rollouts address the student-side exposure bias, it remains unclear whether the teacher score is reliable under these off-distribution contexts. Recent context-matched DMD methods address this issue through Diffusion-Forcing teacher training, exact prefix scoring, and controlled prefix corruption. Does PixReenact feed the exact cached student prefix to both the real and fake score networks? Have the authors considered context corruption or student-generated-context adaptation for the steady-state teacher?
2 comments
Dear Reviewer vEeP,
Thank you again for your time and constructive feedback. We hope that our additional experiments, clarifications, and revised manuscript address your main concerns. If any questions or concerns remain, we would be happy to discuss them further during the remaining discussion period.
We thank the reviewer for recognizing the advantages of direct visual conditioning and the motivation for state-aware cold-start and steady-state teachers. We address the concerns below and provide new quantitative results and clarifications in the revised manuscript. We uploaded a revised manuscript with changes in blue (references below refer to this version).
W1 / Q1 — Long-horizon stability and facial deformation. Thank you for this remark. We evaluated all 28 five-minute sequences against PersonaLive, reporting per-minute ID/IDD, AED/APD, tLP, and FVD (Fig. 10). PixReenact maintains relatively stable identity and motion scores across five minutes and achieves consistently better FVD, while PersonaLive performs slightly better on APD and tLP.
We also quantified facial-structure consistency using SMIRK-estimated FLAME identity-shape coefficients for our method and PersonaLive. We measured deviations relative to each clip's first two seconds. Both methods exhibit stable and similar structural measures (full details in Sec. C, page 24). This is indeed a very challenging case. Our results support long-term structural consistency but some artifacts still remain, which we hope will motivate future research.
W2 — Contribution beyond existing causal diffusion. As stated in Sec. 1, our contribution lies in a novel training framework that enables effective pixel-conditioned, cross-identity streaming head-avatar reenactment. This setting remains challenging for existing causal and offline video diffusion techniques, particularly under extreme conditions and low-latency streaming requirements. Specifically, PixReenact introduces: (i) a new training strategy that makes generated data effective in a task where ground-truth cross-identity supervision is unavailable. To this end, our training strategy combines self-driving and pseudo cross-identity supervision, together with data filtering and corrective objectives, to mitigate driver-identity leakage and artifacts inherited from pseudo-generated data; (ii) long student self-rollouts that expose the model to its own generated history to mitigate training-inference mismatch and error accumulation; and (iii) state-aware dual-teacher distillation, including a dedicated cold-start teacher to handle challenging stream initialization. Together, these contributions enable robust streaming reenactment under extreme poses, expressions, and occlusions, without relying on specialized motion extractors at inference time.
Q2 — Teacher context during self-rollouts. Thank you for this remark. We have corrected the statement in Sec. 3.4 that implied the steady-state teacher always receives clean context. The teacher in fact is exposed to corrupted contexts during training (please see details in Appendix B.4). During DMD, both the frozen real-score teacher and the online fake-score network receive the exact same cached student-generated prefix from the student's self-rollout. We do not further adapt the teacher using student-generated histories, preferring to maintain a student-independent scoring target.