acceptodds
Under review as a conference paper at ICLR 2027

LiveAnimate: Stable Long-Form Streaming Human Animation in Real-Time

Abstract

Pose-driven human animation synthesizes a video of a target person from a single reference image and a driving pose stream. Real-time generation is essential for interactive applications such as live streaming, telepresence, and virtual avatars, yet diffusion-based systems require minutes to hours per clip, precluding responsive interaction. We present LiveAnimate, to our knowledge the first pose-driven human animation system to combine real-time streaming with stable long-form generation at billion scale, built on a 14B-parameter video Diffusion Transformer (DiT). A two-stage training pipeline first adapts a pretrained bidirectional DiT into a block-causal autoregressive generator through Reference-Anchored Teacher-Forcing Adaptation, and then reduces the sampling budget to three steps through Block-wise Self-Forcing Distillation. To preserve appearance over extended streams, we introduce Pose-Retrieval Sink Attention (PR-Sink), a bounded KV-cache mechanism combining a Static Sink that permanently anchors the first generated block, a Dynamic Sink that holds a pose-retrieved historical block, and a three-slot Rolling Window. When a pose recurs, PR-Sink restores the relevant appearance context without retaining the entire sequence, so memory and per-block latency remain constant regardless of stream duration. Together with Ulysses sequence parallelism and operator fusion, these designs enable a DiT throughput of 19.63 FPS on two NVIDIA H100 GPUs, and an end-to-end throughput of 11.2 FPS in a pipelined four-GPU deployment. On a three-minute benchmark, LiveAnimate maintains nearly constant perceptual quality and identity from the first 30 seconds to the final minute (IQA 4.047 vs 4.026), while prior systems degrade substantially or require hours of offline computation for the same rollout. These results establish a new operating point in quality, latency, and duration for interactive full-body animation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Animation, Streaming Video Generation, Diffusion Transformer

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_gjP1OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Built upon a 14B-parameter video Diffusion Transformer (DiT), LiveAnimate adopts a two-stage training pipeline. In the first stage, a pretrained bidirectional model is converted into a block-causal autoregressive generator via Reference-Anchored Teacher-Forcing Adaptation. In the second stage, Block-wise Self-Forcing Distillation compresses the sampling steps down to 3, drastically reducing inference latency. This work proposes Pose-Retrieval Sink Attention (PR-Sink), a fixed-length KV cache composed of a Static Sink, a pose-retrieved Dynamic Sink, and a three-slot Rolling Window, which maintains long-form appearance consistency with constant memory overhead. Combined with optimizations including sequence parallelism and operator fusion, it enables real-time streaming generation. On a three-minute benchmark, this method delivers stable visual quality and identity preservation, with a DiT throughput of 19.63 FPS and an end-to-end throughput of 11.2 FPS.

Critical strengths
  1. Video demos and evaluation metrics overall demonstrate that this work effectively mitigates cumulative error, and well preserves overall high-frequency details and human identity details over a long time horizon.
  2. The dynamic sink mechanism designed based on pose similarity is intriguing, and I believe the research community can draw on this idea to optimize pose-conditioned video generation.
  3. This paper is well organized and easy to understand.
Critical weaknesses
  1. The methodological innovation is relatively limited. The two-stage training paradigm of teacher forcing + self forcing is a prevailing framework in the field of streaming real-time video generation, and numerous works such as Self-Forcing, Self-Forcing++, LiveAvatar, and StreamAvatar adopt a similar approach. Furthermore, the reference-anchored method introduced in Section 3.2 is actually commonly used in streaming image-to-video tasks, so the innovation of simply integrating the reference as a sink is rather incremental.
  2. The video demos contain some imperfections. I observe that nearly all videos under the "Shorter rollouts" section exhibit prominent shifts in color and texture details between the first frame and subsequent frames. I cannot confirm whether this indicates actual changes in human identity, as the supplementary materials do not provide reference images of the subjects to assess the magnitude of identity variation.
  3. Several baselines are missing. Works including SteadyDancer (Zhang et al., 2025) should be included for comparison, since this work also introduces a first-frame preservation mechanism that bears similarities to the attention sink mechanism proposed in this paper.
Questions for authors
  1. Further elaborate on the actual novelties of this work. Revise the paper structure if necessary, with emphasis on Section 3.4 and condensed presentation of Sections 3.2 and 3.3.
  2. Explain the first-frame inconsistency issue in the video demos.
  3. Add more comparative baselines.
1 comment

Thanks for your response, most of my concerns are well addressed. I decided to raise my score.

Reviewer_MCinOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LiveAnimate adapts Wan2.2-Animate-14B into a three-step causal generator for pose-driven streaming human animation. It combines block-wise self-forcing distillation with PR-Sink, a bounded KV-cache mechanism that retains a static appearance anchor, pose-retrieved historical context, and recent frames.

Critical strengths

Combining persistent appearance anchoring with pose-based retrieval is well motivated for this task. The three-minute evaluations and component ablations provide evidence for the effectiveness of the cache design, particularly the static sink. The end-to-end deployment analysis further supports the practical value of the system.

Critical weaknesses

Without direct comparisons to alternative rollout training schemes, the specific advantages of BS-DMD remain insufficiently established. The evaluation would also benefit from further analysis of potential failure modes, including unintended repetition from sink reuse and visual artifacts during later stages of distillation.

Questions for authors
  1. The repeated use of sink blocks raises a concern about temporal flashbacks or repeated motion inconsistent with the driving poses. Have the authors observed this, especially when retrieved blocks change, and how is it handled?

  2. Do artifacts such as overexposure, exaggerated mouth movements, or unintended camera drift appear in the later stages of distillation? If so, what strategies are used to stabilize training?

  3. RAVEN and Self Gradient Forcing offer alternative rollout-based training schemes that allow later-chunk losses to train historical KV representations. How does BS-DMD compare with these approaches, and what advantages does its replay scheme offer?

Reviewer_MQacOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LiveAnimate generates streaming human animation from a single image and a pose sequence. It combines three-step diffusion with pose-based memory retrieval to preserve appearance over long videos. The paper reports stable three-minute generation at 11.2 FPS end-to-end on four H100 GPUs.

Critical strengths

Streaming efficiency with sustained quality. LiveAnimate maintains stable visual quality over three-minute sequences. This combination represents meaningful progress toward interactive human animation, where both generation speed and appearance consistency matter.

Critical weaknesses
  1. Unclear cross-identity evaluation. The paper does not specify whether the reference images and driving videos depict different individuals. This makes it difficult to assess whether the method preserves the target appearance while transferring another person’s motion. Explicit cross-identity results, including subjects with different body proportions, would strengthen the evaluation.
  2. In 06_upper_demo.mp4, the generated hands appear reasonably stable despite substantial instability in the driving hand keypoints. The paper does not adequately explain this behavior.
  3. The provided videos exhibit noticeably choppy motion or excessive blinking, as seen in 06_upper_demo.mp4. These artifacts raise concerns about whether the model adequately maintains temporal consistency.
Questions for authors

Please refer to the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.