- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
EditaLive adapts Wan-Animate for instruction-guided streaming character editing. It combines reconstruction-based training on CharEdit-50K, chunk-wise causal adaptation, and two-step self-rollout distillation with Align Forcing, Fixed RoPE, and first-frame-preserved sparse attention. Short- and long-video benchmarks, component ablations, and a 40-participant user study support improvements in editing quality, motion preservation, and long-term stability.
Critical strengths
-
Effective training formulation. Using real videos as reconstruction targets avoids introducing expression mismatches through synthesized target videos, while enabling appearance editing with explicit body and facial motion control (Sec. 3.1).
-
Well-supported streaming adaptation. The paper includes the relevant Qwen-Image-Edit + Wan-Animate baseline and meaningful long-video ablations: removing Align Forcing reduces ID-SIM from 0.492 to 0.252 (Table 2), and the user study favors EditaLive across all evaluated criteria (Fig. 12).
Critical weaknesses
- The end-to-end real-time claim is not fully established. Timing starts from model-ready conditions at 384 × 672, rather than the 480 × 832 quality-evaluation resolution (App. C.3), without clearly accounting for pose extraction, face localization, or chunk buffering. The standard decoder achieves 14.47 FPS, below the benchmark's 16 FPS input rate; the 16.4 FPS Flash-VAED variant needs a complete pipeline latency measurement.
- Stylization training and evaluation need stronger justification. Stylization uses repeated still images as video targets (App. C.1), but the paper does not show through a targeted ablation how this supervision supports motion-preserving style transfer. Pooled ArcFace/SMIRK scores across realistic and stylized edits, together with omitted consistency scores for two baselines, also make category-specific comparisons difficult to interpret; per-category results are needed.
Questions for authors
-
What are the throughput and capture-to-output latency at 480 × 832 when pose extraction, face localization, and input buffering are included? Can the Flash-VAED configuration sustain a live 16 FPS stream without an increasing backlog?
-
For stylization samples, what reference image, instruction, and motion signals accompany the repeated-image target? Can you provide a stylization-training ablation and per-category results, with baseline consistency scores reported on the categories they support?
-
Sec. 3.3 says the KV cache is constructed from the final prediction, but Algorithm 1 reuses the last denoising-pass KV features for later chunks and performs a clean-input cache forward only for the first chunk. Which caching rule is used, and is it identical during training and inference?
5 comments
Response to Reviewer ZckW (Part 2/3)
RW2 & RQ2: Stylization training inputs, targeted ablation, and per-category evaluation.
Stylization training configuration. In stylization training, we take the original reference frame from the source video and the forward style instruction as conditions, repeat a stylized still image along the temporal dimension to construct a static target video, and extract the accompanying static skeleton and facial motion signals from this target video (clarified in Appendix C.1). These stylization samples are trained jointly with dynamic real-video local-edit samples, which provide supervision for time-varying motion. Since appearance and motion are learned from decoupled conditions, the static targets supervise the style transformation, which combines with dynamic motion signals from the source video at inference.
Stylization-training ablation. Following your suggestion, we conduct a full-scale Stage-1 ablation by replacing all static stylization targets in CharEdit-50K with an equal number of dynamic stylized videos from Ditto-1M under identical settings, using a source frame as reference and extracting character motion from the stylized target video (Appendix D.4, Table 8). Alongside APD and AED, we report Motion Ratio (, closer to is better), the mean Farneback optical-flow magnitude of the edited video divided by that of the source video (values below indicate damped motion, while values above indicate extra motion or temporal jitter):
| Subset | Stylization Target (Stage 1) | ID-SIM | AED | APD | TA | EQ | BC | SR | MR () |
|---|---|---|---|---|---|---|---|---|---|
| Style (46) | Ditto-1M video | 0.268 | 0.586 | 0.152 | 1.920 | 2.391 | 2.348 | 0.283 | 1.338 |
| Style (46) | Static target (Ours) | 0.247 | 0.643 | 0.158 | 2.377 | 2.717 | 2.761 | 0.587 | 1.103 |
| Local (104) | Ditto-1M video | 0.694 | 0.397 | 0.098 | 2.151 | 2.612 | 2.663 | 0.654 | 1.123 |
| Local (104) | Static target (Ours) | 0.697 | 0.405 | 0.100 | 2.321 | 2.590 | 2.756 | 0.740 | 1.104 |
| All (150) | Ditto-1M video | 0.563 | 0.455 | 0.115 | 2.080 | 2.544 | 2.567 | 0.540 | 1.189 |
| All (150) | Static target (Ours) | 0.559 | 0.478 | 0.118 | 2.338 | 2.629 | 2.758 | 0.693 | 1.104 |
Our static-target Stage-1 model achieves better TA, EQ, BC, and SR on both Style and All, more than doubling Style SR ( vs. ). While the Ditto-1M variant obtains slightly better ID-SIM and AED/APD on Style, this comes with substantially lower TA and SR. Regarding motion, the static-target model shows no reduction in average estimated motion magnitude on style edits, attaining virtually the same Motion Ratio on Style () and real-video-supervised Local edits (), with a much smaller mean absolute deviation on Style than the Ditto-1M variant ( vs. ). By contrast, the Ditto-1M variant's ratio exceeds in all Style cases ( to ), indicating consistently higher estimated motion magnitude, potentially including temporal jitter.
Response to Reviewer ZckW (Part 3/3)
RW2 & RQ2: Stylization training inputs, targeted ablation, and per-category evaluation (continued).
Per-category results and baseline consistency scores. Below we report the per-category breakdown for our full model and all baselines on CharEdit-Bench-S (added in Appendix D.2, Table 5, along with CharEdit-Bench-L in Table 6). Bold and denote the best and second-best results within each category (with ties sharing the same rank). Consistent with Table 1, LiveEdit* and LucyEdit* do not support global style editing, so their character-consistency metrics on Style are shown in gray and excluded from ranking:
| Category | Method | ID-SIM | AED | APD | TA | EQ | BC | SR |
|---|---|---|---|---|---|---|---|---|
| Add (41) | LiveEdit | 0.362 | 0.544 | 2.634 | 1.439 | 0.220 | ||
| SANA-Stream. | 0.382 | 0.563 | 0.349 | 2.577 | 1.805 | 1.423 | 0.122 | |
| Stream.V2 | 0.062 | 0.852 | 0.368 | 2.337 | 2.017 | 0.692 | 0.000 | |
| LucyEdit | 0.377 | 0.057 | 2.602 | 1.927 | 2.415 | 0.707 | ||
| UniVideo | 0.748 | 0.483 | 0.308 | 2.699 | 1.927 | 2.000 | 0.537 | |
| Ditto | 0.380 | 0.661 | 0.343 | 2.163 | 1.837 | 1.382 | 0.098 | |
| Qwen.+Wan. | 0.597 | 0.467 | 0.100 | 2.114 | ||||
| Ours | 0.657 | 0.098 | 2.911 | 0.878 | ||||
| Remove (22) | LiveEdit | 0.480 | 0.414 | 0.152 | 1.833 | 1.727 | 0.000 | |
| SANA-Stream. | 0.598 | 0.432 | 0.240 | 1.242 | 1.576 | 1.803 | 0.136 | |
| Stream.V2 | 0.081 | 0.774 | 0.466 | 0.106 | 1.515 | 0.242 | 0.000 | |
| LucyEdit | 0.831 | 0.262 | 0.051 | 0.530 | 1.864 | 2.470 | 0.045 | |
| UniVideo | 0.409 | 0.356 | 2.652 | 1.909 | ||||
| Ditto | 0.646 | 0.922 | 0.606 | 1.576 | 1.303 | 0.000 | ||
| Qwen.+Wan. | 0.663 | 0.422 | 0.119 | 2.895 | 2.030 | 0.864 | ||
| Ours | 0.701 | 0.411 | 0.106 | 2.348 | 0.864 | |||
| Change (41) | LiveEdit | 0.398 | 0.462 | 2.163 | 2.041 | 1.496 | 0.195 | |
| SANA-Stream. | 0.557 | 0.458 | 0.260 | 1.959 | 1.724 | 1.626 | 0.195 | |
| Stream.V2 | 0.051 | 0.832 | 0.399 | 1.593 | 1.943 | 0.496 | 0.000 | |
| LucyEdit | 0.339 | 0.062 | 1.585 | 2.073 | 2.333 | 0.317 | ||
| UniVideo | 0.836 | 0.281 | 2.618 | 2.049 | 2.220 | 0.732 | ||
| Ditto | 0.436 | 0.619 | 0.245 | 1.276 | 1.699 | 1.317 | 0.049 | |
| Qwen.+Wan. | 0.629 | 0.431 | 0.100 | 2.098 | ||||
| Ours | 0.670 | 0.393 | 0.093 | 2.976 | 2.455 | 0.878 | ||
| Style (46) | LiveEdit* | 0.442 | 1.949 | 1.616 | 0.000 | |||
| SANA-Stream. | 0.112 | 0.935 | 0.586 | 2.326 | 1.652 | 1.572 | 0.174 | |
| Stream.V2 | 0.055 | 1.022 | 0.674 | 2.058 | 2.022 | 0.949 | 0.022 | |
| LucyEdit* | 0.884 | 1.268 | 1.543 | 0.000 | ||||
| UniVideo | 0.125 | 1.013 | 0.761 | 2.645 | 2.007 | 1.366 | 0.261 | |
| Ditto | 0.052 | 0.908 | 2.594 | 1.609 | 0.457 | |||
| Qwen.+Wan. | 0.193 | 2.065 | ||||||
| Ours | 0.275 | 0.676 | 0.184 | 2.645 | 2.221 | 2.228 | 0.783 |
EditaLive achieves the best or tied-best SR across all four categories and, among methods supporting global stylization, achieves the best or tied-best score on every metric in Style. On local edits, V2V baselines such as offline LucyEdit and UniVideo achieve higher source-referenced consistency, and streaming LiveEdit attains low APD, whereas EditaLive offers a favorable balance between consistency and edit success across all edit types at real-time speed. Compared with the offline Qwen.+Wan. cascade, which shares the same decoupled formulation, EditaLive achieves better ID-SIM, AED, APD, and BC across all four categories. A similar trend holds on CharEdit-Bench-L (Appendix D.2, Table 6), where EditaLive ranks first in ID-SIM, AED, EQ, and BC across all four categories (and in APD on three categories).
RQ3: Caching rule in Sec. 3.3 vs. Algorithm 1.
Thank you for pointing this out. Algorithm 1 describes the exact caching rule used in our implementation: only the first generated chunk undergoes an additional forward pass on its final denoised prediction to initialize the attention sink, whereas every subsequent chunk directly reuses the keys and values from its last denoising pass. This rule is identical across training and inference, and we have updated Sec. 3.3 to match Algorithm 1.
Response to Reviewer ZckW (Part 1/3)
We thank Reviewer
ZckWfor the detailed and insightful comments. We address each question below.In Table 1 and Appendix C.3, we measured model-side runtime at 384×672 following prior 14B-scale streaming video generation protocols. Following your suggestion, we now benchmark the complete end-to-end pipeline at 480×832 on H100 GPUs (with torch.compile and warm-up), covering input chunk buffering, ViTPose extraction, face localization, motion encoding, two-step DiT denoising, and VAE decoding. Below we report both unthrottled maximum throughput (Max FPS) and steady-state behavior under a continuous 16 FPS simulated camera stream, where the service temporally subsamples buffered frames whenever processing falls behind the 16 FPS arrival rate:
At 480×832, the 1- and 2-GPU setups peak below 16 FPS and stay responsive under a live 16 FPS stream by temporally subsampling incoming frames (31.8% to 31.9% and 13.8% to 14.6%, respectively). Pipelining 1 GPU for preprocessing/VAE with 2 GPUs for parallel DiT denoising (3 GPUs total) raises maximum throughput to 21.77 FPS (Wan-VAE) and 21.95 FPS (Flash-VAED). Under the continuous 16 FPS stream, both 3-GPU setups process all frames at the full 16 FPS rate with zero frame subsampling or dropping and no increasing backlog (steady-state latency slopes of −0.015 and −0.016 s/min), achieving median capture-to-output latencies of 1.75s and 1.44s. At 384×672 on a single GPU, even with chunk buffering and fixed-resolution frontend detectors included, Flash-VAED sustains an end-to-end output rate of 15.88 FPS (1.89s latency, 1.1% subsampled). These full-pipeline results have been added to Appendix D.3 (Table 7).