- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies keyframe-guided video editing. The user edits one frame with any image editor, and the goal is to carry that edit through the whole clip while keeping the motion of the source video. EditCast fully fine-tunes Wan2.1-VACE-1.3B to predict the edited clip in one generator evaluation. The source clip and the edited keyframe, placed at any temporal slot, go in through VACE's control pathway. The authors train only with pixel and perceptual losses on random temporal windows, and they warp the start noise along the source optical flow. The training pairs come from OpenVE-3M, Ditto-1M and a set of generated H3 pairs. The main claim is that the source and the keyframe almost fix the target, so a one-step model is both faster and more accurate than a matched multi-step editor. On KeyEdit-50 (1280×720), EditCast follows the keyframe more than 2 dB better at the last frame than the strongest propagation baseline. It runs at 6.3 FPS on one H200, about 33× faster than the 50-step propagation baselines, and users prefer it in a blind study. The supplementary videos and HTML page make it easy to check the temporal quality.
Critical strengths
-
EditCast makes keyframe-guided video editing much faster. It runs at 6.3 FPS on one H200, about 33× faster than the 50-step propagation baselines (Table 1), and a full edit-and-preview round with FLUX.2 takes about 2.5 seconds of GPU time at 480 × 832 (Appendix F). It also scores best on last-frame following, reference fidelity and the VLM judge among the compared methods (Tables 1 and 6), and users picked it first in 48% of trials (Section 4.2).
-
The experiments show that the keyframe, and not the text, drives the edit. With the source, prompt and seed fixed, 88% of outputs are closest to their own keyframe, against 33% by chance (Table 3). Dropping the text barely changes the result (17.71 vs. 17.86 dB), while dropping the keyframe lowers it to 13.52 dB (Section 4.4). EditCast also keeps the source's dynamic degree, which rules out simply copying the keyframe (Table 5).
-
The method generalizes beyond its training setup. It handles keyframes at any slot and two keyframes at once, which it never saw in training (Table 8). It keeps 93% of the edit on 81-frame clips, 2.5 times the training length (Section 4.5), and keyframes from two image editors that differ by 11.7 dB give the same edit ratio (Appendix F).
-
The reporting is transparent and supports reproducibility. The paper shows a failure case under viewpoint change (Figure 9), includes supplementary videos with an HTML viewer, lists the full data pipeline and filtering rules (Appendix A), and plans to release the code, data lists, scoring masks and model weights.
Critical weaknesses
-
The evidence for better editing quality leans on metrics that are close to the training loss. EditCast trains with pixel and perceptual losses, and Table 1 measures keyframe following with in-mask PSNR against the edited keyframe. At later frames the edited object moves, so a correct output can still differ from the keyframe pixel by pixel. Reference PSNR and LPIPS cover only the 25 KeyEdit-Ref clips (Section 4.1). These scores measure how well the edit is kept. They say little about whether the video stays natural under motion, occlusion or viewpoint change. The VBench scores, the VLM judge, the user study and the keyframe-swap test help here, so this point limits the strength of the quality claim. The paper also lacks a simple baseline that warps the edited keyframe along the source optical flow. Adding this baseline, together with a motion-aware following metric or a video-level reference score, would show how much EditCast gains over simple propagation.
-
The one-step vs. multi-step comparison changes more than the number of steps. Both models in Table 2 start from the same VACE-1.3B checkpoint and share the conditioning, learning rate and batch size. The one-step model trains with pixel and perceptual losses, while the 30-step model trains with flow matching and runs without classifier-free guidance (Section 4.3). So the gain may come from the loss, the step count, or both. The result also depends on training length. At 2,000 steps the multi-step model has the higher anchor PSNR (30.21 vs. 29.93 dB), and at 8,000 steps the one-step model leads (Table 2). A multi-step model with additional reconstruction losses would help separate the effects of the training objective and the number of inference steps. The same issue affects Section 3.5. The paper credits flow-warped noise for lowering the warping error from 1.89 to 1.81, while Table 4 compares two recipes that differ in several settings.
Questions for authors
-
The comparison in Table 2 changes both the training objective and the number of inference steps. Could the authors provide a more controlled comparison, such as adding pixel and perceptual losses to the multi-step model, to clarify which factor explains the improvement? It would also be helpful to evaluate the multi-step model with classifier-free guidance.
-
Could the authors provide a motion-aware keyframe-following metric and an additional video-level quality measure? A simple baseline that propagates the edited keyframe along the source optical flow would also show how much of the improvement comes from the learned model itself.
-
Table 9 and Appendix E report last-frame following per edit category and source motion. Could the authors add bootstrap intervals over clips for the main tables, and report the reference-based metrics per category? This would show how consistent the gains are on clips with occlusion, viewpoint change or large motion.
-
Could the authors provide an ablation comparing i.i.d. noise and flow-warped noise while keeping the remaining training settings fixed? Section 3.5 reports a reduction in warping error from 1.89 to 1.81, but Table 4 changes several training settings, making it difficult to isolate the effect of noise initialization.
-
How were the KeyEdit-Ref reference videos generated? Could the authors clarify whether using reference targets produced by similar generation pipelines to the training data might influence the reference-based fidelity metrics?
Two small points need clarification:
- Section 4.2 states that 14 raters evaluated 20 clips each, which gives 280 trials, while the paper reports 282.
- The abstract says the multi-step editor trails "at equal training steps" on every keyframe-following and fidelity measure, but at 2,000 steps it has the higher anchor PSNR (30.21 vs. 29.93 dB, Table 2).
7 comments
Official Comment by Authors (2/5)
W2. What the loss and noise ablations explain
Following the reviewer's suggestion, we separated these factors: the one-step advantage does not come from the loss alone, the multi-step editor's anchor lead at 2k is an early-training effect, and we will correct the attribution for the initial noise.
- Loss versus step count. Adding the same pixel and LPIPS losses to the 30-step editor improves temporal consistency, but its average last-frame PSNR gain comes mainly from two clips, and most clips barely change. With the same interface, data, and training budget, the one-step recipe still leads this variant by 1.15 dB in last-frame PSNR and 1.75 dB in in-region reference PSNR (see Q1).
- Results at 2k updates. At 2k, the multi-step editor has slightly higher anchor PSNR (30.21 vs. 29.93 dB), but the one-step model already has higher last-frame PSNR; by 8k, the one-step model leads in anchor PSNR, last-frame PSNR, edit ratio, and EWARP (Table 2). At 2k, training is still in progress and the metrics can fluctuate before convergence, so we base our conclusion on the matched 8k comparison. We will state the 8k setting in the abstract and report the 2k results in Section 4.3.
- Initial noise. Our wording in Section 3.5 could imply that flow-warped noise alone reduced EWARP from 1.89 to 1.81. That decrease comes from comparing the two training recipes in Table 4. After both models converged, EWARP was 1.856 with i.i.d. noise and 1.802 with flow-warped noise (see Q4). We will correct the wording in Section 3.5.
Q1. Reconstruction losses and classifier-free guidance
Thank you for your suggestions. We ran both. The multi-step model with added reconstruction losses still trails the one-step recipe (by 1.15 dB in last-frame PSNR). Text CFG lowers all three reported PSNR scores, while EWARP decreases slightly from 2.23 to 2.21. The one-step model is also 6.8 times faster than the 30-step editor (same timing protocol as Table 1).
- Multi-step model with reconstruction losses. The two 30-step models share the data, interface, 8k training updates, and inference setup; the second adds pixel and LPIPS losses on the predicted clean output. With these losses, its EWARP approaches the one-step result, but the one-step recipe still leads by 1.15 dB in last-frame PSNR and 1.75 dB in in-region reference PSNR (table below), so the loss alone does not explain the one-step advantage.
- CFG. We keep the multi-step checkpoint fixed and add text CFG only at inference. At CFG scales of 2.5 and 4.0, KeyEdit-Ref in-region PSNR drops from 22.29 to 21.97 and 21.63 dB, respectively.
Multi-step editor with reconstruction losses.
| Model | Last-frame PSNR, KeyEdit-50 (dB) ↑ | KeyEdit-Ref in-region PSNR (dB) ↑ | LPIPS ↓ | EWARP ↓ |
|---|---|---|---|---|
| 30-step, flow matching | 15.88 | 21.82 | 0.163 | 2.28 |
| 30-step, flow matching + pixel + LPIPS | 16.83 | 22.28 | 0.144 | 1.90 |
| One-step, matched training | 17.98 | 24.03 | 0.116 | 1.84 |
Text CFG at inference.
| Text CFG | Anchor PSNR (KeyEdit-50) ↑ | Last-frame PSNR (KeyEdit-50) ↑ | Ref in-region PSNR ↑ | Ref LPIPS ↓ | Ref EWARP ↓ |
|---|---|---|---|---|---|
| Off | 28.46 | 16.04 | 22.29 | 0.155 | 2.23 |
| 2.5 | 27.93 | 15.97 | 21.97 | 0.176 | 2.21 |
| 4.0 | 27.39 | 15.78 | 21.63 | 0.182 | 2.21 |
We will add the loss ablation to Table 2 and report the CFG comparison in the appendix.
Official Comment by Authors (3/5)
Q2. Motion-aware following and flow propagation
Thank you for your suggestions. We added all three. EditCast still leads on the motion-aware and video-level metrics, and simple optical-flow propagation does not reach its results (6.46 dB lower in-region reference PSNR over frames 1–32).
- Optical-flow propagation baseline. We copy the edited keyframe into frame 0 and warp it to later frames using source optical flow. We use the warped values on co-visible pixels and keep the source elsewhere. Over frames 1–32, EditCast’s in-region reference PSNR is 6.46 dB higher than this baseline’s (24.08 vs. 17.62 dB). EditCast also achieves 3.13 dB higher last-frame PSNR and lower EWARP (1.80 vs. 3.35).
- Motion-aware following metric (MC-PSNR). We align the keyframe to each output frame with the source optical flow and compare only pixels visible in both. On KeyEdit-Ref, EditCast scores 24.2 dB on average and 22.9 dB at the last frame, both higher than PropFly (22.0 and 20.7 dB). On KeyEdit-Img, mean MC-PSNR over frames 1–32 is 24.9 dB for EditCast and 21.0 dB for PropFly. We leave MC-PSNR unreported for direct flow propagation because it uses the same source-flow warp as the metric’s target, giving the baseline an advantage by construction.
- Video-level reference metric. We average DINOv2 features over the whole video and compare them with the edited reference video. EditCast scores 0.990, higher than the 30-step editor (0.977), PropFly (0.968), and flow propagation (0.950).
| KeyEdit-Ref method | In-region reference PSNR (dB) ↑ | LPIPS ↓ | EWARP ↓ | Last-frame PSNR (dB) ↑ | MC-PSNR mean / last (dB) ↑ | Clip DINOv2 ↑ |
|---|---|---|---|---|---|---|
| EditCast | 24.08 | 0.115 | 1.80 | 17.62 | 24.2 / 22.9 | 0.990 |
| 30-step editor | 22.07 | 0.158 | 2.23 | 15.83 | 23.0 / 21.8 | 0.977 |
| PropFly | 20.48 | 0.206 | 2.38 | 15.47 | 22.0 / 20.7 | 0.968 |
| Direct flow propagation | 17.62 | 0.285 | 3.35 | 14.49 | — | 0.950 |
| NOVA | 17.53 | 0.329 | 2.40 | 14.18 | 17.5 / 16.5 | 0.931 |
| AnyV2V | 18.13 | 0.358 | 4.88 | 15.33 | 19.1 / 18.2 | 0.918 |
| VDOT | 11.48 | 0.560 | 2.66 | — | — | 0.852 |
| Copy source | 10.78 | 0.596 | 2.01 | 10.16 | — | 0.824 |
In-region reference PSNR and LPIPS are averaged over frames 1–32 for all methods on the 25 KeyEdit-Ref clips. Direct flow propagation copies the edited keyframe exactly into frame 0.
We will add this baseline and both metrics to the revision.
Official Comment by Authors (1/5)
We thank Reviewer iDek for the thorough review and concrete suggestions, and for the positive assessment of EditCast's speed, generalization, and transparent reporting. We answer each point below.
W1. Evaluation metrics and naturalness
Thank you for this insightful comment. We added a motion-aware following metric, a video-level reference metric, and a direct optical-flow propagation baseline. EditCast leads on both new metrics among the methods scored and outperforms direct flow propagation on reference PSNR, EWARP, and video-level DINOv2 similarity. VBench motion smoothness, VLM visual quality, and the user study also favor EditCast over PropFly.
Evidence beyond the training loss. We leave MC-PSNR unreported for direct flow propagation because the baseline uses the same warped keyframe that defines the metric’s target. This gives it an advantage by construction. The table compares the baseline on in-region reference PSNR, EWARP, and video-level DINOv2 similarity.
Temporal stability on 27 DAVIS clips (first 81 frames). From the DAVIS-2017 annotations, we automatically select clips with occlusion followed by reappearance or large camera moves, together with control clips without that type of event. FLUX.2 [klein] edits frame 0 to produce the keyframes. We measure drift relative to the target’s appearance in the generated clip’s own first frame, where the keyframe is placed, and subtract the corresponding change in the source video. Event effects are differences-in-differences against the corresponding control clips at the same frames. Brackets give 95% bootstrap intervals.
We will clarify in the revision what each metric measures.