- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies human-centric streaming video effect generation, aiming to apply text- or reference-guided effects while preserving the source subject and scene. StreamingEffect builds on Wan2.2-TI2V-5B with an interleaved representation of reference, source, and target latents. A bidirectional teacher is adapted into a causal student, followed by on-policy training with paired-target supervision to reduce sampling to four steps. The paper also introduces VideoEffect-130K, containing 130K paired videos spanning rendered effects and general editing. Evaluation on 2,000 held-out clips compares editing quality against open-source and commercial systems and examines the quality–throughput trade-off across training stages. The final student achieves 14.1 FPS at 720p on a single H200.
Critical strengths
-
The paper addresses a useful deployment problem in which source preservation and responsiveness matter alongside generation quality. The combination of reference-conditioned editing, causal attention, and few-step generation is coherent. Table 2 shows that on-policy training improves all reported VLM dimensions over directly running the Stage-1 student with four steps, providing evidence that this training stage is useful for the proposed setting.
-
The paired dataset is another potentially valuable contribution. Its combination of rendered effects and synthetic edits provides substantial task-specific supervision for a problem that is difficult to address with generic video-generation data alone. Together with the reasonably strong teacher results, this makes the work informative even though the methodological contribution is primarily an integration and adaptation of existing techniques.
Critical weaknesses
1. Selective benchmarking in the primary comparison: Table 1 only evaluates the 50-step offline Teacher (0.14 FPS) against baselines, omitting the proposed 4-step streaming model entirely. When cross-referencing Table 2, the student’s reconstruction quality drops sharply (PSNR falls from 19.4 to 16.7, LPIPS degrades from 0.195 to 0.269), falling behind Kiwi-Edit and Kling O1. Omitting the deployed model from the main comparison obscures the actual quality-latency trade-off of the contribution.
2. Unsubstantiated real-time streaming claims: The reported throughput of 14.1 FPS on an enterprise H200 accelerator falls well below the standard 24-30 FPS input rate, meaning input frames will inevitably back up during continuous streaming. Furthermore, operating on 5-latent-frame chunks introduces an algorithmic buffering delay of roughly 17–20 raw frames (~700–800 ms at 24 FPS) before denoising even starts, yet the paper reports neither end-to-end capture-to-display latency nor time-to-first-frame.
3. Uncontrolled throughput jump in Table 2: The 4-step Causal AR student achieves 7.52 FPS, whereas the 4-step StreamingEffect student reaches 14.1 FPS. Because both models execute the exact same number of sampling steps on identical architectures, this near 2 times speedup is unaccounted for. It remains unclear whether this gain comes from unstated inference modifications (such as dropping CFG or applying compiler optimizations) rather than the proposed training method.
4. Fragile evaluation protocol and lack of human studies: The perceptual evaluation relies heavily on frontier VLMs prompted on concatenated 3-column video grids. VLMs frequently overlook high-frequency temporal flickering, boundary tearing, and fine facial distortions across multi-frame downscaled grids. For an interactive human-centric effect task, the complete absence of human perceptual benchmarks or user preference studies leaves the claimed visual quality weakly supported.
5. Lack of empirical validation for key interactive claims: The paper highlights long-stream generation and dynamic on-the-fly reference switching as core capabilities, but provides no quantitative evaluation for either. Without measuring error drift over extended rollouts or latency/artifact spikes when swapping references within the persistent KV-cache, the feasibility of interactive use remains unproven.
Questions for authors
1. Deployment clarification: Does the reported 14.1 FPS include source encoding, denoising, and output decoding? What are the startup and steady-state end-to-end latencies, including chunk buffering, and how is a continuous 24 FPS input handled?
2. Controlled comparison: What explains the throughput increase from 7.52 to 14.1 FPS between the two four-step students? Are CFG, cache management, numerical precision, and inference optimizations identical?
3. Evidence for the deployed system: Do you have evaluations of the final student on streams substantially longer than the training clips and during online reference changes? In particular, how are cached states handled after a reference update, and what evidence supports the claimed absence of drift and flicker?