- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes StreamCeption, a streaming video generation framework that consecutively appends and denoises noisy latents within a bounded active window. It learns to selectively repair fully denoised internal latents using subsequently developed temporal context to mitigate error accumulation during long rollouts, while leaving previously displayed video unchanged.
The authors evaluate the effectiveness of the proposed method on VBench, where StreamCeption achieves the highest score at the five-second duration on consistency metrics among several baselines, including Causal Forcing, Mask Forcing, and Rolling Forcing. Additional results in the appendix (Table 3) validate the performance gain at the ten-second duration achieved by the proposed repairing mechanism.
Critical strengths
This paper implements a sound and straightforward idea to mitigate temporal drift caused by error accumulation. It learns to repair the history that contains prediction errors and reuses it to generate new frames.
Critical weaknesses
The authors provide only weak experimental evidence, which is insufficient to support their claims.
-
Table 1 does not report Dynamic Degree scores. The higher consistency scores may therefore reflect reduced motion in the generated videos.
-
The paper lacks comparisons with baselines of comparable model size, as Rolling Forcing, Causal Forcing, and Mask Forcing are all implemented using 1.3B models. It also does not report results for Causal Forcing and Mask Forcing at longer durations, such as 10 and 20 seconds.
-
The qualitative comparison in Figure 4 includes only five-second results. As a method for “long streaming video generation,” the paper should provide more qualitative comparisons at longer durations.
-
The main text contains no ablation studies, and the appendix provides only one ablation examining the effect of enabling the repair mechanism.
Questions for authors
Novelty and Positioning.
Beyond the learned repair mechanism in Contribution 2, it would be helpful to clarify what independent methodological novelty Contribution 1 introduces. In particular, I would like to understand how separating video output from internal history modification differs from existing strategies. It appears to be a simple and common approach that refines or re-noises existing history contexts (e.g., Diffusion Forcing, Helios). I am not fully convinced that only outputting newly generated frames first and then reusing them via re-noising or refining is sufficiently novel. It would also be helpful to clarify the technical connection to Flowception, especially since StreamCeption removes Flowception's learned insertion mechanism and instead appends latents sequentially.
Effectiveness of the Repair Operation.
I have several concerns regarding the effectiveness of the repair operation.
-
One concern is whether the error in a frame to be repaired may already have propagated to newer frames, leaving no reliable information from which to repair the target frame. In that case, choosing to repair could lead to worse results, while choosing not to repair may allow the error to continue accumulating and ultimately cause the method to fail.
-
Another concern is that newer frames depend on earlier frames that need repair. It is unclear how one can guarantee that a newer frame predicted from a frame containing prediction errors can help recover a better frame to be generated.
-
It would also be useful to study the effect of performing repair multiple times. As in point 2, if a newer frame predicted from an erroneous frame can effectively repair the frame to be generated, then re-predicting with the repaired frame should yield better results. Conversely, if better repair results can be obtained from a better newly predicted frame, it is worth examining whether multiple repairs lead to better generation results. If multiple repairs make the results worse, this may suggest that the method fails in such cases.
-
During training, repair is always required to construct pseudo-labels. It would be helpful to further clarify the additional training overhead this introduces. In addition, it is unclear how one can ensure that the constructed labels are correct during training, especially in the early stage.
-
It is also unclear why only one latent is repaired at a time. If the motion generated by the model collapses, it is not clear how it could be repaired. More ablation studies would be helpful to explain why only one latent is repaired, since this may make it impossible to repair the motion generated by the model.
Experiments.
-
VBench results should report the Dynamic Degree score to demonstrate that the improvement in consistency is not achieved by generating more static videos.
-
It would be helpful to add more baselines with similar model sizes, such as LongLive-2.0, because Rolling Forcing, Causal Forcing, and Mask Forcing are all 1.3B models.
-
Metrics for measuring drifting should also be added, for example, by computing the difference between the score of the last chunk and that of the first chunk.
-
It would also be helpful to report metrics for selective repair, such as the accuracy and recall of the model's decision on whether repair is needed on controlled or annotated frames.
-
More visualization results at 20 seconds would be helpful, since the results shown on VBench (Figure 4) are only 5 seconds long.