- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Sparse Evolution, a test-time self-evolution framework for few-step video generation that reduces candidate exploration cost by temporally sparsifying the denoising process. The key observation is that after a single full-sequence warm-up step, sparse keyframe previews (retaining only 43–48% of latent temporal positions) preserve the relative rankings of full-video candidates (Spearman ρ = 0.75–0.81). Based on this, Sparse Evolution adaptively selects informative keyframes, restricts subsequent denoising/decoding to retained positions, uses sparse previews for candidate selection (best-of-N or iterative prompt refinement), and completes full-video generation only for the selected candidate. Evaluated on three few-step generators (MiniMax H3-33B, HunyuanVideo1.5-8B, Wan2.1-14B), it achieves 1.75–1.92× speedup over full best-of-8 search while retaining ~91% of quality gains on T2V-CompBench and VBench Semantic.
Critical strengths
-
A clean, well-validated empirical insight that motivates a principled method. The core finding is rigorously established through matched sparse-full pairs across three architecturally distinct models (33B, 8B, 14B). The saturation behavior (Figure 2a) directly motivates the operating-point selection, and the gain-capture analysis (Figure 2c) honestly quantifies what is lost. This is a model of how to motivate a systems-level optimization with empirical evidence.
-
Strong practical impact with broad applicability. The method is training-free, model-agnostic (demonstrated on three different backbones), and composable with both best-of-N search and iterative prompt refinement (including integration into VQQA). The design choice of resuming full generation from saved warm-up states is elegant and avoids quality loss.
Critical weaknesses
-
The adaptive keyframe selection is computed from a single reference seed, introducing a potential bias. The frame selection (Eq. 4) uses the first seed as reference, and all candidates share the same selected positions S. However, different seeds produce different content. The paper shows aggregate ranking preservation but does not analyze whether selection quality degrades for seeds whose content distribution diverges most from the reference. A per-seed analysis of Spearman correlation conditioned on seed–reference content similarity would strengthen the claim that shared selection is robust.
-
Evaluation is limited to instruction-following metrics and does not assess visual quality or temporal coherence of the final videos. T2V-CompBench and VBench Semantic measure prompt alignment, but sparse candidate selection optimizes for the same verifier that scores instruction following. The paper does not evaluate whether the selected candidates also maintain visual quality (FID/FVD), temporal coherence, or motion naturalness compared to full-search selection. It is possible that sparse previews systematically favor instruction-aligned but visually inferior candidates, since motion and temporal coherence are harder to assess from sparse frames.
Questions for authors
-
Does the Spearman correlation between sparse and full rankings vary significantly across seeds that produce very different content from the reference seed used for frame selection? A breakdown of ρ by seed–reference divergence would clarify whether shared keyframe selection introduces systematic bias for outlier seeds.
-
Have you measured FID/FVD or any temporal coherence metric on the final full videos selected by Sparse Evolution vs. full Best-of-8? The current evaluation focuses entirely on instruction-following scores.
-
The warm-up step processes all F positions and saves full states for all N candidates — what is the memory overhead of storing N full warm-up states simultaneously, and how does this scale with candidate count and model size?
-
For the iterative prompt refinement experiments (Table 2), how many refinement rounds does the agent typically use, and does the speedup advantage of sparse candidates compound or diminish over multiple rounds?
2 comments
I appreciate the authors’ response and their careful answers to my questions. Most of my concerns have been addressed, and I have no further questions. Based on this, I will maintain my positive score.
Response to Reviewer GBfQ(1/2)
W1 / Q1 — Reference-seed bias
A1: We compute seed–reference content divergence from full-sequence warm-up features and divide the eight candidates for each prompt into two groups of four, with lower and higher divergence, respectively. We then calculate group-conditioned Spearman correlations based on the original candidate pool's Full/Sparse rankings.
Table 1: Ranking consistency by seed–reference content divergence.
The group-wise correlation differences across the three models are all within 0.02. We observe no pronounced ranking degradation in the higher-divergence group among the tested candidates, supporting the robustness of the shared keyframe-selection strategy.
W2 / Q2 — Visual quality and temporal coherence
A2: We supplement the evaluation with non-semantic VBench quality metrics on the final full-length videos; the results are shown in Table R1. Compared with Best-of-8, the differences in subject and background consistency are both at most 0.03 points, and the maximum reduction in motion smoothness is 0.15 points. These additional results support the preservation of temporal consistency and motion smoothness while also reflecting trade-offs in visual quality.
Table R1: Non-semantic quality of final full-length videos.
Q3 — Warm-up state memory overhead
A3: Each candidate stores only its video latent and the conditioning and state information required to resume sampling; it does not retain Transformer intermediate activations or duplicate model weights. The per-candidate storage requirements for H3, Hunyuan, and Wan are 13.71, 9.64, and 9.08 MiB, respectively. At N=16, the corresponding requirements are 219.36, 154.24, and 145.28 MiB.
These states can be stored in CPU memory, with only the selected candidate's state restored for final generation. Thus, storage scales linearly with the number of candidates, without multiple copies of model weights or Transformer intermediate activations; the memory overhead of warm-up states is negligible relative to the model weights.
Q4 — Refinement rounds and speedup scaling
A4: The Agent performs approximately 1.73 search rounds on T2V-CompBench and 1.31 search rounds on VBench Semantic on average, excluding final full-video generation.
To analyze how speedup changes with the number of search rounds, we use the cost model in Appendix D. Sparse Evolution generates low-cost sparse candidates in each search round and completes the selected full video only once after the search terminates. The theoretical generation speedup is:
Speedup(n)≈nγ+DD−1n.
Therefore, as the number of search rounds increases, the computational savings during candidate generation accumulate, while the fixed cost of full-video completion is amortized over more candidates. Consequently, the relative speedup of Sparse Evolution increases with the number of search rounds and gradually saturates.