acceptodds
Under review as a conference paper at ICLR 2027

Sparse Evolution: Efficient Test-Time Self-Evolution for Few-Step Video Generation

Abstract

Few-step distillation substantially reduces the cost of video generation, yet test-time search remains expensive due to repeated full-video candidate generation. Unlike final video synthesis, candidate selection primarily requires reliable relative rankings, suggesting that generating every candidate in full may be unnecessary. We find that, after a single full-sequence warm-up step, sparse keyframe previews retain discriminative information and largely preserve the rankings of their full-video counterparts. Building on this observation, we introduce Sparse Evolution, an efficient test-time self-evolution framework for few-step video generation. Sparse Evolution adaptively selects keyframes using warm-up features and restricts subsequent denoising and decoding to the retained temporal positions, reducing computation. The resulting sparse previews guide candidate selection and provide evidence for iterative prompt refinement, with full-video generation completed only for the selected prompt–seed pair. Across three few-step video generators, sparse previews achieve mean per-prompt Spearman correlations of 0.75–0.81 with full candidates, while Sparse Evolution accelerates candidate generation by 2.00–2.25× over SVG2. On T2V-CompBench and VBench Semantic, Sparse Evolution achieves 1.74–1.91× end-to-end speedup while retaining 91% of the quality gains from test-time search.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
video generation, test time scaling, acceleration

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GBfQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Sparse Evolution, a test-time self-evolution framework for few-step video generation that reduces candidate exploration cost by temporally sparsifying the denoising process. The key observation is that after a single full-sequence warm-up step, sparse keyframe previews (retaining only 43–48% of latent temporal positions) preserve the relative rankings of full-video candidates (Spearman ρ = 0.75–0.81). Based on this, Sparse Evolution adaptively selects informative keyframes, restricts subsequent denoising/decoding to retained positions, uses sparse previews for candidate selection (best-of-N or iterative prompt refinement), and completes full-video generation only for the selected candidate. Evaluated on three few-step generators (MiniMax H3-33B, HunyuanVideo1.5-8B, Wan2.1-14B), it achieves 1.75–1.92× speedup over full best-of-8 search while retaining ~91% of quality gains on T2V-CompBench and VBench Semantic.

Critical strengths
  1. A clean, well-validated empirical insight that motivates a principled method. The core finding is rigorously established through matched sparse-full pairs across three architecturally distinct models (33B, 8B, 14B). The saturation behavior (Figure 2a) directly motivates the operating-point selection, and the gain-capture analysis (Figure 2c) honestly quantifies what is lost. This is a model of how to motivate a systems-level optimization with empirical evidence.

  2. Strong practical impact with broad applicability. The method is training-free, model-agnostic (demonstrated on three different backbones), and composable with both best-of-N search and iterative prompt refinement (including integration into VQQA). The design choice of resuming full generation from saved warm-up states is elegant and avoids quality loss.

Critical weaknesses
  1. The adaptive keyframe selection is computed from a single reference seed, introducing a potential bias. The frame selection (Eq. 4) uses the first seed as reference, and all candidates share the same selected positions S. However, different seeds produce different content. The paper shows aggregate ranking preservation but does not analyze whether selection quality degrades for seeds whose content distribution diverges most from the reference. A per-seed analysis of Spearman correlation conditioned on seed–reference content similarity would strengthen the claim that shared selection is robust.

  2. Evaluation is limited to instruction-following metrics and does not assess visual quality or temporal coherence of the final videos. T2V-CompBench and VBench Semantic measure prompt alignment, but sparse candidate selection optimizes for the same verifier that scores instruction following. The paper does not evaluate whether the selected candidates also maintain visual quality (FID/FVD), temporal coherence, or motion naturalness compared to full-search selection. It is possible that sparse previews systematically favor instruction-aligned but visually inferior candidates, since motion and temporal coherence are harder to assess from sparse frames.

Questions for authors
  1. Does the Spearman correlation between sparse and full rankings vary significantly across seeds that produce very different content from the reference seed used for frame selection? A breakdown of ρ by seed–reference divergence would clarify whether shared keyframe selection introduces systematic bias for outlier seeds.

  2. Have you measured FID/FVD or any temporal coherence metric on the final full videos selected by Sparse Evolution vs. full Best-of-8? The current evaluation focuses entirely on instruction-following scores.

  3. The warm-up step processes all F positions and saves full states for all N candidates — what is the memory overhead of storing N full warm-up states simultaneously, and how does this scale with candidate count and model size?

  4. For the iterative prompt refinement experiments (Table 2), how many refinement rounds does the agent typically use, and does the speedup advantage of sparse candidates compound or diminish over multiple rounds?

2 comments
AuthorsOpenReview ↗

Response to Reviewer GBfQ(1/2)

W1 / Q1 — Reference-seed bias

Does the Spearman correlation between sparse and full rankings vary significantly across seeds that produce very different content from the reference seed used for frame selection? A breakdown of ρ by seed–reference divergence would clarify whether shared keyframe selection introduces systematic bias for outlier seeds.

A1: We compute seed–reference content divergence from full-sequence warm-up features and divide the eight candidates for each prompt into two groups of four, with lower and higher divergence, respectively. We then calculate group-conditioned Spearman correlations based on the original candidate pool's Full/Sparse rankings.

Table 1: Ranking consistency by seed–reference content divergence.

ModelLow Divergence ρ ↑High Divergence ρ ↑Δρ (High − Low)
H30.780.76−0.02
Hunyuan0.750.76+0.01
Wan0.810.810.00

The group-wise correlation differences across the three models are all within 0.02. We observe no pronounced ranking degradation in the higher-divergence group among the tested candidates, supporting the robustness of the shared keyframe-selection strategy.


W2 / Q2 — Visual quality and temporal coherence

Have you measured FID/FVD or any temporal coherence metric on the final full videos selected by Sparse Evolution vs. full Best-of-8? The current evaluation focuses entirely on instruction-following scores.

A2: We supplement the evaluation with non-semantic VBench quality metrics on the final full-length videos; the results are shown in Table R1. Compared with Best-of-8, the differences in subject and background consistency are both at most 0.03 points, and the maximum reduction in motion smoothness is 0.15 points. These additional results support the preservation of temporal consistency and motion smoothness while also reflecting trade-offs in visual quality.

Table R1: Non-semantic quality of final full-length videos.

ModelMethodSubject Cons. ↑Background Cons. ↑Motion Smooth. ↑Dynamic Degree ↑
H3Best-of-892.5990.1498.3795.75
CachedSearch92.5590.1198.3695.55
Sparse Evolution92.5790.1398.3395.63
HunyuanBest-of-891.4993.4699.0874.10
CachedSearch91.4593.4399.0173.70
Sparse Evolution91.4693.4498.9373.68
WanBest-of-896.4497.2098.3561.20
CachedSearch96.3997.1798.2660.70
Sparse Evolution96.4297.1798.2860.73

Q3 — Warm-up state memory overhead

The warm-up step processes all F positions and saves full states for all N candidates — what is the memory overhead of storing N full warm-up states simultaneously, and how does this scale with candidate count and model size?

A3: Each candidate stores only its video latent and the conditioning and state information required to resume sampling; it does not retain Transformer intermediate activations or duplicate model weights. The per-candidate storage requirements for H3, Hunyuan, and Wan are 13.71, 9.64, and 9.08 MiB, respectively. At N=16, the corresponding requirements are 219.36, 154.24, and 145.28 MiB.

These states can be stored in CPU memory, with only the selected candidate's state restored for final generation. Thus, storage scales linearly with the number of candidates, without multiple copies of model weights or Transformer intermediate activations; the memory overhead of warm-up states is negligible relative to the model weights.


Q4 — Refinement rounds and speedup scaling

For the iterative prompt refinement experiments (Table 2), how many refinement rounds does the agent typically use, and does the speedup advantage of sparse candidates compound or diminish over multiple rounds?

A4: The Agent performs approximately 1.73 search rounds on T2V-CompBench and 1.31 search rounds on VBench Semantic on average, excluding final full-video generation.

To analyze how speedup changes with the number of search rounds, we use the cost model in Appendix D. Sparse Evolution generates low-cost sparse candidates in each search round and completes the selected full video only once after the search terminates. The theoretical generation speedup is:

Speedup(n)≈nnγ+D−1D.\mathrm{Speedup}(n)\approx\frac{n}{n\gamma+\frac{D-1}{D}}.

Therefore, as the number of search rounds increases, the computational savings during candidate generation accumulate, while the fixed cost of full-video completion is amortized over more candidates. Consequently, the relative speedup of Sparse Evolution increases with the number of search rounds and gradually saturates.

I appreciate the authors’ response and their careful answers to my questions. Most of my concerns have been addressed, and I have no further questions. Based on this, I will maintain my positive score.

Reviewer_vCrcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Sparse Evolution, a training-free framework that reduces the cost of test-time search for few-step video generators. After a full-sequence warm-up step, it selects temporal positions from the predicted clean latent and generates sparse previews for candidate ranking and prompt refinement. The selected prompt–seed pair is then completed from its saved full-sequence state, avoiding reconstruction from the sparse preview. Experiments across three generators show favorable quality–latency trade-offs.

Critical strengths
  • The separation between sparse exploration and full-video completion is well designed. Restoring the selected candidate’s full warm-up state avoids having to reconstruct omitted temporal positions, while preserving the original generator’s full-length synthesis procedure.

  • The experiments are relatively thorough. The paper includes equal-budget frame-selection comparisons, warm-up and positional-index ablations, and a controlled study separating evidence-guided refinement from prompt variation alone.

Critical weaknesses
  • On the method side, it is not clear how reliably the sparse preview reflects what will happen in the full video. After warm-up, the two branches follow different denoising paths, and packing nonadjacent frames together changes their temporal relationships. An action that looks successful in the preview may therefore play out differently in the final video. This also matters for prompt refinement: the agent could end up “fixing” a problem that exists only in the preview.
  • On the experimental side, I would like to see a clearer comparison under the same total time budget. Using the same number of candidates shows that sparse previews are cheaper, but does not fully answer whether they are the best way to spend a fixed search budget. Best-of-4 is a useful comparison, although its runtime is not matched and it requires fewer verifier calls. Comparing final quality at matched wall-clock budgets, while varying both candidate count and preview sparsity, would make the practical advantage easier to judge. Table 4 only compares against SVG2, whose clustering overhead may be costly in the few-step setting. More recent training-free methods such as Sol-Attn avoid clustering and reduce routing overhead, making them relevant alternatives here. Including such a baseline under the same sampling settings would help establish whether the reported advantage extends beyond SVG2’s particular implementation and overhead.
Questions for authors
  • Does quality retention remain stable as the candidate pool grows? Please report selection regret distributions and uncertainty across independently sampled candidate pools, rather than only the mean correlation and mean regret at the default budget.
  • Can you provide quality-versus-wall-clock curves that jointly vary candidate count and temporal budget, with all verifier and completion costs included? Comparable sweeps for full-candidate search and CachedSearch would clarify the operating regimes where Sparse Evolution offers the strongest advantage.
5 comments
AuthorsOpenReview ↗

Response to Reviewer vCrc (1/2)

We appreciate the reviewer’s detailed evaluation and helpful suggestions for strengthening the experimental evidence. We respond to the concerns below.


W1 — Sparse-preview and full-video consistency

An action that looks successful in the preview may therefore play out differently in the final video. This also matters for prompt refinement: the agent could end up "fixing" a problem that exists only in the preview.

A1: Sparse previews and full videos start from the same full-sequence warm-up state and subsequently follow different denoising trajectories. The final full video resumes generation from the selected candidate's saved full-sequence state.

In matched prompt–seed experiments, the Sparse–Full ranking correlations are 0.75–0.81, with mean Regret of 0.0127–0.0181. In the Full/Sparse × With/Without Evidence Feedback ablation, Sparse achieves feedback gains of 3.8, 5.3, and 3.5 points, close to the Full backend's 4.4, 5.6, and 3.8 points. These results show that, despite the different denoising trajectories, sparse previews retain effective candidate-ranking and feedback information in the evaluated settings, supporting final full-video selection and prompt refinement.


W2 / Q2 — Matched wall-clock budgets

Can you provide quality-versus-wall-clock curves that jointly vary candidate count and temporal budget, with all verifier and completion costs included? Comparable sweeps for full-candidate search and CachedSearch would clarify the operating regimes where Sparse Evolution offers the strongest advantage.

A2:

The original Figure 2 systematically analyzes candidate ranking consistency and generation cost across different temporal budgets K, establishing the quality–efficiency operating points for each model.

We agree that jointly adjusting candidate count N and keyframe budget K under a fixed total time budget could further optimize search quality, but this is an additional optimization at the level of search-budget allocation. The central objective of this paper is to reduce candidate-generation cost through sparse keyframe previews while preserving as much full-candidate ranking information and final search quality as possible. Building on the existing keyframe-budget analysis, we therefore add a matched total-time budget comparison to assess the practical quality–efficiency advantage of this candidate-generation mechanism.

We set search budgets of approximately 330 s, 220 s, and 102 s for H3, Hunyuan, and Wan, respectively, and compare Best-of-N, CachedSearch, and Sparse Evolution. All methods account for candidate generation, Verifier calls, and final full-video generation.

Compared with Best-of-4, Sparse Evolution improves T2V-CompBench by 1.2–2.4 points and VBench Semantic by 0.6–1.0 points across the three models. Compared with CachedSearch-4, the improvements are 1.5–2.6 points and 0.7–1.1 points, respectively. This shows that the efficiency advantage of Sparse Evolution not only reduces computational cost under a fixed candidate count but can also translate into higher final-video quality under the same search budget.

Table R1: Search quality under shared model-specific time budgets.

ModelMethodBudgetNT2V ↑VBench ↑
H3Best-of-4330 s455.577.0
CachedSearch-4330 s455.276.9
Sparse Evolution330 s856.777.6
HunyuanBest-of-4220 s450.163.8
CachedSearch-4220 s449.963.6
Sparse Evolution220 s852.564.7
WanBest-of-4102 s452.876.3
CachedSearch-4102 s452.476.2
Sparse Evolution102 s854.777.3
AuthorsOpenReview ↗

Response to Reviewer vCrc (2/2)

W2 — Sol-Attn sparse-attention baseline

Table 4 only compares against SVG2, whose clustering overhead may be costly in the few-step setting. More recent training-free methods such as Sol-Attn avoid clustering and reduce routing overhead, making them relevant alternatives here.

A3: First, we would like to clarify that the SVG2 baseline in the original paper has already been extensively optimized. As detailed in Appendix A, we use FlashInfer to implement sparse attention, reuse GPU workspaces, cache token-layout metadata, and reduce synchronization overhead. Thus, the SVG2 results in our paper are based on an optimized implementation rather than an unoptimized original version.

To further rule out the effect of clustering overhead on the experimental conclusions, we add the Sol-Attn baseline, which does not require K-means clustering. Under the same generator, sampling configuration, and candidate-evaluation protocol, Sol-Attn achieves 1.41×, 1.35×, and 1.43× candidate-generation speedups on H3, Hunyuan, and Wan, respectively.

In contrast, Sparse Evolution achieves 2.44×, 2.17×, and 2.14× speedups, corresponding to an additional 1.73×, 1.61×, and 1.50× speedup over Sol-Attn. In candidate evaluation, the Spearman ranking correlations and mean Regret of Sparse Evolution are nearly identical to those of Sol-Attn, demonstrating that our method achieves greater computational efficiency while maintaining high candidate-selection reliability.

Table R2: Comparison of candidate-generation backends.

ModelMethodScore ↑Spearman ρ ↑Regret ↓Candidate Latency (s) ↓Speedup vs. Dense ↑
H3Dense71.151.000.000078.501.00×
SVG270.040.820.011172.591.08×
Sol-Attn70.120.830.010355.671.41×
Sparse Evolution69.860.780.012932.212.44×
HunyuanDense69.911.000.000047.401.00×
SVG268.460.810.014543.641.09×
Sol-Attn68.390.800.015235.111.35×
Sparse Evolution68.100.750.018121.822.17×
WanDense70.211.000.000021.901.00×
SVG269.260.860.009521.411.02×
Sol-Attn69.180.840.010315.311.43×
Sparse Evolution68.940.810.012710.212.14×

Q1 — Candidate-pool scaling

Does quality retention remain stable as the candidate pool grows? Please report selection regret distributions and uncertainty across independently sampled candidate pools, rather than only the mean correlation and mean regret at the default budget.

A4: We extend the candidate pool on Wan2.1-14B to N=16 and additionally report the mean, P90, and 95% confidence interval of the mean Selection Regret at different candidate pool sizes, as follows.

Regarding search gains, the Full Best-of-N T2V-CompBench score increases from 55.1 at N=8 to 56.2 at N=16, an improvement of 1.1 points, while candidate-generation cost nearly doubles. This indicates that the marginal quality gain from further enlarging the candidate pool is limited, and that N=8 provides a reasonable balance between search quality and computational cost.

When N increases from 8 to 16, mean Regret decreases from 0.0127 to 0.0114, and P90 decreases from 0.022 to 0.019. At N=16, the 95% confidence interval for mean Regret is [0.0067, 0.0156]. These results show that, in the evaluated Wan configuration, neither the average nor the high-percentile selection loss increases as the candidate pool expands, supporting the selection reliability of sparse previews with larger candidate pools.

Table R3: Search quality and selection regret at different candidate pool sizes on Wan2.1-14B.

CandidatesFull Best-of-N (T2V ↑)Quality Gain vs. N=1 ↑Sparse Mean Regret ↓Regret P90 ↓95% CI (Mean Regret)
N=149.9————
N=452.8+2.90.00920.017[0.0042, 0.0131]
N=855.1+5.20.01270.022[0.0059, 0.0184]
N=1656.2+6.30.01140.019[0.0067, 0.0156]

Thank you for the detailed response. The explanation of preview and full-video consistency addresses my partial concerns. I have two further questions about your responses:

  • In Table R1, how do you control the concrete time budgets to be exactly the same value, such as the 330s for H3.
  • In Table R2, is the sol-attn baseline also implemented by a few-step backbone? I think it is unfair to compare a full-step acceleration method with Sparse Evolution, which is already based on a few-step generator.
Reviewer_UgLtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Sparse Evolution, a framework for efficient test-time search in few-step video generation. Using a shared full-sequence warm-up step, it adaptively selects keyframe temporal positions and restricts subsequent denoising and decoding to the retained subset, producing low-cost sparse previews for candidate evaluation. Only the selected prompt–seed pair is completed into a full video by resuming from its saved warm-up state. Experiments across three few-step video generators and two benchmarks show the method achieves significant end-to-end speedups while retaining most of the quality gains from full-candidate search.

Critical strengths
  1. The key insight that candidate selection primarily requires reliable relative rankings rather than full-video generation is compelling. Temporal sparsification as a search proxy is a genuinely novel perspective for test-time video generation.

  2. The separation between low-cost candidate exploration and final full-video synthesis is clean. The warm-up reuse mechanism avoids repeated computation during final completion, forming a well-integrated pipeline.

  3. The evaluation spans three model architectures and integrates into multiple search frameworks including best-of-N search, iterative prompt refinement, and VQQA. The ablation studies on warm-up design, RoPE indexing, and temporal selection strategies are particularly thorough.

  4. The method requires no retraining and introduces no additional model components. The speedups are measured end-to-end including verifier and agent calls, making the results credible for real-world deployment.

Critical weaknesses
  1. On T2V-CompBench, Sparse Evolution scores 56.7 vs CachedSearch 56.9. On VBench Semantic, the difference is 77.6 vs 77.7. The method's main advantage is speed, but the quality is essentially on par with or marginally below CachedSearch, raising questions about whether the added complexity is justified.

  2. Only T2V-CompBench and VBench Semantic are used for final quality evaluation. Both measure compositional or semantic alignment. The paper does not evaluate on benchmarks measuring motion quality, temporal consistency, or aesthetic quality, leaving the method's impact on broader video quality unclear.

  3. All ranking and selection results depend on a VQA-based verifier powered by GPT-5.5. The paper does not analyze how sensitive the sparse-preview ranking correlations are to the choice of verifier. A weaker verifier may not preserve the reported 0.75–0.81 Spearman correlations.

  4. The paper reports only mean scores without confidence intervals or significance tests. The quality differences between methods (e.g., 0.2 points on T2V-CompBench) may not be statistically meaningful.

Questions for authors
  1. On T2V-CompBench, Sparse Evolution scores 0.2 points below CachedSearch. Is this difference statistically significant? Could you provide confidence intervals?

  2. How sensitive are the sparse-preview ranking correlations (0.75–0.81) to the choice of verifier? Have you tested with alternative verifiers or human judgments?

  3. The method retains 43%–48% of latent temporal positions. Have you tested how the approach performs on longer videos or with higher temporal resolutions where the sparse subset may miss critical motion?

  4. Could you evaluate on benchmarks measuring motion quality or temporal consistency (e.g., VBench full suite, EvalCrafter) to assess whether temporal sparsification during search affects dimensions beyond semantic alignment?

2 comments
AuthorsOpenReview ↗

Response to Reviewer UgLt (1/2)

We sincerely appreciate the reviewer’s careful assessment and constructive suggestions. We address the comments and questions below.


W1 — Additional complexity and efficiency

The method's main advantage is speed, but the quality is essentially on par with or marginally below CachedSearch, raising questions about whether the added complexity is justified.

A1: We would like to clarify that Sparse Evolution's keyframe selection and sparse temporal execution mechanisms can be integrated into existing generative models in a plug-and-play manner, without additional training or new model components. Across the three models, Sparse Evolution achieves final video quality close to CachedSearch while providing 1.60–1.74× end-to-end speedup, demonstrating the practical value of sparse candidate generation for few-step video search.


W2 / Q4 — Non-semantic video quality

Could you evaluate on benchmarks measuring motion quality or temporal consistency (e.g., VBench full suite, EvalCrafter) to assess whether temporal sparsification during search affects dimensions beyond semantic alignment?

A2: We further evaluate the motion characteristics, temporal consistency, and visual quality of the final full-length videos from all three models. The results are presented below.

Table R1: Non-semantic quality of final full-length videos.

ModelMethodSubject Cons. ↑Background Cons. ↑Motion Smooth. ↑Dynamic Degree ↑
H3Best-of-892.5990.1498.3795.75
CachedSearch92.5590.1198.3695.55
Sparse Evolution92.5790.1398.3395.63
HunyuanBest-of-891.4993.4699.0874.10
CachedSearch91.4593.4399.0173.70
Sparse Evolution91.4693.4498.9373.68
WanBest-of-896.4497.2098.3561.20
CachedSearch96.3997.1798.2660.70
Sparse Evolution96.4297.1798.2860.73

Compared with Best-of-8, the differences in subject and background consistency are both at most 0.03 points, and the maximum reduction in motion smoothness is 0.15 points. Meanwhile, the end-to-end search speedup reaches 1.74–1.91×. These results support the preservation of motion characteristics and temporal consistency in the final videos selected through sparse candidate search.


W3 / Q2 — Verifier robustness

How sensitive are the sparse-preview ranking correlations (0.75–0.81) to the choice of verifier? Have you tested with alternative verifiers or human judgments?

A3: We replace the Verifier with Qwen3.8-27B on the same prompt–seed candidate pools and reevaluate ranking consistency between sparse previews and full videos.

Table R2: Ranking consistency with Qwen3.8-27B as the verifier.

ModelSpearman ρ ↑Top-1 ↑Regret ↓Gain Capture ↑
H30.7557%0.016984.8%
Hunyuan0.7153%0.022777.1%
Wan0.8062%0.013187.2%

The Spearman correlations of the three models are 0.75, 0.71, and 0.80, respectively, while retaining 77.1%–87.2% of the search-quality gains. These results further indicate that ranking consistency between sparse previews and full videos does not depend solely on GPT-5.5 as the specific evaluation model.

AuthorsOpenReview ↗

Response to Reviewer UgLt (2/2)

W4 / Q1 — Statistical significance

On T2V-CompBench, Sparse Evolution scores 0.2 points below CachedSearch. Is this difference statistically significant? Could you provide confidence intervals?

A4: The original experiments generate and compare candidates from multiple seeds for each prompt. We further conduct paired per-prompt statistical analysis and use paired bootstrap to compute 95% confidence intervals for the score differences between Sparse Evolution and CachedSearch.

We define Δ = Sparse Evolution − CachedSearch.

Table R3: Paired score differences between Sparse Evolution and CachedSearch.

ModelT2V-CompBench Δ95% CIVBench Semantic Δ95% CI
H3−0.2[−0.60, +0.20]−0.1[−0.30, +0.10]
Hunyuan−0.1[−0.53, +0.33]0.0[−0.17, +0.16]
Wan+0.2[−0.13, +0.53]+0.1[−0.05, +0.25]

The results show that, on the current sample, the score differences between Sparse Evolution and CachedSearch do not reach statistical significance.


Q3 — Longer videos and temporal resolution

The method retains 43%–48% of latent temporal positions. Have you tested how the approach performs on longer videos or with higher temporal resolutions where the sparse subset may miss critical motion?

A5: The original three-model experiments cover generation configurations at 16 FPS and 24 FPS. We further extend H3 generation from approximately 5 seconds to 10 seconds, keeping the model, sampling steps, spatial resolution, frame rate, and evaluation protocol unchanged. The retained fraction of temporal positions remains approximately 45%.

Table R4: Ranking consistency and efficiency for longer H3 videos.

ModelDurationFKSpearman ↑Regret ↓End-to-end Speedup ↑
H35 s37170.780.01291.91×
H310 s73330.740.01651.93×

Under the new 10-second H3 setting, the ranking correlation is 0.74, mean Regret is 0.0165, and end-to-end speedup is 1.93×, indicating that the method retains effective candidate selection and efficiency advantages in this longer-video setting.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Reviews and Author Responses(1/2)

Dear Area Chair,

We sincerely appreciate your time and effort in handling our submission, as well as the reviewers' thoughtful evaluations and constructive feedback. As the rebuttal phase draws to a close, we would like to briefly summarize the reviews, our responses, and the progress of the discussion to assist in your final decision.

Summary of the Reviews

All three reviewers recognized the motivation, methodological contributions, and experimental strengths of our work. Two reviewers provided positive recommendations, while the remaining reviewer also acknowledged the merits of our sparse-preview framework and experimental design. The reviewers particularly appreciated the insight of preserving candidate rankings through low-cost keyframe previews, the separation of sparse exploration from full-video completion, and the systematic evaluation across three few-step video generators.

The rebuttal discussion has also made positive progress. Reviewer GBfQ explicitly indicated that their main concerns had been addressed, that no further questions remained, and that they would maintain their positive score. Reviewer vCrc acknowledged that our clarification of sparse-preview and full-video consistency had partially addressed their concerns. We subsequently clarified the matched time-budget protocol and the fairness of the few-step Sol-Attn comparison.

The main concerns can be grouped as follows:

  • Evaluation robustness and video quality (Reviewers UgLt, GBfQ). Dependence on a single GPT-5.5 verifier, statistical uncertainty in small quality differences, and evaluation of motion quality and temporal consistency beyond semantic alignment.

  • Sparse-preview reliability (Reviewers UgLt, GBfQ, vCrc). Consistency between sparse previews and final full videos, potential bias introduced by reference-seed keyframe selection, and robustness on longer videos.

  • Quality–efficiency trade-offs and baseline fairness (Reviewers UgLt, vCrc). Practical benefits over CachedSearch, fair comparisons under fixed total time budgets, and comparisons with stronger sparse-attention baselines such as Sol-Attn.

  • Memory overhead and search scalability (Reviewers GBfQ, vCrc). Storage requirements for saved warm-up states, typical Agent refinement rounds, and selection reliability as the candidate pool grows.

AuthorsOpenReview ↗

Summary of Reviews and Author Responses (2/2)

Our Responses and Additional Experimental Evidence

We address these concerns through the following additional experiments and analyses:

  • Broader video-quality evaluation and statistical uncertainty analysis. We supplement the evaluation of final full-length videos with four non-semantic VBench metrics, including subject consistency, background consistency, motion smoothness, and dynamic degree. Compared with Best-of-8, Sparse Evolution shows differences of at most 0.03 points in subject and background consistency, while motion smoothness decreases by at most 0.15 points. A paired bootstrap analysis provides 95% confidence intervals for the final-quality differences between Sparse Evolution and CachedSearch. Across all three generators and both benchmarks, the observed quality differences do not reach statistical significance.

  • Cross-verifier robustness and longer-video evaluation. We introduce Qwen3.8-27B as an alternative verifier, obtaining Sparse–Full Spearman correlations of 0.71–0.80 while retaining 77.1%–87.2% of the full-candidate search-quality gains. For longer videos, we extend H3 generation from approximately 5 to 10 seconds. Sparse Evolution maintains a 0.74 ranking correlation, 0.0165 mean Selection Regret, and 1.93× end-to-end speedup in this setting.

  • Reference-seed bias and selection stability with larger candidate pools. We group candidates according to their content divergence from the reference seed and analyze ranking consistency across the groups. Across all three generators, the Spearman correlation differences between the low- and high-divergence groups are at most 0.02, with no pronounced ranking degradation among candidates with greater content divergence. We extend the candidate pool on Wan2.1-14B from N=8 to N=16. Mean Selection Regret decreases from 0.0127 to 0.0114, while P90 Regret decreases from 0.022 to 0.019. The 95% confidence intervals for mean Regret further quantify statistical uncertainty. These results indicate that expanding the candidate pool does not increase average or high-percentile selection loss in the evaluated setting.

  • Fixed-budget comparisons and the Sol-Attn baseline. Building on the keyframe-budget analysis in the original Figure 2, we compare the search quality of Best-of-4, CachedSearch-4, and Sparse Evolution-8 under shared time budgets. The three models use common wall-clock budget upper bounds of 330s, 220s, and 102s, respectively. Actual runtimes may differ across methods, and all methods account for candidate generation, Verifier evaluation, and final full-video completion. Sparse Evolution improves T2V-CompBench scores by 1.2–2.4 points over Best-of-4 and 1.5–2.6 points over CachedSearch-4, demonstrating that lower candidate-generation costs can translate into higher final-video quality under the same search budget. We further evaluate Sol-Attn on the identical 8/8/4-step few-step generators used by Sparse Evolution. Compared with this training-free sparse-attention baseline, Sparse Evolution achieves an additional 1.50–1.73× single-candidate generation speedup, supporting its quality–efficiency advantage.

  • Memory overhead and multi-round Agent efficiency. We quantify the storage requirements of full-sequence warm-up states, which range from 9.08 to 13.71 MiB per candidate across the three models. The average Agent search rounds are approximately 1.73 on T2V-CompBench and 1.31 on VBench Semantic. The cost model in Appendix D further explains how the one-time full-video completion cost is amortized as more candidates are explored, supporting the efficiency advantages of multi-round search.

These additional experiments and clarifications strengthen the evidence for Sparse Evolution's ranking reliability, robustness, video quality, and practical efficiency, further supporting its central contribution: substantially reducing test-time search costs for few-step video generation through low-cost sparse keyframe previews while preserving most of the ranking information and search-quality gains of full-candidate search.

We sincerely appreciate the constructive feedback from all reviewers and your consideration of our work.

Best wishes,

Authors of Submission 16700

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.