- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes BlockStep, a training-free multi-model routing framework for accelerating autoregressive video diffusion. The method dynamically allocates computation between a large target model and a smaller draft model across both autoregressive blocks and denoising steps. It has three components: a full-target anchor for the first block, an attention-entropy-based router to select subsequent blocks that need more computation, and position-dependent step schedules for upgraded blocks. Experiments using Krea Realtime 14B and Wan2.1-1.3B demonstrate a 2.37× end-to-end inference speedup while maintaining a VBench score comparable to the full target model.
Critical strengths
-
The paper addresses an important computational bottleneck in autoregressive video generation by jointly allocating model capacity across blocks and denoising steps. The method is training-free and achieves a substantial 2.37× end-to-end speedup while maintaining the target model’s VBench score.
-
The ablation results demonstrate that generating the initial block with the target model recovers most of the quality gap between the target and draft models. This provides a useful insight into how approximation errors propagate through autoregressive video generation.
Critical weaknesses
-
The proposed block-level router assumes that higher attention entropy indicates greater uncertainty and therefore a stronger need for target-model computation. However, attention entropy is not a direct measure of draft-model error or the expected benefit of upgrading a block. Although the method outperforms random routing, the paper does not sufficiently establish the relationship between attention entropy and actual target-model improvement. This limits the justification for the central adaptive routing mechanism.
-
The proposed position-dependent step scheduling provides only a marginal improvement over the simpler block-level routing strategy. In Table 1, BlockStep achieves a VBench score of 82.69 and 2.37× speedup, compared with 82.68 and 2.34× for BlockStep without step-level scheduling. Furthermore, the method is evaluated on only one target–draft model pair, making it difficult to assess whether the proposed scheduling principles generalise across different model architectures and generation settings. These limitations raise questions about whether the observed improvements justify the added scheduling complexity.
Questions for authors
Q1. Could the authors provide a direct analysis of the correlation between attention entropy and the actual benefit of upgrading a block to the target model? This would help validate the central routing assumption.
Q2. Given the marginal improvement over block-level routing alone (82.69 vs. 82.68 VBench), could the authors clarify the practical significance of the proposed position-dependent schedules? Are the improvements statistically significant across different prompts and seeds?
Q3. Does BlockStep achieve similar quality–efficiency trade-offs with other target–draft model pairs or different video lengths? Additional evidence would strengthen the generality of the proposed scheduling strategy.
Q4. Since the primary quality claim relies on matching the target model’s aggregate VBench score, could the authors provide more detailed evaluations of temporal consistency, motion quality, and prompt adherence? Human preference comparisons would also help establish whether the accelerated outputs are perceptually comparable to the target model.
2 comments
Detailed Evaluations (Q4)
On the same videos used for the quality-speed comparison, we measure the average per-dimension VBench score across seeds and its standard deviation.
** Per-Dimension VBench Score Comparison**
| Dimension | Target | BlockStep | BlockStep w/o step-level schedule | SDVG | Random routing | Draft |
|---|---|---|---|---|---|---|
| Subject consistency | 95.99 ± 0.13 | 95.64 ± 0.23 | 95.63 ± 0.23 | 89.92 ± 0.66 | 95.25 ± 0.20 | 94.84 ± 0.14 |
| Background consistency | 95.33 ± 0.30 | 95.09 ± 0.33 | 95.10 ± 0.33 | 93.08 ± 0.47 | 94.59 ± 0.23 | 94.47 ± 0.31 |
| Temporal flickering | 96.87 ± 0.19 | 97.85 ± 0.29 | 97.84 ± 0.30 | 98.20 ± 0.25 | 98.00 ± 0.37 | 98.17 ± 0.44 |
| Motion smoothness | 98.38 ± 0.14 | 98.30 ± 0.15 | 98.29 ± 0.16 | 98.43 ± 0.13 | 98.32 ± 0.14 | 98.26 ± 0.16 |
| Dynamic degree | 72.78 ± 4.77 | 71.94 ± 5.68 | 71.94 ± 5.68 | 72.22 ± 1.70 | 73.89 ± 2.85 | 73.61 ± 2.60 |
| Aesthetic quality | 65.15 ± 0.16 | 64.54 ± 0.18 | 64.56 ± 0.18 | 63.70 ± 0.50 | 62.95 ± 0.42 | 62.59 ± 0.53 |
| Imaging quality | 70.06 ± 0.13 | 69.66 ± 0.29 | 69.66 ± 0.28 | 70.12 ± 0.58 | 70.07 ± 0.23 | 69.98 ± 0.36 |
| Object class | 91.98 ± 2.12 | 91.79 ± 1.45 | 91.68 ± 1.41 | 92.25 ± 1.39 | 83.20 ± 2.49 | 81.82 ± 2.03 |
| Multiple objects | 80.53 ± 1.76 | 80.08 ± 1.94 | 80.06 ± 1.93 | 80.98 ± 2.98 | 72.09 ± 1.89 | 69.33 ± 2.62 |
| Human action | 84.80 ± 1.48 | 86.00 ± 1.22 | 86.00 ± 1.22 | 86.00 ± 2.24 | 80.00 ± 2.55 | 76.80 ± 2.39 |
| Color | 86.60 ± 1.69 | 87.44 ± 1.85 | 87.36 ± 1.79 | 88.30 ± 1.38 | 88.22 ± 2.59 | 88.23 ± 2.26 |
| Spatial relationship | 86.13 ± 1.15 | 85.88 ± 1.53 | 85.90 ± 1.58 | 84.15 ± 2.35 | 80.07 ± 3.61 | 77.95 ± 3.16 |
| Scene | 39.84 ± 2.41 | 38.82 ± 3.16 | 38.82 ± 3.16 | 37.95 ± 3.54 | 28.81 ± 1.73 | 28.66 ± 2.06 |
| Appearance style | 19.91 ± 0.06 | 19.85 ± 0.06 | 19.85 ± 0.06 | 19.35 ± 0.11 | 19.32 ± 0.20 | 19.23 ± 0.13 |
| Temporal style | 24.74 ± 0.24 | 24.57 ± 0.29 | 24.57 ± 0.29 | 23.98 ± 0.33 | 23.71 ± 0.24 | 23.51 ± 0.25 |
| Overall consistency | 26.41 ± 0.21 | 26.42 ± 0.22 | 26.42 ± 0.22 | 26.13 ± 0.22 | 24.69 ± 0.27 | 24.34 ± 0.13 |
| Quality score | 84.22 ± 0.24 | 84.25 ± 0.34 | 84.25 ± 0.34 | 82.98 ± 0.12 | 84.12 ± 0.34 | 83.97 ± 0.36 |
| Semantic score | 76.52 ± 0.56 | 76.44 ± 0.75 | 76.41 ± 0.72 | 75.91 ± 0.47 | 71.02 ± 0.96 | 69.74 ± 0.58 |
| Total score | 82.68 ± 0.26 | 82.69 ± 0.29 | 82.68 ± 0.29 | 81.57 ± 0.14 | 81.50 ± 0.22 | 81.12 ± 0.29 |
Overall, though there are significant gaps in score for dimensions such as temporal flickering (BlockStep is higher) and aesthetic quality (BlockStep is lower), when averaged out the total score of BlockStep matches that of the target model.
We are unable to conduct a human study at this time, and agree that the method could follow the target into a shared failure mode. Our method aims to accelerate the target model without quality loss, not necessarily to improve its quality.
Change List
- Revise the entropy explanation.
- Detail the efficiency benefits for adding step scheduling.
- Report the results across model pairs and video lengths.
- Report quality by dimension.
We thank the reviewer CXp5 for their constructive feedback. We are encouraged that the reviewer found BlockStep to have provided “useful insight into how approximation errors propagate through autoregressive video generation".
Below we address the (1) relationship between attention entropy and target-model improvement, (2) practical significance of the proposed position dependent schedules, (3) quality-efficiency trade-offs in different settings, and (4) more detailed evaluations of temporal consistency, motion quality, and prompt adherence.
Response to Concerns and Questions
Relationship Between Attention Entropy and Target-Model Improvement (W1, Q1)
Additional testing shows that while attention entropy is a weak predictor of per-block draft-target discrepancy and quality drop, it acts as a good clip-level allocation signal. Across 18 seeds, mean realized gains ranged from -0.000078 to +0.000083 for deciles 1-6 and +0.000152 to +0.000346 for deciles 7-10. Entropy’s overall rank correlation with the VBench score gain from a block upgrade is small, with a Spearman of +0.018 and per-decision AUC of 0.510. Analyzing the within-prompt discrepancy, however, attention entropy correlates with the decoded draft-target LPIPS of the VBench videos more strongly, with a Spearman of 0.185. In matched-budget offline tests, entropy routing beats random routing and matches or exceeds a LPIPS-based oracle on VBench score. We will describe entropy primarily as a clip-level allocation heuristic, presenting this evidence as motivation.
Practical Significance of Proposed Position Dependent Schedules (W2, Q2)
With a fixed router, implementing the position-dependent schedule instead of restarting every upgraded block only led to a -0.001 decrease in VBench score, with intervals of [-0.013, 0.012] across seeds and [-0.010, 0.010] across prompts. With this negligible decrease in score, the position-dependent schedule saved 0.50 target forward passes and 0.21 seconds per clip. The position-dependent schedules are thus an effective method of increasing efficiency even further.
Quality-Efficiency Trade-Offs in Different Settings (W2, Q2, Q3)
Replacing the draft model to create different model pairs had mixed results. Replacing the drafter with Rolling Forcing, BlockStep’s non-inferiority holds (+0.017 VBench score) and BlockStep beats the random router (+0.104 VBench score). In contrast, replacing the drafter with CausVid 3-step, BlockStep’s non-inferiority to the target model no longer holds (-0.617 VBench score), primarily due to the weaker draft model of CausVid 3-step. BlockStep with CausVid 3-step still beats the anchor-only baseline (+0.175 VBench score) and anchored random routing (+0.120 VBench score).
On 30-s, 40-block clips, BlockStep fails non-inferiority (-0.814 VBench score) and tends to upgrade later blocks more leniently because draft entropy rises with position. BlockStep’s entropy routing algorithm would need to be adjusted in order to address the rise in entropy with block position and make BlockStep more applicable to longer video clips.