- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes SPAD, a method for accelerating autoregressive video diffusion by addressing temporal-context redundancy and uneven denoising difficulty. It profiles layer-wise historical attention and statically prunes selected layers across training and inference, while a lightweight confidence head adaptively allocates denoising steps to each frame or chunk. Probabilistic timestep subsampling during DMD training further prepares the generator for inference-time skipping. Experiments on short- and long-video generation show reduced training and inference costs with competitive generation quality.
Critical strengths
The paper provides a reasonably thorough analysis of two distinct sources of computational redundancy. In particular, the consistency of layer-wise temporal-attention profiles across training stages and datasets motivates a practical pruning strategy rather than relying solely on an intuitive architectural modification.
The experiments show meaningful efficiency gains in both training and inference, including long-video generation. The component ablations also help explain the trade-off between adaptive skipping and generation quality, with timestep subsampling recovering much of the quality lost when adaptive skipping is introduced without corresponding training adaptation.
Critical weaknesses
The novelty of SPAD is somewhat incremental. Static attention pruning based on offline profiles and adaptive denoising for autoregressive video generation have both been explored in related work. The paper's main contribution therefore lies in its particular combination of cross-stage pruning, endpoint-based confidence supervision, and timestep-subsampled DMD. A clearer experimental demonstration of what this combination achieves beyond existing adaptive-generation and attention-acceleration methods would strengthen the contribution.
The ablations are cumulative rather than fully isolating the interaction between the main components. Although Table 3 shows that timestep subsampling recovers quality after confidence-guided skipping is introduced, it does not establish whether the benefit depends on this specific combination or how much the subsampling strategy contributes independently. This limits the conclusions that can be drawn about the necessity of the proposed training adaptation.
The evaluation demonstrates promising results, but the robustness of the reported gains remains unclear. Most short-video comparisons use a single backbone and training configuration, and the paper does not report variability across independent training runs. Moreover, the long-video results use a more aggressive skipping strength than the default short-video setting, making it important to distinguish the benefit of the method itself from the selected quality–efficiency operating point.
Questions for authors
How does SPAD compare with existing adaptive denoising methods, particularly DSA, under matched inference budgets and evaluation settings?
What is the effect of timestep subsampling when evaluated independently of confidence-guided skipping?
How sensitive are the main quality and efficiency gains to the pruning profile and training seed?
3 comments
Response to Reviewer Vrsi (2/2)
W3 & Q3: Robustness of the Reported Gains
- Backbones and settings. SPAD is evaluated on four generation pipelines: Causal Forcing with chunk-wise and frame-wise generation, Rolling Forcing, and LongLive. They differ in temporal grouping, attention window, and video length (5 s and 30 s), and SPAD improves the quality-efficiency trade-off in all of them. All of these pipelines are built on Wan2.1-T2V-1.3B, which is the base model shared by current open-source AR distillation pipelines.
- Variability. We report results over 3 inference seeds with standard deviations in the new Table 7 (§I), where the std of VBench Total is 0.01 to 0.53. We also repeated the DMD stage with a different training seed from the same CD initialization (new Table 8, §I): the second run reaches 83.44 Total and 9.88 VR against 83.07 and 9.73 for the run reported in Table 1, with a slightly lower HPSv3 (9.74 against 9.96), so the result does not depend on one training run.
- Long-video operating point. only scales the predicted confidence at inference and does not change the trained model. The confidence head trained in the long-video setting predicts lower confidence than in the 5-second setting, so we choose to align the average NFE (2.15) with the Fixed 2-step baseline (2.00) and compare them at a similar denoising budget. We also evaluated the 30-second setting at the default : it keeps all four denoising steps (NFE 4.00) and takes 70.53 s, against 2.15 and 45.08 s at , with comparable reward scores (VR 13.67, HPSv3 9.05). For 5-second generation, VBench Total varies only within 82.89 to 83.19 for (Figure 5b), so the gains are not tied to a single operating point.
- Sensitivity to the pruning profile. The profiles are highly consistent across three training stages, four training datasets, and DMD samples and timesteps (Figure 2b), so the selected layer set is insensitive to the profiling data and stage. Replacing the profiled set with uniformly spaced layers at the same budget lowers VBench Total by 0.55 (chunk-wise) and 0.51 (frame-wise) (Table 3), showing that the profile identifies the right layers. The pruning-budget sweep in Figure 5a and Table 6 further shows that quality degrades gradually up to 10 layers.
Q1: Comparison with DSA
DSA has not released its code. To compare under matched settings, we reimplemented it following its paper on the same Causal Forcing checkpoint and training data: a confidence head with learnable tokens cross-attending to DiT blocks 13, 21, and 29, trained jointly with DMD on the fixed four-step grid using the teacher-referenced confidence loss.
The table below compares both DSA variants with SPAD at its default setting.
| Method | Total | Quality | Semantic | VR | HPS | Time (s) | TA | NFE |
|---|---|---|---|---|---|---|---|---|
| DSA (our reimplementation) | 82.91 | 85.07 | 74.23 | 9.13 | 9.70 | 8.59 | 1.00 | 2.66 |
| DSA + profiled TP | 82.89 | 84.88 | 74.92 | 8.14 | 9.42 | 7.42 | 0.67 | 2.76 |
| SPAD (, default) | 83.07 | 85.03 | 75.23 | 9.73 | 9.96 | 6.74 | 0.67 | 2.40 |
At its default setting, SPAD uses fewer forward passes and less wall-clock time than both DSA variants, and scores higher on VBench Total and Semantic and on both reward models. Methodologically, SPAD differs from DSA in its self-endpoint confidence target, its ability to skip intermediate steps instead of only exiting early, its training adaptation through timestep subsampling, and its additional reduction of per-pass cost, as summarized in Table 13.
If you have further questions, we are happy to provide further clarifications during the discussion phase. Thanks for your time and effort!
Thank you for the detailed responses and additional experiments. The new comparisons with DSA, the isolated ablations of adaptive denoising and timestep subsampling, and the additional robustness evaluations address most of my initial concerns. In particular, the results provide stronger evidence for the individual contributions of the proposed components and their effectiveness beyond the original cumulative ablations. While the novelty remains somewhat incremental and evaluation on additional backbones would further strengthen the paper, I find the current evidence sufficient to support acceptance. I will maintain my rating of 3.
Response to Reviewer Vrsi (1/2)
Thank you for acknowledging our thorough analysis of two distinct sources of computational redundancy, the cross-stage and cross-dataset consistency of temporal-attention profiles that motivates a practical pruning strategy, the meaningful efficiency gains in both training and inference, and the component ablations showing that timestep subsampling recovers the quality lost by adaptive skipping. We address your concerns as follows:
W1: Novelty and What the Combination Achieves
SPAD differs from existing attention-acceleration and adaptive-generation methods in four aspects:
These differences are reflected in the existing results:
We have also summarized these differences in a property comparison table (new Table 13, §N).
W2 & Q2: Effect of Timestep Subsampling Independent of Confidence-Guided Skipping
The confidence loss updates only the confidence head, and training trajectories are sampled independently of the confidence prediction. The generator of the "+ Conf." setting in Table 3 is therefore trained on the fixed grid, while the SPAD generator is trained with timestep subsampling, and both use the same confidence-guided inference. Comparing these two rows thus measures the effect of timestep subsampling with the skipping rule held fixed. To separate subsampling from skipping entirely, we additionally run the SPAD generator on the full four-step grid (with confidence-guided skipping disabled). At the same NFE of 4.00, it reaches 83.07 Total against 82.97 for the fixed-grid-trained Profiled TP model of Table 3, so timestep subsampling preserves full-grid quality while improving the generator's robustness to skipped steps.