- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper presents ResiSparse, a training‑free sparse‑attention method for video diffusion transformers. It consists of three cooperating components: QIKC (Query‑Induced Key Clustering), UMDR (Unified Mass–Dispersion Routing), and QPTC (Query‑Prioritized Taylor Compensation). Following the exact‑or‑approximate sparse‑attention paradigm, the method avoids outright discarding unselected token blocks. Instead, it applies second‑order Taylor compensation based on cluster centroids. The goal is to reduce the computational cost of spatiotemporal self‑attention while preserving video generation quality comparable to dense attention. The authors conduct experiments on the Wan2.1 series and HunyuanVideo video models, reporting speedups of 1.59‑2.18× over dense attention with slight degradations in VBench metrics. The authors claim to have theoretically derived the impacts of centroid‑induced logit deviations and intra‑cluster value variation on the attention numerator and softmax denominator, and they use this theory to guide the design of the three modules.
Critical strengths
- The topic addresses a well‑known pain point in video generation: spatiotemporal self‑attention constitutes a major computational bottleneck for video diffusion models. The training‑free sparse acceleration scheme requires no fine‑tuning of model weights and carries practical engineering value.
- The overall pipeline features clear modular design: QIKC for clustering optimisation, UMDR for error‑aware routing, and QPTC for second‑order compensation. Comprehensive speed‑quality evaluations are performed on popular open‑source video models Wan2.1 and HunyuanVideo, with baselines including SVG‑EAR and other exact‑or‑approximate sparse‑attention counterparts.
Critical weaknesses
- Insufficient justification for the valid regime of core theoretical approximations. The second‑order Taylor expansion within QPTC is valid only under . Cases with large intra‑cluster dispersion are not analysed. The compensation term may introduce larger biases at certain denoising steps. Results are supported solely by ablation experiments, lacking theoretical error bounds.
- Incomplete manuscript quality and severe figure defects. Many mathematical symbols in the QPTC block diagram of Figure 2 render incorrectly as question‑mark placeholders. The top‑channel selection mechanism drawn in the figure is not reflected in main theoretical formulas. Inconsistencies among theory, schematic diagrams and implementation reveal inadequate manuscript proof‑reading at review submission time.
- Baseline comparisons are insufficient. Evaluations focus mainly on SVG‑EAR. More granular error comparisons against other low‑cost approximate‑attention variants and alternative sparse‑attention approaches (e.g., linear‑attention‑style approximations) are missing.
Questions for authors
- Please carefully double‑check and fix all formulas and figures in your manuscript.
- Could you provide the complete set of VBench metric dimensions and additional baseline results?
6 comments
Response to Reviewer FHmR (Part2)
W3 & Q2: Additional baseline comparisons and per-dimension VBench results.
Reply:
(1) Additional baselines. ResiSparse achieves higher fidelity to dense outputs at lower latency than PISA [1] and Sol-Attn [4] at the evaluated settings. Given the limited rebuttal time and computational resources, the additional baseline comparison covers Wan2.1-1.3B. All methods use the same prompts, seeds, generation settings, and hardware. We set PISA's target density to 0.20 and Sol-Attn's threshold to to approximately match ResiSparse's attention density rather than being exactly identical.
| Method | Denoising time (s) ↓ | Density (%) | PSNR (dB) ↑ | SSIM ↑ | LPIPS ↓ | VBench (%) ↑ |
|---|---|---|---|---|---|---|
| Dense | 774 | 100.00 | — | — | — | 76.80 |
| PISA | 375 | 27.96 | 20.67 | 0.7530 | 0.2378 | 77.50 |
| Sol-Attn | 368 | 26.81 | 18.76 | 0.6535 | 0.3549 | 74.77 |
| ResiSparse | 355 | 25.99 | 25.49 | 0.8456 | 0.1132 | 76.35 |
For training-free acceleration, our objective is to preserve the pretrained model's dense-attention outputs. PISA scores above dense on VBench (77.50% vs. 76.80%) but has lower PSNR and SSIM and higher LPIPS than ResiSparse. A higher VBench score therefore does not imply better preservation of dense outputs. At the evaluated settings, ResiSparse improves PSNR over PISA by 4.82 dB and over Sol-Attn by 6.73 dB, with shorter denoising time. The added comparisons cover training-free exact-or-approximate sparse attention. Linear-attention-style approximations are outside the evaluated baseline set.
(2) Per-dimension VBench results. Following prior sparse-attention studies [1, 2, 5, 6], we evaluated video quality on six VBench dimensions: subject consistency, motion smoothness, dynamic degree, aesthetic quality, imaging quality, and overall consistency. The reported VBench score is their six-dimension aggregate using the official normalization and weighting procedure [3]. Individual scores below are raw scores multiplied by 100.
| Method | VBench (%) | Subject consistency (%) | Motion smoothness (%) | Dynamic degree (%) | Aesthetic quality (%) | Imaging quality (%) | Overall consistency (%) |
|---|---|---|---|---|---|---|---|
| Dense | 76.80 | 92.60 | 98.56 | 56.94 | 65.30 | 66.59 | 27.31 |
| SVG2 | 76.46 | 91.84 | 98.50 | 55.56 | 65.33 | 65.99 | 27.44 |
| SVG-EAR | 76.30 | 91.99 | 98.56 | 54.17 | 65.20 | 65.89 | 27.34 |
| PISA | 77.50 | 93.12 | 98.51 | 58.33 | 66.03 | 67.80 | 27.58 |
| Sol-Attn | 74.77 | 92.93 | 98.71 | 47.22 | 62.62 | 62.58 | 27.06 |
| ResiSparse | 76.35 | 92.36 | 98.56 | 54.17 | 65.13 | 66.16 | 27.22 |
Manuscript revisions. We have revised the manuscript and highlighted the changes in blue. We corrected Figure 2, added QPTC validation across denoising steps and logit-variance ranges, and expanded the baseline comparisons and six-dimension VBench reporting.
References
[1] Li et al. PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers. arXiv:2602.01077, 2026.
[2] Li et al. Compact Attention: Exploiting Structured Spatio-Temporal Sparsity for Fast Video Generation. arXiv:2508.12969, 2025.
[3] Huang et al. VBench: Comprehensive Benchmark Suite for Video Generative Models. CVPR 2024.
[4] Li et al. Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification. arXiv:2607.24027, 2026.
[5] Yang et al. Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation. NeurIPS 2025.
[6] Zhou et al. SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing. arXiv:2603.08982, 2026.
Follow-up on our response
Dear Reviewer FHmR,
Thank you again for your comments. Our response focuses on the following concerns:
- QPTC's validity beyond small logit deviations. We clarify the condition under which QPTC reduces approximation error, make the bound derived from the Taylor remainder explicit, and report whether the correction increases errors across sampled denoising steps and logit-variance ranges.
- Figure defects and consistency with the formulas and implementation. We corrected Figure 2, rechecked the manuscript's formulas and figures, and identified where top-channel selection was already defined in the original equations and algorithm.
- Missing baselines and VBench details. We added PISA and Sol-Attn comparisons on Wan2.1-1.3B and reported individual scores for all six VBench dimensions evaluated in our experiments.
The corresponding manuscript revisions are highlighted in blue. We would appreciate your feedback on the revised manuscript and our response. Please let us know if any points need further clarification.
Best regards, The authors
Response to Reviewer FHmR (Part1)
We thank the reviewer for the comments on theoretical validity, presentation, and experimental completeness.
W1: QPTC validity with large logit deviations and across denoising steps.
Reply:
(1) Taylor approximation and error bound. QPTC can improve on the uncorrected centroid estimate without requiring ∣δℓ∣≪1. The original Appendix A.4 retains the exact Taylor remainder in Eqs. (36)–(37), and Eq. (42) separates the remaining error into this remainder and the selected-channel variance-estimation error. Eq. (43) gives the improvement condition 0<σℓ,ij2/2<2(Zij/Zij−1), which permits non-negligible higher-order terms. Improvement is conditional on this inequality. We have also made the error bound obtained directly from the same Taylor remainder explicit in Appendix A.4.
(2) Results across denoising steps. We evaluated QPTC on Wan2.1-1.3B using five prompts, five layers, and four denoising steps, yielding 100 sampled locations in total. With clustering and routing fixed, we compute relative L2 error for each query and attention head, then average within each step. Bold percentages indicate reductions in these mean per-query errors:
(3) Results across logit-variance ranges. The step-wise results above measure the total numerator, denominator, and normalized-output errors. To directly test the approximation underlying QPTC, we compare Zsjcorr with its exact value Zsj for the key clusters assigned to approximate computation. Here, σℓ,sj2 is the variance of attention logits across keys in cluster j for a fixed query s. The table groups these estimates by σℓ,sj2 and reports how often QPTC increases the estimation error:
Error increases become less frequent at larger logit variances, with none observed for σℓ,sj2≥4. Together with lower mean numerator, denominator, and normalized-output errors at every sampled step, these measurements show that the correction remains useful beyond the small-deviation regime in the evaluated setting.
W2 & Q1: Figure corrections and the top-channel selection mechanism.
Reply:
(1) Figure correction. We apologize for the rendering defects in Figure 2. The figure was created manually and then exported to PDF using WPS. It rendered correctly in WPS, but some symbols and formulas were corrupted in other PDF viewers. Our repeated pre-submission checks did not cover different PDF viewers, so this compatibility issue went unnoticed. We have re-exported the figure, verified the corrected version, and rechecked all figures and formulas in the manuscript.
(2) Top-channel selection in the original submission. Top-channel selection was already specified in the original submission. In the main text, Section 4.3, Eq. (13) computes QPTC's variance estimate over the selected channel set Ω; Appendix A.4, Eq. (39) defines Ω through query-energy ranking; and Appendix B and Algorithm 3 use the same selected-channel variances in the implementation. Thus, the theory, diagram, and implementation describe the same mechanism.