- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the substantial computational overhead of 3D spatio‑temporal attention in video generation. It points out that conventional Top‑K/Top‑P oracle policies adopt hard truncation and discard the continuous tail of the attention distribution, which introduces structural errors that accumulate across diffusion iterations. The authors propose the Low‑Delta Oracle Policy and theoretically prove that sparse attention can achieve zero error when aggregating tokens with identical attention scores. Built upon this theory, they design the Unified Tail Aggregation (UTA) method to aggregate logits with very small score differences. UTA can seamlessly integrate with the FlashInfer high‑performance attention kernel with less than 1% additional FLOP overhead. Experiments on Wan and HunyuanVideo text‑to‑video models show that UTA substantially reduces the MSE of attention outputs, effectively preserves the structural integrity of attention distributions, and noticeably improves video generation quality.
Critical strengths
- Solid theoretical contributions: The paper puts forward the zero‑error aggregation theorem for sparse attention. It identifies renormalization distortion and discarded‑tail probability loss as intrinsic drawbacks of the traditional Top‑K oracle. The Low‑Delta Oracle serves as a new theoretical upper‑bound, refreshing the evaluation thinking for sparse attention with prominent theoretical novelty.
- Practical and efficient engineering solution: The proposed UTA algorithm is easy to implement and can be directly plugged into existing sparse‑attention frameworks including FlashInfer and SVG2, with extra FLOP overhead below 1%. Improvements are validated in both oracle simulation and real‑world video‑generation settings. Sufficient quantitative and qualitative evidence is provided via attention‑distribution plots and attention‑map visualizations.
Critical weaknesses
- Despite its novelty, the overall field‑wide impact is limited. Plenty of existing sparse‑attention works for DiT‑based video generation realize extreme sparsity above 90% with minor quality degradation versus dense attention. Combined with kernel optimizations, those approaches can achieve speed‑ups of over 2×. The comparative advantages of the proposed method are not obvious.
- Experimental evaluation can be further strengthened: the set of compared baselines is limited; detailed breakdowns for VBench metrics are missing.
Questions for authors
- Compared with alternative approaches that achieve extremely high sparsity and large speed‑up ratios, is the proposed method mainly distinguished by theoretical insights while having limited practical value?
- Could the authors supply comparisons against more baselines?
- Can the authors report per‑dimension detailed scores for VBench?