acceptodds
Under review as a conference paper at ICLR 2027

QUASAR: Query-Adaptive Sparse Attention with Recovery for Video Generation

Abstract

Long spatiotemporal attention in video Diffusion Transformers becomes an inference bottleneck as sequences grow. Existing sparse attention methods can be tightly coupled to sparse-operator implementations, while accuracy compensation may be insufficient or overly complex, limiting the speed-accuracy trade-off of sparse attention. We propose QUASAR, which uses a block-relevance score to select KV blocks for exact computation and restores the contributions of the remaining blocks. Selection has two variants: Full-Domain scores all legal KV blocks, while Candidate-Constrained applies the same score within a candidate set provided by an external sparse-attention method. Recovery estimates the contributions of unselected blocks from Value-block summaries and combines them with exact attention under the same normalization, with Recovery states at multiple query granularities. We prove output-error bounds for both Candidate-Constrained and Recovery. In Wan2.1, QUASAR with per-query Recovery at sparsity achieves VBench AVG5 of (Dense: ) with end-to-end speedup. At 720p, QUASAR reaches up to end-to-end speedup with more than sparsity and remains about VBench AVG5 drop relative to dense attention. Results across sparse-attention backends, video models, KV-block geometries, and hardware demonstrate QUASAR's portability.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Sparse Attention, Video DiT

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_japDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

QUASAR studies attention acceleration for video Diffusion Transformers. It scores KV blocks using mean-pooled Q/K representations, computes attention exactly on a Top-K subset, and approximates the remaining numerator and denominator using K/V block summaries before joint normalization. Recovery weights can be shared across an entire query block, shared within smaller groups, or computed per query. These choices vary the extent of query sharing while keeping the exact selected set and KV summaries fixed. Selection can operate over all legal blocks or over candidates supplied by another method; blocks outside the candidate set receive no Recovery.

The paper analyzes candidate coverage and attention-output error and evaluates implementations across models, backends, and hardware. On Wan2.1 at 480p, it reports an AVG5 of 0.8301, compared with 0.8347 for dense attention, at 84.77% sparsity and a 1.54× E2E speedup. At 720p, the fastest per-query configuration achieves a 2.41× speedup with an AVG5 of 0.8215, compared with 0.8268 for dense attention. Ablations show that summary-based recovery improves over dropping unselected blocks in the evaluated settings.

Critical strengths
  1. The ablations provide good evidence for the role of recovery. In Table 3, under the same selection configuration and at 89.84% sparsity, per-query Recovery raises AVG5 from 0.7926 to 0.8270 while E2E latency changes from 167.832 s to 167.957 s. Similar gains occur at 84.77% sparsity. Table 5 holds Q/K/V and the exact selected set fixed and shows lower attention-output error with recovery; the appendix also reports recovery gains with another seed. These results help attribute the quality improvement to recovery rather than changes in selection.

  2. The implementation is fairly complete, and the reported acceleration includes the main online costs. The paper describes summary computation, online-softmax state merging, and the kernel implementation of per-query recovery. E2E timing includes candidate generation, selection, layout conversion, Recovery, and VAE decoding, making the reported acceleration practically relevant. Complete attention-call breakdowns, execution tests across backends, and validation on a second model strengthen the evidence beyond isolated kernel timings in a single configuration.

Critical weaknesses
  1. The distinction from existing compensation methods is not sufficiently established. Exact computation on selected blocks combined with summary compensation for the remaining blocks is directly related to PISA’s exact-plus-approximate framework and SVG-EAR’s centroid compensation. Showing that recovery improves over dropping unselected blocks does not establish what the method adds over existing compensation approaches. Fixed-block summaries, recovery granularity, and execution choices may offer advantages, but their differences from prior methods and the significance of those differences require a clearer comparison.

  2. Multi-granularity primarily refines the accuracy–cost trade-off. The different granularities change the scope of query sharing while retaining the same recovery formula and KV summaries. The paper theoretically explains how reducing query sharing improves the approximation and provides supporting experiments. These results establish a useful refinement, but do not establish a major methodological contribution distinct from the recovery mechanism. Its contribution to the overall novelty is therefore limited.

  3. The practical case for candidate reuse remains unclear. Candidate generation adds overhead, and blocks outside the candidate set receive no Recovery. In the representative Wan comparisons, candidate modes are slower and yield lower quality than Full-Domain. The evidence therefore primarily establishes interface compatibility. The paper should explain when candidate reuse is preferable in practice, since scoring fewer blocks does not by itself improve overall inference performance.

  4. The theory explains the method but contributes less to the overall novelty. The analysis identifies errors arising from query sharing, within-block variation, and normalization, and the paper numerically validates its bounds. These results are useful. However, the tighter bounds mainly support offline analysis, while the practical guidance provided by the input-summary-based bounds remains unclear. I regard the theory as supporting the method’s rationale, but do not yet see a sufficiently significant new analytical result to change my assessment of the contribution.

Questions for authors
  1. What are the main mathematical and implementation differences between Recovery, SVG-EAR’s centroid compensation, and PISA’s approximation? Do these differences reduce overhead, broaden applicability, or improve the quality–latency trade-off under comparable settings? Please focus on the distinction that most clearly establishes the paper’s added value.

  2. In the representative Wan comparisons, candidate modes are slower and yield lower quality. Under what deployment conditions are they preferable to Full-Domain? If their practical value depends on candidate indices already being available at no additional generation cost, please clarify where that assumption applies.

  3. For the input-summary-based bounds that do not require exact attention information, what do the existing results establish about their practical tightness? Can these bounds help determine when finer-grained recovery is worthwhile, or is their current role primarily to provide a qualitative explanation?

4 comments
AuthorsOpenReview ↗

Response of the Review from Reviewer japD (Part 1/3)

Thank you for your detailed review. The core innovation of QUASAR is a portable, controllable-recovery sparse-attention operator introduced through Candidate integration, together with its adaptation to DiT. We address its mechanism, deployment value, and theoretical guidance through PISA-0th/SVG-EAR comparisons, Meta Comment C1/C2 Full-KV evidence, and input-bound analysis.

Q1.1 What are the main mathematical and implementation differences between Recovery, SVG-EAR’s centroid compensation, and PISA’s approximation?

A1.1 QUASAR combines three mechanisms: Candidate selection, independently controlled recovery coverage/granularity, and cross-backend recovery-state reuse.

① Controllable recovery granularity. PISA-0th and SVG-EAR use per-query compensation; QUASAR supports G1, G2, and per-query modes via g(i)g(i), adjusting accuracy and cost through group sharing or per-query weights.

② Independent candidate and recovery coverage. Exact selection remains within the Candidate, while Full-KV recovery covers all unselected valid blocks. With matching Exact set, granularity, summaries, and normalization, Candidate and Full-Domain produce identical outputs.

③ Recovery-state reuse across backends. Exact and Recovery states merge under the same scaling and integrate with Magi and FlexAttention; per-query recovery uses a query-block tiled kernel, while coarse recovery reuses group computation.

See Meta Comment C2 for related-work comparison; Theorem 2, F.2, and K.12 cover output error, granularity, and out-of-domain omissions. Quantitative effects appear in A3.2 and Meta Comment C1.

Q1.2 Do these differences reduce overhead, broaden applicability, or improve the quality–latency trade-off under comparable settings?

A1.2 The practical benefits are:

① Reduced execution overhead. With PISA-0th-aligned selection rules and Exact budget, QUASAR’s per-query invocation decreases from 12.280 to 11.422 ms. The table compares the full 96-prompt pipeline; matched settings are in Meta Comment C2.

MethodAVG5E2E (s)
PISA-0th0.835443205.169
SVG-EAR0.834158318.681
QUASAR matched0.835210201.247

② Extended Candidate Support. The same Recovery incorporates Draft and XAttention candidates; see A2.1.

③ Improved Quality–Latency Trade-off. Appendix Table 13(a) shows 720p per-query K=39 at AVG5 0.8215/428.712 s, outperforming G1 K=52 (0.8186/435.907 s) and K=64 (0.8210/452.561 s). Table 5 validates fixed-support control; generation shows the Pareto gain from fewer Exact blocks.

Q1.3 Please focus on the distinction that most clearly establishes the paper’s added value.

A1.3 The core distinction is transferability combined with controllable recovery. Decoupling candidates, Exact selection, Recovery, and backend lets different selection methods reuse one operator; at fixed sparsity, G1/G2/per-query adjust accuracy and cost independently.

Figure 3 shows competitive Full-Domain, Draft, and XAttention Pareto points across approximately 10%–45% Exact-attention computation.

W1 R1: This weakness has been addressed in Q1.1–Q1.3: C2 distinguishes between the zero-order endpoint in PISA-0th, the clustering/routing in SVG-EAR, and the combination of candidates, granularity, and state in QUASAR; Q1.2 provides an execution comparison with the same support.

W2 R2: This weakness has been addressed in Q1.1–Q1.2 and Q3.1–Q3.2: G1/G2/per-query make precision–cost a configurable axis, supported by fixed support error and 720p Pareto results.

Q2.1 In the representative Wan comparisons, candidate modes are slower and yield lower quality. Under what deployment conditions are they preferable to Full-Domain?

A2.1 For existing candidate workflows, the Candidate interface retains the original selection design and achieves quality close to Full-Domain through Full-KV recovery. Meta Comment C1 includes operator and E2E comparisons with candidate-generation costs.

Q2.2 If their practical value depends on candidate indices already being available at no additional generation cost, please clarify where that assumption applies.

A2.2 Candidate indices are available at the output of the methods in Figure 1. Draft/XAttention generates CC within the same attention call, which QUASAR consumes; generation cost is included in total latency. Appendix Table 9 compares 75%-sparse providers:

MethodSparsityAVG5E2E/s
XAttention75.00%0.8169204.345
XAttention Candidate75.00%0.8273202.928

In-domain recovery already improves AVG5 by 0.0104 while reducing E2E. Full-KV recovery further compensates outside the candidate domain, achieving quality close to Full-Domain (Meta Comment C1).

W3 R3: This weakness has been addressed in Q2.1–Q2.2: C1 and the provider comparison show that Candidate selection is retained while Full-KV recovery restores quality close to Full-Domain.

AuthorsOpenReview ↗

Response of the Review from Reviewer japD (Part 2/3)

Q3.1 For the input-summary-based bounds that do not require exact attention information, what do the existing results establish about their practical tightness?

A3.1 We answer your question through the following logical chain: “What tightness conclusions do existing results establish?”—“A review of the input-summary-based bound”—“How tight is this bound, and where is the margin?”—“How should the bound be tightened based on the margin, and what optimization direction follows?”

Existing results and additions in this round. Main text §3.3 Theorem 2 and Eqs. (10)–(11) provide the output-error decomposition and hierarchical bounds; K.13 gives the input-summary analysis; Appendix J Table 29(b) reports bound tightness. With Q/K/V, the Exact set, and granularity fixed, we quantified the margin from substituting Eq. (87) into Eq. (85).

The table reports “normalized upper bound − actual attention error” for G1, G2, and per-query, using the same Dense denominator across resolutions.

Upper Bound480p Gap720p Gap
Input Summary6.954824–6.9592857.995186–8.006168
HCR0.230101–0.2423980.330051–0.333757
Pairwise0.007868–0.0095710.026900–0.029461

The input summary bound is 37.52–47.33 times the actual error; the Pairwise bound, which preserves more complete output information, is only 5.09%–14.17% higher.

Appendix K.13 of the paper provides a complete derivation of the input bounds, namely Eqs. (84), (85), and (87); all subsequent tightness analyses are based on these formulas.

Where is the margin? On the same query/head set I\mathcal I, take the square sum as per Eq. (81), take the square root, and divide by ∥OIΩ∥F\|O_{\mathcal I}^{\Omega}\|_F. The two margins are:

The margin gained by substituting the input statistics for the actual logit range:

\sqrt{\sum_{i\in\mathcal I}[\overline D_{V,D}\tanh(\epsilon_i/2)]^2} -\sqrt{\sum_{i\in\mathcal I}[\overline D_{V,D}\tanh(h_i/4)]^2} }{\|O_{\mathcal I}^{\Omega}\|_F} \approx 0.002856\text{–}0.004623.$$ The margin from the actual logit range to the output error: $$\frac{\sqrt{\sum_{i\in\mathcal I}[\overline D_{V,D}\tanh(h_i/4)]^2}-\Vert\widehat O_{\mathcal I}-O_{\mathcal I}^{\Omega}\Vert_F}{\Vert O_{\mathcal I}^{\Omega}\Vert_F}\approx 6.951134\sim 8.002457.$$ The range comes from two captures and three granularities. **The main margin lies in the transformation from actual logit range to output error**, including the slack from replacing $D_{V,D}$ with $\overline D_{V,D}$; replacing it with actual $h_i$ produces only a small reduction. This explains the tightening direction: Pairwise/HCR bounds preserve more output-distribution or Value-geometry information and lie closer to the true error; the main opportunity is output-side estimation rather than redefining input-summary terms. **Future work should prioritize tightening the output-side estimate in Eq. (85)** by using finer Value geometry and weight information from the input summary; Pairwise/HCR provide offline references. **Conclusion: The margin lies primarily on the output side; preserving Value geometry and error cancellation significantly tightens the bounds for the same output, providing direction for optimizing input bounds.**
AuthorsOpenReview ↗

Response of the Review from Reviewer japD (Part 3/3)

Q3.2 Can these bounds help determine when finer-grained recovery is worthwhile, or is their current role primarily to provide a qualitative explanation?

A3.2 We answer four questions in sequence: (1) where the effect of Recovery stops; (2) how much upper-bound improvement the input bound predicts for each Recovery granularity and which positions are better suited to that granularity; (3) under the same budget, how many times the actual improvement from bound-based selection exceeds random selection; and (4) where the theoretical guidance is applied and how it determines the recovery positions. The following analysis is based on Eqs. (10)–(11) in Theorem 2 of the main text, Eq. (17) in Appendix F.2, and Eqs. (85) and (87) in Appendix K.13.

1. What the sharing granularity controls. Theorem 2 Eqs. (10)–(11) and F.2 Eq. (17) provide the hierarchical decomposition and query-sharing terms. With KV summaries fixed, G1/G2/per-query alter only the sharing scope of query-sharing logits and recovery weights; they share one state across 128, 64, and 1 queries, respectively. Per-query removes the shared terms, while Eq. (87) retains intra-block key variation. Thus, granularity refinement targets query-sharing bias, and KV-summary granularity controls intra-block variation.

2. Deriving Recovery Priorities from Residual Output Bounds. Eq. (87) yields ϵi\epsilon_i, and Eq. (85) yields the target-granularity residual bound D‾V,Dtanh⁡(ϵi/2)\overline D_{V,D}\tanh(\epsilon_i/2). The common Exact-only starting point is constrained by Eq. (44) and the Value diameter:

\le D_{V,D}\le\overline D_{V,D}.$$ Therefore, for a complete query tile $G_r$, the following is computed at a given target granularity: $$\sum_{i\in G_r} \left[\overline D_{V,D}^{\,2} -\big(\overline D_{V,D}\tanh(\epsilon_i/2)\big)^2\right].$$ This score is the sum of unrecovered-bound squares minus target-granularity-bound squares. **A larger score means a larger reduction at that location.** It applies to G1, G2, and per-query, incorporating shared bias, intra-block variation, and Value scale; same-budget experiments verify the resulting gains. **3. Actual Benefits Under the Same Budget.** Starting from the same Exact-only baseline, we selected a specified number of 32 complete Q128 tiles to enable the target granularity and reported $$\frac{\text{Exact-only Rel-L2}-\text{Rel-L2 with Bound Selection}} {\text{Exact-only Rel-L2}-\mathbb E[\text{Randomly Selected Rel-L2}]}.$$ For the same target granularity and tile count, additional computation is fixed. **The ratio measures how many times the error improvement from bound-selected locations exceeds random allocation.** When the random average reduction is positive, a value above 1 indicates lower final error. **480p** | Number of restoration tiles (32 total) | G1 | G2 | Per-query | |---|---:|---:|---:| | 4 | 3.80× | 3.25× | 2.87× | | 5 | 3.12× | 3.72× | 3.26× | | 6 | 3.73× | 3.57× | 3.26× | | 8 | 3.18× | 3.35× | 3.15× | | 16 | 1.87× | 1.85× | 1.76× | **720p** | Number of tiles restored (32 total) | G1 | G2 | Per-query | |---|---:|---:|---:| | 4 | 2.58× | 2.91× | 2.71× | | 5 | 2.77× | 3.10× | 2.59× | | 6 | 2.50× | 2.83× | 2.66× | | 8 | 2.70× | 2.73× | 2.54× | | 16 | 1.46× | 1.52× | 1.45× | **4. Where the guidance is applied.** Calculate the Step 2 score from input summaries and enable the target granularity on the highest-scoring query tiles. Since Eq. (81)’s quadratic upper bound is additive across tiles, **this selection minimizes the overall quadratic upper bound** for a fixed granularity and tile count. This step determines the recovery priority based on the residual bound at the target granularity, and the true gain per tile is given by $$\sum_{i\in G_r}\left( \|O_i^M-O_i^{\Omega}\|_2^2 -\|\widehat O_i-O_i^{\Omega}\|_2^2\right)$$ The table reports Spearman correlation between input scores and true gains, testing whether locations with larger bound reductions improve more; true gains are used only for evaluation. | Recovery Granularity | 480p Spearman | 720p Spearman | |---|---:|---:| | G1 | 0.650 | 0.416 | | G2 | 0.629 | 0.513 | | Per-query | 0.730 | 0.485 | **Conclusion: The formulas in the main text provide residual bounds for each granularity, which are used to rank and select recovery positions; experiments with the same budget verify that this guidance is effective for G1, G2, and per-query.** **W4** R4: This weakness has already been addressed in Q3.1–Q3.2: Theorem 2/K.13 transforms the multi-granularity error decomposition into a criterion for determining recovery positions in the input summary, and its computational allocation role has been validated by 30 configurations. Thank you for your question, which has helped us clarify the practical guidance provided by our theory and identify directions for future research.

Thank you for your rebuttal and the additional experiments. However, my concerns about the incremental novelty and the practical value of candidate reuse remain. The comparison with PISA-0th shows a modest gain, while candidate reuse has not demonstrated a clear quality–latency advantage over full-domain. I therefore maintain my original score.

Reviewer_KFNAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

QUASAR is a training-free sparse-attention framework for video DiTs. It uses mean-pooled query/key scores to select KV blocks for exact computation, either over the full domain or within externally supplied candidates. Unselected blocks within the selection domain are approximated using key/value summaries, and their contributions are merged with exact attention under a shared normalization. Recovery supports query-block, grouped, and per-query granularities. The paper provides candidate-coverage guarantees, bounds for changes in selected blocks, and a hierarchical decomposition of recovery error. Validation includes several models, multiple attention backends, and H200/B200 hardware. Representative configurations achieve 1.54x and 2.41x end-to-end speedups, respectively, with AVG5 reductions of 0.0046 and 0.0053 relative to dense attention. Controlled ablations show substantial recovery gains over exact-only sparse attention.

Critical strengths
  • Controlled comparisons directly support the motivation for recovering omitted contributions. Table 3 compares recovery with exact-only execution under the same block selection. This provides clear evidence that compensation addresses information loss left unresolved by sparse selection.
  • The separation of selection, recovery, and execution provides a useful deployment abstraction. The method explicitly distinguishes logical block selection from backend-specific execution. This makes candidate reuse and recovery granularity accessible through a common formulation. The deployment experiments support the practical relevance of this modular design.
Critical weaknesses
  • The novelty of the core recovery formula is insufficiently distinguished from existing work. For fixed blocks and masks, Full-Domain per-query recovery matches the exact-plus-zeroth-order component of PISA. Comparing complete methods does not isolate improvements beyond this shared component.

  • The candidate-reuse motivation is not matched by a demonstrated end-to-end advantage over Full-Domain selection. The main comparisons generally favor Full Domain in both latency and quality. Candidate coverage therefore shows selection compatibility without establishing the practical benefit of candidate restriction.

  • The added significance of finer recovery granularity is only partially established. Table 5 shows that block-shared recovery already captures much of the improvement over exact-only attention. Per-query recovery further reduces attention error, but its additional generation-quality benefit is comparatively modest. The main evidence does not clearly identify when finer granularity materially improves the quality–latency trade-off.

Questions for authors
  • What additional benefit does QUASAR provide over exact-plus-zeroth-order compensation? Which contribution should be regarded as its principal novelty?

  • In what regime does Candidate-Constrained selection outperform Full-Domain after accounting for candidate generation cost and lost recovery context?

  • Which attention characteristics make per-query recovery materially preferable to block-shared recovery, and do the main quality–latency results substantiate this distinction?

3 comments
AuthorsOpenReview ↗

Response of the Review from Reviewer KFNA (Part 1/2)

Thank you for your detailed review of our paper. The core innovation of QUASAR is a portable, controllable-recovery sparse-attention operator introduced through Candidate integration, together with its adaptation to DiT. The operator output follows Eq. (7) in the main text:

{Z_i^M+\widehat Z_{r,g(i)}^R}.$$ Here, the exact numerator and denominator over $M_r$ are merged with the recovery state over $R_r$; $g(i)$ identifies the recovery group. G1 (block-shared) and G2 correspond to $G_{\mathrm{rec}}=1,2$ groups per query block, while per-query recovery uses one group per query. Candidate and backend layers are composable, and the granularity is controllable; DiT benefits are validated by quality–latency experiments. **Q1.1 What additional benefit does QUASAR provide over exact-plus-zeroth-order compensation?** A1.1 **Compared to PISA-0th, QUASAR offers recovery accuracy, composable candidates and backends, and lower full-execution cost.** The three benefits are: 1. **Faster inference.** With aligned block selection and Exact budgets, QUASAR/PISA-0th full invocations are 11.422/12.280 ms; 96-prompt E2E is 201.247/205.169 s; and AVG5 is 0.835210/0.835443 (Meta Comment C2). 2. **Quality–latency choices.** With Exact and KV fixed, G1, G2, and per-query modes are available; at 720p, per-query improves both quality and latency over some G1 configurations with less Exact computation (A3.2). 3. **Reuse of existing sparse schemes.** Draft/XAttention retains candidate selection, while Full-KV recovery covers all unselected valid blocks and brings quality close to Full-Domain; a shared state interface supports backend replacement and recovery-state reuse (Meta Comment C1, Appendices F.2–F.3). **Q1.2 Which contribution should be regarded as its principal novelty?** A1.2 The main innovation is the **portable, controllable-recovery sparse-attention operator introduced through Candidate integration, together with its DiT adaptation**, established by four contributions in the submitted paper: 1. **Unified selection and execution interface.** Eqs. (1)–(2) apply the same scoring and Top-K rules to both Full-Domain and external Candidates; Figure 1 separates the candidate selection method from the sparse executor. 2. **Controllability of candidate error.** Theorem 1 provides coverage conditions for maintaining Full-Domain exact computation; Appendix I bounds selection changes when coverage is incomplete. 3. **Multi-granularity recovery.** With Exact and KV fixed, G1, G2, and per-query parameters are selected to adjust recovery accuracy and cost; this is based on Eqs. (6)–(7), Theorem 2, and Appendix F.2. 4. **Portable DiT Validation.** We validate the unified operator across backends, models, KV geometries, and hardware, and present quality–latency Pareto points based on comprehensive generative experiments (Figure 3). **Q2.1 In what regime does Candidate-Constrained selection outperform Full-Domain after accounting for candidate generation cost and lost recovery context?** A2.1 Full-KV recovery lets Draft/XAttention retain its selection logic while achieving quality close to Full-Domain with virtually no added generation time relative to the original recovery configuration. Meta Comment C1 includes candidate-generation costs and the measured comparisons.
AuthorsOpenReview ↗

Response of the Review from Reviewer KFNA (Part 2/2)

Q3.1 Which attention characteristics make per-query recovery materially preferable to block-shared recovery?

A3.1 g(i)g(i) controls the granularity of shared recovery weights; per-query recovery is more valuable when queries within a group differ along the recovery block’s mean-key direction. G1/G2 share one/two weight sets, while per-query weights are independent. Following the Flash-Attention-2+ Q-tile outer loop and KV-tile inner loop, per-query recovery executes in the outer loop, outside the inner loop, to control additional cost.

Using the identity underlying Eq. (87) and dividing by d\sqrt d gives the logit deviation for token jj in recovery block bb:

sij−ρr,g(i),bΓ=(qi−qˉr,g(i))⊤kˉbd+qi⊤(kj−kˉb)d.s_{ij}-\rho^\Gamma_{r,g(i),b}=\frac{(q_i-\bar q_{r,g(i)})^\top\bar k_b}{\sqrt d}+\frac{q_i^\top(k_j-\bar k_b)}{\sqrt d}.

The two terms represent query sharing and within-block key variation. Per-query sets qˉr,g(i)=qi\bar q_{r,g(i)}=q_i, eliminating the first term. F.2 Eq. (17) shows that nested partitioning reduces the shared logit-error component, while Corollary 7 links logit bias to output error. The table compares fixed captured Q/K/V tensors and Exact sets:

KG1 Rel-L2per-query Rel-L2Error ReductionG1 Operator Runtime (ms)per-query Operator Runtime (ms)
260.2803320.2745502.06%6.2637286.455200
520.1889670.1853001.94%10.05753610.391408
640.1635610.1604351.91%11.81017612.263360

KV-summary granularity is a separate control axis: in the 720p Table 22 control, KV128→KV32 changes per-query error from 0.208438 to 0.203051 and G1 from 0.222351 to 0.224492.

Q3.2 Do the main quality–latency results substantiate this distinction?

A3.2 Yes. Figure 3 and Appendix Table 13(a) demonstrate stronger per-query Pareto operating points. The table below shows Wan 720p results from Table 13(a) (96 prompts, seed 42); at the same budget, per-query achieves higher AVG5 with a 2.23%–2.78% E2E increase.

720p Sparsity (K)G1 AVG5G1 E2E (s)per-query AVG5per-query E2E (s)E2E Improvement
93.40% (39)0.8158419.3440.8215428.712+2.23%
91.20% (52)0.8186435.9070.8224447.636+2.69%
89.17% (64)0.8210452.5610.8247465.129+2.78%

At K=39, per-query reaches AVG5 0.8215 in 428.712 s, outperforming G1 K=52 (0.8186 in 435.907 s) and K=64 (0.8210 in 452.561 s). Thus fine-grained recovery improves the quality–latency balance at higher sparsity; E2E changes are measured relative to same-budget G1.

W1 R1: This weakness has been addressed in responses to Q1.1–Q1.2: We acknowledge the shared recovery endpoints of PISA-0th under a fixed support and position the added value as Candidate access, controllable recovery, and portable execution.

W2 R2: This weakness has been addressed in Q2.1: C1’s Full-KV recovery results retain candidate selection and achieve quality close to that of Full-Domain; see Meta Comment C1 for exact point estimates and candidate costs.

W3 R3: This weakness has been addressed in Q3.1–Q3.2: Under fixed support, G1/G2/per-query isolate query-sharing errors, and Rel-L2 and 720p Pareto results provide applicable operating points for fine-grained recovery.

Thank the authors for the response. However, the results show that the recovery component is equivalent to PISA-0th under fixed support, and the end-to-end gain is minimal. Candidate integration also brings no benefit over full domain. My concerns about novelty therefore remain, and I will keep my current score.

Reviewer_GFxuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents QUASAR, a training-free sparse-attention framework for accelerating video Diffusion Transformers. It uses mean-pooled query/key representations to select KV blocks for exact computation, either over the full legal domain or within candidates supplied by an external sparse-attention method. Unselected blocks within that domain are approximated using key/value summaries, and their contributions are combined with exact attention under a common softmax normalization. Recovery can share a state across a query block, use smaller query groups, or compute query-specific weights. The paper provides candidate-coverage conditions and attention-output error bounds, together with implementations across multiple attention backends. Experiments primarily on Wan2.1 report substantial recovery gains at fixed sparse selections and end-to-end speedups of up to 1.54× at 480p and 2.41× at 720p, with small decreases in the reported five-metric VBench average. Additional experiments examine hardware transfer, another video model, recovery granularity, and operator costs.

Critical strengths
  1. At 89.84% sparsity on Wan 480p, adding per-query recovery improves AVG5 from 0.7926 to 0.8270 with nearly unchanged reported end-to-end latency (Table 3). The paper also varies recovery granularity while holding captured Q/K/V tensors and exact-computation sets fixed. These comparisons provide more direct evidence for the benefit of recovering omitted contributions than an aggregate speedup comparison alone. The 720p results establish useful faster operating points, including a 2.41× end-to-end speedup with a 0.0053 AVG5 decrease relative to the matched dense baseline.

  2. The evaluation includes online component timings, fixed-selection comparisons across FlexAttention, BSA, and Magi, hardware-transfer experiments, and supplementary independent-seed results. The generation timing includes selection, layout conversion, recovery, and VAE decoding. The analysis also separates within-block approximation, between-block weighting, and exact/recovery branch-mass errors, while Appendix K.12 explicitly accounts for the omitted candidate domain. Together, these results make the framework and its limitations reasonably inspectable and support the practical value of the implementation.

Critical weaknesses
  1. With matched block partitions, exact/recovery sets, valid-token multiplicities, selection domains, and attention scaling, QUASAR’s per-query recovery in Eqs. (6)–(7) is algebraically equivalent to PISA’s explicitly evaluated zeroth-order variant: PISA’s Eqs. (3)–(5) already provide the same denominator and exact-plus-summary numerator. SVG-EAR likewise uses query-specific centroid compensation under a common normalization. QUASAR’s adjustable query-sharing granularity, candidate-coverage analysis, and composable execution interface could nevertheless constitute useful contributions. However, the strongest reported results use the previously established per-query endpoint, while the exact-only ablations primarily demonstrate the general value of recovery. Table 1 includes PISA, but the evaluated approximation variant and relevant implementation settings are not sufficiently specified, and the comparison does not isolate PISA-0th under matched masks and execution conditions. SVG2 is evaluated, but the more directly relevant SVG-EAR is not. Consequently, the current evidence does not clearly establish how much improvement comes from QUASAR-specific additions, rather than an existing approximation combined with different selections, configurations, or implementations.

  2. At matched 79.69% sparsity on Wan 480p, Full-Domain achieves 0.8314 AVG5 in 184.304 s, compared with 0.8249 in 187.234 s for Draft candidates and 0.8264 in 195.215 s for XAttention candidates (Table 1). Table 15 explains this result: candidate construction costs more than the scoring work it saves, and both candidate routes have higher complete attention-call latency at both resolutions. Moreover, near-perfect recall of the Full-Domain exact blocks does not preserve the recovered output, because blocks outside the candidate domain receive no recovery. Table 2 shows appreciable quality variation with candidate size despite approximately 100% selection recall. Appendix K.12 correctly analyzes this additional error, so the concern is not a missing theoretical term. Rather, the demonstrated benefit of this principal component is primarily interface compatibility, while a deployment setting with a net efficiency or quality advantage has not been established. This limits the significance of the candidate interface as a major contribution, particularly given the overlap of the recovery operator with prior work.

References:

[1] Haopeng Li et al. “PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers.”

[2] Xuanyi Zhou et al. “SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing.”

Questions for authors
  1. Which PISA approximation variant, code version, and execution settings were used in Table 1? For matched block partitions and exact/recovery sets, do the authors agree that per-query QUASAR recovery is equivalent to PISA-0th? Please identify the precise additional algorithmic or implementation mechanisms and provide a focused comparison that isolates their benefit, controlling the block geometry, precision, dense prefix, and timing boundary. Separating approximation differences from backend differences would materially affect my assessment.

  2. Given the matched results in Table 1 and the online costs in Table 15, under what evaluated deployment condition is Candidate-Constrained preferable to Full-Domain? Is the intended benefit compatibility with existing candidate providers, reuse of candidates already available at negligible additional cost, or improved scaling? Please distinguish benefits demonstrated by the current experiments from anticipated benefits, taking into account both candidate-construction cost and the loss of recovery outside the candidate domain.

  3. Please reconcile the 96-prompt main evaluations with the 11 suites of eight prompts described for bootstrap analysis in Appendix C, and identify the samples used for the corresponding quality comparisons. Can the authors report the paired confidence intervals already described in the paper for the key near-dense quality gaps and the smaller recovery-granularity gains? Since AVG5 covers five visual dimensions and can average over improvements and regressions, please also clarify the scope of the quality-preservation conclusion, particularly regarding prompt adherence and compositional correctness.

3 comments
AuthorsOpenReview ↗

Response of the Review from Reviewer GFxu(Part 1/2)

Thank you for your detailed review of our paper. The core innovation of QUASAR is a portable, controllable-recovery sparse-attention operator introduced through Candidate integration, together with its adaptation to DiT. Figure 3, Table 5, and Appendix F.3 provide Pareto, granularity, and backend evidence; Meta Comment C1/C2 and the support, Full-KV, and 96-prompt controls establish the corresponding benefit attribution.

Q1.1 Which PISA approximation variant, code version, and execution settings were used in Table 1?

A1.1 Table 1 uses the general configuration from the official PISA implementation, specifically full recovery (non-0th order), version 655648df86cafd75042dc32d6b6be78c4ea0eca8; it uses H200, BF16, block size 64, density 0.15 (85% sparsity), and no bias term. See Q1.2 for the zero-order variant.

Q1.2 For matched block partitions and exact/recovery sets, do the authors agree that per-query QUASAR recovery is equivalent to PISA-0th?

A1.2 The recovery computation for QUASAR per-query Full-Domain is identical to PISA-0th when the support set is fixed; the relative output difference is 0.01287%. The operator design and full execution path are compared in Meta Comment C2.

Q1.3 Please identify the precise additional algorithmic or implementation mechanisms and provide a focused comparison that isolates their benefit, controlling for block geometry, precision, dense prefix, and timing boundaries.

A1.3 Three controlled comparisons follow.

A. Candidate Integration and Independent Recovery. Separating candidate generation, Exact selection, and Recovery lets one operator integrate candidates from different sources. Table 3 compares fixed-Exact results before and after recovery at 89.84% sparsity:

Selection PathExact-only AVG5per-query AVG5AVG5 GainExact-only E2E/sper-query E2E/s
Full-Domain0.79260.8270+0.0344167.832167.957
Draft Candidate0.78030.8144+0.0341172.087173.103
XAttention Candidate0.79210.8180+0.0259185.114185.697

Full-KV recovery extends over all unselected valid blocks while retaining Candidate and Exact selections; see Q2 and Meta Comment C1 for the Full-Domain comparison.

B. Controllable Recovery Granularity. Appendix F.2 fixes the Exact set, KV summaries, and normalization for G1, G2, and per-query weights. The table uses Table 5’s 79.69% sparsity and fixed Wan Q/K/V and Exact sets.

Recovery GranularityNumber of recovery groups per query blockWan capture Rel-L2AVG5E2E/s
G110.18900.8305182.075
G220.18860.8307184.005
per-queryOne group per query0.18530.8314184.304

C. Portable Execution Interfaces and Implementations. Appendices F.1–F.3 define the interface and backend evidence; Meta Comment C2 gives query-block optimizations and matched-capture costs.

Q1.4 Separating approximation differences from backend differences would materially affect my assessment.

A1.4 The same selection and recovery schemes run across Attention backends and integrate different candidates, supporting Full-Domain deployment and Candidate-Constrained reuse. Appendix Table 18 fixes the Exact set; Theorem 1 and Meta Comment C1 support candidate coverage and reuse.

Q2.1 Given the matching results in Table 1 and the online costs in Table 15, under what evaluated deployment condition is Candidate-Constrained preferable to Full-Domain?

A2.1 With an existing Candidate provider, Full-KV recovery retains Draft/XAttention selection while achieving quality close to Full-Domain with virtually no added generation time; operator and E2E results are in Meta Comment C1.

Q2.2 Is the intended benefit compatibility with existing candidate providers, reuse of candidates already available at negligible additional cost, or improved scaling?

A2.2 The verified benefits are provider compatibility, candidate reuse, and quality improvements from independent recovery. Draft/XAttention produces CC within one call; QUASAR selects MM and recovers on Ω∖M\Omega\setminus M, with candidate-generation cost included. Eqs. (1)–(2), Table 15, and Theorem 1 connect scoring range, construction cost, and coverage.

Q2.3 Please distinguish the benefits demonstrated by the current experiments from anticipated benefits, taking into account both the candidate-construction cost and the loss of recovery outside the candidate domain.

A2.3 Meta Comment C1 reports operator and E2E results before and after Full-KV recovery, including candidate-construction costs and omitted candidate-domain contributions.

AuthorsOpenReview ↗

Response of the Review from Reviewer GFxu (Part 2/2)

Q3.1 Please reconcile the 96-prompt main evaluations with the 11 suites of eight prompts described for bootstrap analysis in Appendix C, and identify the samples used for the corresponding quality comparisons.

A3.1 The main evaluations, quality comparisons, and paired statistics all use the same set of 96 prompts: 8 prompts for each of the 11 VBench dimensions, plus 8 held-out prompts randomly selected from other dimensions.

Q3.2 Can the authors report the paired confidence intervals already described in the paper for the key near-dense quality gaps and the smaller recovery-granularity gains?

A3.2 Using the same 96 prompts, we performed 10,000 with-replacement paired resamples within each layer and used the 2.5th and 97.5th percentiles of the mean difference as the 95% CI.

① Figure R1 reports per-query−Dense differences for the three shared budgets at 480p/720p and per-query−G1 gains at 720p; Dense AVG5 is 0.834713 and 0.826837, respectively. The 720p values are:

Exact Budgetper-query − Dense95% CIper-query − G195% CI
K39−0.005331[−0.010760, −0.000056]+0.005718[+0.002889, +0.008616]
K52−0.004436[−0.009610, +0.000599]+0.003794[+0.000703, +0.007045]
K64−0.002166[−0.007334, +0.002922]+0.003662[+0.001096, +0.006285]

Supplementary Figure R1. Points are paired ΔAVG5 values with 95% CI error bars; the three panels show the comparisons above. K is the number of Exact blocks, and G1 shares one recovery state per query block. All three 720p per-query−G1 intervals are above zero.

② Complete pipeline matching results (AVG5 with five metrics equally weighted; E2E timing covers the entire pipeline):

ComparisonΔAVG595% CIΔE2E/s95% CI
QUASAR matched − PISA-0th−0.000233[−0.000830, +0.000356]−3.922[−3.998, −3.848]
Full-Domain − PISA-0th−0.001914[−0.003286, −0.000578]−5.802[−5.862, −5.743]
Full-Domain − QUASAR matched−0.001681[−0.002997, −0.000416]−1.880[−1.939, −1.818]
Full-Domain − SVG-EAR−0.000629[−0.002080, +0.000787]−119.314[−119.976, −118.703]
SVG-EAR − PISA-0th−0.001285[−0.003026, +0.000432]+113.512[+112.909, +114.169]

The paired intervals support acceleration. Delay intervals for QUASAR matched−PISA-0th and Full-Domain−SVG-EAR are entirely below zero, while the corresponding AVG5 intervals include zero.

Q3.3 Since AVG5 covers five visual dimensions and can average improvements and regressions, please also clarify the scope of the quality-preservation conclusion, particularly regarding prompt adherence and compositional correctness.

A3.3 C1 reports the five AVG5 metrics and Dynamic Degree separately. We additionally evaluated Overall Consistency and Temporal Flickering under the paper’s VBench protocol; 480p results are:

MethodOverall Consistency ↑Temporal Flickering ↑
Dense0.2390640.986623
QUASAR Full-Domain K390.2444520.985091
QUASAR Full-Domain K520.2435410.985037
Draft Candidate0.2437020.985043
XAttention Candidate0.2428850.985172
PISA-0th0.2410590.986855
SVG-EAR0.2388820.986569

QUASAR’s Overall Consistency exceeds PISA-0th, SVG-EAR, and Dense; see Figure 4 and the appendix.

W1 R1: This weakness has been addressed in Q1.2–Q1.4: Meta Comment C2 first identifies the shared recovery endpoints of PISA-0th, then demonstrates the QUASAR-specific additions through controlled experiments involving Candidate integration, multi-granularity recovery, and a portable backend.

W2 R2: This weakness has been addressed in Q2.1–Q2.3: Meta Comment C1 presents Full-KV recovery results that account for candidate costs, demonstrating the quality of recovery and the value of route reuse after candidate incorporation; specific point estimates are provided in C1 above.

The rebuttal clarifies the relationship to PISA-0th and addresses my questions about the evaluation. However, the matched comparison shows a modest 1.9% reduction in end-to-end latency, and the candidate interface still does not show a clear quality-latency advantage over Full-Domain. I appreciate the implementation work, but remain unconvinced that these additions make a strong enough contribution. I will therefore keep my score at 2 (weak rejection).

Comments on the paper

2 comments
AuthorsOpenReview ↗

Meta Comment for the Common Questions of the Reviewers (Part 1/2)

(Part 1/2 of Meta Comment)

We thank the reviewers for their feedback. This response addresses questions raised by multiple reviewers. We invite the PC, AC and the reviewers to examine these clarifications; the relevant content and analysis will be incorporated into the appropriate sections of the revised manuscript.

C1 Candidate-Constrained vs. Full-Domain

In response to concerns from three reviewers (Q2.3 from Reviewer GFxu, Q2.1 from Reviewer KFNA, and Q2.1 from Reviewer japD) regarding the recovery coverage and deployment benefits of the "Candidate-Constrained" (CC) approach, we have conducted additional experiments using a "Full-KV" recovery strategy. These results demonstrate that it is possible to achieve recovery performance comparable to the Full-Domain(FD) approach while still retaining the candidate selection process.

Let CC denote the set of candidate blocks, M⊆CM \subseteq C the set of blocks selected for exact attention computation, and Ω\Omega the set of all KV blocks (the full domain). The recovery coverage for the full-domain approach is Ω∖M=(C∖M)∪(Ω∖C)\Omega \setminus M = (C \setminus M) \cup (\Omega \setminus C), while maintaining the original logic for candidate and exact selection. Given consistent settings for block partitioning, summarization, recovery granularity, and normalization, we have:

M_r^{\mathrm{CC}}=M_r^{\mathrm{FD}},\qquad R_r^{\mathrm{CC}}=R_r^{\mathrm{FD}} =\Omega_\Gamma\setminus M_r^{\mathrm{FD}} \\ \Longrightarrow\quad \widehat O_i^{\mathrm{CC}}=\widehat O_i^{\mathrm{FD}}. \end{gathered}$$ In the table below, $C \setminus M$ represents recovery within the original candidate domain, while $\Omega \setminus M$ represents recovery across the full KV domain. **Operator Results.** Each cell in the table below shows the execution time (in ms) and the Rel-L2 error (evaluated on an H200 GPU using the Wan 480p dataset). | | C∖M | Ω∖M | |---|---:|---:| | Full-Domain | 11.0603/0.185249 | 11.0020/0.185249 | | Draft | 11.4586/0.189781 | 11.5461/0.185251 | | XAttention | 11.6635/0.189740 | 11.5701 /0.185247 | **E2E and VBench results.** 96 prompts, seed 42, H200, sparsity approximately 80% (K52); timing boundaries are the same as in Table 1. | Configuration | AVG5 | Subject | Background | Motion | Aesthetic | Imaging | Dynamic | E2E/s | |---|---:|---:|---:|---:|---:|---:|---:|---:| | Full-Domain K52 | 0.831410 | 0.965059 | 0.966537 | 0.991573 | 0.552620 | 0.681261 | 0.416667 | 184.304 | | Draft, Original Recovery $C \setminus M$ | 0.826205 | 0.957470 | 0.963131 | 0.990838 | 0.553280 | 0.666305 | 0.437500 | 184.881 | | Draft, Full-KV Recovery $\Omega \setminus M$ | 0.830481 | 0.964733 | 0.964696 | 0.991629 | 0.551978 | 0.679368 | 0.406250 | **184.057** | | XAttention, Original Recovery $C \setminus M$ | 0.826537 | 0.958828 | 0.961846 | 0.991365 | 0.552465 | 0.668180 | 0.416667 | 197.994 | | XAttention, Full-KV Recovery $\Omega \setminus M$ | **0.831439** | 0.965745 | 0.965690 | 0.991701 | 0.554894 | 0.679166 | 0.406250 | 196.400 | **From attention operator output to DiT generation, Full-KV candidate recovery achieves results nearly identical to those of the FD approach, while preserving the original candidate generation and exact selection logic.** Consequently, existing sparse attention methods can retain their own selection designs while benefiting from full recovery.
AuthorsOpenReview ↗

Meta Comment for the Common Questions of the Reviewers (Part 2/2)

(Part 2/2 of Meta Comment) C2: Advantages of QUASAR (per-query) over zero-order SOTA methods

The core innovation of QUASAR lies in its introduction of an attention mechanism for portable candidate, controllable recovery within sparse attention operators—along with its adaptation for DiT. All three approaches incorporate query-specific compensation for imprecisely computed regions, but QUASAR integrates external candidate input, recovery granularity control, and exact/recovery states into a composable sparse attention operator. Here, “query-specific compensation” refers to the calculation of compensation for imprecise regions based on the query.

Algorithm Differences.

Comparison ItemPISA-0th[1]SVG-EAR[2]QUASAR
Selection Domain/RoutingPISA Routing ScoreClustering + error-aware routingFull-Domain / Candidate + mean pooling Q/K (query/key) Top-K
Recovery RepresentationExact + zero-order summaryV-centroid compensationValue-block summary + merged exact/recovery states (see Eq. (7))
Query Sharing Granularityper-queryper-queryG1 / G2 / per-query (optional; Table 5 & Appendix F.2)
Recovery Granularityper-queryper-queryG1 / G2 / per-query ($G_{\mathrm{rec}}=1/2/
Execution CombinationPISA-specific pathCluster/reorder + FlashInfer/TritonDecoupling of candidate, exact executor, and backend
Key CapabilitiesZero-order recovery endpointsError-aware center-of-mass routingPortable, controllable repair after candidate integration

Results consistent with the protocol. All Rel-L2 values below refer to the relative output error compared to Dense.

Comparison ItemPISA-0thSVG-EARQUASAR
Proportion of Exact Computations20.31%17.08%20.31%
Wan capture Rel-L20.1755840.0322980.175744
attention call median12.280 ms38.307 ms11.422 ms
96-prompt E2E/Vbench AVG5205.169 s / 0.835443318.681 s / 0.834158201.247 s / 0.835210

Figure 3 shows the Pareto results for the three per-query approaches: Full-Domain, Draft Candidate, and XAttention Candidate; Appendix F.2/F.3 provides the interface design and modular verification.

The implementation employs three pipeline optimizations for query block operators: ① Concatenating the selected block indices directly to form the execution range; ② Having the computation kernel write directly to the output buffer held by the caller; ③ Compressing variable-length indices only once, with multiple fields of the execution descriptor reusing this result.

Below, with fixed input and 128-token blocks in the original token order, using the Exact budget, full-domain recovery is employed. The first two rows use the same fixed Exact set and count only Attention computation and restoration; the last two rows include their respective block-selection costs. Rel-L2 represents the relative Dense output error, and the execution time is the median of 20 CUDA event timings.

Timing RangeExecutorRel-L2Full Call (ms)
block selection not includedPISA-0th0.17558410.80
block selection not includedQUASAR (Magi)0.17558510.73
Block selection phase includedPISA-0th0.17558412.35
Block selection phase includedQUASAR (Magi)0.17574411.48

When support is fixed, the error and execution time of the two methods are similar; After block selection, QUASAR’s full execution reduces the time by 0.87 ms (approximately 7%), while the output error remains essentially unchanged, demonstrating the efficiency advantage of the full execution approach.

96-prompt E2E results. QUASAR’s matched recovery employs the same block division and repair logic as PISA-0th, using QUASAR’s per-query block selection logic.

ApproachAVG5E2E/s
QUASAR Full-Domain0.833529199.367
QUASAR matched recovery0.835210201.247
SVG-EAR0.834158318.681
PISA-0th0.835443205.169

After aligning with the PISA-0th configuration, the operator efficiency advantage further translates into E2E acceleration.

References:

[1] Haopeng Li et al. “PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers.”

[2] Xuanyi Zhou et al. “SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing.”

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.