- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
SPARE proposes a training-free sparse attention method for few-step diffusion transformers. The authors select the attention support by combining top-k sparse attention with randomly selected keys and demonstrate that it achieves improved performance in video-generation and other video-related tasks under a fixed sparse budget.
Critical strengths
- The paper addresses an important problem of efficiency in diffusion models.
- The method is plug-and-play and allows to use ready pretrained models.
- The method itself is very simple and the results are somewhat convincing over the baselines that the authors compared to.
Critical weaknesses
- Limited evaluation against other sparse attention methods. The paper mostly compares to SLA, but there are several methods that use quite similar approaches, like DFSAttn, SpargeAttention or XAttention.
- Hence, the research gap that the authors actually address is not clear, because a lot of the claims seem to be covered by the previous work.
- The paper's writing is extremely hard to follow. The structure of the paragraphs is chaotic.
Questions for authors
- Could you compare to the baselines I mentioned (or explain why these are not comparable if you think this is the case) and other relevant plug-and-play sparse attention baselines for video generation?
- Could you clearly state the scientific claim the paper is presupposing and explain how it was not covered before SPARE and how your results demonstrate it?