- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper draws on the property of low correlation between distant pixel patches in natural images, and proposes to pool distant token patches within the attention mechanism to reduce the computational cost of attention. To this end, the authors design a hierarchical pooling strategy and apply it to image generation, achieving a notable speedup compared with existing attention mechanisms.
Critical strengths
The phenomenon observed in the paper (the interaction strength between distant tokens is significantly lower than that between adjacent tokens) is intuitive and consistent with common sense. The proposed scheme is also intuitive and achieves noticeable performance improvements in some experiments.
Critical weaknesses
-
The presentation of the paper is disorganized and appears to be AI-generated. The paper does not seem to be carefully prepared, organized or formatted. I even doubt whether the authors have read through their own manuscript. Some notable issues are listed below: a) Panels (b)/(c) of Figure 1 appear before Figure 2(c)/(d) in layout, but their citations in the main text come much later than those for Figure 2(c)/(d). Figure 2(c)/(d) is presented in the introduction section (page 2), while Figure 1(b)/(c) only appears in the work section (page 4). This arrangement is inconsistent with the reading flow for readers. b) Figure 1(b)/(c) contain numerous terms that are undefined in the main text and must be looked up in the appendix. Furthermore, one panel shows theoretical results and the other shows experimental measurements, yet they use entirely different terminology with substantially divergent definitions in the appendix. It is unclear why separate terms are introduced for theoretical and measured results, which violates standard writing conventions. c) The plot in Figure 1(d) is asymmetric. Neither the caption nor the main text provides a detailed description of the setup or the rationale for this asymmetry. d) There are excessive and abnormal appendix citations throughout the main text. Three or more citations pointing to different appendices appear on nearly every page, severely hindering readability. e) The paper contains an excessively large appendix packed with abundant terminology and descriptions, even though the core of the proposed method is not complicated and can be fully understood from Figure 1(e). This raises suspicion that most of the appendix content was generated by an LLM. f) The abstract cites experiments on SD3.5 as supporting evidence, yet these experiments are not included in the main manuscript.
-
In addition, there are flaws in the authors’ experiments: a) Comparisons against state-of-the-art methods such as REPA and JiT are missing. b) The speed evaluation is problematic. The claimed 200×+ speedup for 256k resolution in forward/backward passes is misleading, as no practical model generates images at 256K resolution. The actual speed values shown in Table 1 have only minor differences compared with the full baseline.
Questions for authors
See above.