- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Cross-Layer Sparse Attention (CLSA), which computes top-k routing once and shares the resulting indices across layers in the YOCO architecture, avoiding repeated per-layer routing computation. The indexer is adapted to the model through a two-stage training procedure. The method reduces the overhead of proxy score computation while preserving the architectural advantages of YOCO. Experimental results demonstrate substantial speedups with limited performance degradation.
Critical strengths
- The paper explores a novel and timely research setting. Given the growing interest in the YOCO architecture, improving its efficiency is a meaningful contribution.
Critical weaknesses
-
The methodological novelty appears limited. Cross-layer reuse of routing indices has already been explored in Transformer-based sparse attention methods, such as IndexCache and HySparse. The paper primarily adapts this idea to the YOCO architecture, without sufficiently establishing the additional technical novelty beyond this architectural adaptation.
-
The two-stage training procedure follows a conventional warmup-and-joint-adaptation recipe used in prior architectures. Moreover, the experiments do not isolate the contributions of S1 and S2, leaving the necessity of indexer warmup and the incremental benefit of joint adaptation unclear.
Questions for authors
-
What additional technical novelty does CLSA introduce beyond adapting existing cross-layer routing reuse to YOCO? How does sharing one routing index across all cross-decoder layers compare with sharing an index every layers in terms of model quality and inference efficiency?
-
Could the authors provide ablation studies to clarify the individual contributions of the S1 and S2 training stages?