- Rating
- 2
- Confidence
- 2(+1)
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This work presents a library called LeetDecoding that implements a set of causal linear attention methods. The authors carefully analyze the complexity of each method and conduct a comprehensive evaluation of these methods in terms of scalability, latency, and numerical stability. Experiments show that each method has its own advantages and no single method is superior in all aspects.
Critical strengths
- This work implements a set of causal linear attention methods and performs a unified evaluation of these methods.
- The proposed library is easy to use and can be a useful tool for researchers and practitioners in the field of efficient transformers.
Critical weaknesses
Disclaimer: I am not an expert in this area, and my comments are only an educated guess based on my understanding of the paper.
- Under the topic of linear attention, I am wondering what is the strength of the leetdecoding library compared to other famous existing libraries such as fla [1]. The only comparison I can find is in Table 13, where the fla implementation has the similar performance as the leetdecoding implementation. But in main results I cannot find any further comparison with fla. As far as I know, fla implements most of the well-known linear attention methods and is also a highly optimized library. I am wondering if the authors can provide more discussion on the comparison with fla, and what is the strength of leetdecoding compared to it. If there is no significant strength, I do not think this work has enough contribution to be accepted as a paper in ICLR.
- I feel like this work does not bring much new insights to the field of linear attention. One thing that the authors emphasize is that Cdotp is faster than LA under RTX5090 with fp32 and batch size 16, with an accelerate of about 1.2x (Section C.1). I can hardly say that this is a significant contribution to the field -- this is only a narrow observation on a specific hardware and a specific setting.
[1] FLA: A Triton-Based Library for Hardware-Efficient Implementations of Linear Attention Mechanism. Yang, Songlin and Zhang, Yu
Questions for authors
See the weakness section.
4 comments
Follow-up on Response
Dear Reviewer XrES,
As the discussion period draws to a close in roughly two days, we are following up to check whether our rebuttal has addressed your questions regarding our contributions relative to FLA.
Specifically, we detailed how LeetDecoding complements FLA's horizontal architectural breadth with vertical cross-paradigm depth (where our optimized kernel also outperforms FLA's retention baseline in Table 13) and explained how our findings challenge the assumption that chunk-wise methods universally dominate. We remain fully available for any further clarifications and would greatly appreciate your feedback before the interaction window closes.
Response to Authors
I thank the authors for their dedicated explanation. I fully understand that the contribution of this work is to propose a linear attention benchmarking library, not to provide a fast production library. The authors draw the conclusion "no single method universally dominates across operational contexts" based on a specific configuration (RTX 5090, FP32, batch size 16) and recogize this as a meaningful contribution.
I believe this work is sound and the authors faithfully report the result. But instead of "no single method universally dominates across operational contexts", the experiments actually show that in general chunk-wise implementations are indeed the optimal. This perhaps is a contribution, but I do not think this brings insight to the majority of the community and is enough to accepted as a conference paper in ICLR. After consulting experts in this area, I decide to raise the soundness score from 2 to 3 and raise confidence from 1 to 2. I respectfully suggest the authors consider publishing this paper as a workshop paper or at venues like AACL or EACL.
Dear Reviewer XrES,
We thank you for engaging in the discussion and for raising the soundness score of our work to 3, confirming the rigor of our theoretical derivations and empirical evaluations. However, we respectfully correct two critical mischaracterizations in your concluding remarks:
1. Reducing a multi-dimensional benchmark to a single hardware configuration.
Our conclusions do not rest solely on the RTX 5090 FP32 setting. Even where chunk-wise implementations hold a broad latency advantage, scientifically establishing its operational boundaries (rather than treating it as folklore) requires a controlled cross-paradigm benchmark. More importantly, focusing purely on raw latency overlooks the pillars of our study:
- Theoretical foundation: Unifying disparate paradigms under an explicit non-iterative matrix formulation and mathematically proving the suboptimal complexity of recursive splitting.
- Intra-paradigm system insights: Demonstrating that hardware-aware memory prefetching and framework choices alter latency by over 50% for an identical mathematical algorithm.
- Model-level numerical fidelity: Exposing that mathematically equivalent kernels can induce severe perplexity degradation (e.g., PPL rising from 14.02 to 333.30) and up to 92% next-token disagreement when substituted into pretrained LLMs without recovery training.
2. Misalignment of the suggested venues.
This work investigates low-level CUDA/Triton kernel engineering, hardware memory hierarchies, asymptotic time-space complexity, and floating-point numerical stability. Suggesting computational linguistics venues such as AACL or EACL reflects a fundamental mismatch with the technical nature of our contribution. Rigorous algorithmic taxonomies and hardware-aware operator benchmarks sit squarely within core machine learning and the explicit scope of the Datasets and Benchmarks track.
We appreciate the time you took to review our submission and your acknowledgment of its technical soundness.
We appreciate your useful comments on the contribution of our work. By further highlighting the comparative evaluation and its practical implications, we hope to address your concerns.
W1: Strength relative to FLA
We respectfully refer the reviewer to General Response R1 for a comprehensive discussion.
To directly answer your question regarding our core strength: our contribution is establishing a rigorous scientific benchmark across fundamentally different algorithmic paradigms, rather than merely engineering a single faster production library.
While FLA provides horizontal architectural breadth (primarily chunk-wise kernels), LeetDecoding provides vertical algorithmic depth by unifying token-wise CUDA recurrence (
Cdotp), VQ-style block methods (Bb), and O(NlogN) recursive splitting (Recur) alongside end-to-end prefill breakdowns and zero-shot numerical evaluations. Furthermore, in Table 13, our optimized block-wise kernel (LA) performs slightly faster than FLA's retention reference (LA_ref, 351.92 s vs. 366.88 s at batch size 1), confirming that our implementation is competitive with production baselines.W2: Limited insight
We kindly refer the reviewer to General Response R2. The reviewer observes that the 1.2x advantage of Cdotp over LA is tied to a specific configuration (RTX 5090, FP32, batch size 16). However, we respectfully argue that identifying this specific regime constitutes a meaningful contribution.
Because chunk-wise implementations are assumed to be universally optimal, demonstrating a specific regime where a token-wise CUDA kernel outperforms an optimized chunk-wise Triton kernel underscores the value of context-aware benchmarking. We will revise the discussion to frame this clearly within our broader trade-off analysis.