- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies attention scheduling when requests have different sequence lengths. TailorAttention changes the kernel to reduce chunk-boundary overhead and make chunk latency more predictable. It then uses a latency model to choose chunk sizes and assign work to CTAs, including the cost of merging partial results. The paper compares with FlashAttention and FlashInfer on Hopper and Blackwell, includes kernel/scheduler ablations, and shows a serving improvement in SGLang.
Critical strengths
-
I like the connection between the kernel and scheduler. Making the kernel faster is useful by itself, but making its latency easier to predict also helps the scheduling decision. The discussion of V/O buffers and the transposed computation makes the idea fairly clear.
-
The evaluation covers several query lengths, GQA settings, and workload mixes, using strong kernel baselines. The kernel-only versus full-system comparison is helpful. I think there is already a useful contribution here, even though a same-kernel scheduling comparison would make the source of the gains clearer.
Critical weaknesses
There are one detail I could not follow in the serving setup. Section 5 says the weights are BF16, but Section 5.4 runs gpt-oss-120b on one H100. The official model card lists 117B parameters and MXFP4 MoE weights for single-GPU deployment, so a fully resident BF16 copy would not fit the stated memory. Is BF16 here only referring to attention? This may be a wording issue, but the actual formats and residency need to be clear. It would also be good to report numerical agreement with the reference attention implementation.
For the theory, my understanding is that Theorem 1 bounds the processing makespan T, not the full T+R objective. Please make the scope consistent in the surrounding text. I do not see this as invalidating the kernel results; it just needs a more precise claim.
Questions for authors
-
What weight and KV formats are actually used for the single-H100 gpt-oss-120b run? How closely do the attention outputs match the reference implementation?
-
Does the reported serving time include building and transferring the schedule? Please also clarify the scope of the bound. If you already have results for a length-based scheduler on the same TA kernel, they would be useful to see.
Author-reviewer discussion period is ending soon
Dear Reviewer HYzr,
If you haven't done so, please read through authors' response and let authors know if you need further clarification or have more questions. Beginning 11/19, authors are not allowed to respond any more, so this is good time to review the response and ask more.
Thank you.
-AC