- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the memory overhead of recurrent-state checkpoints in hybrid linear-attention LLM serving, where storing many checkpoints for prefix reuse causes cache eviction and expensive recomputation. It proposes WEAVE, which stores checkpoints as low-rank residuals from a shared anchor and constructs them from the model’s native state updates. Experiments on two hybrid linear-attention LLMs show up to 4.26× checkpoint compression with minimal quality degradation, along with up to 6.96× lower TTFT and 3.14× higher token goodput.
Critical strengths
- The paper addresses a practical problem in hybrid linear-attention LLM serving, where recurrent-state checkpoints consume substantial cache memory. WEAVE stores a shared anchor and low-rank residuals for subsequent checkpoints, constructed from the model’s native state updates.
- The evaluation simulates prefix-cache reuse by repeatedly restoring checkpoints and continuing inference. WEAVE achieves over 4× compression with little quality loss on both GDN (Qwen3.5) and KDA (Kimi-Linear) models, while maintaining higher task scores than the INT4 baselines.
- The implementation includes vLLM integration and optimized GPU kernels. End-to-end serving experiments show lower TTFT and higher goodput than uncompressed checkpoints at the same checkpoint density under memory pressure.
Critical weaknesses
- The paper does not compare against existing prefix caching methods for hybrid models, such as cache admission/eviction or checkpoint placement, which address the same memory pressure. The main baselines are only uncompressed checkpoints and simple quantization, so it is unclear how WEAVE compares to prior work.
- The quantization baselines are limited to basic per-column INT8 and INT4 schemes, so the comparison does not show whether WEAVE outperforms a stronger quantization method.
Questions for authors
Could the authors add comparisons with other baselines, such as existing prefix caching methods for hybrid models and stronger quantization schemes? This would make it clearer how WEAVE performs relative to existing approaches. A brief discussion of the concurrent work DASC, which also compresses GDN/KDA checkpoints, would also be helpful.
Minor: Some figures (e.g., Figures 1, 2, and 5) are hard to read because of their small fonts and dense panels. Please improve them in the future versions.
2 comments
Thanks for the additional experiments. The comparisons with Marconi, stronger INT4 baselines, and DASC help address my main concerns. I appreciate the authors' efforts and will keep my rating at 3.
Dear Reviewer, Just a gentle follow-up on our rebuttal. We've included additional experiments on stronger caching and quantization baselines. We'd be happy to hear your thoughts when you have a chance. Thank you!