- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The main claim of this paper is to use SSDs to improve the performance of LLM systems.
To avoid the garbage collection operations may interfere with LLM workload, Zoned Namespace (ZNS) SSDs was propose to reduce migration by exposing data placement and space reclamation to users.
The authors proposed KVReclaim, a host–SSD coordinated KV-cache management framework for ZNS SSDs that takes advantage of KV semantics across the storage system. In particular, KVReclaim manages KV states at the conversation granularity. It flushes the data into zones according to their runtime access behavior and recomputation overhead. Due to these operations, during garbage collection instant, the system can directly discardsselected KV states instead of migrating so to reduce GC-induced overheads.
Critical strengths
The strength of this paper is in presenting a simple algorithm to improve performance of LLM systems by reducing the amount of GC's overheads during garbage collection time.
Critical weaknesses
The major weaknesses include:
- The proposed idea of using zone-based SSDs and techniques in reducing GC's overhead is quite simple and easily done. So there is a lack of novelty in the work since there was no clever algorithm or smart data-structures were proposed.
- The performance of KVReclaim hinges on the precision of state conversation granularity. The authors didn't illustrate the impact and pitfalls of such.
- The evaluation is poor since the authors only compared with some simple SSD's GC algorithms, while there are many clever GC algorithms out there.
- What is the relationship between the ZNS backend and layout gains?
Questions for authors
Please address the following:
- The proposed idea of using zone-based SSDs and techniques in reducing GC's overhead is quite simple and easily done. So there is a lack of novelty in the work since there was no clever algorithm or smart data-structures were proposed.
- The performance of KVReclaim hinges on the precision of state conversation granularity. The authors didn't illustrate the impact and pitfalls of such.
- The evaluation is poor since the authors only compared with some simple SSD's GC algorithms, while there are many clever GC algorithms out there.
Response to Reviewer rMbe for Novelty, Conversation-Level Granularity, and Advanced GC Comparison
Concluding remarks: The main claim of this paper is to use SSDs to improve the performance of LLM systems. To avoid the garbage collection operations may interfere with LLM workload, Zoned Namespace (ZNS) SSDs was propose to reduce migration by exposing data placement and space reclamation to users. The authors proposed KVReclaim, a host–SSD coordinated KV-cache management framework for ZNS SSDs that takes advantage of KV semantics across the storage system. In particular, KVReclaim manages KV states at the conversation granularity. It flushes the data into zones according to their runtime access behavior and recomputation overhead. Due to these operations, during garbage collection instant, the system can directly discardsselected KV states instead of migrating so to reduce GC-induced overheads.
Thank you very much for your time and effort in reviewing our manuscript and for your valuable comments and suggestions.
We have submitted a revised version of the manuscript incorporating our responses to the reviewers' comments.
Question 1: What is the novelty of KVReclaim beyond using zone-based SSDs to reduce GC overhead? What new algorithms or data structures distinguish it from a straightforward implementation?
Thank you for this insightful question. ZNS provides host-side control over SSD data placement and space reclamation, while leaving application-specific data management and optimization to the host software. It does not itself provide KV-aware management policies. A straightforward approach is to organize KV cache data by conversation to reduce data migration during subsequent garbage collection.
Compared to straightforward ZNS implementation, we emphasize that KVReclaim contributes coordinated placement, eviction, and recovery policies Its DropCost-aware placement groups conversations with similar eviction costs into the same placement cohort. Then, it couples garbage collection with conversation-level KV eviction, directly discarding low-cost KV states to reclaim space while reducing page migration. Furthermore, its adaptive recovery policy selects between SSD reads and KV recomputation based on their estimated runtime costs, reducing KV recovery overhead. These policies jointly exploit the regenerability of KV states to improve storage management. The results of ablation study verities the efficitiveness of KVReclaim compared to the naive ZNS implementation. We have revised the manuscript by adding more description to help reader better understand our contribution.
Please refer to Sections 3.2–3.4 and 4.3.3 at pp. 5–8 and pp. 9–10, respectively.
Question 2: How does the precision of conversation-level state granularity affect KVReclaim's performance? What are the impacts and potential pitfalls of this granularity?
Thank you for this insightful question. Conversation-level granularity defines the unit of KV eviction and invalidation. Invalidating a selected conversation’s cached KV across its occupied zones can make one or more zones directly resettable without page migration. The trade-off is reduced flexibility in retaining individual KV states: evicting a long conversation may discard KV that will soon be reused, increasing subsequent recomputation. KVReclaim accounts for this cost through DropCost, which combines access activity and recomputation overhead. Besides, DropCost-aware placement groups conversations with similar eviction costs, and ReclaimScore balances the aggregate eviction cost against the reclamation benefit. Recovery operates on the required KV range, so conversation-level invalidation does not necessarily require recomputing the entire conversation. These mechanisms balance migration savings against the cost of regenerating discarded KV.
Question 3: Why is KVReclaim compared only with simple SSD garbage-collection algorithms? How does it perform against more advanced GC algorithms?
Thank you for this helpful suggestion. Advanced GC optimizations, such as low-I/O-intensity-aware partial GC scheduling [1], reduce interference with foreground I/O by scheduling GC operations during periods of low I/O activity. In contrast, KVReclaim incorporates KV eviction and recomputation costs into reclamation decisions, directly discarding selected KV states and avoiding valid KV migration during GC.
Following your suggestion, we have extended our experiments to include this advanced GC scheduling scheme, integrating it into both CachedAttention and FlashGen. The results show that KVReclaim achieves the lowest mean E2E recovery latency in 11 of the 12 evaluated configurations, demonstrating the effectiveness of KV-aware reclamation beyond GC scheduling optimizations. We have revised the manuscript to include these comparisons.
Please refer to Appendix A.3.4 and Figure 4 for more details.
[1] Z. Sha et al. Low I/O Intensity-aware Partial GC Scheduling to Reduce Long-tail Latency in SSDs. ACM Transactions on Architecture and Code Optimization, 18(4).