- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
ACache enables cross-request KV cache reuse for shared text spans—including prefixes, infixes, and suffixes—in diffusion language models with bidirectional attention. It selects a small subset of Anchor Tokens for recomputation using mask-to-affix attention, while sharing the remaining KV states across requests. Integrated with dKV-Cache and Fast-dLLM, ACache reduces recomputation latency by up to 56.7%, achieves up to 1.71× end-to-end throughput, and reduces peak KV cache memory by up to 45.8% in the main few-shot experiments. Accuracy preservation varies across models and tasks.
Critical strengths
- The method extends cross-request cache reuse to different text positions in DLLMs, addressing the request-dependent nature of shared KV states under bidirectional attention.
- The system design has practical value: the read-slot map combines shared and private KV states without duplicating the entire affix cache for each request.
- The evaluation includes accuracy–efficiency trade-offs, Anchor stability analysis, and random-selection ablations, supporting the method’s effectiveness in the evaluated settings.
Critical weaknesses
- The algorithmic novelty needs further clarification. CacheBlend already explores selective recomputation, MaskKV uses mask-based attention scores for token importance, and concurrent work BiCache studies cross-request prefix reuse in DLLMs. Although ACache contributes support for different affix positions and a practical system design, the additional novelty of its core selection strategy remains insufficiently clear.
- The evaluation scope remains limited. Experiments cover only 7B/8B models, generally use 256-token generation, and reuse a fixed shared affix within each configuration. Effectiveness on larger models, longer generations, and serving workloads with multiple affixes or low cache hit rates remains unverified.
Questions for authors
- What is ACache’s key algorithmic contribution beyond CacheBlend’s selective recomputation, MaskKV’s mask-based attention scoring, and BiCache’s cross-request cache reuse?
- Why should mask-to-affix attention magnitude predict the benefit of recomputing a token? Could you validate the relationship between the scores and actual accuracy recovery, and justify normalizing only over affix tokens and aggregating equally across layers and heads?
- The current findings mainly concern 7B/8B models and 256-token generation. Does selecting Anchors only once remain reliable for larger models, longer contexts, and longer generations? Could you compare the quality and overhead of fixed versus periodically updated Anchors?
- With multiple affixes, varying cache hit rates, and cold starts, how many requests must reuse an affix to achieve a net benefit after accounting for cache precomputation? Could you characterize the break-even conditions in terms of shared-span length, concurrency, and cache hit rate?