- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MaskAhead, a training-free KV-cache management method for block diffusion language models. Its central observation is that the model's own mask-token queries can inform two distinct decisions: temporary cache selection for the current block and irreversible cache eviction for future blocks. MaskAhead uses current-block queries for sparse reads and a no-write probe of upcoming masked blocks to estimate which prefix entries should be retained. Both decisions employ a CAOTE-inspired value-aware attention score. The authors further introduce Q-MaskAhead, which combines recurrent eviction with four-bit KV quantization, performing both ranking and attention directly from the quantized cache.
Experiments on Fast-dLLM-v2-7B, supplemented by preliminary cross-model results, cover mathematical reasoning, long-context question answering, retrieval, and serving efficiency. The method achieves substantial KV-memory reductions while retaining much, but not all, of dense-cache performance. The strongest gains occur under restrictive retention budgets, whereas competing methods become competitive or superior at wider budgets.
Critical strengths
I think one strength of the paper is that it makes a useful distinction between reducing cache reads and reducing memory storage. The proposed method uses information from both current and future blocks to make these decisions separately. I find the idea of looking ahead without storing extra KV states particularly interesting. It also doesn't require additional training or a separate prediction model, which makes the approach quite practical.
Another strength is that the paper provides fairly detailed experiments and ablation studies. The authors compare different query sources, scoring methods, memory budgets, and quantization settings. One encouraging result is that, with a cache capacity of 128, MaskAhead achieves 51% accuracy on MATH-500, compared with 39.4% for H2O. The authors also report statistical evidence supporting this improvement. I also appreciate that they discuss negative results, including limited benefits from value-aware scoring and weaker performance on some other models. Although I'm not an expert in this specific area, I think this makes the evaluation more balanced and convincing.
Critical weaknesses
My first concern is that the benefit of using future-mask queries has not been fully demonstrated.
I also have some concerns about the choice of the lookahead horizon. Once a cached entry is removed, it cannot be recovered later. So, looking ahead only until the next eviction checkpoint might miss information that becomes important further in the future.
I think the authors should explain more clearly why this horizon is appropriate and how it relates to the long-term importance of cached information.
Questions for authors
How sensitive is the method to the lookahead horizon? Since random-token probes perform almost as well, how can we tell whether the improvement really comes from future-mask queries?
Once a cache entry is removed, it cannot be recovered. So why is looking ahead only to the next eviction checkpoint enough? Could you also clarify how the current block is handled during eviction?
How does Q-MaskAhead compare with other methods, such as DapQ and H2O+CAOTE, when they all use the same four-bit quantization? Would the improvement still hold under the same memory and runtime budgets?
Could you explain how question-conditioned prefill is implemented? Does the reported memory usage include the extra memory and computation needed for probing? How would other retention methods perform with the same strategy?
How many eviction steps actually happen in your experiments? Could you report the output lengths and number of eviction checkpoints? This would help us understand whether the gains come from repeated lookahead decisions or mainly from initial cache compression. Could you also clarify how reproducible the preliminary DreamReasoner and LLaDA results are?
Dear Reviewers,
As we are approaching the end of reviewer-author discussions, please go through authors' responses and raise questions that you would like authors to reply.
Thanks,
AC