- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the compatibility of prefix KV cache reuse with adaptive sparse prefill, where attention selection may depend on the input. Although the retained attention connections remain causal, queries from suffix tokens can influence the attention patterns within the shared prefix, potentially resulting in different prefix KV states across requests with identical prefixes.
The authors conduct a series of controlled experiments to investigate this issue. On 80 constructed document tasks, switching cache donors changes the generated outputs in 11/80 cases under sparse attention and 2/80 cases under dense attention. Since these differences may also involve execution-shape effects, the authors further conduct matched-shape interventions, showing that dynamic attention selection can produce different prefix KV states, while sharing the same attention connections eliminates these differences.
The paper also derives sufficient conditions for preserving mathematical and bitwise equivalence during cache substitution and proposes a diagnostic protocol for cache reuse. In their reference implementation, restricting reuse to complete prefill-call boundaries recovers the outputs of the corresponding cold runs, at the cost of additional token computation.
Overall, this work primarily provides an empirical diagnosis and a formal characterization of prefix-cache compatibility in adaptive sparse prefill, rather than introducing a new sparse attention algorithm or an optimized caching system.
Critical strengths
S1. A meaningful observation supported by controlled experiments.
The paper identifies an interesting compatibility issue between adaptive attention selection and prefix KV reuse. In particular, the matched-shape intervention that fixes attention connections while independently recomputing Q/K/V provides useful evidence that suffix-dependent attention selection can affect prefix representations. The distinction between cache-source effects, restart effects, and attention-selection effects is also valuable.
Critical weaknesses
W1. Limited novelty and insufficient motivation for the practical significance of the problem.
While the observation is interesting, the technical novelty appears limited. The proposed sufficient conditions largely formalize the requirement that equivalent dependency graphs and matching execution procedures should produce equivalent results. Similarly, restricting cache reuse to complete prefill-call boundaries seems to be a relatively straightforward conservative strategy rather than a fundamentally new caching method.
Furthermore, the paper does not sufficiently establish the practical significance of the identified compatibility issue. It remains unclear how frequently such incompatibilities occur in realistic serving workloads, how severely they affect downstream performance, and why the proposed compatibility framework offers substantial advantages over existing dependency-aware or conservative caching strategies. Consequently, the motivation and overall contribution of the work are not sufficiently convincing.
W2. Limited generalizability and incomplete empirical validation.
The experiments are primarily conducted using a single model, Qwen2.5-7B-Instruct, and a local adaptation of MInference with uniform attention budgets and masked-dense execution. This setup differs from the released sparse attention implementation and does not evaluate native sparse kernels. The paper also reports substantial discrepancies between its local fp32 selector and the released bfloat16 selector under the main budget configuration.
In addition, the main output-level experiments involve only 80 constructed tasks, while the matched-shape mechanism intervention is limited to four input pairs. The observed donor-swap output differences cannot be directly attributed to suffix-dependent selection, since producer execution shapes also vary. The separate mechanism experiment does not evaluate generated answers.
These limitations make it difficult to assess whether the identified behavior and proposed reuse rule generalize across sparse attention methods, model architectures, execution backends, and realistic workloads. The reported token-computation overhead also does not establish the actual serving performance of the proposed approach.
Questions for authors
Q1. Generalizability.
Could the authors demonstrate that the observed suffix-dependent prefix KV differences and the proposed compatibility rule generalize beyond the current MInference adaptation? In particular, would the same conclusions hold for other adaptive sparse attention methods, the released MInference configuration, different model architectures, and native sparse attention kernels?
**Q2. Causal attribution. **
Could the authors quantify how much of the output-level variation observed in the 80 tasks is actually caused by suffix-dependent attention selection, rather than differences in producer execution shapes or numerical execution? For example, could the shared-connection intervention be extended to end-to-end generation under matched execution conditions?
Q3. Novelty and practical impact.
What is the concrete advantage of the proposed compatibility framework over existing conservative or dependency-aware caching strategies? Could the authors demonstrate that the proposed conditions enable meaningful additional cache reuse or practical serving benefits, especially considering the reported 43.9% increase in consumer-side token computation relative to page-aligned reuse?