- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a KV cache compression method that combines Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT). It employs a one-time attention calibration process to determine the compression strategy and utilizes hierarchical routing to reduce the computational overhead of attention mechanisms. The authors evaluate the approach across various models in terms of language modeling performance, long-context task capabilities, and system efficiency, which aims to balance cache capacity and computational costs without the need for fine-tuning.
Critical strengths
- The method accounts for both the size of cached representations and the cost of attention-based access. It is designed to balance these two types of overhead, with a clear system-level motivation.
- The experiments encompass multiple models and diverse task types, and include ablation studies on compression and routing, which helps clarify the method's scope of applicability.
Critical weaknesses
- In Appendix E.7, the query-dependent importance scores for certain baselines are replaced with scores derived solely from key vectors. This modification could significantly alter the performance of the original methods. However, the main tables continue to use the original method names, making it difficult to assess the actual advantages over the original, unmodified versions.
- There is a clear inconsistency in the "SnapKV" row of Table 2: the arithmetic mean of the 14 listed task scores is approximately 6.40, yet the reported average is 37.40, a value higher than any of the individual task scores in that row. Furthermore, improvements in average performance relative to baselines can be skewed by the inclusion of weaker baselines. It is more meaningful to prioritize reporting gains relative to the strongest comparable method.
Questions for authors
- Regarding the task scores and their averages for SnapKV in Table 2, which specific values require correction? Does this correction affect the primary comparisons or the improvement percentages cited in the abstract?
- Could you clearly distinguish between the "vanilla" (original) baselines and the query-agnostic modified versions? Furthermore, which of these comparisons do your current results actually support?
- Does the reduction in attention score computations in WaveKV lead to tangible end-to-end inference acceleration? It is recommended to provide data on decoding throughput and peak VRAM usage across various context lengths under consistent cache budget and hardware conditions, while accounting for the additional overheads introduced by the wavelet transform and cache maintenance.
2 comments
My concerns have been adequately addressed
Thanks for the detailed response and additional experiments. My concerns have been adequately addressed, and I have therefore raised my score from 2 to 3.
Response to reviewer questions regarding numerical consistency, baselines for comparison, system performance.
We thank the reviewer for the constructive questions. We have corrected the reported averages, clarified the baseline protocols, and distinguished attention-scoring reduction from measured inference performance.
Q1. SnapKV correction and abstract comparisons
The 14 task-level scores for query-independent SnapKV in Table 2 are correct. Only its LLaMA-3.1-8B LongBench average changes from 37.40 to 6.40. Each LongBench average is weighted by the number of evaluation samples in each task. We have also reconciled the Full KV scores across Tables 2 and 12 (Table 10 in initial submission). Neither correction changes WaveKV's measured scores. We recalculated the abstract's LongBench comparison against all 12 baselines in the query-independent evaluation, rather than selecting the next-best method. These comprise GemFilter, SnapKV, PyramidKV, LaCache, FastKV, CaM, D2O, ThinK, StarKV, WaveFilter, FreqKV, and PyramidInfer. Query-dependent selection is adapted to key-only importance where applicable, while inherently query-independent components remain unchanged (Appendix E.7). We first compute each method's sample-weighted average over 14 LongBench tasks, then average equally across the 12 baselines and three models. The resulting baseline mean is 27.36, compared with 41.62 for WaveKV, giving 52.1% relative improvement, revised from the original 40% abstract claim. The remaining abstract comparisons are unaffected by this LongBench correction. We have clarified the aggregation and corrected Tables 2 and 12.
Q2. Vanilla, unmodified, and query-independent baselines
Full KV (vanilla) denotes inference without compression. Unmodified baselines use each published method's native importance scoring, token selection, and compression. The query-independent variants in Appendix E.7 replace query-derived importance scores with key-only statistics, preserving their cache budgets and other compression settings. The latter variants isolate the effect of query-independent cache construction. Several existing approaches use attention from recent queries to select and irreversibly evict states, potentially discarding information needed by later queries at arbitrary context positions. WaveKV instead compresses independently of the decoding query and uses query-dependent hierarchical routing over its retained representation. The query-independent comparisons therefore serve as a controlled evaluation, not a replacement for the published implementations. We additionally evaluated the unmodified methods under comparable cache budgets. On LLaMA-3.1-8B, WaveKV scores 44.56, versus 43.08 for PyramidInfer. On Qwen3-8B, it scores 43.19, versus 41.55 for PyramidInfer and 41.36 for FastKV. On Phi-3-medium-128k, WaveKV scores 37.12, while SnapKV (40.38) and PyramidKV (40.36) perform better. Thus, the relative advantage varies by model. Importantly, LongBench averages alone do not characterize retrieval. At 16K, WaveKV achieves 86.57% RULER accuracy on LLaMA-3.1-8B, versus 45.66% for PyramidInfer and 89.08% for Full KV. Figure 10 also demonstrates stronger Needle-in-a-Haystack retrieval across needle positions and context lengths, consistent with retaining compressed context rather than irreversible eviction. We distinguish both protocols in Section 4.1.2, Tables 2 and 12, and Appendices E.7 and G.2 of the revised submission.
Q3. Attention-scoring reduction and system performance
The requested system measurements were included in Section 4.3 (Section 4.2 in the initial submission) and Appendices E and F of the initial submission. Figure 11 reports 11.7× fewer attention-scoring operations at 32K, while Tables 5–8 report measured decoding throughput, peak GPU memory, and prefill latency. Attention-workload reduction is not equivalent to an end-to-end speedup because wavelet routing, cache maintenance, and other inference operations also contribute to runtime. These costs are reflected in the measured throughput. At 16K/32K/64K, WaveKV achieves 22.4/23.1/22.4 tokens/s, versus 25.3/25.4/24.9 for optimized Full KV, a 9–12% throughput difference. At 16K, WaveKV is approximately 70% faster than FreqKV (22.4 versus 13.2 tokens/s), with comparable throughput at 32K–64K. At 16K, WaveKV uses 46.8 GB versus 42.3 GB for Full KV. At 32K, peak GPU memory reduces by 15.3% (32.1 versus 37.9 GB), reaching 40.1% at 64K (33.2 versus 55.4 GB). At 64K, WaveKV also reduces TTFT from 10.70 to 9.42 s relative to Full KV. At 128K, WaveKV uses 35.4 GB of peak memory and sustains 21.5 tokens/s, whereas Full KV encounters OOM and achieves lower TTFT than FreqKV (19.30 versus 22.02 s). These results demonstrate joint savings in KV storage and attention computation with competitive long-context decoding and improved memory scalability.
The rebuttal questions also address the critical weaknesses.