- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes EchoPress, a training-free KV cache compression scheme that approximates the KVzip reconstruction phase via virtual reconstruction. By rotating the query and key vectors computed during the prefill phase, the relative RoPE shift of the reconstruction query and key vectors can be accommodated. This allows EchoPress to demonstrate KVzip-comparable accuracy while maintaining speeds comparable to training-based accelerated methods.
Critical strengths
- With the ever-increasing context length, compressing KV cache is timely and important.
- The proposed method shows significant speedups with decent accuracy, without any training.
- Approximating the reconstruction phase with a rotation to match the RoPE difference is a simple yet clever and practical trick.
Critical weaknesses
- The paper would benefit from some revision to make it easier for readers to understand. A more detailed explanation of RoPE and how its rotational properties enable virtual reconstruction would be helpful. Refining Figure 2 would also improve clarity; in particular, a side-by-side comparison of KVzip and EchoPress using the scoring example in Figure 2(b) could better illustrate how each component of EchoPress approximates KVzip.
Questions for authors
-
Please consider providing a more detailed explanation of RoPE to clarify how rotation-based virtual reconstruction works.
-
Is there a justification for applying the quantile mapping derived from the first chunk throughout the entire reconstruction process? Figure 3 shows that score distortion varies across layers and KV heads, and Figure 4 illustrates differences between prefill and reconstruction queries across chunks. However, I could not find direct evidence that the quantile mapping derived from the first chunk generalizes to subsequent chunks.
-
Fast KVzip reports requiring only 0.59 H100 GPU-hours to train Qwen3-8B, which does not appear to be a substantial cost. I do recognize that EchoPress is training-free and eliminates this training overhead, but I would be interested in understanding how its accuracy compares with training-based methods and whether eliminating the training cost justifies potential accuracy trade-offs. Could the authors provide additional accuracy comparisons with training-based methods beyond Figure 1(a)?
-
The accuracy evaluations are limited to 8B models and context lengths of up to 64K tokens. Can EchoPress scale to larger models and scenarios with longer context lengths?
2 comments
Thank you for your response. The response has resolved my issues. I think the paper did a great job capturing the essence of KVZip that prior training-based methods missed, and the authors provided a solid training-free approximation. Therefore, I will raise my score to 4.
Thank you for the encouraging review. We are glad that you find the RoPE rotation "a simple yet clever and practical trick". We ran new experiments for Q2 to Q4, all in the paper's evaluation setup.
W1 / Q1. RoPE and Figure 2. To make the method easier to follow, we will add a RoPE primer to Section 2.2 and a new figure that puts KVzip and EchoPress side by side on the scoring example of Figure 2(b) and highlights where the two differ.
Q2. Justification for the first-chunk quantile map. The map transfers to later chunks because the scale error it corrects is mostly a property of the head. The heads whose virtual scores are too large or too small in the first chunk are the same ones in later chunks.
Figure 3 shows the scale error of each head in the first chunk. We ran a new experiment that extends this measurement to later chunks, on 12 LongBench contexts per model. For each layer and KV head we take the same ratio between exact and virtual scores and correlate it across heads between the first chunk and a later chunk. On Qwen3-8B the correlation is 0.88 for the second chunk, 0.80 for the third and 0.71 from the fourth on. On Llama-3.1-8B it is 0.94, 0.92 and 0.84.
As a result, the first-chunk map is close to an ideal map fitted on all chunks. Agreement of the KV pairs kept in later chunks with the ones KVzip keeps (Qwen3-8B):
Q3. Training-based methods. We evaluated the released Fast KVzip gates (Kim et al., 2026a) in KVPress with global allocation. Cells are KVzip / Fast KVzip / EchoPress.
Without any training, EchoPress is on par with Fast KVzip up to r = 0.75 and more accurate at r = 0.9 in all six settings: by 6.4 and 4.0 points on Qwen3-8B, 3.0 and 2.2 on Qwen3-14B, and 43 and 26 at 32K and 64K. Fast KVzip's one clear task-level win at r = 0.9 is LCC (61.4 vs 37.6), which comes largely from its policy of never evicting the most recent 2% of the context, with the same window EchoPress reaches 62.2.
We agree that the training cost per model is small. Our point is that removing training does not cost accuracy at these budgets, and that no training is needed. We also believe a training-free approximation gives insight into where KVzip's scores come from: they can be largely recovered from the queries and keys of the ordinary prefill, once the position of the repeated context is accounted for. As shown in Figure 7, adding this RoPE position offset raises the rank correlation with KVzip's scores from 0.305 to 0.678 on Qwen3-8B and from 0.288 to 0.740 on Llama-3.1-8B.
Q4. Larger models and longer contexts. Qwen3-14B is in the table above. On Qwen3-32B RULER-4K, KVzip / EchoPress score 95.6 / 95.5 at r = 0.75 and 95.2 / 94.7 at r = 0.9 (uncompressed 95.7). At 32K and 64K we now use 100 examples per task for Llama-3.1-8B and 50 for Qwen3-8B, instead of 20. At 128K we use 20 examples per task because of the long running time. EchoPress minus KVzip, with a 95% confidence interval estimated by resampling the test examples (an interval that contains 0 is within noise):
Up to 64K EchoPress stays within 2 points of KVzip. The Qwen differences are within noise and the Llama differences go both ways, so there is no systematic loss. At 128K EchoPress is within noise of KVzip at r = 0.75 on both models and at r = 0.9 on Qwen. At r = 0.9 on Llama both methods fall far below the uncompressed cache (KVzip 55.5 and EchoPress 45.6, against 77.9), and EchoPress loses 9.9 points more. This budget is too aggressive for Llama at 128K with either method.
We will add these results to the revised paper. We hope we have answered all your questions, and we would be glad to answer any further questions.