acceptodds
Under review as a conference paper at ICLR 2027

EchoPress: Query-Agnostic KV Cache Pruning via Virtual Context Reconstruction

Abstract

KV cache pruning reduces long-context inference memory usage by evicting less important key–value pairs. KVzip estimates importance through context reconstruction: prompting a model to repeat the context chunk by chunk. This achieves strong compression quality at the cost of additional forward passes. Learned approximations reduce this cost but require model-specific training. We analyze how KVzip identifies important cached information and show how to approximate its reconstruction scores using information already computed during prefill. These findings motivate EchoPress, a training-free method that approximates reconstruction attention using queries and keys from standard prefill. For each request, it reconstructs only the first chunk to calibrate importance scores for the remaining context. Experiments on LongBench and RULER with Qwen3-8B and Llama-3.1-8B-Instruct show that EchoPress matches KVzip in task accuracy across eviction ratios from 50% to 90%, while reducing compression overhead by 1.7–19.6× and total prefill time by up to 2.9×. Code is available at https://anonymous.4open.science/r/echo-press/.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
KV cache compression, KV cache pruning, long-context LLMs

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_NM5DOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes EchoPress, a training-free KV cache compression scheme that approximates the KVzip reconstruction phase via virtual reconstruction. By rotating the query and key vectors computed during the prefill phase, the relative RoPE shift of the reconstruction query and key vectors can be accommodated. This allows EchoPress to demonstrate KVzip-comparable accuracy while maintaining speeds comparable to training-based accelerated methods.

Critical strengths
  • With the ever-increasing context length, compressing KV cache is timely and important.
  • The proposed method shows significant speedups with decent accuracy, without any training.
  • Approximating the reconstruction phase with a rotation to match the RoPE difference is a simple yet clever and practical trick.
Critical weaknesses
  • The paper would benefit from some revision to make it easier for readers to understand. A more detailed explanation of RoPE and how its rotational properties enable virtual reconstruction would be helpful. Refining Figure 2 would also improve clarity; in particular, a side-by-side comparison of KVzip and EchoPress using the scoring example in Figure 2(b) could better illustrate how each component of EchoPress approximates KVzip.
Questions for authors
  • Please consider providing a more detailed explanation of RoPE to clarify how rotation-based virtual reconstruction works.

  • Is there a justification for applying the quantile mapping derived from the first chunk throughout the entire reconstruction process? Figure 3 shows that score distortion varies across layers and KV heads, and Figure 4 illustrates differences between prefill and reconstruction queries across chunks. However, I could not find direct evidence that the quantile mapping derived from the first chunk generalizes to subsequent chunks.

  • Fast KVzip reports requiring only 0.59 H100 GPU-hours to train Qwen3-8B, which does not appear to be a substantial cost. I do recognize that EchoPress is training-free and eliminates this training overhead, but I would be interested in understanding how its accuracy compares with training-based methods and whether eliminating the training cost justifies potential accuracy trade-offs. Could the authors provide additional accuracy comparisons with training-based methods beyond Figure 1(a)?

  • The accuracy evaluations are limited to 8B models and context lengths of up to 64K tokens. Can EchoPress scale to larger models and scenarios with longer context lengths?

2 comments

Thank you for the encouraging review. We are glad that you find the RoPE rotation "a simple yet clever and practical trick". We ran new experiments for Q2 to Q4, all in the paper's evaluation setup.

W1 / Q1. RoPE and Figure 2. To make the method easier to follow, we will add a RoPE primer to Section 2.2 and a new figure that puts KVzip and EchoPress side by side on the scoring example of Figure 2(b) and highlights where the two differ.

Q2. Justification for the first-chunk quantile map. The map transfers to later chunks because the scale error it corrects is mostly a property of the head. The heads whose virtual scores are too large or too small in the first chunk are the same ones in later chunks.

Figure 3 shows the scale error of each head in the first chunk. We ran a new experiment that extends this measurement to later chunks, on 12 LongBench contexts per model. For each layer and KV head we take the same ratio between exact and virtual scores and correlate it across heads between the first chunk and a later chunk. On Qwen3-8B the correlation is 0.88 for the second chunk, 0.80 for the third and 0.71 from the fourth on. On Llama-3.1-8B it is 0.94, 0.92 and 0.84.

As a result, the first-chunk map is close to an ideal map fitted on all chunks. Agreement of the KV pairs kept in later chunks with the ones KVzip keeps (Qwen3-8B):

KV pairs keptno mapfirst-chunk mapideal map (fitted on all chunks)
50%82.6%88.4%91.1%
25%67.8%79.7%82.0%
10%54.1%70.2%70.7%

Q3. Training-based methods. We evaluated the released Fast KVzip gates (Kim et al., 2026a) in KVPress with global allocation. Cells are KVzip / Fast KVzip / EchoPress.

r = 0.75r = 0.9
Qwen3-8B RULER-4K95.1 / 94.8 / 94.987.3 / 80.9 / 87.3
Qwen3-8B LongBench49.8 / 49.5 / 49.639.6 / 37.1 / 41.1
Qwen3-8B RULER-32K87.3 / 84.9 / 87.471.3 / 29.3 / 72.3
Qwen3-8B RULER-64K79.9 / 78.9 / 78.567.8 / 42.5 / 68.9
Qwen3-14B RULER-4K96.1 / 95.9 / 96.192.2 / 88.9 / 91.9
Qwen3-14B LongBench50.2 / 50.0 / 49.943.9 / 42.4 / 44.6

Without any training, EchoPress is on par with Fast KVzip up to r = 0.75 and more accurate at r = 0.9 in all six settings: by 6.4 and 4.0 points on Qwen3-8B, 3.0 and 2.2 on Qwen3-14B, and 43 and 26 at 32K and 64K. Fast KVzip's one clear task-level win at r = 0.9 is LCC (61.4 vs 37.6), which comes largely from its policy of never evicting the most recent 2% of the context, with the same window EchoPress reaches 62.2.

We agree that the training cost per model is small. Our point is that removing training does not cost accuracy at these budgets, and that no training is needed. We also believe a training-free approximation gives insight into where KVzip's scores come from: they can be largely recovered from the queries and keys of the ordinary prefill, once the position of the repeated context is accounted for. As shown in Figure 7, adding this RoPE position offset raises the rank correlation with KVzip's scores from 0.305 to 0.678 on Qwen3-8B and from 0.288 to 0.740 on Llama-3.1-8B.

Q4. Larger models and longer contexts. Qwen3-14B is in the table above. On Qwen3-32B RULER-4K, KVzip / EchoPress score 95.6 / 95.5 at r = 0.75 and 95.2 / 94.7 at r = 0.9 (uncompressed 95.7). At 32K and 64K we now use 100 examples per task for Llama-3.1-8B and 50 for Qwen3-8B, instead of 20. At 128K we use 20 examples per task because of the long running time. EchoPress minus KVzip, with a 95% confidence interval estimated by resampling the test examples (an interval that contains 0 is within noise):

r = 0.75r = 0.9
Llama 32K−2.0 [−2.8, −1.2]+1.8 [+0.5, +3.2]
Llama 64K+2.0 [+1.0, +3.0]−1.7 [−3.3, −0.3]
Llama 128K−0.7 [−3.6, +2.1]−9.9 [−13.6, −6.0]
Qwen 32K+0.1 [−1.6, +1.9]+1.0 [−1.8, +3.5]
Qwen 64K−1.5 [−3.1, +0.2]+1.2 [−1.3, +3.7]
Qwen 128K−1.6 [−4.1, +0.9]−1.7 [−5.7, +2.3]

Up to 64K EchoPress stays within 2 points of KVzip. The Qwen differences are within noise and the Llama differences go both ways, so there is no systematic loss. At 128K EchoPress is within noise of KVzip at r = 0.75 on both models and at r = 0.9 on Qwen. At r = 0.9 on Llama both methods fall far below the uncompressed cache (KVzip 55.5 and EchoPress 45.6, against 77.9), and EchoPress loses 9.9 points more. This budget is too aggressive for Llama at 128K with either method.

We will add these results to the revised paper. We hope we have answered all your questions, and we would be glad to answer any further questions.

Thank you for your response. The response has resolved my issues. I think the paper did a great job capturing the essence of KVZip that prior training-based methods missed, and the authors provided a solid training-free approximation. Therefore, I will raise my score to 4.

Reviewer_FD6nOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

EchoPress is a KV cache compression method that approximates the importance scores—derived from context reconstruction in KVzip—by reusing the query and key tensors computed during the prefill phase, without requiring additional training. It calculates "virtual" reconstruction scores by adjusting for token positions and attention spans; it then performs actual reconstruction only for the first chunk, while calibrating the remaining scores on a per-layer and per-KV-head basis. The method is evaluated using Qwen3-8B and Llama-3.1-8B-Instruct on the LongBench and RULER benchmarks, demonstrating a reduction in compression overhead for context lengths of up to 128K tokens.

Critical strengths
  1. It demonstrates a practical benefit by significantly reducing the overhead of cache construction, without requiring any training. For Qwen3-8B with a 128K context length and a 50% removal ratio, the additional cost for compression drops from 77.02 seconds (with KVzip) to 3.94 seconds, while the total prefill time—including compression—decreases from 112.16 seconds to 38.16 seconds. This performance gap persists even when KVzip is executed using a full-prefill approach, confirming that the speed improvement is not merely a result of the baseline’s chunked prefill configuration.
Critical weaknesses
  1. It has not been verified whether the reported improvements in accuracy and speed can be achieved simultaneously within the same implementation. In Table 10, accuracy is evaluated using KVPress/SDPA, while execution time is measured using KVzip/FlashAttention-2; furthermore, the chunk sizes and prompt/prefix configurations differ between these setups. Appendix B explicitly states that the equivalence of downstream accuracy between these two configurations has not been established. Therefore, it is necessary to verify whether the reported task performance is maintained in implementations that demonstrate significant speed improvements.

  2. Significant performance degradation occurs in some tasks, yet the underlying causes have not been sufficiently analyzed. Under the 90% removal condition, the LCC score for Qwen3-8B is 37.6 for EchoPress compared to 47.0 for KVzip, while the score for common-word extraction in RULER-4K is 59.4 compared to 78.2 (Tables 2–3). Although the authors acknowledge these performance drops, it remains unclear whether the primary cause is the approximation of virtual scores or the correction method based on the first chunk. This suggests that, even if average scores appear similar, EchoPress may not be a suitable replacement for KVzip across all types of tasks.

Questions for authors
  1. Regarding Weakness 1: Could you evaluate the accuracy of representative tasks using the specific FlashAttention-2 implementation employed for timing measurements, and demonstrate whether performance remains comparable to that of the KVPress/SDPA implementations?

  2. Regarding Weakness 2: For the 90% pruning condition, could you provide an analysis that distinguishes whether the performance degradation observed in LCC and common-word extraction stems from the approximation of virtual scores or from the correction method based on the first chunk?

2 comments

Thank you for the careful review, and for recognizing the practical benefit of cutting the compression overhead without any training. We ran new experiments for both points.

W1 / Q1. Is the accuracy kept in the implementation used for timing?

We use two code bases so that each result is measured in its reference implementation. We measure accuracy in KVPress, where many compression methods are integrated, so it is easy to compare with them. We measure latency in KVzip's official code (FlashAttention-2). The KVzip inside KVPress is a minimal re-implementation that masks evicted keys instead of removing them, and its documentation points to the official code for the optimized version. Timing the baseline there would be unfair to KVzip, so we timed KVzip in its authors' code and integrated EchoPress into the same code. We kept the defaults of each code base, which is why the chunk size (2,048 vs 2,000 tokens) and the prompt format differ. These differences are an engineering convenience and not part of the method.

To check that they do not matter, we ran the accuracy benchmark inside the timing implementation: RULER-4K on Qwen3-8B (13 tasks x 500 examples) in KVzip's official code, once with KVzip as released by its authors and once with the EchoPress integration timed in Section 3.3.

The KVPress numbers here are re-runs and can differ slightly from the paper's tables, by at most 0.3 points on the RULER-4K averages and 1.2 points on single tasks.

implementationmethodr = 0.5r = 0.75r = 0.9
KVzip's official code (timing)KVzip94.493.885.1
KVzip's official code (timing)EchoPress94.393.684.4
KVPress (accuracy in the paper)KVzip95.295.187.3
KVPress (accuracy in the paper)EchoPress95.294.987.3
  • In the timing implementation EchoPress is within 0.1, 0.2 and 0.7 points of KVzip at r = 0.5, 0.75 and 0.9. In KVPress the gap is at most 0.3 points. So the timing implementation keeps the accuracy.
  • All scores in KVzip's code are 1 to 3 points lower than in KVPress. This also holds without compression (94.4 vs 95.4), so it does not come from either method.

We will add this table to Appendix B with a footnote in Table 10.

W2 / Q2. Do the losses on LCC and CWE at r = 0.9 come from the virtual score approximation or from the first-chunk calibration?

To separate the two potential reasons, we replaced the first-chunk quantile map by an oracle map, fitted on exact scores that EchoPress does not have at run time. We find that on LCC (code completion) the loss comes from the virtual scores, and on CWE (common word extraction) from the calibration. LCC cells are r = 0.75 / 0.9, and CWE is at r = 0.9.

Qwen3-8B LCCLlama-3.1-8B LCCQwen3-8B CWE
1. KVzip62.9 / 47.949.0 / 41.377.7
2. EchoPress55.2 / 37.743.2 / 30.458.2
3. oracle map fitted on all chunks55.7 / 38.142.1 / 31.775.1
4. one oracle map per chunk56.2 / 37.142.5 / 31.978.5
5. no map58.0 / 41.143.1 / 35.169.6
6. EchoPress, most recent 2% of tokens never evicted65.3 / 62.254.3 / 51.565.4

LCC. Oracle maps do not help: rows 3 and 4 are within 1.5 points of row 2. In row 4 the score scale of every chunk is exact and only the order of tokens inside a chunk is still virtual, yet LCC stays 6.5 to 10.8 points below KVzip. So the loss comes from the virtual ranking inside a chunk. The virtual ranking can drop KV pairs that the task needs. We find that never evicting the most recent 2% of tokens, the default rule of Fast KVzip, prevents this: it lifts EchoPress above KVzip on LCC (row 6 vs row 1).

CWE. Oracle maps bring CWE back to 75.1 and 78.5, against 77.7 for KVzip, so the virtual scores are adequate. The first-chunk map is the cause: it is even worse than no map (58.2 vs 69.6).

In summary, the virtual ranking and the first-chunk calibration can each cost accuracy on specific tasks. EchoPress can therefore underperform KVzip on some tasks, even when its average score is similar or higher. We will add this analysis and state this limitation in the paper.

We hope these results address both of your concerns, and we would be glad to answer any further questions.

Thank you for the detailed response and additional experiments. The new results substantially address my concerns regarding implementation consistency and provide a clearer understanding of the task-specific performance degradation. Although some limitations remain, I appreciate the additional analysis and will maintain my current rating.

Reviewer_zCYjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EchoPress is a training-free method for query-agnostic KV cache compression aiming to minimize the computation cost of KVzip's reconstruction-based importance scores. To avoid the second prefill-like forward pass employed by KVzip, EchoPress reuses the query and key tensors already computed during ordinary prefill, combined with a correction for RoPE position, to approximate KVzip's scores (Eq. 1 to 4). Because virtual scores and true reconstruction scores differ in scale across layers and KV heads, EchoPress performs one exact KVzip-style reconstruction on the first chunk per request, and uses it to build a per-layer, per-KV-head empirical quantile map as a calibration for all later virtual scores.

On Qwen3-8B and Llama-3.1-8B-Instruct, across 12 LongBench tasks and 13 RULER-4K tasks, EchoPress tracks KVzip very well at eviction ratios r=0.5/0.75/0.9, and stays within 2.8 points of KVzip at the longer-context RULER-32K/64K settings. In terms of efficiency improvement, EchoPress was tested on Qwen3-8B and demonstrated compression overhead reduction by 1.7× to 19.6× for context lengths ranging from 4K to 128K tokens (Figure 1b). An ablation (Figure 7, Table 1) traces the gain to specific mechanisms and finds that the RoPE position correction is responsible for the largest improvement.

Critical strengths
  1. A simple, training-free approximation and calibration scheme that is demonstrably effective. Reusing prefill's own query/key tensors plus a RoPE correction, combined with a cheap per-request, per-layer, per-KV-head quantile calibration, delivers a large, consistently measured efficiency win — comparable to methods that require model-specific training (Figure 6).
  2. Clear ablation isolating where the approximation gain comes from. The paper isolates three specific corrections (key visibility, RoPE shift, causal virtual-copy) via a controlled ablation (Figure 7) and shows the RoPE position correction alone drives Spearman correlation with true KVzip scores from ~0.3 to ~0.68–0.74.
Critical weaknesses
  1. Insufficient empirical validation. EchoPress is only ever compared against the single baseline it approximates (KVzip) — not against directly comparable related work, most notably Fast KVzip (Kim et al., 2026a), which targets the exact same cost with a trained gate instead of a training-free approximation. The authors are evidently aware of this related work, since it appears in Figure 1a's leaderboard overlay and supplies one latency data point in Figure 6 — but no accuracy comparison at any of EchoPress's own tested budgets is ever reported, and no discussion weighs the trade-offs (training-free vs. trained, accuracy vs. speed) between the two approaches. Overall coverage is also thin on the model side (only 2 mid-size backbones, Qwen3-8B and Llama-3.1-8B), on sample size at the longer contexts that most motivate the method (RULER-32K/64K uses only 20 examples/task, Table 8) — exactly where EchoPress's relative performance is noisiest (e.g., −2.15 points vs. KVzip at 32K/r=0.9 on Qwen) — and on the eviction-ratio grid itself, tested only at r=0.5/0.75/0.9 (plus a leaderboard-matched 0.25/0.875 in Figure 1a), leaving the lower-eviction, more aggressive-compression regime largely unexamined.
  2. Quantization compatibility is untested, despite the baseline it approximates already demonstrating it. All results are reported against a slot-count eviction ratio r, not a byte/memory budget, and the KV cache's precision (BF16) is never treated as a methodology axis. KVzip itself shows its reconstruction scoring integrates with 4-bit quantization with negligible added degradation — but EchoPress never tests or discusses whether its own virtual-scoring approximation composes similarly, despite citing KV quantization as a distinct, orthogonal lever in related work (§4).
  3. Extension to decoding-time compression is only briefly acknowledged as a limitation, never attempted or evaluated. Section 5 notes that "EchoPress compresses the prefilled context but performs no further eviction during decoding, so the cache grows with generated tokens," and that extending the scorer to this setting "would require buffering query states and choosing a scoring frequency" — but this is left entirely to future work, with no experiment, design sketch, or even a rough cost estimate for what such an extension would require.
Questions for authors

please see Weaknesses above

3 comments

Thank you for the detailed review. We are glad that you find EchoPress simple and effective. We ran new experiments for every concern you raised.

W1a. Fast KVzip. The submission compares accuracy with Fast KVzip only in Figure 1(a), and we agree that this is inadequate. We evaluated the released Fast KVzip gates in KVPress with global allocation. Cells are KVzip / Fast KVzip / EchoPress.

r = 0.75r = 0.9
Qwen3-8B RULER-4K95.1 / 94.8 / 94.987.3 / 80.9 / 87.3
Qwen3-8B LongBench49.8 / 49.5 / 49.639.6 / 37.1 / 41.1
Qwen3-8B RULER-32K87.3 / 84.9 / 87.471.3 / 29.3 / 72.3
Qwen3-8B RULER-64K79.9 / 78.9 / 78.567.8 / 42.5 / 68.9
Qwen3-14B RULER-4K96.1 / 95.9 / 96.192.2 / 88.9 / 91.9
Qwen3-14B LongBench50.2 / 50.0 / 49.943.9 / 42.4 / 44.6

At r = 0.5 (RULER-4K 95.2 / 95.3 / 95.2, LongBench 48.5 / 49.2 / 48.2) and r = 0.75 the three methods are within 1 point at 4K and on LongBench. At r = 0.9 Fast KVzip is below EchoPress in every row: by 6.4 and 4.0 points on Qwen3-8B, 3.0 and 2.2 on Qwen3-14B, and 43 and 26 at 32K and 64K. Fast KVzip's one clear task-level win at r = 0.9 is LCC (61.4 vs 37.6). Our ablation traces this to its policy of never evicting the most recent 2% of the context, not to its scorer, with the same window EchoPress reaches 62.2 and KVzip 62.4.

Trade-off. Fast KVzip is faster (1.08 s vs 3.94 s to compress 128K tokens on Qwen3-8B) but needs a gate trained per model (0.59 H100 GPU-hours reported for Qwen3-8B) and loses accuracy at r = 0.9 on LongBench and RULER. EchoPress needs no training, runs on any model KVzip supports and stays at KVzip's level of accuracy. EchoPress is derived from KVzip's reconstruction pass, so it also helps to explain where KVzip's scores come from. As Figure 7 shows, the prefill states largely reproduce their ranking once the RoPE position shift of the repeated context is applied.

W1b. Models. Qwen3-14B is in the table. On Qwen3-32B RULER-4K, which has no released Fast KVzip gate, KVzip / EchoPress score 95.6 / 95.5 at r = 0.75 and 95.2 / 94.7 at r = 0.9 (uncompressed 95.7). EchoPress tracks KVzip from 8B to 32B.

W1c. Sample size at 32K/64K. We now use 100 examples per task for Llama-3.1-8B and 50 for Qwen3-8B, instead of 20. The table gives EchoPress minus KVzip with a 95% confidence interval, estimated by resampling the test examples. An interval that contains 0 means the difference is within noise.

r = 0.75r = 0.9
Llama 32K−2.0 [−2.8, −1.2]+1.8 [+0.5, +3.2]
Llama 64K+2.0 [+1.0, +3.0]−1.7 [−3.3, −0.3]
Qwen 32K+0.1 [−1.6, +1.9]+1.0 [−1.8, +3.5]
Qwen 64K−1.5 [−3.1, +0.2]+1.2 [−1.3, +3.7]

EchoPress stays within 2 points of KVzip in every cell. The Qwen differences are within noise and the Llama differences go both ways, so there is no systematic loss. The −2.15 points at 32K and r = 0.9 on Qwen that the review cites become +1.0.

W1d. Eviction ratios. We added r = 0.95:

r = 0.95KVzipFast KVzipEchoPress
Qwen3-8B RULER-4K37.931.046.8
Qwen3-8B LongBench16.621.520.1
Llama-3.1-8B RULER-4K63.6no gate62.2
Llama-3.1-8B LongBench23.8no gate27.1

EchoPress is ahead of KVzip in three of the four rows (by 3.3 to 8.9 points), ahead of Fast KVzip on RULER-4K and close to it on LongBench.

W2. KV quantization. We ran KVzip and EchoPress on a 4-bit quantized cache (HQQ), with scoring, eviction and generation on the quantized context. Change in RULER-4K accuracy from bf16 to 4-bit, Qwen3-8B / Llama-3.1-8B:

KVzipEchoPress
r = 0.75−0.3 / −0.3−0.2 / −0.3
r = 0.9+0.2 / −0.8+0.6 / −0.9

The uncompressed cache changes by −0.4 / −0.2. EchoPress is always within 0.4 points of KVzip's change, so it composes with quantization as KVzip does.

W3. Decoding-time eviction. We implemented a prototype of the scoring step for decoding-time eviction and measured its cost. It works on a buffer that holds the queries of the last C_dec generated tokens, scores this chunk with EchoPress's virtual scorer, applies the calibration map fitted at prefill and re-runs the global selection. No reconstruction pass is needed. Cost of scoring one chunk on Llama-3.1-8B (A100), at an 8K / 32K cache:

C_decdecode stepsper generated tokenquery buffer
2560.58 / 0.3649 / 50 µs64 MiB
5120.94 / 0.5939 / 41 µs128 MiB
10243.3 / 2.068 / 69 µs256 MiB

A 512-token chunk costs under one decode step, or 0.1 to 0.2% of decode time.

We will add these results and the trade-off discussion to the paper. We hope we have addressed all your concerns and would be glad to answer any further questions.

AuthorsOpenReview ↗

Follow up to W3. Decoding-time eviction

Your comment W3 asked for an evaluation of decoding-time compression. Our first reply gave the cost of the scoring step. We have now implemented decoding-time eviction in full and measured its accuracy.

W3. Does EchoPress keep its accuracy when the cache is evicted during decoding?

We added EchoPress to the official decoding code of Fast KVzip and kept its evaluation setup: AIME24 and MATH with Qwen3-8B in thinking mode, up to 32K and 16K tokens per solution, and a fixed KV cache budget that is enforced every 128 generated tokens over the question and the generated tokens together. The two methods differ only in how they score KV pairs. EchoPress scores each new block of 128 generated tokens with its virtual scorer, from the queries and keys of the ordinary decoding pass.

Accuracy on AIME24 (30 problems) and MATH (500 problems). Each problem is solved once with the same sampling seed for all methods.

TaskKV cache budgetUncompressedFast KVzipEchoPress
AIME244,09666.770.076.7
AIME242,04866.760.070.0
MATH2,04893.894.093.0

EchoPress is on par with Fast KVzip on both tasks. On AIME24 it solves 23 and 21 problems, Fast KVzip 21 and 18, and the uncompressed cache 20. On MATH it solves 465, Fast KVzip 470, and the uncompressed cache 469. EchoPress stays at the level of the uncompressed cache on AIME24 and within one point of it on MATH. Our reproduced Fast KVzip scores on AIME24 match the numbers reported in its paper (72.7 and 63.1).

We hope this completes our answer to W3 and would be glad to discuss any remaining point.

Thank you for the thorough response and the new experiments. The comparison with Fast KVzip across six settings, the added Qwen3-14B and Qwen3-32B results, the larger 32K/64K samples with confidence intervals, the r=0.95 results, the 4-bit quantization experiment, and the decoding-time implementation address my concerns about baselines, model coverage, sample size, eviction ratios, quantization compatibility and decoding. Based on the response, I raise my score from 2 to 3. My assessment of the contribution is otherwise unchanged: EchoPress is a simple and effective way to approximate KVzip at much lower cost, but as an approximation its value lies in efficiency, not in accuracy beyond KVzip, which, in my opinion, limits the contribution and is why I do not raise my score further.

Comments on the paper

1 comment

Dear Reviewers zCYj,

The authors have provided their rebuttal, but no follow-up comments have been posted yet. As the author-reviewer discussion period is ending soon, please review and acknowledge the rebuttal. I also encourage all reviewers to engage in further discussion if any concerns remain unresolved.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.