acceptodds
Withdrawn from ICLR 2027

CAPE: COMPRESSED ATTENTION WITH PRESERVED ENTROPY

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reasoning models answer hard problems by generating long chain-of-thoughts (CoT), the KV-cache grows with the reasoning length and becomes the binding memory constraint on inference. KV-cache compression aims to maintain the same model performance while keeping only a bounded number of cached entries. Eviction-based methods have made this practical: scoring cached keys by the at- tention they receive from recent decode queries retains much of the uncompressed accuracy at a small fraction of the cache. We show that under a reduced cache bud- get, Softmax attention over the remaining keys is sharper than the distribution the model is calibrated for at its logical length, and the mismatch widens as the trace grows. A second issue is the loss of the objective: because keys are scored by the attention of recent queries, we found the model becomes less certain of the ques- tion as generation length grows under this eviction schema. We propose CAPE (Compressed Attention with Preserved Entropy), which restores the entropy of compressed attention to its uncompressed level, and reserves a bounded share of the budget for objective anchors selected by context reconstruction. Across three reasoning models on MATH-500 and AIME, CAPE recovers most of the accuracy lost under eviction with negligible additional cost.

Primary area:
foundation or frontier models, including LLMs
Keywords:
KV cache compression, efficient LLM inference, reasoning models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_v2sPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the performance degradation caused by KV-cache compression during long chain-of-thought (Long-CoT) reasoning. The authors identify two major failure modes associated with existing attention-based KV-cache eviction methods: Lost Objective and Commitment Collapse.

To address these issues, the paper proposes CAPE, a training-free method that employs context reconstruction to select and preserve important KV entries from the original question, thereby mitigating objective loss. Meanwhile, it adjusts the Softmax temperature according to the logical sequence length and physical cache size to alleviate the attention entropy deficit induced by KV-cache compression.

Experimental results demonstrate that CAPE generally outperforms existing compression baselines under matched cache budgets, recovering part of the reasoning accuracy lost under compression while substantially reducing KV-cache memory consumption.

Critical strengths
  1. Valuable analysis of failure modes. The paper goes beyond final-answer accuracy by identifying and analyzing two failure modes, Lost Objective and Commitment Collapse, providing a more fine-grained understanding of KV-cache compression failures during long-CoT reasoning.

  2. Simple and complementary method design. CAPE combines objective preservation with attention temperature adjustment to address different failure modes. Both components are training-free and can be readily integrated into existing dynamic KV-cache compression frameworks.

  3. Competitive experimental performance. The experiments cover three reasoning models and three mathematical reasoning benchmarks. CAPE consistently matches or outperforms the evaluated compression baselines under the reported settings.

  4. Comprehensive ablation and diagnostic analyses. The paper provides question-recall evaluations, commitment-delay analysis, attention entropy interventions, and component ablation studies, offering useful insights into the effectiveness of the proposed method.Critical Weaknesses.

Critical weaknesses
  1. Insufficient direct validation of the entropy-matching mechanism.The temperature scaling derivation relies on assumptions such as i.i.d. and Gaussian attention logits, which may not hold after importance-based KV eviction. Moreover, although temperature scaling improves accuracy, the paper does not directly compare the actual attention entropy of CAPE with that of FullKV. Therefore, the proposed entropy-matching mechanism remains insufficiently validated.

  2. Limited methodological novelty. CAPE's temperature scaling and context reconstruction components are closely related to existing methods such as InfoScale, AhaKV, and KVzip. Furthermore, Random Attention (Wang et al., 2026) demonstrates that preserving the original prompt while randomly evicting reasoning tokens can also achieve competitive performance. Although CAPE provides some ablation studies, more comprehensive comparisons with these simpler strategies are needed to clarify the additional value of reconstruction-based anchor selection and the overall combined design.

  3. Limitations in measuring Commitment Collapse. The paper defines a trace as answer-ready when the correct answer first appears in the reasoning text, but this does not necessarily indicate that the model has completed its reasoning. Furthermore, counting expressions such as "Wait" and "Let me check" may not reliably distinguish legitimate verification from unnecessary reasoning loops. Additional manual validation or more reliable semantic evaluation would strengthen the analysis.

  4. Limited evidence of generalizability. The experiments are restricted to MATH-500 and AIME24/25. It remains unclear whether fixed Objective Anchors are effective for tasks such as code generation, long-document reasoning, and complex instruction following, where multiple constraints or objectives may need to be preserved. Further evaluation is needed to establish the generalizability of the proposed method.

Questions for authors
  1. Could the authors directly measure and compare the actual attention entropy of FullKV, standard KV-cache compression methods, and CAPE across different decoding lengths and attention layers to verify whether the proposed temperature scaling achieves the intended entropy matching?

  2. How much additional benefit does reconstruction-based Objective Anchor selection provide compared with simply preserving the entire prompt? Could the authors include fair comparisons with simpler approaches such as Random Attention?

  3. How reliable is the definition of answer-ready based on the first occurrence of the correct answer in the reasoning text? Have the authors conducted manual inspections to distinguish genuine Commitment Collapse from legitimate subsequent verification?

  4. How does CAPE perform on non-mathematical tasks, such as code generation and long-document reasoning? Are fixed Objective Anchors still effective when prompts contain multiple complex constraints?

Reviewer_We2vOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CAPE for KV-cache compression during long-chain-of-thought reasoning. It combines reconstruction-selected question anchors with length-dependent attention temperature adjustment to mitigate question loss and delayed answer commitment. Experiments on three reasoning models and mathematical benchmarks report improved accuracy over the evaluated compression baselines.

Critical strengths

1.Identifies relevant failure modes of KV-cache compression, supported by targeted diagnostic experiments. 2.Proposes a simple, training-free approach, with ablations demonstrating the contributions of both components. 3.Matches or outperforms the evaluated compression baselines across three models and three mathematical benchmarks.

Critical weaknesses

1.Table 3 is inconsistent with the stated efficiency conclusion. The two runtime ratios appear to be swapped, which cannot be explained by rounding, while lines 468–469 claim that CAPE achieves the shortest single-stream runtime among the compressed methods. 2.The experiments cover three models but evaluate only mathematical reasoning, leaving applicability to other domains untested. Although the main results are averaged over multiple runs, no confidence intervals are reported. 3.The ablation shows that temperature adjustment improves performance, but lacks a comparison with a tuned constant temperature. It therefore remains unclear whether the gains stem from the proposed length-dependent formula or whether a simple temperature adjustment could achieve similar results.

Questions for authors

1.Please clarify the inconsistency between Table 3 and the corresponding text, as noted in W1. 2.Could you evaluate the method on datasets beyond mathematical reasoning and report results across multiple random seeds? 3.Could you add the ablation study suggested in W3?

Reviewer_zeUWOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CAPE addresses KV-cache compression during long reasoning. It protects prompt tokens selected through reconstruction and adjusts attention temperature according to logical sequence length and physical cache size. Experiments cover three reasoning models on MATH-500 and AIME24/25, with component ablations and analyses of question retention and answer commitment.

Critical strengths
  1. Table 1 reports useful gains over existing compressors. Qwen3-8B MATH-500 accuracy reaches 77.0%, compared with 67.6% for TriAttention and 60.8% for SnapKV. Improvements extend across several models and benchmarks.
  2. The anchor and temperature ablations, random-anchor control, and FullKV temperature intervention help investigate the proposed failure mechanisms.
Critical weaknesses

The reported results disagree. The reported results need reconciliation. Figure 3C shows approximately 16.7% for CAPE on Qwen3-8B/AIME25 at B=512, while Table 4, also labelled B=512, and Section 5.5 report 43.3%. SnapKV is approximately 3.3% in the figure versus 16.7% in Table 4. The text does not explain whether these use different settings or runs. Table 3 also lists SnapKV at 25.74 hours and CAPE at 26.95 hours, contradicting the claim that CAPE has the shortest compressed-method wall time. Their printed FullKV-relative ratios are reversed relative to the listed hours. These discrepancies affect the accuracy and efficiency claims. This is CRITICAL! I need authors to take this seriously. It could be careless mistakes or fabrications. But such disagreement appears in more than one places in a paper shows that the paper lacks proper proof reading.

The entropy argument needs more evidence. The derivation assumes i.i.d. logits, but attention-dependent selection need not preserve their distribution. The identity in Appendix A.1 does not by itself establish the subsequent assertion that the entropy gap increases as retained mass decreases. Although the paper includes a FullKV entropy-deficit intervention, it does not report measurements showing how closely Equation 7 matches FullKV entropy across layers and heads.

Questions for authors

Please reconcile Figure 3C with Table 4 at B=512, and Table 3's hours, ratios, and fastest-method claim. Which settings or entries differ?

How was the answer-readiness detector validated? A gold value appearing somewhere in a trace can be an intermediate quantity or rejected candidate, rather than evidence that the problem has been solved.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.