- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CCoR, a context compression framework that prioritizes evidence supporting downstream reasoning. It uses sufficiency- and necessity-based scores to identify important chunks, preserves their token-level representations, and compresses the remaining chunks into query-guided summary vectors. The framework is trained through supervised alignment and reasoning-preservation reinforcement learning. Experiments report improvements over several compression baselines on three QA benchmarks.
Critical strengths
The distinction between query relevance and reasoning utility is a meaningful motivation, particularly for multi-hop questions requiring intermediate evidence. The reported average improvements across two model families and multiple compression ratios are encouraging. The paper also provides ablations and qualitative examples to help explain the approach.
Critical weaknesses
-
The evaluated contexts are too short to substantiate the broader long-context claims. Table 4 reports average context lengths of 2,933, 1,018, and 1,402 tokens for NaturalQuestions, 2WikiMQA, and HotpotQA, respectively. Even the maximum lengths remain below approximately 4.4K tokens. Although the evaluation contains a substantial number of examples, it primarily tests compression of relatively short contexts.
-
The results of the “w/o RS” ablation require clarification. In Table 2, the w/o RS variant has exactly the same F1 scores as RAM in Table 1 across all three datasets: 62.77, 57.50, and 65.65. Its NaturalQuestions and 2WikiMQA EM scores also match RAM exactly, while its HotpotQA EM differs slightly: 52.58 versus 52.28. This pattern is not sufficient to conclude that the results are incorrect, but it raises an important question about the ablation’s provenance.
-
Section 4 evaluates a relevance-only selection heuristic rather than the existing methods characterized in the text. The pilot study ranks chunks by query–chunk embedding cosine similarity and retains the top-ranked chunks. This experiment can demonstrate a limitation of that particular selection heuristic. It does not, by itself, establish the same limitation for existing compression methods, which may incorporate additional objectives.
-
The architectural relationship to RAM is insufficiently emphasized. The framework basically follows RAM: some chunks are fully retained, and others are compressed, but this is not well-recognized in the paper.
Questions for authors
See the weakness section.
5 comments
W4: Architectural Relationship to RAM
We sincerely thank the reviewer for this helpful comment. We agree that the relationship between CCoR and RAM deserves to be more explicitly discussed. Indeed, RAM is a strong and influential work in context compression, and its hybrid representation paradigm, which combines explicit retention of important chunks with latent compression of the remaining context, partially inspired our forward compression design. We will clarify this connection in the revised manuscript.
More importantly, CCoR focuses on improving how critical chunks are identified, rather than proposing an entirely new hybrid representation architecture. To further validate this contribution, we extend CCoR's reasoning-based selection criterion and training framework to three representative compression architectures: RAM, COMI, and SeCo (Jiwei Tang and Zhijing Huang, arXiv:2605.09463). Using LLama3.1-8B-Instruct under the same 4× compression setting, we observe consistent improvements across all three architectures. Specifically, the average EM gains are 8.14, 17.26, and 9.24 points, respectively, while the average F1 gains are 8.15, 17.82, and 4.62 points. These results further highlight the effectiveness of CCoR's reasoning-based selection beyond its original architecture. We appreciate the reviewer’s suggestion, which helps us better clarify the relationship between our architectural design and methodological contributions. These experiments have been added to section 6.6 of the revised manuscript.
| Methods | NaturalQA | HotpotQA | 2WikiMQA | Avg | ||||
|---|---|---|---|---|---|---|---|---|
| EM | F1 | EM | F1 | EM | F1 | EM | F1 | |
| RAM | 51.15 | 62.77 | 52.28 | 65.65 | 49.51 | 57.50 | 50.98 | 61.97 |
| CCoR-RAM | 60.68 | 71.29 | 53.18 | 67.56 | 63.51 | 71.51 | 59.12 | 70.12 |
| COMI | 49.83 | 62.09 | 37.58 | 51.72 | 41.60 | 50.31 | 43.00 | 54.71 |
| CCoR-COMI | 58.04 | 69.96 | 57.86 | 73.52 | 64.89 | 74.12 | 60.26 | 72.53 |
| SeCo | 54.80 | 69.12 | 51.79 | 70.74 | 48.41 | 63.49 | 51.67 | 67.78 |
| CCoR-SeCo | 63.88 | 74.74 | 56.50 | 71.56 | 62.34 | 70.89 | 60.91 | 72.40 |
Table 4. Performance comparison of CCoR applied to different compression architectures. All EM and F1 scores are reported inpercentage (%).
We sincerely appreciate your valuable feedback and insightful discussion! We hope our response has been helpful to you. As the discussion period is drawing to a close, we warmly welcome any further questions from the reviewer. We would be delighted to provide additional clarification!
We sincerely thank the reviewer for the constructive comments and feedback. We appreciate that our work is considered “meaningful motivation” and “improvements are encouraging”. We hope our response can address your concerns.
W1: Longer Text Compression
We sincerely thank the reviewer for this valuable suggestion. To further evaluate CCoR under longer-context scenarios, we construct a sub-testset of NarrativeQA by sampling examples with an average context length of approximately 30K tokens and conduct additional experiments under the same 4× compression setting using LLama3.1-8B-Instruct.
As shown in Table 1, CCoR-RAM achieves an F1 score of 25.98, outperforming RAM (24.12), LongLLMLingua (21.29), and LLMLingua (20.92). These results provide additional evidence of CCoR's effectiveness in substantially longer-context compression scenarios.
Table 1. Experimental results with longer-context compression scenarios. All F1 scores are reported inpercentage (%).
W2: Clarification of the w/o RS Ablation
We sincerely thank the reviewer for carefully identifying this inconsistency. We apologize for the reporting error in the original manuscript, which resulted from a mistake during manuscript preparation and caused the original w/o RS results to be incorrectly reported as matching the RAM baseline. We appreciate the reviewer for bringing this issue to our attention.
To address this concern, we have conducted a new controlled ablation. Specifically, the revised w/o RS variant is independently trained using the same compressor, decoder, and training stages as CCoR, with only the reasoning-aware selection criterion replaced by query–chunk semantic relevance, a selection signal commonly adopted in existing methods such as RAM. The corrected results are presented in Table 2. They show consistent performance drops across all three datasets compared with CCoR, further demonstrating the contribution of reasoning-aware selection. We sincerely appreciate the reviewer’s careful examination, which helped us identify this reporting error and improve the reliability of our experimental evaluation. These experiments have been added to section 6.2 of the revised manuscript.
Table 2. Ablation study on LLama3.1-8B-Instruct under 4×compression. All EM and F1 scores are reported inpercentage (%).
W3: Limitations of Existing Task-Aware Selection Criteria
We sincerely thank the reviewer for this thoughtful comment. We appreciate the reviewer’s observation that existing context compression methods may incorporate richer selection objectives beyond query–chunk relevance. For example, LongLLMLingua employs query-conditioned perplexity, while COMI incorporates semantic relevance and redundancy. To further examine the effectiveness of these criteria in preserving reasoning-relevant information, we conduct additional experiments.
To further investigate this concern, following the experimental setting in Section 4, we extend the pilot study by incorporating these additional selection criteria, using Reasoning Perplexity and Evidence Sufficiency Rate as evaluation metrics. As shown in table 3, The results indicate that incorporating these objectives can indeed alleviate reasoning information loss to some extent compared with relevance-only selection, but their performance still falls short of CCoR. This suggests that improving query relevance or accounting for information redundancy alone may not be sufficient to preserve the evidence required for valid reasoning trajectories. CCoR addresses this limitation by explicitly modeling the contribution of context chunks to reasoning through sufficiency and necessity. These experiments have been added to Appendix C.5 of the revised manuscript.
Table 3. Pilot study on the information gap between existing task-aware selection criteria and CCoR.