acceptodds
Under review as a conference paper at ICLR 2027

CCoR: Reasoning-Aware Context Compression via Submodular Information Gain

10 views

Abstract

Context compression has emerged as a promising paradigm to reduce high computational costs and redundant noise in Large Language Models (LLMs) across data-intensive scenarios. Existing methods span task-agnostic and task-aware approaches. Task-agnostic methods sacrifice fine-grained details by operating independently of downstream queries, whereas current task-aware paradigms remain strictly query-centric. Consequently, only text semantically relevant to the query is selectively retained, inadvertently discarding critical logical scaffolding required for multi-step derivations. In this work, we propose **CCoR** (**C**ontext **C**ompression f**o**r **R**easoning), a novel framework that preserves essential reasoning evidence rather than relying solely on query semantic relevance. Specifically, CCoR formulates context importance evaluation as a submodular maximization problem to rigorously capture sufficiency and necessity for model reasoning. Building upon this, CCoR introduces an adaptive compression mechanism that preserves high-importance chunks into distinct representations while condensing redundant content into latent summaries, optimized jointly via supervised alignment and reasoning-preservation reinforcement learning. Extensive experiments show that CCoR substantially outperforms existing baselines, achieving robust reasoning preservation under stringent compression rates. Notably, under compression, CCoR achieves an average 9.23% F1 gain over state-of-the-art (SOTA) across three long-context benchmarks using LLaMA-3.1-8B-Instruct.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Context Compression, LLM Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_CnhxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CCoR, a context compression framework that prioritizes evidence supporting downstream reasoning. It uses sufficiency- and necessity-based scores to identify important chunks, preserves their token-level representations, and compresses the remaining chunks into query-guided summary vectors. The framework is trained through supervised alignment and reasoning-preservation reinforcement learning. Experiments report improvements over several compression baselines on three QA benchmarks.

Critical strengths

The distinction between query relevance and reasoning utility is a meaningful motivation, particularly for multi-hop questions requiring intermediate evidence. The reported average improvements across two model families and multiple compression ratios are encouraging. The paper also provides ablations and qualitative examples to help explain the approach.

Critical weaknesses
  1. The evaluated contexts are too short to substantiate the broader long-context claims. Table 4 reports average context lengths of 2,933, 1,018, and 1,402 tokens for NaturalQuestions, 2WikiMQA, and HotpotQA, respectively. Even the maximum lengths remain below approximately 4.4K tokens. Although the evaluation contains a substantial number of examples, it primarily tests compression of relatively short contexts.

  2. The results of the “w/o RS” ablation require clarification. In Table 2, the w/o RS variant has exactly the same F1 scores as RAM in Table 1 across all three datasets: 62.77, 57.50, and 65.65. Its NaturalQuestions and 2WikiMQA EM scores also match RAM exactly, while its HotpotQA EM differs slightly: 52.58 versus 52.28. This pattern is not sufficient to conclude that the results are incorrect, but it raises an important question about the ablation’s provenance.

  3. Section 4 evaluates a relevance-only selection heuristic rather than the existing methods characterized in the text. The pilot study ranks chunks by query–chunk embedding cosine similarity and retains the top-ranked chunks. This experiment can demonstrate a limitation of that particular selection heuristic. It does not, by itself, establish the same limitation for existing compression methods, which may incorporate additional objectives.

  4. The architectural relationship to RAM is insufficiently emphasized. The framework basically follows RAM: some chunks are fully retained, and others are compressed, but this is not well-recognized in the paper.

Questions for authors

See the weakness section.

5 comments

We sincerely thank the reviewer for the constructive comments and feedback. We appreciate that our work is considered “meaningful motivation” and “improvements are encouraging”. We hope our response can address your concerns.

W1: Longer Text Compression

We sincerely thank the reviewer for this valuable suggestion. To further evaluate CCoR under longer-context scenarios, we construct a sub-testset of NarrativeQA by sampling examples with an average context length of approximately 30K tokens and conduct additional experiments under the same 4× compression setting using LLama3.1-8B-Instruct.

As shown in Table 1, CCoR-RAM achieves an F1 score of 25.98, outperforming RAM (24.12), LongLLMLingua (21.29), and LLMLingua (20.92). These results provide additional evidence of CCoR's effectiveness in substantially longer-context compression scenarios.

MethodsNarrativeQA
RAM24.12
LongLLMLingua21.29
LLMLingua20.92
CCoR-RAM25.98

Table 1. Experimental results with longer-context compression scenarios. All F1 scores are reported inpercentage (%).

W2: Clarification of the w/o RS Ablation

We sincerely thank the reviewer for carefully identifying this inconsistency. We apologize for the reporting error in the original manuscript, which resulted from a mistake during manuscript preparation and caused the original w/o RS results to be incorrectly reported as matching the RAM baseline. We appreciate the reviewer for bringing this issue to our attention.

To address this concern, we have conducted a new controlled ablation. Specifically, the revised w/o RS variant is independently trained using the same compressor, decoder, and training stages as CCoR, with only the reasoning-aware selection criterion replaced by query–chunk semantic relevance, a selection signal commonly adopted in existing methods such as RAM. The corrected results are presented in Table 2. They show consistent performance drops across all three datasets compared with CCoR, further demonstrating the contribution of reasoning-aware selection. We sincerely appreciate the reviewer’s careful examination, which helped us identify this reporting error and improve the reliability of our experimental evaluation. These experiments have been added to section 6.2 of the revised manuscript.

MethodsNaturalQAHotpotQA2WikiMQA
EMF1EMF1EMF1
CCoR60.6871.2953.1867.5663.5171.51
w/o RL57.1468.3952.5766.9963.2470.86
w/o ECP43.8456.2641.7755.7655.1664.16
w/o RS46.2959.1448.2862.5854.4462.74

Table 2. Ablation study on LLama3.1-8B-Instruct under 4×compression. All EM and F1 scores are reported inpercentage (%).

W3: Limitations of Existing Task-Aware Selection Criteria

We sincerely thank the reviewer for this thoughtful comment. We appreciate the reviewer’s observation that existing context compression methods may incorporate richer selection objectives beyond query–chunk relevance. For example, LongLLMLingua employs query-conditioned perplexity, while COMI incorporates semantic relevance and redundancy. To further examine the effectiveness of these criteria in preserving reasoning-relevant information, we conduct additional experiments.

To further investigate this concern, following the experimental setting in Section 4, we extend the pilot study by incorporating these additional selection criteria, using Reasoning Perplexity and Evidence Sufficiency Rate as evaluation metrics. As shown in table 3, The results indicate that incorporating these objectives can indeed alleviate reasoning information loss to some extent compared with relevance-only selection, but their performance still falls short of CCoR. This suggests that improving query relevance or accounting for information redundancy alone may not be sufficient to preserve the evidence required for valid reasoning trajectories. CCoR addresses this limitation by explicitly modeling the contribution of context chunks to reasoning through sufficiency and necessity. These experiments have been added to Appendix C.5 of the revised manuscript.

Selection CriteriaNaturalQA (4x)NaturalQA (8x)2WikiMQA (4x)2WikiMQA (8x)
PPLESR %PPLESR %PPLESR %PPLESR %
Relevance28.5272.4146.8157.5831.2669.8750.4355.48
LongLLMLingua25.6969.0339.5965.7630.6873.3347.9757.27
COMI20.7887.8737.0473.4429.2684.1742.4761.84
CCoR17.4292.3327.9388.3123.0792.7033.5270.36

Table 3. Pilot study on the information gap between existing task-aware selection criteria and CCoR.

W4: Architectural Relationship to RAM

We sincerely thank the reviewer for this helpful comment. We agree that the relationship between CCoR and RAM deserves to be more explicitly discussed. Indeed, RAM is a strong and influential work in context compression, and its hybrid representation paradigm, which combines explicit retention of important chunks with latent compression of the remaining context, partially inspired our forward compression design. We will clarify this connection in the revised manuscript.

More importantly, CCoR focuses on improving how critical chunks are identified, rather than proposing an entirely new hybrid representation architecture. To further validate this contribution, we extend CCoR's reasoning-based selection criterion and training framework to three representative compression architectures: RAM, COMI, and SeCo (Jiwei Tang and Zhijing Huang, arXiv:2605.09463). Using LLama3.1-8B-Instruct under the same 4× compression setting, we observe consistent improvements across all three architectures. Specifically, the average EM gains are 8.14, 17.26, and 9.24 points, respectively, while the average F1 gains are 8.15, 17.82, and 4.62 points. These results further highlight the effectiveness of CCoR's reasoning-based selection beyond its original architecture. We appreciate the reviewer’s suggestion, which helps us better clarify the relationship between our architectural design and methodological contributions. These experiments have been added to section 6.6 of the revised manuscript.

MethodsNaturalQAHotpotQA2WikiMQAAvg
EMF1EMF1EMF1EMF1
RAM51.1562.7752.2865.6549.5157.5050.9861.97
CCoR-RAM60.6871.2953.1867.5663.5171.5159.1270.12
COMI49.8362.0937.5851.7241.6050.3143.0054.71
CCoR-COMI58.0469.9657.8673.5264.8974.1260.2672.53
SeCo54.8069.1251.7970.7448.4163.4951.6767.78
CCoR-SeCo63.8874.7456.5071.5662.3470.8960.9172.40

Table 4. Performance comparison of CCoR applied to different compression architectures. All EM and F1 scores are reported inpercentage (%).

We sincerely appreciate your valuable feedback and insightful discussion! We hope our response has been helpful to you. As the discussion period is drawing to a close, we warmly welcome any further questions from the reviewer. We would be delighted to provide additional clarification!

Reviewer_tRFEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CCoR is a training-based, two-stage reasoning-aware context-compression framework for long-context / multi-document QA. Stage 1 formulates chunk selection as a submodular maximization over f(S)=(1−λ)·Suff(S)+λ·Nec(S), where Suff(S) is the decoder's teacher-forced likelihood of the gold reasoning chain given the subset and Nec(S) is a leave-S-out drop in that likelihood; a greedy pass yields per-chunk importance labels. Stage 2 builds a hybrid context (critical chunks kept as full hidden-state sequences; non-critical chunks compressed into a single query-guided pooled vector), distills the offline importance into a lightweight scoring head, and optimizes decoder utilization via SFT (generation + BCE score calibration) and GRPO. On NaturalQA/2WikiMQA/HotpotQA with LLaMA-3.1-8B and Qwen3-8B, CCoR outperforms task-agnostic and task-aware baselines across 4×–32× compression (~+9.23% average F1 over SOTA at 8×), shows a monotone scaling trend, and remains robust at 32× (46.0% EM on NaturalQA vs. COMI's 6.0%).

Critical strengths

1.A well-motivated and empirically validated core signal. The claim that query semantic relevance ≠ reasoning necessity is crisply supported by the §4 pilot study (relevance-only selection: ESR drops to ~56–58% and gold-reasoning perplexity surges at 8×), and the signal is validated by the ablation: removing reasoning-aware selection (w/o RS) drops 2WikiMQA F1 by ~14 points — exactly the multi-hop setting where bridging evidence is weakly query-aligned. This is consequential because it isolates a concrete, reusable idea (score chunks by their contribution to the gold reasoning chain, not query similarity) that demonstrably helps on the hardest cases.

2.Consistent results with transparent ablations. CCoR wins across both backbones, all three datasets, and 4×–32× ratios, shows a monotone scaling trend, and degrades gracefully at 32× where competitors collapse. The component ablation (Table 2) makes per-component attribution auditable, which materially aids assessment.

Critical weaknesses

1.The main performance driver (the hybrid "ECP" architecture) is inherited, not novel, and this is not made explicit. The w/o ECP ablation is by far the largest drop (NaturalQA F1 71.29→56.26, −15 pts; HotpotQA 67.56→55.76, −12 pts), so the hybrid "full hidden states for critical chunks + a single query-guided pooled vector for the rest" is what carries performance. Yet this architecture — including the non-critical compression operator in Eq. 6 (w_{i,t}=softmax_t(sim(h_{i,t},r_q)); v_i=Σ_t w_{i,t} h_{i,t}, with r_q from query mean-pooling) — appears symbol-for-symbol identical to RAM's skimming operator, and the hard/soft hybrid paradigm is RAM's; RAM is CCoR's own SOTA baseline. This is consequential because the paper's novelty, as framed, overstates the architectural contribution: CCoR's genuine delta is confined to the selection signal and training recipe, and the paper does not quantify how much of its gain comes from the new signal versus the inherited architecture.

2.Overstated theoretical rigor around a non-submodular objective. The contribution is repeatedly framed as "rigorous/principled submodular maximization" with a (1−1/e) guarantee (Appendix A.2), but the authors themselves concede in §5.1 and A.2 that f(S) does not strictly satisfy submodularity for LLM likelihoods; the proof is the standard Nemhauser argument under an assumed submodularity that does not hold for their objective. This is consequential because a central framing claim ("rigorous/principled") is not actually established — the theory decorates a well-motivated heuristic rather than guaranteeing it, and a reader could over-trust the method's optimality properties.

3.CCoR is not training-free — Stage 1 needs correctness-filtered gold reasoning chains and many likelihood evaluations, training uses 8×A800, and inference relies on a distilled scoring head whose approximation quality, not the submodular greedy, is the real ceiling;

4.and the "marginal gain" terminology collides with COMI's MIG while fixed-length chunking can fracture evidence.

Questions for authors

1.Clarification affecting assessment. Can you isolate the true contribution by transplanting only the reasoning-likelihood signal into the RAM/COMI architecture (same backbone/budget)? How much of CCoR's gain is the signal versus the inherited hybrid architecture?

2.Clarification affecting assessment. Do you claim any formal approximation guarantee for the greedy maximization of f(S) given that Suff/Nec from LLM likelihoods are not submodular? If not, please restate the (1−1/e) result as illustrative and remove "rigorous/principled" from the central claim.

3.Clarification affecting assessment. Please report end-to-end training cost, the dependence on gold reasoning chains (and degradation when they are unavailable/noisy), and the accuracy gap between the offline submodular selection and the distilled scoring head used at inference.

4.Clarification affecting assessment. How do you differ, in principle and empirically, from COMI (MIG = relevance − redundancy), and how sensitive are results to the fixed-length chunk size L_seg given the acknowledged risk of splitting bridging evidence?

5 comments

We sincerely thank the reviewer for the constructive comments and feedback. We appreciate that our work is considered “well-motivated and empirically validated core signal” and “consistent results with transparent ablations”. We hope our response can address your concerns.

W1 & Q1: Isolating the Contribution of CCoR

We sincerely thank the reviewer for this thoughtful comment. We would first like to clarify that the w/o ECP ablation removes explicit preservation of critical chunks and instead uniformly compresses all chunks into soft representations. This creates a relatively weak baseline, as even reasoning-critical evidence loses its explicit representation. Therefore, the substantial performance drop reflects the importance of preserving critical evidence. We also acknowledge RAM as an influential work in context compression, whose hybrid representation paradigm combining explicit text retention and latent compression provided valuable inspiration for our design. Our primary methodological contribution lies in how critical chunks are identified, rather than in the hybrid representation paradigm itself. Specifically, CCoR introduces reasoning-based selection through sufficiency and necessity, together with a corresponding training framework to identify and preserve evidence essential for valid reasoning trajectories. We appreciate the reviewer’s suggestion and will further clarify the relationship between our architectural design and prior work, as well as more precisely distinguish the architectural choices from our methodological contributions.

Motivated by the reviewer's suggestion, we further evaluate the effectiveness of our core reasoning-based selection and training criterion independently of the underlying compression architecture. Specifically, using LLama3.1-8B-Instruct as the backbone under 4×compression, we adapt CCoR's reasoning-based data construction and training framework to RAM, COMI, and SeCo (Jiwei Tang and Zhijing Huang, arXiv:2605.09463), while keeping the backbone and compression budget consistent. This allows us to evaluate the effectiveness of the proposed selection criterion beyond our original architecture. The results are presented in Table 1. The results show that CCoR consistently improves all three architectures, achieving average EM gains of 8.14, 17.26, and 9.24 points on RAM, COMI, and SeCo, respectively. These results demonstrate that reasoning-based selection generalizes beyond CCoR's original architecture. These experiments have been added to section 6.6 of the revised manuscript.

MethodsNaturalQAHotpotQA2WikiMQAAvg
EMF1EMF1EMF1EMF1
RAM51.1562.7752.2865.6549.5157.5050.9861.97
CCoR-RAM60.6871.2953.1867.5663.5171.5159.1270.12
COMI49.8362.0937.5851.7241.6050.3143.0054.71
CCoR-COMI58.0469.9657.8673.5264.8974.1260.2672.53
SeCo54.8069.1251.7970.7448.4163.4951.6767.78
CCoR-SeCo63.8874.7456.5071.5662.3470.8960.9172.40

Table 1. Performance comparison of CCoR applied to different compression architectures. All EM and F1 scores are reported inpercentage (%).

W2 & Q2: Theoretical Clarification

We sincerely thank the reviewer for this careful and constructive comment. We would like to clarify that we do not claim a formal approximation guarantee for the greedy maximization of the actual utility function f(S)f(S). As discussed in §5.1 and Appendix A.2, we acknowledge that the utility derived from LLM likelihoods may not strictly satisfy submodularity. Accordingly, the (1−1/e)(1-1/e) result in Appendix A.2 is established under the standard assumptions of non-negativity, monotonicity, and submodularity, and serves as an illustrative theoretical result for the greedy strategy under these conditions, rather than a guarantee for our actual objective. We apologize if our original wording gave the impression of a stronger theoretical claim than intended.

We appreciate the reviewer’s suggestion regarding the use of terms such as "rigorous" and "principled." Although our intention was to emphasize the theoretically motivated formulation of the selection objective, we agree that these expressions could be interpreted as implying stronger guarantees than we establish. We will therefore refine the relevant wording throughout the manuscript, making it clearer that greedy selection is an efficient heuristic for our reasoning-based utility, while the approximation result is conditional on the standard assumptions. We sincerely appreciate the reviewer’s guidance in helping us present the theoretical contribution with greater precision.

W3 & Q3: Dependence on Gold Reasoning Chains and Scoring Accuracy

We sincerely thank the reviewer for this valuable suggestion. To further investigate CCoR's dependence on gold reasoning chains, we conduct two additional experiments. In CCoR w/o GR (Gold Reasoning), we remove the use of ground-truth for reasoning-path selection. In CCoR w Pert., we introduce paragraph-level perturbations by rewriting the original reasoning chains to examine the impact of noisy reasoning supervision [1].

As shown in Table 2, removing gold reasoning chains and introducing perturbations both lead to performance degradation, indicating the importance of reliable reasoning supervision. Nevertheless, both variants still outperform RAM and COMI in average EM and F1, demonstrating the robustness of CCoR under imperfect or unavailable reasoning supervision. Furthermore, we report the accuracy of the distilled scoring head in the last row, achieving an average accuracy of 85.24% across the three datasets. This provides an empirical assessment of how well the scoring head approximates the offline selection decisions and supports its use during inference. These experiments have been added to Appendix C.4 of the revised manuscript.

MethodsNaturalQAHotpotQA2WikiMQAAvg
EMF1EMF1EMF1EMF1
COMI49.8362.0937.5851.7241.6050.3143.0054.71
RAM51.1562.7752.2865.6549.5157.5050.9861.97
CCoR60.6871.2953.1867.5663.5171.5159.1270.12
CCoR w/o GR58.3668.7352.4466.3960.3067.8257.0367.83
CCoR w Pert.57.1068.1250.2464.3155.8366.4154.3966.28
Scoring Acc.89.7882.3383.6085.24

Table 2. Performance comparison of CCoR on LLaMA-3.1-8B under 4×compression applied to different settings. All EM, F1 and ACC scores are reported inpercentage (%).

[1] Zhou Z, Tao R, Zhu J, et al. Can language models perform robust reasoning in chain-of-thought prompting with noisy rationales?[J]. Advances in Neural Information Processing Systems, 2024, 37: 123846-123910.

Q3: End-to-End Training Cost

We sincerely thank the reviewer for this suggestion. To provide a clearer picture of the computational overhead, we report the end-to-end training time and average performance of CCoR and several representative compression baselines, including PCC, COMI, and RAM. All experiments were conducted using 8×A800 GPUs. As shown in Table 3, CCoR requires 2h52m for SFT and an additional 3h32m for RL, achieving an average performance of 59.12/70.12 (EM/F1). Although RL introduces additional training cost, it brings substantial performance improvements. Moreover, compared with PCC, which requires an additional pretraining stage of 32h30m, CCoR achieves stronger performance with a considerably shorter overall training time. These results suggest that the additional cost of RL is reasonable given the performance gains it brings.

MethodsPretrainSFTRLPerformance
PCC32h30m7h52m-33.22/44.90
COMI-3h13m-43.00/54.71
RAM-2h48m-50.98/61.97
CCoR-2h52m3h32m59.12/70.12

Table 3. Performance comparison of end-to-end training cost. Performance metrics are EM/F1.

W4 & Q4: Length Sensitivity of CCoR

We appreciate the reviewer’s concern regarding the potential impact of chunk length on CCoR. To investigate this, we evaluated CCoR using different chunk length, with results summarized in Table 4. Specifically, under a 4× compression ratio with LLama3.1-8B-Instruct as the backbone, we report both Exact Match (EM) and F1 scores across all three QA benchmarks, together with the population standard deviation (Std) of CCoR's performance across different chunking settings on each dataset. The results show that CCoR maintains consistently strong performance across varying segmentation granularities, while exhibiting relatively low Std on all benchmarks. This suggests that its effectiveness is not sensitive to a particular chunk length, providing further evidence of its robustness to different segmentation choices. These experiments have been added to section 6.7 of the revised manuscript.

Chunk LengthNaturalQA2WikiMQAHotpotQA
F1EMF1EMF1EM
5071.2960.6871.5163.5167.5653.18
10069.8258.4572.2464.9066.9053.05
15070.3959.8370.6062.4266.4852.28
Std.0.610.910.671.010.450.40

Table 4. Experimental results with different segment length for CCoR.

W4 & Q4: Differents between CCoR and COMI

We thank the reviewer for pointing out this potential ambiguity. CCoR's marginal gain differs fundamentally from COMI's Marginal Information Gain (MIG). COMI defines MIG as query relevance minus maximum semantic redundancy, i.e., (MIG(ci)=Rel(ci,Q)−max⁡j≠iSim(ci,cj)\mathrm{MIG}(c_i)=\mathrm{Rel}(c_i,Q)-\max_{j\neq i}\mathrm{Sim}(c_i,c_j)), aiming to preserve relevant and non-redundant information. In contrast, CCoR defines a set-level reasoning utility, (f(S)=(1−λ)Suff(S)+λNec(S)f(S)=(1-\lambda)\mathrm{Suff}(S)+\lambda\mathrm{Nec}(S)), where sufficiency and necessity measure how selected chunks support valid reasoning and how reasoning capability degrades when they are removed, respectively. Its marginal gain, (Δf(ci∣S)=f(S∪ci)−f(S)\Delta f(c_i\mid S)=f(S\cup{c_i})-f(S)), therefore captures each chunk's incremental contribution to reasoning preservation. In short, COMI measures semantic novelty, whereas CCoR measures reasoning contribution.

To qualitatively illustrate the differences between COMI and CCoR, as well as the advantage of having a reasoning awareness mechanism, we show representative examples from three benchmarks: MultihopQA, HotpotQA, and NaturalQuestions. Overall, these cases illustrate that COMI primarily captures what appears lexically relevant to the query, which can easily mislead the generation model. CCoR transcend surface-level alignment and prioritize preserving the fundamental evidence according to its actual contribution to the downstream reasoning path. For example, in Case 1 (2WikiMultihopQA), which requires comparing the ages of two entities, COMI is distracted by the high lexical overlap of the name ``Bausch''. In contrast, CCoR precisely identifies the birth dates of both target individuals (April 18, 1945 and April 29, 1758), providing the necessary logical bridge for the comparison.

2WikiMQA:

Query: Who is younger, Richard Bausch or Johann Heinrich Lips?
Chain-of-Thought: Step 1: Richard Bausch was born on April 18,1945. Step2: Johann Heinrich Lips was born on April 29,1758.
Gold Answer: Richard Bausch
CCoR top-1 chunk: ...Richard Bausch (born \reason{April 18, 1945}) is an American novelist and short story writer, and Professor in the Writing Program at Chapman University...
COMI top-1 chunk: ...He has published twelve novels, eight short story collections, and one volume of poetry and prose. Bausch holds a B.A. from George Mason University, and an M.F.A. from the Iowa Writers' Workshop at the University of...

HotpotQA:

Query: Which actor and film director is from German Jewish descent, Kurt Gerron or Dell Henderson?
Chain-of-Thought: Step 1: Kurt Gerron was a German Jewish actor and film director.Step 2: Dell Henderson was a Canadian-American actor, director, and writer, with no mention of German Jewish descent.
Gold Answer: Kurt Gerron
CCoR top-1 chunk: Kurt Gerron (11 May 1897 -- 28 October 1944) was a German Jewish actor and film director. My Wife, the Impostor...
COMI top-1 chunk: ...in an avalanche while climbing the Piz Palü mountain, and spends the next few years searching the mountain alone for her body. Four years later he meets a young couple who agree to accompany him on his next climb. \"The White Hell of P...

NaturalQA

Query: who turned out to be the mother on how i met your mother?
Chain-of-Thought: Step 1: The character known as ``The Mother'' in How I Met Your Mother is Tracy McConnell. Step 2: Tracy McConnell is played by Cristin Milioti.
Gold Answer: Tracy McConnell
CCoR top-1 chunk: The Mother (How I Met Your Mother). Tracy McConnell, better known as ``The Mother'', is the title character from the CBS television sitcom How I Met Your Mother...
COMI top-1 chunk: a snowstorm in 2024. They pass the time by telling each other stories, but come to realize that they have told. The Mother (How I Met Your Mother)\nseason 8 having never watched \"How I Met Your Mother...
Reviewer_J9MCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to compress long contexts while preserving the information needed for downstream LLM reasoning. The authors argue that existing task-aware compression methods primarily rely on query relevance, which may overlook intermediate evidence required for multi-step reasoning.

To address this problem, the paper proposes CCoR, a reasoning-aware context compression framework. It estimates the importance of context chunks using a utility function combining reasoning sufficiency and necessity, and applies greedy selection to identify critical evidence. The framework then constructs a hybrid compressed representation by preserving important chunks at the token level while summarizing less important chunks into latent vectors. A lightweight scoring head approximates the importance scores at inference time, and the compressor and decoder are trained through supervised alignment followed by reasoning-preservation reinforcement learning.

The authors evaluate CCoR on three QA benchmarks using LLaMA-3.1-8B and Qwen3-8B under different compression ratios. The reported results show substantial improvements over existing context compression baselines, particularly under aggressive compression. The paper also includes ablation studies, scaling experiments, and analyses of reasoning preservation and computational efficiency.

Critical strengths
  1. The paper addresses a real difficulty: evidence needed for an intermediate reasoning step may have little surface similarity to the original question. Keeping such evidence explicitly available to the decoder is a sensible design choice.

  2. The empirical gains are substantial. At 8× compression, average F1 exceeds RAM by 9.23 points on LLaMA-3.1-8B and 6.13 points on Qwen3-8B. The results across compression budgets make a convincing case that the complete system is effective.

Critical weaknesses
  1. The introduction draws too broad a distinction between prior task-aware compression and reasoning-aware selection. LongLLMLingua uses question-conditioned perplexity, and COMI considers marginal information gain and redundancy; these are richer signals than independent semantic similarity. IterCOMP (ACL 2026) also addresses reasoning-aware compression for multi-hop QA, using answerability estimation and follow-up questions to gather evidence. Its relationship to CCoR deserves discussion.

Since RAM already combines explicit text retention with latent compression, the main contribution seems to be the reasoning-based selection criterion and its integration into training. I would find the novelty easier to assess if the paper focused on this distinction. The strong overall results do not by themselves tell me how much the new criterion adds.

  1. Two details make that contribution difficult to assess. First, Appendix A.2 proves the standard greedy guarantee under explicit submodularity and monotonicity assumptions, but the paper does not establish those assumptions for its own utility function. The authors acknowledge this limitation. The issue is that the surrounding discussion still presents the guarantee as support for CCoR's selection procedure. Complementary evidence in multi-hop QA makes diminishing returns particularly uncertain. The method may work well as a heuristic; the theoretical claim should reflect that.

Second, the "w/o Reasoning-aware Selection" row in Table 2 exactly matches the RAM results in Table 1 across the three datasets. This may have an explanation, but I could not tell whether it is an independently trained ablation with only the selection criterion changed. If the rest of the pipeline is held fixed, this would address much of my concern about isolating the contribution. If RAM results were reused, the comparison would not establish the same point.

Questions for authors
  1. How would you distinguish CCoR's contribution from RAM's hybrid compression and IterCOMP's reasoning-aware evidence selection? A focused discussion would help clarify the novelty.

  2. Was the "w/o Reasoning-aware Selection" variant trained with the same compressor, decoder, and training stages as CCoR? Please explain why its scores match RAM exactly. If these are reused baseline results, a controlled comparison changing only the selection criterion on one setting would be useful.

  3. Does the approximation guarantee apply to the actual utility in Eq. (5), or is it background for the greedy heuristic? If it is the latter, please make that limitation explicit wherever the guarantee is invoked.

4 comments

We sincerely thank the reviewer for the constructive comments and feedback. We appreciate that our work is considered “addresses a real difficulty” and “the empirical gains are substantial”. We hope our response can address your concerns.

W1 & Q1: Comparison with Related Work

We sincerely thank the reviewer for this insightful comment. We agree that our original characterization of prior task-aware compression methods was overly broad. In particular, LongLLMLingua employs query-conditioned perplexity, while COMI incorporates marginal information gain and semantic redundancy, both of which go beyond simple semantic similarity. We will revise the Introduction and Related Work to provide a more accurate characterization of these approaches.

We also appreciate the reviewer for pointing out IterCOMP (Yun and Kim, ACL 2026), which is closely related to our work. We will include it in our Related Work and clarify the distinction. While both approaches explore reasoning-aware context compression, they address complementary aspects of the problem. IterCOMP adopts an iterative evidence-refinement paradigm, using answerability estimation and targeted follow-up questions to progressively accumulate sufficient evidence for multi-hop QA. In contrast, CCoR explicitly models the contribution of context chunks to reproducing valid reasoning trajectories through reasoning sufficiency and necessity, and further employs hybrid context representations to preserve critical evidence while compressing less important content into latent representations. Thus, IterCOMP focuses on iterative reasoning-guided evidence acquisition, whereas CCoR focuses on reasoning-trajectory-based evidence importance modeling and adaptive representation optimization. We thank the reviewer for helping us improve the positioning of our work. These discussions have been added to the intro and related work sections of the revised manuscript.

W2: Novelty Beyond RAM and Contribution of the Selection Criterion

We sincerely thank the reviewer for this insightful comment. We agree that RAM is a strong and closely related work in the field of context compression. Indeed, our forward compression design was partially inspired by RAM's hybrid representation strategy, which combines explicit text retention with latent compression.

The reviewer’s insightful suggestion also prompted us to recognize that our reasoning-based selection criterion and its integration into training are not inherently tied to a specific compression architecture, but can be extended to different compression frameworks. To further investigate its effectiveness beyond the overall performance of our original framework, we extend CCoR to three representative compression architectures, namely RAM, COMI, and SeCo (Jiwei Tang and Zhijing Huang, arXiv:2605.09463). Specifically, using LLama3.1-8B-Instruct as the backbone under 4×compression, we adapt CCoR data construction and training framework to each architecture, enabling a more direct evaluation of the contribution of reasoning-based selection across different compression designs. The experimental results are presented in Table 1.

The results show that incorporating CCoR's reasoning-based selection framework consistently improves all three compression architectures. Specifically, compared with their original counterparts, CCoR yields average EM improvements of 8.14, 17.26, and 9.24 points on RAM, COMI, and SeCo, respectively, with corresponding average F1 improvements of 8.15, 17.82, and 4.62 points. These consistent gains across different architectures provide further evidence that the effectiveness of reasoning-based selection is not limited to CCoR's original hybrid architecture, but can be transferred to diverse compression frameworks. These experiments have been added to section 6.6 of the revised manuscript.

MethodsNaturalQAHotpotQA2WikiMQAAvg
EMF1EMF1EMF1EMF1
RAM51.1562.7752.2865.6549.5157.5050.9861.97
CCoR-RAM60.6871.2953.1867.5663.5171.5159.1270.12
COMI49.8362.0937.5851.7241.6050.3143.0054.71
CCoR-COMI58.0469.9657.8673.5264.8974.1260.2672.53
SeCo54.8069.1251.7970.7448.4163.4951.6767.78
CCoR-SeCo63.8874.7456.5071.5662.3470.8960.9172.40

Table 1. Performance comparison of CCoR applied to different compression architectures. All EM and F1 scores are reported inpercentage (%).

W3 & Q3: Theoretical Clarification

We sincerely thank the reviewer for this insightful comment. We agree that the theoretical guarantee should be clearly distinguished from the behavior of our actual utility function. To clarify, the (1−1/e)(1-1/e) approximation guarantee in Appendix A.2 is a conditional theoretical result, rather than a formal guarantee for the actual utility function in Eq. (5). Our utility combines reasoning sufficiency and necessity to quantify the contribution of context chunks to reproducing valid reasoning trajectories. While inspired by submodular optimization, we do not establish that it strictly satisfies submodularity and monotonicity over all possible subsets. As already acknowledged in Appendix A.2, the proof assumes these standard properties and illustrates the theoretical behavior of greedy optimization under ideal conditions.

We also agree that these assumptions can be particularly difficult to guarantee in practical multi-hop reasoning scenarios, where evidence chunks may exhibit complex dependencies and their marginal contributions do not necessarily diminish as more evidence is selected. Accordingly, we view greedy selection in CCoR as an efficient heuristic for optimizing our reasoning-based utility function, rather than a procedure with a strict approximation guarantee for the actual objective. Following the reviewer's suggestion, we will further clarify this distinction in both the main text and the appendix, explicitly stating the conditional nature of the theoretical bound and revising relevant descriptions to avoid overstating the theoretical guarantees.

W4 & Q2: Ablation Study

We sincerely thank the reviewer for carefully identifying this inconsistency. We apologize for the error in our paper, which was caused by a mistake during manuscript preparation. The original w/o Reasoning-aware Selection results were incorrectly reported as identical to the RAM baseline. We appreciate the reviewer for bringing this issue to our attention and will carefully check the revised manuscript to prevent similar errors.

Following the reviewer's suggestion, we have conducted a new controlled ablation. Specifically, the revised w/o RS variant is independently trained using the same compressor, decoder, and training stages as CCoR, with only the reasoning-aware selection criterion replaced by query–chunk semantic relevance, a commonly adopted selection signal in existing methods, including RAM. The corrected results are presented in Table 2. The results show that removing reasoning-aware selection consistently reduces performance across all three datasets, highlighting its contribution beyond the shared architecture and training pipeline. We sincerely appreciate the reviewer’s suggestion, which helped us identify this reporting error and strengthen the validity of our ablation study. These experiments have been added to section 6.2 of the revised manuscript.

MethodsNaturalQAHotpotQA2WikiMQA
EMF1EMF1EMF1
CCoR60.6871.2953.1867.5663.5171.51
w/o RL57.1468.3952.5766.9963.2470.86
w/o ECP43.8456.2641.7755.7655.1664.16
w/o RS46.2959.1448.2862.5854.4462.74

Table 2. Ablation study on LLama3.1-8B-Instruct under 4×compression. All EM and F1 scores are reported inpercentage (%).

We sincerely appreciate your valuable feedback and insightful discussion! We hope our response has been helpful to you. As the discussion period is drawing to a close, we warmly welcome any further questions from the reviewer. We would be delighted to provide additional clarification!

We sincerely thank the reviewer again for the time and care invested in this review. As the discussion period is coming to a close, we want to make sure that our responses have reached you and that no concern remains unaddressed. We remain glad to provide any further clarification.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response — Thanks to All Reviewers and AC for Their Constructive Feedback

We sincerely thank the Area Chair and all reviewers for their careful reading and constructive feedback. We appreciate the reviewers' recognition of CCoR's motivation and empirical effectiveness in preserving reasoning-critical evidence under context compression. Reviewer J9MC recognized the importance of preserving intermediate evidence for multi-hop reasoning and highlighted the substantial empirical gains over existing compression methods. Reviewer tRFE appreciated the well-motivated and empirically validated core idea, as well as the consistent results and transparent component ablations. Reviewer Cnhx also acknowledged the meaningful distinction between query relevance and reasoning utility, and found the improvements across different models and compression ratios encouraging.

The reviewers also raised important questions regarding the novelty of our selection criterion beyond existing hybrid compression architectures, the theoretical justification of the greedy optimization, and the robustness and practical applicability of CCoR. Their feedback motivated additional experiments and clarifications, which we summarize below.

  1. Generalization beyond existing compression architectures (Reviewers J9MC, tRFE, and Cnhx). We further clarified that our primary contribution lies in reasoning-based evidence selection and its integration into training, rather than introducing an entirely new hybrid compression architecture. Motivated by the reviewers' suggestions, we extended CCoR's reasoning-based selection and training framework to three representative architectures: RAM, COMI, and SeCo. Under the same backbone and compression budget, reasoning-based selection consistently improves all three architectures, achieving average EM gains of 8.14, 17.26, and 9.24 points, respectively, and average F1 gains of 8.15, 17.82, and 4.62. These results provide further evidence that the proposed selection criterion generalizes beyond CCoR's original architecture.

  2. Clarification of related work and selection criteria (Reviewers J9MC and Cnhx). We refined our discussion of existing task-aware compression methods, particularly LongLLMLingua and COMI, and added a discussion of IterCOMP to better position CCoR relative to existing reasoning-aware methods. Furthermore, we expanded the pilot study by incorporating query-conditioned perplexity and semantic relevance with redundancy modeling. Using Reasoning Perplexity and Evidence Sufficiency Rate on NaturalQA and 2WikiMQA under different compression ratios, we show that although these criteria help preserve reasoning information, CCoR achieves consistently better results by explicitly modeling the contribution of evidence to reasoning paths.

  3. Clarification of theoretical guarantees (Reviewers J9MC and tRFE). We clarified that the (1−1/e)(1-1/e) approximation guarantee in Appendix A.2 is conditional on the standard assumptions of monotonicity and submodularity, rather than a formal guarantee for CCoR's actual utility function. Since these assumptions are not established for our LLM-likelihood-based objective, we explicitly characterize greedy selection as an efficient heuristic and refine the relevant theoretical descriptions to avoid overstating the guarantees.

  4. Controlled ablations and supervision robustness (Reviewers J9MC, tRFE, and Cnhx). We identified and corrected a reporting error in the original w/o Reasoning-aware Selection ablation and conducted a new independently trained controlled experiment, keeping the compressor, decoder, and training stages unchanged while replacing only the selection criterion. The corrected results show consistent performance degradation without reasoning-aware selection. We also evaluated settings without gold reasoning supervision and with perturbed reasoning chains. Both variants retain advantages over RAM and COMI, while the distilled scoring head achieves 85.24% average accuracy in approximating offline selection decisions.

  5. Practical efficiency and longer-context evaluation (Reviewers tRFE and Cnhx). We reported end-to-end training costs, including 2h52m for SFT and 3h32m for RL on 8×A800 GPUs, and examined the sensitivity to chunk length, observing stable performance across different segmentation settings. To address concerns about the relatively short contexts in the original benchmarks, we additionally constructed a NarrativeQA subset with approximately 30K-token contexts. Under 4× compression, CCoR-RAM achieves an F1 score of 25.98, outperforming RAM, LongLLMLingua, and LLMLingua.

We believe these additional experiments and clarifications strengthen the empirical validation and positioning of CCoR. We sincerely appreciate the reviewers' constructive feedback, which helped us identify weaknesses in our original presentation, improve the rigor of our experimental evaluation, and clarify the scope of contributions. We thank the Area Chair and all reviewers again for their time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.