acceptodds
Under review as a conference paper at ICLR 2027

COPS: First-Principles Long-Context Code Compression via Constructing Context Optimal Preservation Sets

Abstract

Code large language models incur substantial computational costs and suffer from interference caused by irrelevant context during long-context reasoning. Existing code compression methods typically reduce the problem to a static ranking task based on a single context–query relevance criterion, rather than deriving the compression objective from first principles as a dynamic subset-selection problem. Consequently, they suffer from severe performance degradation when the token budget is aggressively reduced, while still retaining considerable information redundancy that interferes with downstream tasks. In this paper, we revisit long-context code compression from an information-theoretic perspective and formulate it as constructing Context Optimal Preservation Sets. Through a top-down decomposition of the mutual-information objective, we identify three fundamental principles that govern effective context selection: task relevance, dependency coverage, and information gain. Based on these principles, we propose COPS, a training-free, plug-and-play framework that instantiates all criteria with Approximated Mutual Information (AMI) and constructs preservation sets through a principled coarse-to-fine pipeline: (i) AMI-based recall followed by greedy preservation-set construction over candidate contexts; (ii) an information-saturation trigger that dynamically terminates context expansion via a one-sided concentration bound; and (iii) perplexity-anchored semantic block partitioning with positive-advantage iterative pruning for intra-function refinement. Extensive experiments on long-context benchmarks for code completion, summarization, and question answering demonstrate that COPS consistently establishes a superior Pareto frontier between compression ratio and downstream performance, achieving up to compression with negligible degradation while significantly reducing inference latency and memory consumption.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Long-Context Code Compression, Prompt Compression, Mutual Information, Code LLMs, Code Intelligence

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JukTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes COPS, a training-free framework for long-context code compression. It combines perplexity-based scoring, greedy context selection, adaptive stopping, and semantic-block pruning. Experiments on code completion, summarization, and question answering show promising trade-offs between compression and downstream performance.

Critical strengths
  1. The method addresses a practical problem and does not require additional training.
  2. The paper reports end-to-end latency and memory consumption alongside task performance.
Critical weaknesses
  1. The paper has limited novelty in its techniques. AMI is simply perplexity reduction. For a fixed query, its ranking is equivalent to conditional-perplexity ranking, already used in LongCodeZip. The information-theoretic terminology does not clearly establish a new scoring method.
  2. The paper makes insufficient distinction from prior work. Balancing relevance and redundancy in context selection is an established idea. COMI already combines these criteria through marginal information gain in a coarse-to-fine framework. The paper needs to explain what COPS contributes beyond this principle and standard greedy selection.
Questions for authors

What distinguishes COPS from existing relevance–redundancy compression approaches?

3 comments

We thank the reviewer for suggesting that we clarify the differences between our method and prior work.

We agree that AMI-based relevance scoring and the balance between relevance and redundancy are established components of existing methods. COMI primarily targets general text, whereas COPS targets code, which differs in both structure and semantics. In general text, low similarity to the query vector is more likely to indicate weak relevance. In code, however, two context snippets with low similarity may provide complementary information for predicting the query, making both worth retaining. Code also contains implicit dependencies: as the selected context changes, the utility of other snippets that depend on it often changes as well. Static selection may therefore be insufficient. COMI requires training to compress general-text representations, whereas COPS is training-free and plug-and-play, and selects code snippets dynamically.

Our contributions include evaluating candidate snippet utility as the retained context evolves, integrating this evaluation with directed coverage and adaptive stopping, and introducing a novel method for pruning code at the semantic-block level.

1. Selection conditioned on the retained context versus LongCodeZip’s static relevance ranking

For a fixed query, AMI ranking is indeed equivalent to ranking by conditional perplexity. We do not claim that AMI itself is a new estimator. However, using the same relevance metric does not imply an identical selection procedure. LongCodeZip’s coarse-grained stage ranks functions by their individual relevance to the query, whereas COPS also evaluates each candidate’s additional contribution given the currently selected context.

Specifically, our query-gain term simplifies to

\left[ \operatorname{PPL}(Q\mid S) -\operatorname{PPL}(Q\mid S\cup\{c_i\}) \right]_+.$$ Thus, even a candidate with high individual relevance may provide little additional gain if its useful information is already present in $S$. This distinction matters for code generation and understanding. Independent ranking may repeatedly select similar usage examples while overlooking definitions that are less directly relevant to the query but specify parameter constraints, exceptional conditions, or resource-management requirements. The generated code may consequently follow inappropriate API usage patterns, while code understanding may miss preconditions or exceptional execution paths. These possible failure modes motivate evaluating complementary evidence alongside relevance. COPS addresses this limitation within the retrieved candidate pool through scoring conditioned on the currently selected context and coverage across snippets. **2. Different operational definitions of redundancy in COMI and COPS** COMI defines redundancy using the maximum cosine similarity between a unit and other units in the relevant context or group. COPS’s conditional query gain measures how much additional predictive utility a candidate provides given the assembled selected context. These criteria are not equivalent, because similarity between representations does not necessarily imply that snippets are interchangeable for a particular code task. For example, two highly similar functions may encode different boundary conditions, making both worth retaining. Conversely, structurally different snippets may provide redundant evidence for the same query. Our conditional gain can reflect these distinctions when the scoring model captures them. The coverage score provides another signal. Directed perplexity affinity evaluates how one snippet helps predict another, rewarding additional coverage of snippets. This differs from penalizing similarity between candidates. The affinity remains a model-dependent proxy for dependency information. The methods also differ architecturally. COMI compresses learned representations through token merging and model adaptation. COPS directly extracts the original code text without task-specific training for compression, so the compressed context can be used by a different downstream model. **3. Contributions beyond standard greedy selection** Greedy selection itself is not our claimed innovation. Our contribution is its specific integration with directed coverage, query gain conditioned on the currently selected context, and a saturation-triggered stopping mechanism in the later inclusion stage. Together, these mechanisms address two complementary questions: which snippets provide additional evidence, and whether further context expansion is worthwhile. The ablation results in Table 2 support the practical value of combining these selection signals. On code completion, the full method achieves ES/EM of 56.94/31.40, compared with 56.22/30.40 for relevance-only selection. On RepoQA, the corresponding accuracies are 85.5 and 84.6.

Dear Reviewer, The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.

Best, AC

Thanks for the reply. I would like to keep my rating, as my concern on the novelty stays.

Reviewer_WD7xOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

COPS is a training-free framework for compressing long code contexts. It motivates context selection through a mutual-information objective and operationalizes task relevance, dependency coverage, and marginal information gain using reductions in language-model perplexity. Candidate functions are selected through a greedy coarse stage, an adaptive saturation rule determines when selection should stop, and retained functions are further pruned into semantic blocks. Experiments cover code completion, module summarization, and repository-level QA across open and closed generation models.

Critical strengths
  1. The method addresses a practically important problem and combines query relevance, coverage, redundancy control, and within-function refinement in a coherent pipeline. It is training-free and can be transferred across compressor and generator models.

  2. The empirical evaluation is broad. In addition to unconstrained operating points, the authors include matched-retention comparisons, multiple code tasks, multiple backbones, ablations, closed-model generation, and latency/memory measurements.

Critical weaknesses
  1. The claimed first-principles derivation is not sound as written. Since the selected set SS is a subset/function of CC, maximizing I(S;C,Q)I(S;C,Q) largely reduces to maximizing information or entropy in SS; it does not establish that downstream reasoning from (S,Q)(S,Q) is consistent with reasoning from (C,Q)(C,Q). Likewise, perplexity reduction is a model-dependent heuristic rather than an estimator of the stated mutual-information quantities. These issues undermine the theoretical interpretation of relevance, coverage, and gain, even if the resulting heuristic is empirically useful.

  2. The statistical interpretation of the saturation trigger is insufficiently justified. Cantelli’s inequality is applied to the evolving sequence of perplexities, but these values are adaptively generated, dependent, and not presented as samples from a fixed distribution. The variance correction and threshold formula therefore do not appear to provide the stated false-truncation control. The method should be framed and validated as a heuristic unless a valid probabilistic argument is supplied.

Questions for authors
  1. What random-variable model makes I(S;C,Q)I(S;C,Q) the appropriate objective when S⊆CS\subseteq C, and how does this objective imply preservation of the downstream answer rather than merely high entropy of the selected subset?

  2. Under what assumptions does Equation 11 provide a false-truncation guarantee for an adaptive, dependent perplexity sequence? Can the authors provide either a proof under explicit assumptions or remove the statistical-guarantee language?

  3. How many conditional LM evaluations and inter-fragment affinity evaluations are required as a function of the number of fragments? Please report compressor latency and memory across context lengths, including the 1M-token setting.

  4. How does COPS compare with a simpler relevance-plus-diversity objective and a validation-tuned stopping threshold? This would clarify whether the more elaborate information-theoretic formulation contributes beyond standard subset selection.

3 comments
AuthorsOpenReview ↗

Rebbuttal to Reviewer WD7x [part 1]

We thank the reviewer for the helpful comments. We address the weaknesses and questions below:

[Q1/W1] Scope and interpretation of the selection criterion

We would like to clarify that COPS is an uncertainty-based, query-conditioned extractor for long contexts built on a general-purpose LLM backbone. Deterministic and query-free extractors are outside the scope of this study. We select a context subset S from the full context C using a mutual-information-based criterion involving S, C, and the query Q. This criterion is decomposed into task relevance and dependency coverage, with an information-gain term further incorporated into the selection process. Specifically, we first retrieve candidates based on their relevance to Q. We then evaluate the combined criterion on the top-ranked candidates, rerank them, and iteratively construct S. Both task relevance and information gain explicitly account for the relationship with Q.

Following Longllmlingua [1] and LongCodeZip [2], we respectfully clarify that perplexity reduction is not a model-dependent heuristic method and it quantifies the change in predictive uncertainty about Q under the scoring model when additional context is provided. We use the difference between unconditional and conditional perplexity as an approximated mutual-information (AMI) score. A higher AMI score indicates that a context snippet cic_i makes Q more predictable under the scoring model and thus provides greater predictive utility for the query. Accordingly, our method evaluates how the selected context changes the model’s predictive uncertainty about Q, rather than measuring the information content or entropy of S in isolation.

Furthermore, our experiments evaluate our method COPS using base models with three different architectures—DeepSeek-Coder, Qwen2.5-Coder, and Seed-Coder, demonstrating its generalizability and broad applicability.

[Q2/W2] Statistical interpretation of the saturation-triggered stopping mechanism

Regarding Eq. (11), based on the relevant literature [3,4,5,6], Cantelli’s inequality does not assume a particular distribution or require independent and identically distributed observations. It applies to a random variable with a finite mean and second moment (variance).

Using the mean and standard deviation of the relevant random variable, Cantelli’s inequality yields a one-sided probability bound. When applied to the stopping event under the assumptions of our analysis, this bound controls the probability of premature truncation. The corresponding derivations and supporting proofs are provided in Appendix A.2.

References:

  1. Jiang, Huiqiang, et al. "Longllmlingua: Accelerating and enhancing llms in long context scenarios via prompt compression." Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

  2. Shi, Yuling, et al. "Longcodezip: Compress long context for code language models." 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2025.

  3. Mazumdar, R. R. (2010). Stochastic Processes in Engineering Science, Chapter 1.

  4. Howard, S. R., Ramdas, A., McAuliffe, J., & Sekhon, J. (2021). Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics, 49(2), 1055–1080.

  5. Cantelli, F. P. (1928), "Sui confini della probabilita," Atti del Congresso Internazional del Matematici, Bologna, 6, 47-5.

  6. Boucheron, Stéphane (2013). Concentration inequalities : a nonasymptotic theory of independence. Gábor Lugosi, Pascal Massart. Oxford: Oxford University Press. ISBN 978-0-19-953525-5. OCLC 829910957.

AuthorsOpenReview ↗

Rebbuttal to Reviewer WD7x [part 2]

[Q3] Compressor latency and memory.

We thank the reviewer for requesting a clearer analysis of computational overhead. Relative to the baseline compression pipeline, COPS restricts its additional LM-based candidate scoring and inter-fragment affinity evaluations to the top-8 candidate fragments. This limits the scope of additional scoring rather than exhaustively evaluating all candidates. The number of affinity evaluations also depends on the target fragments considered and the reuse of cached scores.

The following table summarizes compressor latency and memory for the LongCodeZip and COPS on LongCodeQA.

Input lengthLongCodeZip latency (s)LongCodeZip GPU memory (GB)COPS latency (s)COPS GPU memory (GB)
32K42.9918.4250.7321.57
64K99.6818.65105.8221.86
128K204.6718.81220.2922.23
256K320.6018.79331.3222.00
512K712.2218.81740.4522.31
1M808.1718.81853.6722.46

[Q4] COPS compare with a simpler relevance-plus-diversity objective and a validation-tuned stopping threshold.

We thank the reviewer for suggesting this comparison. Our existing ablations in Table 2 partially address this question by isolating the contributions of directed coverage, selected-context-dependent query gain, and adaptive stopping.

ConfigurationCompletion ESCompletion ACURepoQA accuracy (%)RepoQA ACU
Relevance only56.220.5384.60.45
Without conditional query gain56.350.5484.90.45
Without directed coverage56.410.5485.10.45
Without saturation trigger56.390.4185.80.37
COPS (full)56.940.5585.50.45

Selection objective. The configuration without conditional query gain is the closest existing internal comparison to a simpler relevance-plus-complementarity objective: it retains relevance and directed coverage while removing the candidate’s additional query-perplexity reduction conditioned on the selected context. Adding conditional query gain increases completion ES from 56.35 to 56.94 and RepoQA accuracy from 84.9% to 85.5%. Removing directed coverage also reduces performance, to 56.41 ES and 85.1% accuracy. These results support the complementary utility of the two signals: coverage evaluates additional contributions toward other fragments, whereas conditional query gain evaluates additional predictive utility for the query.

Stopping criterion. Removing the saturation trigger decreases completion ACU from 0.55 to 0.41 and RepoQA ACU from 0.45 to 0.37. The completion compression ratio also decreases from 6.4× to 4.8×. These results support the trigger’s role in improving compression efficiency.

Overall, the ablations support the practical contribution of the proposed scoring and stopping mechanisms.

Dear Reviewer, The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.

Best, AC

Reviewer_sudWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

COPS is a training-free framework for compressing long code contexts. It combines perplexity-based relevance scoring, greedy fragment selection, an adaptive stopping rule, and iterative block pruning. Experiments cover code completion, summarization, and question answering, with additional evaluations of cross-model transfer and inference efficiency.

Critical strengths

The paper addresses a practically relevant problem: reducing inference costs and distracting information in long code contexts.

Evaluation spans multiple tasks and both open-source and closed-source generators, providing useful evidence of applicability.

Critical weaknesses

Limited methodological novelty. LongCodeZip already uses the same AMI definition, function-level selection, and perplexity-based block compression. COMI also jointly considers relevance and redundancy. The distinct contribution therefore rests on the added selection and stopping mechanisms, whose novelty requires clearer justification beyond the “first-principles” framing. (Sections 2–4; LongCodeZip, COMI)

The information-theoretic objective does not establish task preservation. For a deterministic extractor (S=f(C,Q)), (I(S;C,Q)=H(S)). Thus, Eq. (1) maximizes the entropy of the retained content, which does not necessarily preserve answer-relevant information or downstream predictions. The paper also leaves the underlying random-variable distributions unspecified. (Section 3.1, Eqs. (1)–(5))

The block-pruning procedure cannot reach the claimed nonempty fixed point. Not every element of a nonempty set can strictly exceed its own mean, so the stopping condition (S=L) is impossible. Repeated pruning eventually empties the set, making the next mean undefined. For example, scores ([1,2,3]) retain only the third block, which is then discarded in the next iteration. (Section 4.4, lines 280–290)

The implemented coverage score differs from the function analyzed theoretically. Equation (8) uses (a(c_j c_k)) for baseline coverage, whereas Appendix A.1 requires (a(c_k c_j)). These quantities are not interchangeable because perplexity-based affinity is asymmetric. Equation (8) also sums only over unselected fragments. Consequently, the submodularity argument does not directly justify the actual selection rule. (Eq. (8); Appendix A.1)

The stopping rule has an undefined initialization and unverified theoretical assumptions. Algorithm 1 initializes the history with one perplexity value, yielding zero standard deviation before dividing by it. Moreover, Section 4.3 includes the current observation in threshold estimation, whereas the proof requires a threshold determined beforehand. The assumed upper bound on conditional variance is not empirically validated. (Section 4.3; Appendix A.2; Algorithm 1, lines 2 and 10–14)

Reporting inconsistencies weaken reproducibility. Default EM is 31.40 in Tables 1–2 but 31.00 in Table 7. Corresponding cross-model results also differ between Tables 1 and 3 without an explanation of configuration changes. Small performance improvements lack uncertainty estimates or significance analysis. Presentation additionally needs proofreading, including “rethinked” and a comma splice in Section 6. (Tables 1–3 and 7; Section 6)

Questions for authors

Please provide the implemented block-pruning procedure. How does it avoid an empty set and retain multiple blocks?

Which coverage direction is implemented? How are zero variance and the timing of threshold estimation handled?

Please reconcile the inconsistent default results and specify decoding settings, random seeds, and the actual sample counts, context-length distribution, and models used for Long Code QA.

4 comments
AuthorsOpenReview ↗

Rebbuttal to Reviewer sudW [part 1]

We thank the reviewer for the helpful comments. We address the weaknesses and questions below:

[W1] Limited methodological novelty.

We agree that AMI, function-level selection, perplexity-based segmentation, and balancing relevance and redundancy are established components. However, the novelty of our method lies in proposing an extractor that extracts an optimal subset SS of the full context CC, given CC and the current query QQ, from a first-principles perspective. We decompose the aforementioned mutual-information formulation into task relevance and dependency coverage metrics and introduce an information-gain metric. We model the entire procedure as a dynamic subset-selection process, iteratively selecting the candidate that achieves the highest score at each step and adding it to the selected set. Specifically:

Our intended contributions lie in the specific selection and stopping procedures, semantic-block-level pruning, and the introduced dynamic selection mechanism. Compared with LongCodeZip’s static function selection based on relevance ranking, COPS updates candidate priorities using their incremental directed coverage of the recalled pool and their additional reduction in query perplexity given the already selected context. It then monitors the assembled context to determine whether to accept additional candidates.

COMI performs learned representation compression through group reallocation and token merging using a training-based approach. Its primary application domain is general text, whereas assessing the structure and semantics of code contexts involves different considerations. Our method, COPS, primarily targets code contexts and uses generative-model scores to perform training-free, extractive code-fragment selection.

COMI indeed considers both relevance and redundancy; however, COPS handles redundancy through different scoring mechanisms. COMI defines redundancy as the maximum cosine similarity between a unit’s representation and those of other units in the corresponding context or group:

RCOMI(xi;X)=max⁡xj∈X,,j≠ixi⊤xj∣xi∣∣xj∣R_{\mathrm{COMI}}(x_i;X)=\max_{x_j\in X,\\,j\neq i}\frac{x_i^\top x_j}{\\|x_i\\|\\|x_j\\|}

GCOMI(xi,q,X)=xi⊤q∣xi∣∣q∣−RCOMI(xi;X)G_{\mathrm{COMI}}(x_i,q,X)=\frac{x_i^\top q}{\\|x_i\\|\\|q\\|}-R_{\mathrm{COMI}}(x_i;X)

In contrast, COPS combines conditional query gain with directed dependency coverage to evaluate a candidate’s contribution relative to the selected context. Its conditional query gain is:

di(S)=[PPL⁡(Q∣S)−PPL⁡(Q∣S∪ci)]+d_i(S)=\left[\operatorname{PPL}(Q\mid S)-\operatorname{PPL}(Q\mid S\cup\\{c_i\\})\right]_+

where SS denotes the selected context, cic_i is a candidate fragment, and [z]+=max⁡(0,z)[z]_+=\max(0,z). This term measures whether the candidate provides additional predictive utility for the query beyond the evidence already available in SS.

Coverage provides a complementary signal through directed inter-fragment affinity:

a(ci→cj)=PPL⁡(cj)−PPL⁡(cj∣ci)a(c_i\rightarrow c_j)=\operatorname{PPL}(c_j)-\operatorname{PPL}(c_j\mid c_i) For a directionally consistent coverage formulation, the existing coverage of target fragment cjc_j and the candidate’s incremental coverage are defined as: bS(cj)=max⁡(0∪a(ck→cj):ck∈S)b_S(c_j)=\max\left(\\{0\\}\cup\\{a(c_k\rightarrow c_j):c_k\in S\\}\right) cov⁡i(S)=∑cj∈Crecall[a(ci→cj)−bS(cj)]+\operatorname{cov}_i(S)=\sum_{c_j\in C_{\mathrm{recall}}}\left[a(c_i\rightarrow c_j)-b_S(c_j)\right]_+ Thus, a candidate receives coverage credit only when its affinity toward a target exceeds the coverage already supplied by the selected fragments. Unlike cosine similarity, this affinity is directional: helping predict another fragment need not imply an equivalent contribution in the reverse direction. The combined selection score is:

Score⁡(ci;S,Q)=r~i+λ,cov⁡~i(S)+α,d~i(S)\operatorname{Score}(c_i;S,Q)=\widetilde{r}_i+\lambda\\,\widetilde{\operatorname{cov}}_i(S)+\alpha\\,\widetilde{d}_i(S)

where the tildes denote min–max normalization. COMI therefore penalizes representation similarity, whereas COPS evaluates additional query utility and additional directed coverage relative to the selected context. These signals capture different aspects of overlapping information and remain model-dependent proxies, rather than formal guarantees of dependency or task preservation.

Taken together, the core methodology and claims of COPS differ substantially from those of these two papers.

AuthorsOpenReview ↗

Rebbuttal to Reviewer sudW [part 2]

[W2] The information-theoretic objective does not establish task preservation.

We respectfully clarify that COPS is an uncertainty-based, query-conditioned long-context extractor built upon general-purpose LLM backbones. Deterministic and query-free extractors therefore fall outside the scope of our study and are unrelated to our method. Consequently, the assertion that “Equation (1) maximizes the entropy of the retained content” and the subsequent conclusions do not hold.

[Q1/W3] The block-pruning procedure.

In each round of our iterative block-pruning procedure, we subtract the current mean from each element’s score and set any negative result to zero. When computing the mean in the next round, we continue to use the total number of blocks as the denominator.

For example, consider three blocks with scores ([1,2,3]). In the first round, the mean is (6/3=2). Subtracting this mean and clipping negative values to zero yields ([0,0,1]), retaining only the third block. In the second round, the mean is (1/3=0.33), so the updated scores become ([0,0,0.67]). The retained block indices are unchanged from the previous round: only the third block is retained. The updates therefore stop, and the final result retains the third block, yielding a nonempty set.

[Q2/W4] The implemented coverage score.

We sincerely thank you for pointing out the incorrect expression in Equation (8), which resulted from a typographical error. We have revised Equation (8) as follows:

cov⁡i(S)=∑cj∈Crecall[a(ci→cj)−bS(cj)]+\operatorname{cov}_i(S)=\sum_{c_j\in C_{\mathrm{recall}}}\left[a(c_i\rightarrow c_j)-b_S(c_j)\right]_+

We have also corrected the corresponding results in the manuscript PDF.

[Q2/W5] The stopping rule.

Regarding zero variance at initialization, as described in Algorithm 1 and Section 4.3, we initialize the history with the first v0v_0 perplexity values rather than a single value. Furthermore, to prevent the algorithm from stopping immediately at the initial set, we skip threshold computation and the stopping test when the first fragment from the remaining set is added. Therefore, our method does not experience abnormal termination of the iteration due to zero variance.

Regarding the timing of threshold estimation, as described in Section 4.3 and the proof in Appendix A.2, we use the previous observation Ht−1H_{t-1} rather than the current observation HtH_t when estimating the threshold. We then substitute the threshold expression into the one-sided concentration bound to derive an inequality for αt\alpha_t, from which we obtain the updated threshold.

Regarding the assumption that the conditional variance has an upper bound, the perplexity in our setting is bounded, and the global system perplexity follows a generally decreasing trajectory with fluctuations within a certain range. Context fragments that cause perplexity to increase beyond that range are discarded. Therefore, the conditional variance has an upper bound.

AuthorsOpenReview ↗

Rebbuttal to Reviewer sudW [part 3]

[Q3] Thank you for pointing out these inconsistencies. The discrepancies in the default results across Tables 1–3 and 7 were due to typographical errors, which we have corrected in the revised manuscript.

The decoding settings and random-seed configurations are summarized below.

TaskDecoding settingsMaximum new tokensRandom seed
Long Code CompletionGreedy decoding; temperature=0; top p=0.9512842
Long Module SummarizationGreedy decoding; temperature=0; top p=1.0, top k=50204842
RepoQAGreedy decoding; temperature=0;102442
Long Code QAtemperature=0; top p=0.9525642

The separate GPT-4o-mini evaluator for module summarization uses temperature=0.3 and max_tokens=1, without an explicitly specified seed.

Our Long Code QA evaluation uses the following configuration:

ItemConfiguration
Actual number of samples128
Mean context length before compression80K tokens
Base modelQwen2.5-Coder-7B-Instruct

Dear Reviewer, The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.

Best, AC

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of reviewers feedback and authors response

We thank the reviewers for their feedback. We summarize the main contributions, rebuttal clarifications, and remaining discussion concerns below.

  • COPS contributes dynamic, selected-context-dependent compression—not merely another relevance ranking. It combines task relevance, directed dependency coverage, and conditional query gain with adaptive stopping and semantic-block pruning. Unlike LongCodeZip’s static relevance ranking, COPS evaluates candidates’ additional utility given the context already selected. Unlike COMI’s learned representation compression for general text, COPS performs training-free, plug-and-play extraction of code context, with different redundancy modeling.

  • Our rebuttal provides concrete clarifications, corrections, and additional evidence. We explained the implemented block-pruning procedure and stopping initialization, corrected the coverage expression and inconsistent table entries, and documented decoding settings, seeds, and evaluation configurations. We also supplied compressor latency and memory measurements from 32K to 1M tokens. Ablations support the practical contribution of our mechanisms: compared with relevance-only selection, COPS improves completion ES from 56.22 to 56.94 and RepoQA accuracy from 84.6% to 85.5%. Adaptive stopping improves the completion compression ratio from 4.8× to 6.4×. These results complement the evaluation across multiple tasks and model architectures.

  • Several substantive responses remain insufficiently discussed. Reviewers sudW and WD7x have not provided follow-up responses in the discussion record despite the AC’s reminders. Reviewer JukT maintained the original rating but did not explain why the detailed distinctions from prior work failed to resolve the novelty concern. Consequently, important clarifications have not received substantive examination. We respectfully request that novelty be assessed against the actual selection, stopping, and pruning mechanisms, and that remaining theoretical objections identify the precise unresolved issues after considering our responses.

We appreciate your consideration and ask that the final assessment account for the corrected manuscript, additional evidence, and substantive rebuttal, while distinguishing remaining limitations from concerns already addressed through clarification or correction.

Sincerely,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.