- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes COPS, a training-free framework for long-context code compression. It combines perplexity-based scoring, greedy context selection, adaptive stopping, and semantic-block pruning. Experiments on code completion, summarization, and question answering show promising trade-offs between compression and downstream performance.
Critical strengths
- The method addresses a practical problem and does not require additional training.
- The paper reports end-to-end latency and memory consumption alongside task performance.
Critical weaknesses
- The paper has limited novelty in its techniques. AMI is simply perplexity reduction. For a fixed query, its ranking is equivalent to conditional-perplexity ranking, already used in LongCodeZip. The information-theoretic terminology does not clearly establish a new scoring method.
- The paper makes insufficient distinction from prior work. Balancing relevance and redundancy in context selection is an established idea. COMI already combines these criteria through marginal information gain in a coarse-to-fine framework. The paper needs to explain what COPS contributes beyond this principle and standard greedy selection.
Questions for authors
What distinguishes COPS from existing relevance–redundancy compression approaches?
3 comments
Dear Reviewer, The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.
Best, AC
Thanks for the reply. I would like to keep my rating, as my concern on the novelty stays.
We thank the reviewer for suggesting that we clarify the differences between our method and prior work.
We agree that AMI-based relevance scoring and the balance between relevance and redundancy are established components of existing methods. COMI primarily targets general text, whereas COPS targets code, which differs in both structure and semantics. In general text, low similarity to the query vector is more likely to indicate weak relevance. In code, however, two context snippets with low similarity may provide complementary information for predicting the query, making both worth retaining. Code also contains implicit dependencies: as the selected context changes, the utility of other snippets that depend on it often changes as well. Static selection may therefore be insufficient. COMI requires training to compress general-text representations, whereas COPS is training-free and plug-and-play, and selects code snippets dynamically.
Our contributions include evaluating candidate snippet utility as the retained context evolves, integrating this evaluation with directed coverage and adaptive stopping, and introducing a novel method for pruning code at the semantic-block level.
1. Selection conditioned on the retained context versus LongCodeZip’s static relevance ranking
For a fixed query, AMI ranking is indeed equivalent to ranking by conditional perplexity. We do not claim that AMI itself is a new estimator. However, using the same relevance metric does not imply an identical selection procedure. LongCodeZip’s coarse-grained stage ranks functions by their individual relevance to the query, whereas COPS also evaluates each candidate’s additional contribution given the currently selected context.
Specifically, our query-gain term simplifies to
\left[ \operatorname{PPL}(Q\mid S) -\operatorname{PPL}(Q\mid S\cup\{c_i\}) \right]_+.$$ Thus, even a candidate with high individual relevance may provide little additional gain if its useful information is already present in $S$. This distinction matters for code generation and understanding. Independent ranking may repeatedly select similar usage examples while overlooking definitions that are less directly relevant to the query but specify parameter constraints, exceptional conditions, or resource-management requirements. The generated code may consequently follow inappropriate API usage patterns, while code understanding may miss preconditions or exceptional execution paths. These possible failure modes motivate evaluating complementary evidence alongside relevance. COPS addresses this limitation within the retrieved candidate pool through scoring conditioned on the currently selected context and coverage across snippets. **2. Different operational definitions of redundancy in COMI and COPS** COMI defines redundancy using the maximum cosine similarity between a unit and other units in the relevant context or group. COPS’s conditional query gain measures how much additional predictive utility a candidate provides given the assembled selected context. These criteria are not equivalent, because similarity between representations does not necessarily imply that snippets are interchangeable for a particular code task. For example, two highly similar functions may encode different boundary conditions, making both worth retaining. Conversely, structurally different snippets may provide redundant evidence for the same query. Our conditional gain can reflect these distinctions when the scoring model captures them. The coverage score provides another signal. Directed perplexity affinity evaluates how one snippet helps predict another, rewarding additional coverage of snippets. This differs from penalizing similarity between candidates. The affinity remains a model-dependent proxy for dependency information. The methods also differ architecturally. COMI compresses learned representations through token merging and model adaptation. COPS directly extracts the original code text without task-specific training for compression, so the compressed context can be used by a different downstream model. **3. Contributions beyond standard greedy selection** Greedy selection itself is not our claimed innovation. Our contribution is its specific integration with directed coverage, query gain conditioned on the currently selected context, and a saturation-triggered stopping mechanism in the later inclusion stage. Together, these mechanisms address two complementary questions: which snippets provide additional evidence, and whether further context expansion is worthwhile. The ablation results in Table 2 support the practical value of combining these selection signals. On code completion, the full method achieves ES/EM of 56.94/31.40, compared with 56.22/30.40 for relevance-only selection. On RepoQA, the corresponding accuracies are 85.5 and 84.6.