- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CodeComp is a training-free KV cache compression method for coding agents. It targets historical READ results and uses Joern-derived program structure to prioritize code regions and supplement attention-based token scores. Compression is applied after full-prompt prefill, leaving stored messages unchanged. Experiments on 100 SWE-bench Verified tasks with two models show reduced peak KV cache memory and task resolution rates close to those of the uncompressed agent.
Critical strengths
1.The paper addresses a practical problem in coding agents. Using program structure to guide cache retention is a reasonable approach, and the method requires no additional model training.
2.The evaluation includes complete agent runs, comparisons with attention-based and chunk-based compression, and component ablations. The reported end-to-end latency includes structural analysis overhead.
Critical weaknesses
1.The contribution beyond simpler code-aware retention is unclear. CodeComp converts program structure into token priorities and an additive scoring term. While this is a useful design, the experiments do not establish whether control-flow and data-dependency analysis is necessary. ChunkKV provides a relevant comparison, but retaining complete statements or functions would be a more direct alternative. Without such comparisons, the gains could mainly reflect preserving coherent code regions rather than exploiting program dependencies.
2.The structural analysis does not sufficiently establish better preservation of task-relevant information. Higher retention of Joern-annotated tokens shows that the method follows its structural priorities, but does not demonstrate that these tokens are necessary for the repair. Differences between attention scores and structural scores alone do not establish harmful pruning. Moreover, compression occurs after full prefill, so removing a token’s KV is not equivalent to removing that token from the original input. The ablations are also mixed: hard protection alone reduces resolved tasks from 24 to 20, while the full method reaches 28. The interaction between the components needs a clearer explanation.
3.The empirical advantage is modest and its reliability remains uncertain. Compared with ChunkKV, CodeComp resolves only one additional task on Qwen and three on Devstral, with small additional memory savings. These results come from single runs on 100 tasks; ablations use only 50 tasks and one model. Repeated runs and comparisons across multiple retention budgets are needed to establish a reliable advantage. The practical benefit is also bounded: full-prompt prefill and context-window requirements remain unchanged, and latency improves over the native agent on only one of the two models.
Questions for authors
1.How does CodeComp compare with retaining complete statements or functions under the same KV budget? What additional benefit comes specifically from dependency information?
2.Can controlled experiments on identical agent histories show that structural retention better preserves correct edits or tool decisions?
3.Do the gains over ChunkKV persist across repeated runs and different retention budgets? Can fixed-history profiling separate direct compression savings from changes in agent trajectories?
Thank you for these suggestions. We appreciate your careful reading and detailed feedback.