- Rating
- 3(+1)
- Confidence
- 3(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
this paper trains a (small language) model for context compression in long horizon llm agents. to achieve this it introduces a context compaction layer (CAL) that first transforms each harness's history into a list of turns. then the model would label each turn as one of the following: visible, compressed, merged, hidden.
this model (used for turn labeling) is trained with SFT on some trajectories generated by a larger model (deepseek v4 pro) and then with RL. in the RL stage, a frozen executor would continue the task after context compaction. evaluations on coding, deep research, QA, and claw like personal ai benchmarks + two recent harnesses (opencode and pi agent) show good numbers over selected baselines while using fewer tokens.
Critical strengths
+ timely topic in the domain of agents: context compression is an important problem for ai agents, especially when horizon is long. many recent papers on this topic.
+ reasonable design. SFT+RL is a very legit process for the kind of turn labeling model authors want.
+ very detailed discussion of the training process (of the turn classifier model). this is strongly encouraged since the insights could help the community even if people might not follow the same training recipe or base language model or dataset.
+ reported results are decent across multiple chosen benchmarks (different applications as well) and language models.
Critical weaknesses
- choice of baselines: could you justify why you pick the current baselines? the practice in this research direction is to compare to published papers or well-known papers with high citations; most naturally, even if you build you own baselines, you can include a baseline where a strong llm just summarizes or drops things; and as i know there are a lot more, either trained compressors or test-time approaches; just to name a few, you don't have to present results for these, ACON (trained), LLM-Lingua and its follow-ups (both), etc.; is your resum baseline trained or not?; the current baselines are pretty simple or relatively less known ones (CoMem is the only one that gets formally published, at icml 2026). without legit baselines i am not sure how i could evaluate this work in the context of the general research direction.
- it is unclear to me if the transfer gains (to unseen harnesses and domains) are strong enough; the train -> transfer evidence is weak in current evaluation. for example, held-out harness is only tested on repo generation (table 1).
Questions for authors
See "weaknesses". I am always ready to be convinced.