- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes FOCUS, a training-free method for compressing the interaction history of LLM agents. A draft model generates future plans and identifies the historical spans needed by those plans; a defensive verification step additionally retains information that may prevent future errors. The paper connects this procedure to counterfactual utility and an information bottleneck objective, and evaluates it on several agent benchmarks.
Critical strengths
- The paper is clearly written and well organized. The motivation and overall procedure are easy to follow.
- FOCUS is simple and practical. Its training-free design and verbatim retention of selected spans make it straightforward to integrate into existing LLM agents.
Critical weaknesses
- The connection between the theoretical objective and the implemented algorithm is not sufficiently established. I do not see a result showing that Algorithm 1 approximately optimizes the proposed information bottleneck objective. Proposition 1 concerns maximizing counterfactual utility under a budget, whereas FOCUS thresholds citation frequencies from draft plans and adds defensively retained spans. Appendix A.2 acknowledges that dependency scores need not preserve the ordering of counterfactual utilities. The one-sided safety bound provides a conditional justification for deletion, but does not establish an approximation guarantee for this selection procedure. Likewise, the concentration results and proxy agreement in A.2.6 do not close this gap. Since the theoretical connection is presented as a central contribution, I find the support for the claimed optimization interpretation insufficient.
- The comparisons with prior methods leave substantial uncertainty about FOCUS’s empirical advantage. For OfficeBench, Appendix A.9 specifies a compression trigger of 2,048 tokens, whereas ACON uses 4,096 tokens for its history-compression methods and baselines. Nevertheless, every baseline entry in Table 1(b) matches ACON’s Table 2(a), including accuracy, steps, peak tokens, and dependency. This is difficult to reconcile with the statement that the experimental settings are the same. The paper needs to clarify the source and configuration of these baseline results. Until this discrepancy is resolved, I cannot determine whether the reported gains reflect the method or differences in compression settings. PAACE is also a relevant missing baseline. It already uses future-plan relevance to guide compression and evaluates the same three principal benchmarks. Table 11 explains methodological differences, but provides no evidence of how FOCUS compares with this closely related approach. A direct comparison would help establish whether FOCUS’s simpler, training-free design offers a competitive accuracy–efficiency trade-off.
Questions for authors
- Which result establishes an approximation guarantee for the citation-based selection rule in Algorithm 1, rather than for a selector using the true counterfactual utilities? If no such result is intended, what precise theoretical claim is being made about the implemented algorithm?
- What compression trigger was actually used for each OfficeBench method in Table 1(b)? Were the baseline results reproduced or taken from ACON, and how should the 2,048/4,096 discrepancy be reconciled?
- Can the authors provide a controlled comparison with PAACE, or explain any substantive obstacle to making this comparison?
2 comments
Replying to authors
I appreciate the authors’ response and their answers to my questions. However, I still believe my initial overall rating reflects my assessment of the paper, and therefore I would like to keep it.
Request to review our responses
Dear Reviewer
CAmawe request you to please review our responses and would love to further engage with you in a constructive discussion if you have any clarifying questions. If all your concerns are addressed we would highly appreciate if you would consider increasing your score.