acceptodds
Under review as a conference paper at ICLR 2027

FOCUS: Training-Free Decision-Preserving Context Compression for LLM Agents

Abstract

LLM agents accumulate interaction histories that grow linearly with task length, causing quadratic inference cost scaling and performance degradation from attention dilution. Existing context-compression methods learn what to discard offline: by contrastively optimizing guidelines, distilling compressors, or training compression policies. This incurs a substantial cost. Further, the compression policy is learned a priori and is not dynamically conditioned on the evolving test-time trajectories. In this paper we ask a complementary question: Which past interactions causally shape the agent's future decisions? We recast context compression as a causal decision preservation problem over discrete interaction units and introduce FOCUS, a training-free context compression framework that operates entirely at test time. Our method requires no offline data collection or fine-tuning, and is architecture-agnostic, attaching to any closed-API frontier model as a modular compression layer. We evaluate FOCUS on diverse agentic benchmarks including API and tool-calling, QA, web domain and multi-turn dialogue. Our method establishes new state of the art performance, cutting peak context by up to 48% and dependency by 73% while improving task success by up to 8.9 percentage points over uncompressed execution.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Context Compression, Long-horizon task, LLM agent

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_CAmaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FOCUS, a training-free method for compressing the interaction history of LLM agents. A draft model generates future plans and identifies the historical spans needed by those plans; a defensive verification step additionally retains information that may prevent future errors. The paper connects this procedure to counterfactual utility and an information bottleneck objective, and evaluates it on several agent benchmarks.

Critical strengths
  1. The paper is clearly written and well organized. The motivation and overall procedure are easy to follow.
  2. FOCUS is simple and practical. Its training-free design and verbatim retention of selected spans make it straightforward to integrate into existing LLM agents.
Critical weaknesses
  1. The connection between the theoretical objective and the implemented algorithm is not sufficiently established. I do not see a result showing that Algorithm 1 approximately optimizes the proposed information bottleneck objective. Proposition 1 concerns maximizing counterfactual utility under a budget, whereas FOCUS thresholds citation frequencies from draft plans and adds defensively retained spans. Appendix A.2 acknowledges that dependency scores need not preserve the ordering of counterfactual utilities. The one-sided safety bound provides a conditional justification for deletion, but does not establish an approximation guarantee for this selection procedure. Likewise, the concentration results and proxy agreement in A.2.6 do not close this gap. Since the theoretical connection is presented as a central contribution, I find the support for the claimed optimization interpretation insufficient.
  2. The comparisons with prior methods leave substantial uncertainty about FOCUS’s empirical advantage. For OfficeBench, Appendix A.9 specifies a compression trigger of 2,048 tokens, whereas ACON uses 4,096 tokens for its history-compression methods and baselines. Nevertheless, every baseline entry in Table 1(b) matches ACON’s Table 2(a), including accuracy, steps, peak tokens, and dependency. This is difficult to reconcile with the statement that the experimental settings are the same. The paper needs to clarify the source and configuration of these baseline results. Until this discrepancy is resolved, I cannot determine whether the reported gains reflect the method or differences in compression settings. PAACE is also a relevant missing baseline. It already uses future-plan relevance to guide compression and evaluates the same three principal benchmarks. Table 11 explains methodological differences, but provides no evidence of how FOCUS compares with this closely related approach. A direct comparison would help establish whether FOCUS’s simpler, training-free design offers a competitive accuracy–efficiency trade-off.
Questions for authors
  1. Which result establishes an approximation guarantee for the citation-based selection rule in Algorithm 1, rather than for a selector using the true counterfactual utilities? If no such result is intended, what precise theoretical claim is being made about the implemented algorithm?
  2. What compression trigger was actually used for each OfficeBench method in Table 1(b)? Were the baseline results reproduced or taken from ACON, and how should the 2,048/4,096 discrepancy be reconciled?
  3. Can the authors provide a controlled comparison with PAACE, or explain any substantive obstacle to making this comparison?
2 comments
AuthorsOpenReview ↗

Request to review our responses

Dear Reviewer CAma we request you to please review our responses and would love to further engage with you in a constructive discussion if you have any clarifying questions. If all your concerns are addressed we would highly appreciate if you would consider increasing your score.

Reviewer_CAmaOpenReview ↗

Replying to authors

I appreciate the authors’ response and their answers to my questions. However, I still believe my initial overall rating reflects my assessment of the paper, and therefore I would like to keep it.

Reviewer_za1yOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces a training-free method for compressing histories while preserving information relevant to future decisions for LLM agent. It samples multiple plans using a lightweight draft model and estimates each historical span's importance based on how frequently it is cited by the plans. Experiments on five benchmarks showed that the framework improves task performance while reducing context usage compared to existing baselines.

Critical strengths

The paper offers a principled approach to context compression by focusing on preserving information that may influence an agent’s future decisions, rather than simply removing redundant content.

Moreover, it is a training-free approach that does not require additional model training which is relatively easy to integrate into existing LLM agents.

Critical weaknesses
  • Although the retained sets show high overlap with the KL counterfactual reference, the reported rank correlations are relatively low (0.39 on OfficeBench and 0.48 on AppWorld in appendix 2.6). This raises some concerns about whether the frequency of citation reliably reflects how much a span actually affects the main agent’s decisions. While the paper provides a qualitative failure case, more analysis of where the two estimates disagree and whether these differences lead to important information being dropped would be helpful.

  • The paper discusses when defensive verification is beneficial and provides qualitative failure cases, but its effectiveness is not directly analyzed in terms of how often it recovers important spans missed by the initial selection. While FOCUS-D substantially improves accuracy on AppWorld, it provides little benefit on OfficeBench and 8-QA despite retaining more context. A more detailed analysis of the spans recovered by verification and their impact on task success would help clarify the value of this additional step.

Questions for authors

Could you address the concerns raised in the weaknesses, particularly regarding the reliability of the dependency score and the effectiveness of defensive verification?

4 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We thank the reviewer for the thoughtful feedback. We respond point-by-point below.

W1: Analysis on the two estimates (Draft and KL counterfactual)

We thank the reviewer for suggesting the analysis. The 0.39/0.48 values correlate binary keep/drop decisions, so they penalize extra retention as heavily as missed spans. Decomposing the disagreement on the same 30 events per benchmark: the draft keeps spans KL would drop (53 OfficeBench / 56 AppWorld) far more often than it drops spans KL would keep (22 / 15), indicating a safe approach taken by the draft. The draft retains 70% / 76% of KL-important spans.

Across 30 events (span kept/dropped) by Draft and KL:

MetricOfficeBenchAppWorld
Both keep5147
Draft keeps, KL drops (extra context, not lost information)5356
KL keeps, draft drops2215
Both drop85113
Share of KL-important spans the draft keeps0.700.76

Low draft score implies low counterfactual impact. Binning spans by draft score, spans the draft drops (score < 0.3) have low KL utility (mean 0.17 / 0.11) and are judged important by KL only 21% / 12% of the time. For spans the draft scores highly, this rises to around 60% on both benchmarks. So we can say that the draft's dropping decision aligns well with the KL counterfactual, and acts safely by tending to keep more spans than KL.

Is important information dropped? As mentioned in the paper, the KL counterfactual is intractable in practice and we try to approximate it using a sampling-based proxy (it looks 3 actions ahead and uses leave-one-out), which can itself be noisy. Nevertheless, even in this approximation, we see that the draft decisions do not result in KL required spans getting dropped, with KL important spans the draft keeps being 0.7 and 0.76 on the above events.

Together, these show that the residual disagreements rarely translate into lost decision-relevant information (along with the presented results in Tables 1,2,3 where FOCUS performs better than the baselines on accuracy as well as efficiency metrics)

AuthorsOpenReview ↗

Rebuttal (2/2)

Analysis and Usefulness of Defensive configuration (W2)

We thank the reviewer for this suggestion. We analyze defensive verification in three ways: how often it fires and what it rescues, how many needed spans it recovers, and why its benefit differs across benchmarks.

(1) How often verification fires and what it rescues.

MetricAppWorldOfficeBench
Events where verification rescues >=1 span94%39%
Share of spans kept: plan → plan + rescue0.38 → 0.780.54 → 0.66
Spans kept because of rescue per event3.60.7

On AppWorld, the rescued spans are mostly API documentation (39%) and credentials or access tokens (32%). These are state the agent needs repeatedly across apps but rarely cites in a short plan. On OfficeBench, rescues are mostly file listings and app switches and there is little credential state.

(2) How many needed spans verification recovers. We take successful uncompressed gpt-4.1 AppWorld trajectories and find every point where compression would fire (history > δ\delta). This gives 189 decision points across 65 tasks. At each decision point tt, a stronger model (GPT-5.5) examines the agent's actual subsequent steps (the complete steps from beginning to end) and labels which spans preceding tt were used in those step. We then use the smaller draft model (gpt-4.1) to score the same spans.

Metrics. Let (D) and (G) denote the sets of spans retained by the draft and gold model, respectively. We measure their alignment using:

  • Recall: ∣D∩G∣∣G∣\frac{|D \cap G|}{|G|}, the fraction of gold retained spans also retained by the draft model.
  • Precision: ∣D∩G∣∣D∣\frac{|D \cap G|}{|D|}, the fraction of gold retained spans also retained by the gold model.

GPT-4.1 as draft

MethodPrecisionRecallF1
Plan only0.580.640.61
Plan + defensive verification0.470.930.62

The verification step increases the recall to 0.93 covering almost all the spans marked required by gpt-5.5 indicating it restores important information required for the agent to successfully complete the task, with a recovery rate of 80% of the needed spans the plan misses.  We would like to point out that low precision score points that the draft model acting safely tends to overkeep spans and does not reflect poor performance.

We also observe that the increased recall also aligns and reflects the better performance of the agent (Table 1, main text)

(3) Why the benefit differs across benchmarks.

  • AppWorld is strongly stateful. Credentials, IDs and API signatures obtained early must persist across many steps and apps, and short-horizon plans tend to under-cite them. Verification fires at almost every event and recovers most of what the plan misses.
  • OfficeBench tasks are shorter and carry little persistent state. The plan already covers most of what is needed, as reported above verification step rarely fires and adds little.

We are happy to address any further concerns. If all your issues are addressed, we would highly appreciate it if you consider increasing the score.

Best!

AuthorsOpenReview ↗

Request to review our responses

Dear Reviewer za1y we request you to please review our responses and would love to further engage with you in a constructive discussion if you have any clarifying questions. If all your concerns are addressed we would highly appreciate if you would consider increasing your score.

Thank to the authors for the rebuttal. The authors addressed my concerns, and I would like to maintain my positive score.

Reviewer_spb3OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary
  • The paper proposes a practical approach to managing agent execution histories accumulated over multiple turns by summarizing them and selecting spans that are likely to be useful in future interactions.
  • The key idea is to use a draft model at inference time to automatically identify spans with high potential for future use. This approach offers a cost-efficient alternative to existing methods that either require a dedicated summarization model to be trained in advance or rely on a large model at every inference step.
Critical strengths
  • Using a draft model to automatically identify spans that are likely to be useful in the future offers a cost-efficient alternative to similar approaches that require a dedicated summarization model to be trained in advance or repeatedly invoke a large model at inference time.
  • Experiments on multiple agent benchmarks demonstrate improvements in task accuracy and efficiency, including reduced execution time and fewer interaction turns.
  • The proposed method can be applied at inference time without additional training. Although its execution process is more complex than conventional summarization using a large model, it achieves cost efficiency by leveraging a relatively small draft model.
Critical weaknesses
  • The paper lacks a comparison with a direct history summarization approach performed at inference time using a relatively small draft model. It is necessary to investigate whether the proposed method can maintain its cost efficiency compared with this alternative.
  • Plans generated by the draft model are expected to be less comprehensive than those generated by the target model. An analysis is needed to determine the extent to which the spans selected by the draft model align with those likely to be used by the target model's actual plans.
Questions for authors
  • (Line 291) As the number of stochastic plans (N) increases, the inference cost could become substantial even when using a draft model. Could the authors analyze the resulting computational overhead? A comparison against a baseline that directly invokes a small draft model at inference time to summarize the execution history would help clarify whether the proposed method provides a meaningful efficiency advantage.
  • (Tables 1–3) Please explain the abbreviations used in the tables, such as Stp, Steps, Pk, Peaks, and Dep, in the corresponding captions.
  • (Tables 1–3) Descriptions of the models used as baselines are missing. Please provide sufficient information about these models to facilitate interpretation and reproducibility.
  • (Lines 341–342) If there are preliminary analyses evaluating the accuracy of the draft model in identifying useful spans, please share the results. Such analyses would help clarify how reliably the draft model identifies spans that are likely to be useful for the target model.
2 comments
AuthorsOpenReview ↗

Request to review our responses

Dear Reviewer spb3 we request you to please review our responses and would love to further engage with you in a constructive discussion if you have any clarifying questions. If all your concerns are addressed we would highly appreciate if you would consider increasing your score.

AuthorsOpenReview ↗

Request to review our responses

Dear Reviewer spb3 as the author-reviewer discussion window is coming to a close, we request you to please review our responses. We would love to further engage with you if you have any clarifying questions. If all your concerns are addressed, we would highly appreciate if you would consider increasing your score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Rebuttal (2/2)

Baselines (Q3)

We thank the reviewer for pointing this out. The baseline results in Tables 1–3 are taken from ACON (Kang et al., 2025). We will add the descriptions to the appendix in the revised paper.

Following are the baselines description (taken from ACON Kang et al 2025): 1. No Compression: Retains the full, uncompressed interaction history. 2. FIFO: Keeps the 5 most recent interaction turns and discards older ones. 3. Retrieval: Keeps the last turn plus the 4 past turns most similar to the current query, using OpenAI text-embedding-3-large embeddings. 4. LLMLingua: Extractive token-level compression using an encoder-only LM with a keep rate of 30%. 5. Prompting: The same model (as agent=GPT-4.1) summarizes the interaction history using a generic, hand-written compression instruction and ACON configurations. As suggested in W1, we additionally evaluate a summarization approach using a smaller draft model, we will also include the results(W1 Table) in the updated paper.


Draft-model span selection analysis (Q4)

We thank the reviewer for this question. We conducted an analysis to evaluate how accurately the draft model identifies spans that the agent subsequently uses.

Setting (OfficeBench): We used successful uncompressed GPT-4.1 agent trajectories. We identify points where the compression rule would trigger (history exceeds δ\delta tokens), yielding 99 decision points across 35 tasks. At each decision point tt, a stronger model (GPT-5.5) examines the agent's actual subsequent steps (the complete steps from beginning to end) and labels which spans preceding tt were used in those steps, including values, file paths, IDs, results, or errors that influenced later decisions. We then use GPT-4.1-mini as a draft model to score the same spans (N=3N=3, τ=1/3\tau=1/3).

MethodPrecisionRecallF1
FOCUS0.480.800.60

The results show that even a lightweight draft model achieves 80% recall, successfully identifying most spans that the agent subsequently uses. The precision of 0.48 indicates that the draft model also retains additional spans. This reflects a conservative retention strategy that prioritizes preserving potentially decision-relevant information over aggressive compression.


We are happy to address any further concerns. If all your issues are addressed, we would highly appreciate it if you consider increasing the score.

Best!

AuthorsOpenReview ↗

Overall Summary of Our Responses

We sincerely thank all reviewers for their constructive feedback and thoughtful suggestions. We summarize the conducted additional experiments and analyses to address the major concerns raised.

  1. Comparison with lightweight summarization (spb3 W1): We evaluated direct inference-time history summarization using a smaller draft model (GPT-4.1-mini). Compared to this Summarization approach, FOCUS achieves 9.5% points higher accuracy on OfficeBench while reducing cumulative dependency by ~30%, demonstrating a stronger accuracy-efficiency trade-off.

  2. Reliability of draft-model span selection (spb3 W2,Q4): We analyzed alignment between draft and target-model span selections. GPT-4.1-mini as draft model achieves a recall of 0.9, recovering most spans selected by the target model (GPT 4.1). An additional analysis with a stronger model (GPT 5.5) with access to complete agent trajectories yields a recall of 0.8 with the small draft model (gpt-4.1-mini), supporting the reliability of lightweight draft models.

  3. Effectiveness of defensive verification (za1y W2): We quantified the spans recovered by verification and their importance. On AppWorld, verification recovers 80% of needed spans missed by the initial plan, improving recall from 0.64 to 0.93 (from plan only to plan + verify). We also analyzed why its benefits differ across benchmarks, particularly due to differences in persistent state requirements.

  4. Computational overhead (spb3 Q1): We extended the analysis (Table 4) of stochastic plan sampling (NN) with API costs. With GPT-4.1-mini (Table 5) as the draft, the draft overhead accounts for only ~4% of total API cost.

  5. Citation frequency and counterfactual utility (za1y W1): We further analyzed disagreements between draft-model citation scores and KL-based counterfactual estimates. Most disagreements involve additional retention rather than potentially important information being dropped. With the share of KL important spans the draft keeps being 0.7 and 0.76 on the above OfficeBench and AppWorld events. These align with the overall performance of the method (Tables 1,2,3)

  6. Controlled comparison with ACON (CAma W2): We re-evaluated FOCUS on OfficeBench using the same compression threshold as ACON (δ=4096\delta=4096). FOCUS achieves +3.2 percentage points higher accuracy (77.9% vs. 74.7%), with fewer interaction steps, lower peak context, and reduced dependency, despite requiring no offline training.

  7. Theoretical clarification (CAma W1, Q1): We clarified the two-stage connection between the information bottleneck objective, counterfactual utility, and draft-model citation frequencies.

  8. Comparison with PAACE (CAma W2, Q3): We clarified key methodological differences: PAACE requires offline compressor training on 1.2M synthetic workflows, whereas FOCUS is training-free. PAACE also relies on externally supplied plans, while FOCUS generates its own plan sketches. We explained the challenges in making a controlled comparison due to no available code implementation of PAACE.


We hope these additional results and clarifications address the reviewers' concerns and strengthen the empirical evidence supporting FOCUS. We sincerely appreciate the reviewers' time and consideration and would highly appreciate a response from the reviewers, and a consideration in increasing score if their concerns are addressed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.