acceptodds
Under review as a conference paper at ICLR 2027

QUOTA: Learning Downstream-Aware Context Allocation for Tool-Using Language Agents

Abstract

Tool-using language agents accumulate large volumes of tool outputs over long-horizon interactions. Existing compression methods primarily optimize immediate context length, but removing critical information can trigger task failures or redundant tool re-invocations, causing local token savings to increase end-to-end cost. We introduce QUOTA, a query-conditioned framework that learns how to allocate context budgets across individual tool outputs. At each compression event, QUOTA assigns a continuous retention quota to every output based on the query, execution state, and output characteristics. The policy is trained with REINFORCE using an objective that jointly considers task success, agent token consumption, and post-compression tool re-invocations, while keeping the agent and compressor frozen. To address delayed credit assignment, QUOTA learns an outcome model that compares each selected quota against a full-retention counterfactual, producing output-specific signals for policy optimization. Across held-out queries from a production data-analysis agent and three compressor backends, QUOTA reduces agent token consumption by 29%-46% relative to full retention, at success rates of 0.75-0.81 against 0.75 for full retention. It adds 15%-18% savings over a tuned tool-type-conditional policy. Interventional rollouts validate the learned outcome estimates, while cross-backbone and cross-compressor experiments demonstrate policy transfer. A separately trained policy reduces agent tokens by 18%-25% on five unseen LOCA-bench environments at success rates of 0.38-0.48 against 0.37 for full retention, showing that downstream-aware context allocation improves the efficiency of long-horizon language agents.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agent, Context Compression, Reinforcement Learning, Counterfactual Credit Assignment, Long-Horizon Task

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qjaDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces QUOTA, a method that reduces token consumption during multiple rounds of tool use by dynamically adjusting the retention ratio of each tool output. QUOTA keeps the parameters of both the agent and the compressor fixed. It makes retention decisions based on the current query and task progress. The reinforcement learning reward accounts for both task success and total token consumption throughout the interaction. It also penalizes additional tool calls. Furthermore, QUOTA uses an outcome prediction model to perform counterfactual comparisons. This model estimates the potential downstream costs of compressing individual tool outputs and provides more fine grained feedback for retention decisions. Experiments show that QUOTA reduces token consumption by approximately 29% to 46% compared with retaining all tool outputs, while maintaining or improving task success rates.

Critical strengths
  1. QUOTA considers the impact of context compression on subsequent tool calls rather than optimizing only the immediate token compression ratio. Its optimization objective accounts for task success rate, total token consumption, and additional tool calls caused by compression. This allows QUOTA to better balance immediate token savings against the recovery costs associated with information loss. It also better reflects the requirements of practical agent applications.

  2. QUOTA dynamically adjusts the retention ratio for each tool output based on query requirements, execution state, and tool output content without retraining the agent or compressor. By separating retention ratio allocation from the actual compression process, QUOTA offers greater flexibility and can be easily adapted to different agents and compressors.

  3. The experiments cover a variety of compressors and tasks. Ablation studies evaluate the contribution of counterfactual credit assignment, while intervention experiments verify the consistency between counterfactual predictions and actual execution outcomes. Overall, the experimental design is relatively comprehensive.

Critical weaknesses
  1. Counterfactual credit assignment relies on a learned outcome prediction model whose estimates lack rigorous causal guarantees. In addition, compressing different tool outputs may introduce complex interactions that could affect the accuracy of credit assignment.

  2. The paper primarily reports token savings achieved by the agent without fully accounting for the cost of compressor calls in its main experimental metrics. Since different compression backends vary in computational overhead and latency, further evaluation is needed to determine whether these token savings translate into lower deployment costs.

Questions for authors
  1. The paper emphasizes credit assignment for individual tool outputs. However, when the same tool is called multiple times, how does the method determine which historical output a subsequent call is intended to recover? If a single call recovers information from multiple outputs, how is credit assigned without double counting? These rules directly affect the training objectives of the credit assignment model, but they are not clearly explained in the paper.

  2. The layout of the figures and tables on page 9 is cluttered and should be revised for clarity.

Reviewer_HJPBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

QUOTA learns a continuous retention ratio for each tool-output instance when a tool-using agent compresses its context. It keeps the agent and compressor fixed, trains the allocation policy from task success, agent tokens, and excess tool calls, and uses a learned outcome model to estimate the counterfactual cost of fully retaining one output. The evaluation includes 40 held-out production queries, three compressor backends, single-output interventions, backbone and compressor transfer, and a separately trained policy on LOCA-bench.

Critical strengths
  1. The paper addresses a real gap between immediate compression and downstream recovery calls. In Table 1, QUOTA reduces agent tokens beyond a tuned tool-type-conditional allocation under all three compressors, supporting the value of decisions at the output-instance level.
Critical weaknesses
  1. Equation 7 combines an explicit local-saving preference with learned counterfactual cost, but Table 2 turns the entire channel on or off. Table 3 validates the model's predictions against interventions; it does not establish that those predictions, rather than the local-saving term or an additional training signal, cause the token gain over RL-only.
  2. The 29%–46% savings in Table 1 count agent tokens but exclude Summarization model calls and Self-Info scoring. The evidence does not yet establish that the full system saves cost while preserving task success.
Questions for authors
  1. Under the same training budget and on the same held-out records, what happens to success, agent tokens, and excess calls when the local-saving term and the learned counterfactual-cost term in Equation 7 are removed separately?
  2. How does QUOTA compare with a query-conditioned scalar quota or a lightweight per-output policy given the same state features and training budget?
  3. For each compressor, what are per-task billed cost and latency after including compressor calls, scoring compute, and agent input/output tokens, relative to keep-all and tool-type-conditional allocation?
Reviewer_p1XMOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Motivation: Tool-using agents compress accumulated tool outputs when context length is close to limit. However, this local compression may lead to eventual higher token cost. proposes QUOTA, a small MLP that assigns a Beta-distributed retention ratio to every tool-output instance at each compression event and is trained with REINFORCE on success, session tokens and event-local excess calls, while the agent and compressor stay frozen. The major novelty is: using tool conditioned RL training method to assign continuous quotas to individual tool output instances.

Critical strengths
  1. Applying the training-based Reinforce method to a training-free dominant solution problem (agent context compression) is a novel application.
  2. In general the writing is easy to follow and definitions are clear
  3. Experiments shows confidence interval and multi-seeds results demonstrating rigorous
  4. Light-weight solution and easy to deploy with a 120k MLP useable by CPU
Critical weaknesses

Major:

  • Is it a submission to other conferences as the supplementary shows "AAAI"?
  • Narrow and questionable reproducibility in terms of evaluation. The main claims rest on 40 held-out queries from one private agent. The success differences of 0–6 points versus keep-all correspond to only a few query–seed records and are not tested (App. I).
  • Small gain from the novel component. The counterfactual channel adds only 3.4–4.7% token savings over RL-only
  • Too little training to trust that the policy learned. Although limited by the number of verified outputs written by human, each seed trains on only 100 sessions drawn from 80 queries, most queries are seen once, it is hard to believe that training is really effective with counterfactuals.

Minor:

  • Weak comparisons. The heuristic baselines are not tuned and mostly lose even to no compression; e.g., "Recency" keeps only 10% of the oldest output. There is no tuned single ratio for all outputs and no other learned context manager (e.g., ACON, AdaCoM). Minor:
  • The method limits to a fixed number (maximum 16) tool outputs, which is not flexible and not generalizable which may only work for limited settings with fewer tool calls/types.
  • The choice of learning method and the condition used are not justified, and the new part adds little. They could be cherry picked.
Questions for authors

Questions

  1. Why REINFORCE? Was any other learner tried, such as picking ratios directly with the learned cost model? Even a heuristic explanation helps.
  2. How does QUOTA compare with a single tuned ratio for all outputs, with a uniform ratio set to QUOTA's own average, and with the LLM-judge baseline in the code?
  3. Were the results produced with one ratio per output or one ratio per tool type (the 16 network slots match the 16 tool types)? How were sessions with more than 16 outputs handled on LOCA-bench, and was the LOCA policy trained with the same method?
  4. Which test and data produced p = 0.008 and Table A3? How is NMAE defined? Why is no-compression success 0.64 in Table 5?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.