acceptodds
Withdrawn from ICLR 2027

: earning onstant-size atent ontext for Language Model Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language model agents have become highly capable across many areas. They accumulate increasingly long contexts of observations, actions, tool outputs, and intermediate reasoning. Without proper context management, agents' performance can degrade significantly as the context grows. Another major problem is that long context can drive costs up, making language model agents prohibitive in practice. In this work, we ask: can we effectively compress the agent context without affecting its performance? In this work, we propose : a framework for learning a constant-size latent context for a frozen language model agent. We leverage the strong representational power of the continuous latent token space: even as the rounds of agent interaction increase, the number of (latent) tokens representing the context remains constant. Concretely, we train a lightweight compressor to compress past context history into a constant number of latent tokens that we insert directly into the frozen language model's embedding sequence. We use a two-stage training procedure: behavioral supervised learning first teaches the compressor to preserve the full context, followed by reinforcement learning that directly optimizes the compressor using the downstream task reward. We argue that this training procedure can learn a compact latent representation of an agent's context, rather than relying on pure natural-language text summarization. Results on three widely used agentic benchmarks show that our method can improve performance up to 10% while reducing the token count to one-tenth of the original.

Primary area:
foundation or frontier models, including LLMs
Keywords:
language model agents, latent context representation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zTfrOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes (LC)², which uses a small trainable compressor to recursively compress a language model agent’s growing interaction history into a fixed number of latent tokens. These tokens are inserted into a frozen large model, so the context length no longer grows with the number of turns. The compressor is first trained with supervised learning on successful trajectories and then optimized with GRPO using task rewards. On ALFWorld, WebShop, and HotpotQA, the method matches or exceeds ReAct while using far fewer input tokens.

Critical strengths

The direction is well motivated. Extending latent compression from static documents to multi-turn, stateful agent contexts is a natural next step, and optimizing the compressed representation for future actions rather than for reconstructing the original text is a sensible design choice.

The method is clearly designed and modular. The large model stays frozen, and only a small compressor and two mapping layers are trained. The ablation of the two training stages also shows what BSL and RL each contribute.

Critical weaknesses

The comparisons are unfair and the baselines are insufficient. The proposed method is trained with both supervised learning and RL on each task, while the three baselines involve no training at all. The performance gains may therefore come mainly from task adaptation rather than from compression itself. Two important groups of baselines are missing: latent compression methods (ICAE, Gist tokens, Activation Beacon) and learned context-management methods (MEM1, ACON, AgentFold). In addition, the reported “token count” only covers the frozen model’s input. It ignores the compressor’s computation at every turn, and the paper does not report latency, FLOPs, or training cost, so the efficiency claims are incomplete.

The experimental scope is limited, and some analyses are not rigorous. None of the three tasks involves very long interactions, so the long-context and context-rot problems that motivate the paper are not really tested. Only one model pair (Qwen2.5-7B with a 0.5B compressor) is used, and on HotpotQA the K=16 variant is below ReAct in EM. Figure 2b compares success rates grouped by trajectory length, but length itself depends on whether the task succeeds (failed episodes tend to run longer), so the conclusion that the method is better on long horizons is not reliable. There are also inconsistencies: the text says the KL term is omitted, yet Table 3 lists a KL coefficient of 0.04, and the paper does not explain where the successful training trajectories come from.

Questions for authors

With the same data and training pipeline, how would the method compare with directly fine-tuning the frozen model (e.g., with LoRA), or with training an adapter of similar size that sees the full ReAct history? This would show whether the gains come from compression or from task-specific training.

Could you report the end-to-end inference cost including the compressor (FLOPs or wall-clock latency) and the training cost? Please also explain which model generated the successful trajectories used for BSL, and how they were collected.

Reviewer_jbFMOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces a method for constant size latent compaction LC2LC^2. The key differentiation to a regular latent compaction paper is that the number of latent tokens put into the decoder model is constant even as the context increases. The authors keep the backbone decoder llm frozen and train a compressor model which outputs K latent tokens. They also train 2 auxiliary networks: one to feed the latent tokens back into the compressor to be iterated on with the next action, and one to feed the latent tokens into the decoder network. There is 2 stages to training an offline stage followed by online GRPO, for GRPO a TBPTT is used with a max of 8 to keep stability during training. The authors compare to ReAct, sliding window and summarisation baselines. LC2LC^2 actives the highest accuracy on all 3 benchmarks considered. The authors also show the choice of K is important as large K can cause overfitting.

Critical strengths
  1. Interesting concept to try constant size compression
  2. Method well explained
Critical weaknesses
  1. Related work on latent compaction outdated.
  2. The baselines are not given equal training compute. Making it very hard to discern whether the improvements of LC2LC^2 come from training in a harness or from the actual method. For example, a summarisation model training in a harness in the same 2 stage way may be able to perform much better.
  3. Models used (Qwen 2.5) are a little old.

My main concern is the lack of compute matched baselines, this makes it extremely difficult to discern the merits of the method.

Questions for authors
  1. For the methods in table 1 what are the actual runtimes of the methods? For example if summarisation happens infrequently compared to LC2LC^2, then it could actually be faster.
  2. Constant size compression is inherently lossy, we don’t see this in the experiments so far. What context lengths might we expect to see this occur?
  3. How many training tokens for BSL and iterations for GRPO are the models trained on?
Reviewer_g8ugOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LC2 learns a fixed number of continuous memory tokens for a frozen language-model agent. A small transformer updates these tokens from the previous memory and the latest interaction, with separate mappings into the compressor and agent embedding spaces. The compressor is trained first on successful trajectories and then with task-reward reinforcement learning. Experiments on ALFWorld, WebShop, and HotpotQA report competitive task performance with substantially fewer agent input tokens.

Critical strengths
  • The recurrent interface keeps historical memory bounded without changing the agent weights. Reading out a fixed set of compressor states and mapping them into the frozen agent is a simple, understandable design with potential practical value.

  • The approach is evaluated in three different interaction settings, and the ALFWorld training ablation shows why initialization matters. Supervised training alone reaches 55.6, while the full two-stage procedure reaches 63.8, giving concrete evidence that the proposed optimization can improve the learned interface.

Critical weaknesses
  • The performance comparison confounds memory representation with additional task training. Although the agent weights are frozen, its action distribution is changed by a prefix optimized using demonstrations and downstream rewards. ReAct, the sliding window, and the prompted summarizer do not receive comparable adaptation. The training ablation shows that reinforcement learning matters, but does not show that retaining the interaction history in recurrent latent tokens is the reason for the improvement. A basic control would reset the recurrent memory while retaining the same compressor and training procedure, to test how much the model gains from history rather than a learned task-specific prefix. A comparably trained memory baseline would strengthen the conclusion further. The demonstration provenance and training splits also need to be specified so that the generalization results can be assessed.
Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.