acceptodds
Under review as a conference paper at ICLR 2027

LAM: An Efficient And Controllable Lossy Agent Memory Framework

Abstract

Agent memory grows as agents read inputs, reason, and call tools. Longer histories increase inference cost and eventually exceed the context window. LLM-based summarization reduces this history but adds latency and provides no explicit bound on information loss. We propose LAM, a Lossy Agent Memory system with three components: a deterministic deduplication rule with a substitution bound on retrieval scores — a bound on score perturbation, not a certificate of unchanged ranking, a memory manager that preserves the cached prefix and overlaps compaction with inference, and a performance model that estimates compaction costs before deployment. On 600 agent trajectories, LAM removes 22.47% of observation tokens while retaining 99.984% of the measured gold-patch evidence. Evaluation on end-to-end latency shows that LAM achieves a 71.4×–91.6× end-to-end speedup from removing records before prefill, validating LAM's effectiveness and efficiency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, agent memory, context compression, memory compaction, error-bounded lossy compression, rate-distortion

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_geNcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes an agent memory compression method that deduplicates similar records during agent trajectory generation instead of post-generation summarization, which avoids re-prefilling the context. The deduplication step involves byte-exact hashing to remove exact records in the trajectory, and an embedding similarity-based near-deduplication process to remove similar records in the trajectory and replace them with a retained representative. They also provide a bound on the replacement based on the retrieval scores. Another design choice is about when the compression should occur. They propose to remove the duplicate records when they arrive and before prefilling, which makes the KV cache valid without the re-prefilling cost of summarization-based compression methods. Experiments show that their method removes 22.47% of observation tokens while retaining 99.984% of the measured gold-patch evidence.

Critical strengths
  • The idea of applying compression techniques before the records enter the trajectory and get prefilled can reduce computation cost compared to summarization-based methods is valid.
Critical weaknesses
  • Poorly written. The paper looks heavily AI-generated, with a lot of AI writing artifacts, and much of the content is hard to follow.
  • Lack of novelty. The proposed method looks like engineering tricks rather than a novel research work.
  • Insufficient experiments. The evaluation is mostly conducted via proxies rather than end-to-end experiments.
Questions for authors
  • Can the authors run end-to-end experiments to report the actual solve rate and behavior change on the compacted context instead of using proxies?
Reviewer_EPYVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes LAM, an online memory management method for long-horizon agents. It removes exact and near-duplicate tool observations before they enter the model's context, replacing removed content with short references. The method provides a mathematical bound on changes to retrieval scores, although this does not guarantee unchanged task performance.

The paper evaluates LAM on recorded SWE-smith trajectories and reports 22.47% token removal while preserving 99.984% of measured patch evidence. It also studies the contribution of different compression layers, token costs, and runtime efficiency. A cost model predicts a 71.4–91.6× speedup from removing redundant content before prefill rather than afterward.

Critical strengths

The method is simple and well motivated. Its mathematical guarantee is clear, and checking each removed record against a retained record avoids errors from repeatedly merging similar records.

The experiments provide useful analysis of token removal, evidence retention, and runtime costs. Near-duplicate matching removes about 4.5 additional percentage points of tokens compared with exact hashing alone.

Critical weaknesses

The reported 71.4–91.6× speedup is based on a cost model rather than actual end-to-end measurements. The paper describes a 32K context budget but uses 360K-token cache rebuilding costs in its analysis, making it unclear how realistic the reported speedup is.

The practical benefit over simpler methods is not fully demonstrated. Exact hashing already removes about 18% of tokens, and the SemDeDup comparison does not include the same exact-hashing stage. The paper also does not measure whether LAM preserves task success in actual agent runs.

Questions for authors

Can the authors explain why the 32K-budget experiment uses 360K-token rebuilding costs and provide actual end-to-end runtime measurements?

Can the authors compare LAM against exact hashing alone and SemDeDup with the same exact-hashing stage, using live agent runs to measure both task success and runtime?

Reviewer_pEX3OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

this paper focuses on agent context compaction and retention. the authors introduce the LAM framework to shrink llm agent history. LAM has three layers: L0 removes exact duplicates with hashing, L1 uses an embedding similarity threshold to drop duplicate chunks, L2 compresses stored embeddings. evaluations on swe smith trajectories show that LAM could remove 22.47% of observation tokens while retaining 99.984% of the measured gold-patch evidence. filtering before prefill also gives much lower latency per turn (as compared to deleting things from prefilled context).

Critical strengths

+ timely topic. agent context compression is important, and we need not just semantic level considerations but also infra-level considerations (like KV cache, prefill/decode, which this paper partially covers).

+ redundancy measurement is a major insight: a lot of tool outputs are repeated. very legit motivation for the design.

+ the layered design is clear and reasonable, and i can follow the motivation and design for each layer very well.

+ very good results on the selected swe-smith trajectories.

Critical weaknesses

two critical weaknesses that prevent me from believing in the evaluation takeaways of this paper

- choice of dataset and evaluation subset: this paper uses one dataset for evaluation (swe-smith); the scope is quite narrow and it is unclear whether the proposed method would generalize to datasets outside swe-smith and domains beyond software engineering. on swe-smith, the authors pick 600 trajectory subset out of 6026 ("unless stated otherwise we use a fixed 600-trajectory subset"): this is highly suspicious. please at least clearly state why and how you choose this subset.

- every speedup number in this paper is computed from a mathematical formula instead of measured from real empirical runs (section 2.6). there is also very little justification or endorsement on why this performance model could hold true in empirical runs (this formula is derived from a synthetic session). the authors do mention this but only in captions and in appendix. the abstract and the intro (including the contribution list) presents speedup results as if they are measured empirically without bringing up the performance model. this is highly misleading and confusing to readers.

Questions for authors
  • could you explain why and how you select the 600 trajectory subset from 6026 swe-smith trajectories?
  • could you present results on datasets beyond swe-smith?
  • could you explain why your performance model would realistically reflect empirical measurements?

Also see "weaknesses".

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to reviews

Dear reviewers,

We thank your valuable suggestions and comments. We agree that two major issues of the submitted paper: lack of end-to-end run and major writting refinement. We will improve the work according to the reviews.

Best,

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.