acceptodds
Desk rejected at ICLR 2027

HasMem: Hard-Origin Adaptively Softened Memory for Long-Term LLM Agents

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Text-based memory and context compression support reuse of past interactions. Resizing continuous memory changes the input to a frozen LLM, coupling capacity allocation with readout. We propose Hard-Origin Adaptively Softened Memory (HasMem). Frozen hard-prompt embeddings provide a verifiable initial state. A controller adjusts memory widths, a Writer re-encodes resized entries, and Reader and Global provide readout adaptation and cross-turn state. On all questions in a reconstruction probe derived from the Multi-Session Chat (MSC) development split, the main configuration achieves lexical F1 of ( percentage points) at of the hard reference's framed memory positions. With approximately matched per-question target body budgets, six configurations at mean per-entry retention around – exceed rule-based re-encoding by – exact-match (EM) percentage points. With fixed model parameters and rule target width ratio , Global's EM gain passes a user-level exact paired test with Bonferroni correction over eight comparisons. On all LongMemEval-S questions, local lexical F1 rises from the hard reference's to , and answer negative log-likelihood (NLL) falls from to . F1 gains accompany lower EM on both evaluations.

Primary area:
foundation or frontier models, including LLMs
Keywords:
long-term memory, LLM agents, interative agents, memory-augmented language models, prompt compression, soft prompts

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ZcGWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper proposes HasMem, an adaptive continuous memory framework for long-term LLM agents. It initializes memories using token embeddings from a frozen LLM and stores them in a Soft Bank. A Controller decides whether to KEEP, SHRINK, or EXPAND each memory, while a Writer re-encodes memories when their lengths change. Reader and Global are used to improve memory readout and maintain information across interactions. The framework is trained using QA supervision and memory capacity costs. Experiments show that HasMem reduces memory usage and improves lexical F1 and answer NLL in some settings. Learned memory updates also outperform rule-based re-encoding at several compression levels, although Exact Match drops on both benchmarks.

Critical strengths
  • HasMem allows memory sizes to be adjusted over time and also addresses how the LLM can effectively read compressed representations. Jointly considering memory updates and readout is a useful aspect of the design. This is a useful direction for long-term LLM agents that need to manage growing interaction histories.
Critical weaknesses
  • The framework has several components (Controller, Writer, Reader, and Global), making it hard to follow the overall process. I suggest adding a short overview at the beginning of Section 3.1 to explain what each component does and how they work together. A simple end-to-end example would also help readers understand the workflow.
  • The paper already compares HasMem with rule-based re-encoding and some existing memory methods. However, it would be useful to include simpler baselines, such as keeping only the most recent memories (Recent-k / Sliding Window) or selecting informative tokens using methods like Selective Context [1].
  • The MSC-derived dataset contains at most seven memory records per user, and the questions mainly involve simple information retrieval. Real-world agents may need to handle hundreds or thousands of interactions. It is unclear whether HasMem can maintain memory quality over such long histories.
  • Although the Controller supports KEEP, SHRINK, and EXPAND, the main experiment reports zero EXPAND operations. This makes it unclear whether EXPAND is actually useful.
  • HasMem improves F1 but reduces EM on both benchmarks. This raises concerns about whether compressed memories can preserve exact factual information, especially names, numbers, and other specific details.

[1] Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. Compressing Context to Enhance Inference Efficiency of Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6342–6353, Singapore. Association for Computational Linguistics.

Questions for authors
  • Could you provide a step-by-step example showing how a new memory is stored, updated, and used to answer a question?
  • How does HasMem perform with longer interaction histories? Does repeated compression lead to information loss over time?
  • Why is EXPAND never selected? Could you provide an example or experiment showing when EXPAND helps?
Reviewer_hoBvOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper aims to compress the memory stored as the soft prompt for the LLMs equipped with long-term memory. To achieve this, the paper proposes a dynamic compression framework named Hard-Origin Adaptively Softened Memory (HasMem). More specifically, HasMem dynamically decides whether to compress each scheduled memory item during the inference stage. Since its compression is applied on the level of soft embedding instead of the hard prompt, the compressed embedding may be unreadable in the text level and thus hard for the LLM to understand. To mitigate this issue, HasMem introduces a Writer component to recover the semantics of the compressed embeddings, together with two low-rank readout modules (Reader and Global) that help the LLM to understand them. Evaluation results show that HasMem can achieve comparable or better performance in some metrics on MSC-derived dataset (built by the authors) and LongMemEval-S, compared with the case without memory compression.

Critical strengths

The problem authors focus on is important and may have big impact on real-world applications. A well-compressed long-term memory can save the context window of LLM and thus accelerate the inference speed.

Critical weaknesses

The critical weaknesses of this paper are listed as follows.

  1. The experimental result of this paper is not convincing. Firstly, most of the results on the main paper in on MSC-derived dataset, a dataset built by the authors. The authors should include more results on the widely used benchmarks such as LongMemEval-S in the main paper (beyond the single comparison in Table 2) to show the effectiveness of HasMem. Secondly, the major baseline is the hard reference, which is the case without any compression. To show the superiority of HasMem, the authors should show more comparison with existing SOTA on long-term memory compression, especially the baselines that also operate on the soft prompt level such as FlexComp and the context compressors you mentioned in the related work section.

  2. The experimental settings are not close to the real-world application of long-term memory. The self-built MSC-derived dataset only has at most 7 records per user, and each record is also relatively short. This is not a common case of long-term memory, which usually has much more record. Moreover, in most of the tables, the maximum retrieved-entry count k (how many record to be retrieved from the memory) is set to 8, which means all the records (at most 7) will be leveraged by the LLM. It further enlarges the gap to the real-world setting. Moreover, in Table 1, there is a setting that k is 2 where we can observe a significant drop on EM for HasMem. It also makes the effectiveness of HasMem doubtable.

  3. According to Table 1 with k =8, the saving on the size of memory is only 6.4%. This reduction is not big enough to bring significant benefit for LLM inference with long-term memory.

Questions for authors

Your evaluation on LongMemEval-S shows that the case without any memory compression (hard reference) performs poorly. Could you explain why? Could you find a setting on LongMemEval that hard reference can perform well and see whether HasMem can achieve comparable performance?

Reviewer_5w7jOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes HasMem, a long-term LLM memory architecture that stores records as continuous embeddings and learns to KEEP, SHRINK, or EXPAND their representations over time. It combines a Controller for memory-width decisions, a Writer for re-encoding, and Reader and Global modules for adapting the frozen LLM's readout. The results show improved lexical F1 with fewer memory positions, although exact-match accuracy decreases compared to the uncompressed baseline.

Critical strengths

The hard-origin initialization provides a clear equivalence guarantee with the original prompt embeddings when no memory modifications occur. The authors provide several ablation studies, comparisons with rule-based compression, and analyses across different configurations and random seeds. The paper is also relatively transparent about unfavorable results and acknowledges several experimental limitations.

Critical weaknesses

The scope, novelty, and practical relevance of the proposed method are questionable. The main functionality is shrinking and expanding stored embeddings, yet the main experiment never uses EXPAND (1,470 KEEP, 516 SHRINK, zero EXPAND). Thus, a major part of the proposed method is not demonstrated. What remains is essentially a compression strategy that repeatedly revisits memories through deterministic round-robin scheduling, regardless of which records are contextually relevant. This introduces additional computation without a demonstrated advantage over simpler alternatives. The growing number of stored memories raises scalability concerns, and repeated re-encoding could progressively degrade stored information, which is not adequately investigated. Furthermore, existing systems already support broader and more context-aware memory operations. A-MEM (NeurIPS 2025) identifies related memories and evolves their contextual representations; Memory-R1 (ACL 2026) learns ADD, UPDATE, DELETE, and NOOP operations; Mem0 (ECAI 2025) retrieves semantically related memories before deciding how to update them; and MemRefine (2026) supports content-aware merging, deletion, and preservation under storage budgets. Although these systems use different representations, the paper does not convincingly explain why repeated width adjustment is a preferable or sufficiently useful contribution. The experimental evaluation does not establish a convincing quality–efficiency trade-off. The main configuration saves only 6.4% of framed memory positions, while exact match decreases from 88.1% to 84.3% and answer NLL worsens. The reported F1 improvement is partly associated with fewer abstentions rather than better reconstruction. On LongMemEval-S, exact match and brevity-constrained correctness also decrease. Furthermore, the learned method's advantage over rule-based resizing does not translate into outperforming the uncompressed reference. Comparisons with existing compression methods are insufficient and largely relegated to the appendix. LLMLingua-2 is evaluated at a substantially different compression ratio (0.33 versus HasMem's 0.936) and with different input ordering, while ICAE uses a different backbone and substantially different memory budgets. Comparisons with simpler textual summarization, re-encoding, and stronger adaptive compression baselines at matched budgets are missing. Finally, the main reconstruction benchmark contains at most seven records per user, with a retrieval limit of eight, meaning all records can be retrieved. This does not adequately test retrieval, interference, or memory maintenance over long histories. Actual inference latency, storage overhead, and end-to-end efficiency are also not measured. These limitations are particularly concerning given the additional architectural complexity.

Questions for authors

Can the authors demonstrate a practical scenario in which EXPAND is actually useful and performed by the model? How does memory fidelity evolve after repeated shrinking and expansion? Can the authors compare deterministic round-robin scheduling against context-aware memory selection, particularly over longer and continuously growing histories? Can the authors provide stronger comparisons with existing compression approaches under matched memory budgets and report actual inference and storage costs? Such results could materially affect my assessment of the method's practical relevance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.