acceptodds
Withdrawn from ICLR 2027

MemTact: Query-Aligned and Cost-Aware Dynamic Memory for LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-term memory allows large language model agents to reuse past information without repeatedly reading an ever-growing interaction history. Yet memory systems must solve two fundamentally different problems: they must make relevant evidence easy to find, and they must make frequently used information cheap to consume. These objectives are often entangled in a single compressed representation, even though retrieval and direct reading impose different requirements and costs. Moreover, the right representation cannot be determined once at storage time: future queries reveal both how information is searched for and which evidence is worth keeping readily accessible. We introduce MemTact, a use-oriented framework that treats navigation and information residency as distinct cost decisions. Navigation uses source-linked semantic hierarchies to connect concrete evidence with queries at broader levels of abstraction. Residency adapts to observed query usage, weighing retrieval, recovery, reading, and update costs. The framework distinguishes recoverable, findable, and resident service states; the current implementation combines a fixed hierarchical index with query-driven residency updates. Because queries can require complementary evidence, we formulate joint evidence placement and derive an exact maximum-weight-closure solution under conjunctive demands, additive state costs, and no hard capacity constraint. Across 460 questions from RULER, EventQA, and LongMemEval, MemTact achieves 61.52% accuracy, exceeding RAPTOR and A-MEM by 19.35 and 11.09 percentage points, respectively. It uses 62.6% fewer online API tokens than A-MEM and 66.4% fewer than QueryLink, demonstrating a favorable balance between answer accuracy and serving cost.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM memory, memory compression, adaptive memory, query–memory alignment, cost-aware memory, long-horizon agents

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_UcJoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MemTact, which separates long-term LLM memory into two parts: Navigation Memory and Information Memory. The first part uses hierarchical semantic representations for retrieval, and the second part dynamically decides which evidence should be kept in the readable memory based on previous queries and recovery costs.

The paper also introduces a joint evidence placement optimization problem and shows that it can be solved exactly using minimum cut under certain assumptions.

Experiments compare MemTact with RAPTOR, A-MEM, and QueryLink on three benchmarks. MemTact achieves 61.52% overall accuracy, which is better than RAPTOR and A-MEM but worse than QueryLink. Its online token usage is lower than A-MEM and QueryLink but higher than RAPTOR.

Critical strengths

1. The paper considers both retrieval and reading costs.

I think separating findability and residency is a reasonable system design. The paper points out that information which can be retrieved does not necessarily need to stay in the memory that the LLM reads every time. This is a practical consideration for long-term memory systems, although I am not convinced this is a new idea.

2. Some experimental results are positive.

MemTact gets better accuracy than RAPTOR and A-MEM, and uses fewer online tokens than A-MEM and QueryLink. So the combined system does show some practical benefits in the evaluated settings.

Critical weaknesses

1. I have a hard time understanding what the actual methodological contribution is.

The paper presents Navigation Memory and Information Memory as two main contributions. But to me, Navigation Memory is basically a retriever with hierarchical semantic representations. Hierarchical retrieval, query expansion, and multi-view retrieval are already quite standard. It is difficult to see what is fundamentally new here.

For Information Memory, isn't this essentially a cache? The system keeps some frequently useful evidence, recovers information when needed, and updates what should remain in memory based on previous accesses. This is very similar to cache admission and replacement, or simply taking notes about what should be remembered for future queries.

I understand that the authors have some specific implementation choices, but why should these be considered methodological contributions rather than a combination of existing components? Most existing long-term memory systems already need retrieval and some form of memory update. Simply separating them into two modules does not seem sufficient.

2. The theoretical formulation and experimental claims are not convincing to me.

I find Section 3.4 particularly confusing. The paper introduces an exact optimization formulation using maximum-weight closure and minimum cut. But then the authors explicitly say that the actual online controller is a separate practical policy and does not solve this optimization problem.

Then why introduce this exact optimization formulation in the first place? If the implemented system neither solves it nor empirically validates its relevance, what is the actual contribution of this formulation? I cannot see how it connects to the algorithm being evaluated.

The experiments also do not convincingly support the claimed cost-optimal trade-off.

MemTact does not achieve the best accuracy, and it does not achieve the lowest online token usage either. Of course, I understand that a method does not need to be the best on every metric. It can be valuable if it achieves a better accuracy-cost trade-off.

But how do we know this is actually a better trade-off?

The paper only compares one operating point for each method. Maybe QueryLink with a smaller retrieval budget can achieve higher accuracy than MemTact at a similar token cost. Maybe another configuration of RAPTOR can reach a better accuracy-cost point. Without showing the accuracy-cost curves or comparing the Pareto frontiers, how can the paper conclude that MemTact provides a better balance?

Even the dynamic residency experiment does not clearly demonstrate cost savings. It improves accuracy, but also increases Reader token usage. The fact-reuse ablation shows token savings on a small selected set of target queries, but excludes the prefix-query cost required to build up the dynamic memory state from its reported target usage.

To me, the paper has not clearly demonstrated either the benefit of its optimization formulation or the claimed end-to-end cost advantage of its memory controller.

Questions for authors
  1. What exactly is new about the two proposed memory components?
  2. Why is the exact optimization formulation introduced if the system does not use it?
  3. How can the authors claim a good accuracy-cost trade-off based on only one operating point per method?
  4. Does dynamic residency actually save cumulative serving cost?
Reviewer_1zZWOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper separates finding evidence from deciding what evidence should remain in the memory that the model reads. MemTact uses hierarchical retrieval views to locate source content. The reader first uses compact memory and can recover the original text when needed. Cited source sentences can then be retained, with block-level budgets adjusted using observed recovery and reading costs. The question is whether this cost-aware decision improves on simply saving previously used evidence.

Critical strengths

Text that helps retrieve a record is not necessarily enough to answer a question, and evidence that was useful once may not be worth reading every time its block is retrieved. Separating these decisions is a reasonable way to study memory design. The paper also clearly distinguishes the restricted joint-placement problem from the practical online controller.

Critical weaknesses

The added value of the cost-based retention rule over simply saving cited sentences is still unclear. The main dynamic-versus-fixed comparison introduces two changes together: allowing evidence to persist across queries and adjusting the retention budget based on cost. If the dynamic arm performs better, the current comparison cannot tell us how much comes from saving old evidence and how much comes from making better cost-based choices. A direct comparison would also retain cited sentences but omit the cost-feedback rule.

On the 300-question stream, dynamic memory answers 191 questions correctly versus 184 for fixed memory, with only 0.89% more reader tokens. This is a positive quality result at similar usage. However, no eviction occurs in this stream, and the selected reuse experiment mainly shows later savings. The experiments do not yet clearly show when the controller rejects or replaces evidence that was previously useful but is not worth keeping. The description of how budgets grow and are reduced by repeated reading costs is also qualitative.

This is central to my assessment because deciding what is worth keeping is the more interesting research claim. If simply retaining cited sentences gives similar results, the system may still be useful, but the contribution of cost-aware control would be weaker.

Questions for authors

Could the authors specify the actual budget-update rule and compare it on the existing query stream with a strategy that retains cited sentences without cost-based budget adjustment? It would be helpful to show when their retention or replacement decisions differ, and whether these differences improve accuracy versus reader-token usage.

Reviewer_CSYSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MemTact, separating "index for query" and "things to read after query" for agent memory systems, and dynamically adjust the later based on problems.

(1) Construction time: Offline indices are constructed. A 7B model is trained to do these extraction.

(2) Query time: related indices retrieves related text summarization blocks. These summarizations are fed into LLMs first, and if not enough, original text pieces can be provided.

(3) The memory is updated with queries. This is like updating cache. Authors use an optimal control model to model this situation.

On benchmarks like RULER, EventQA and LongMemEval, author's method show good accuracy-cost balance.

Critical strengths

(1) Separating "easy to be extracted" and "Easy/cheap to read" proposes a good view of analyzing the agent memory problem.

(2) The dynamic method which updates through optimizing the cost function is interesting.

Critical weaknesses

(1) MemTact uses trained GritLM-7B, while other methods use GritLM-7B. So the improvement might come from, (a) trained model, (b) multi-layer index, (c) compression process, (d) dynamic updates. There are relatively weak ablation.

(2) (Minor) The overall well-performance mainly comes from EventQA, and using only one answering model is a bit unenough.

Questions for authors

Please see weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.