acceptodds
Under review as a conference paper at ICLR 2027

LiveMem: Maintaining LLM Long-running Memory by Living Memory State

Abstract

Long-running assistants and agents stand or fall by memory; yet prevailing memory systems keep memory outside the model: they store the interaction history as textual records and retrieve them back into the context on demand. This paradigm preserves access to history but leaves inference itself stateless; and the preserved texts are also limited to the information that can be verbalized. This paper studies an alternative approach to long-running memory: a memory state that lives inside the computation, which is updated online, is persistent across context turnovers, and continues to influence the model's behavior after its source evidence has left the context. We introduce LiveMem, which augments a pretrained full-attention LLM with a parallel recurrent memory branch: the branch maintains a living memory state across the lifecycle, while the original attention path provides exact access to a bounded working context. LiveMem updates and preserves this state as the active context turns over, and uses memory-oriented post-training to teach the model how to use the state. Experiments show that LiveMem achieves leading overall performance compared with RAG and other intrinsic memory methods. On LongMemEval, LiveMem answers questions from its memory state even when the supporting evidence has been fully removed from the context, and its test-time learning results indicate that the state carries patterns distilled from past experience rather than only retrievable records. Therefore, LiveMem establishes a performant implementation of state-based long-running memory as a practical complement to existing retrieval-based memory systems.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, long-term memory, stateful inference, recurrent memory, context turnover, long-running inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MKgeOpenReview ↗
Rating
3(+1)
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper argues that current LLM memory systems keep memory outside the model as text records that are retrieved on demand. Inference itself therefore stays stateless. The authors propose LiveMem, which attaches a parallel Gated DeltaNet-2 recurrent branch to every attention layer of a frozen Qwen3-4B. The attention path covers a bounded working context that is turned over FIFO by evicting KV chunks, while the recurrent state is meant to carry information forward after its source tokens have been evicted. Only the branch is trained, with SFT followed by GRPO-style RL, and eviction is applied during training so the model has to rely on the state. Evaluation covers 18 datasets in four groups: multi-hop and multi-question Wiki QA, conversational memory, test-time learning and long-document QA, under 8K/32K active budgets. LiveMem reaches the best overall average (0.519 vs. 0.458). Supporting analyses condition on how much evidence has been evicted (LongMemEval) and compare a zero-state truncated input (Trunc) against the full state.

Critical strengths

Training under real eviction on a frozen pretrained backbone. The recurrent branch is zero-initialized, so training starts exactly from the base model. Training-time masking matches inference-time KV release, which stops the model from bypassing the state through attention. This is a practical way to add memory without pretraining a hybrid architecture, and it is the main reason I see value in the work.

Critical weaknesses

The gains are not shown to come from the memory state. In Table 2, the zero-state Trunc variant already recovers most of the improvement over Qwen3-4B. For example, on 2Wiki-MQ the base model scores 0.336, Trunc 0.545 and the full state 0.587. Across tasks the state itself adds only about 1–4 points. The training mixture also contains MuSiQue, 2Wiki, NarrativeQA and classification data that overlap with the evaluation tasks, while none of the baselines were trained on this data. Without a control that fine-tunes Qwen3-4B on the same data and active budget but without the branch, the headline result may mainly reflect in-distribution post-training. This bears directly on the paper's central claim.

Baseline protocols make the comparison unfair.

δ-Mem is run under a protocol that discards out-of-window input. It never sees the evicted history, so it behaves like a truncation baseline (3.1% on the fully-evicted subset). RAG uses top-1 retrieval for multi-hop QA (2Wiki 0.117 vs. 0.817 for the base model) and is omitted from MQ and from the eviction analysis in Fig. 2. This is despite RAG outperforming LiveMem on LongMemEval-S (0.548 vs. 0.302). The Recurrent baseline is untrained and scores 0.000 on Banking and CLINC. The strongest baseline overall is the base model with no memory mechanism at all. Taken together, the claim of "leading performance over RAG and intrinsic memory methods" is not established.

Questions for authors

1: How does Qwen3-4B score when trained on the same data and active-context budget, but without the recurrent branch?

2: How does δ-Mem perform under a protocol where it streams the full input? How does RAG perform with per-question retrieval and a larger top-k (e.g., top-5) on multi-hop, MQ and the eviction-conditioned subsets of Fig. 2?

3:On single-question Wiki QA, the inputs fit in the window and no eviction happens, yet LiveMem underperforms the base model (HotpotQA 0.846→0.772, MuSiQue 0.461→0.383). Why does the branch hurt when it is not needed?

3 comments

Overall Response

Thank you for your review comments, we will respond to your concerns in the rebuttal. We strengthen the attribution of performance gains to memory state and decompose the gains from the post-training part. At the same time, we also re-adjust the baseline configurations to address the baseline setup. The adjusted overall results still support the conclusion that LiveMem is comprehensively leading.

Strengthening the Attribution of Performance Gains

We summarize your concern as follows: the gains of LiveMem come from the introduction of memory state and from post-training, but these two parts are mixed in the table without separate attribution. Therefore, we further refine this attribution. We select representative recall-style QA tasks (LongMemEval, LoCoMo, 2Wiki-MQ, HotpotQA-MQ, MuSiQue-MQ), filter samples whose supporting evidence is entirely outside the context window, and rerun generation with LiveMem after clearing the memory state on these samples, so as to compare the gain brought by Memory State. Then, on the same samples, we compare the QA scores of LiveMem without memory state and the Qwen3-4B backbone under the same conditions, in order to distinguish the gain brought by post-training.

LiveMem w. Memory StateLiveMem w.o. Memory Statew./w.o state diffQwen3-4BLiveMem w.o. Memory State vs Qwen3-4B diff
LongMemEval0.1650.121+4.4%0.031+9.0%
LoCoMo0.1460.081+6.5%0.059+2.2%
2Wiki-MQ0.4060.279+12.7%0.095+18.4%
HotpotQA-MQ0.2410.229+1.2%0.082+14.7%
MuSiQue-MQ0.2470.047+20%0.146-9.9%

The table shows that the gains of LiveMem overall come from two parts: one is the gain brought by memory state, and the other is the gain brought by post-training.

The gain brought by Memory State is in line with our expectations, but the gain brought by post-training may be somewhat confusing. We share an explanation starting from information leakage: for wiki-style QA, although we performed document deduplication, some knowledge still exists across documents, and this knowledge is memorized by the model during training; for LME, its synthesis process itself may leak evidence information into other sessions, and post-training strengthens the model’s ability to guess the answer from such information. On LoCoMo, we can see relatively clean attribution: the change brought by post-training is significantly smaller than the performance improvement after introducing memory state. After further verification, we will put this explanation in the appendix as lessons learned from data engineering pitfalls.

Baseline Improvements—RAG & δ\delta-Mem

We agree with your suggestion that, on wiki-style multi-hop tasks, retrieving only one document with RAG is indeed unreasonable. We investigated the number of supporting evidence documents in 2Wiki, HotpotQA, and MuSiQue, and confirmed that top-k=3 is an appropriate value. In addition, we switched to Qwen3-Embedding-4B, a stronger embedding model. We evaluated the RAG baseline under this more reasonable setting than before and present the evaluation results.

oldnew
2Wiki0.1170.639
HotpotQA0.3990.724
MuSiQue0.1170.318

The improved RAG scores are overall more reasonable.

In paper we used the trunc. setting for δ\delta-mem, while we agree with you suggestion that δ\delta-mem should be aligned with LiveMem here. We have added the aligned evaluation results.

δ\delta-memδ\delta-mem (trunc.)diff
2Wiki0.8010.824-2.3%
2Wiki-MQ0.2460.232+1.4%
HotpotQA0.8520.853-0.1%
Hotpot-MQ0.2810.226+5.5%
MuSiQue0.4960.483+1.3%
MuSiQue-MQ0.1480.151-0.3%
LoCoMo0.2900.256+3.4%
LongMemEval-s0.2580.230+2.8%
FactCons0.1890.171+1.8%
Banking0.8400.770+7.0%
Clinc0.8800.890-1.0%
NLU0.7900.770+2.0%
Trec-C0.7000.740-4.0%
Trec-F0.3100.510-20.0%
Redial0.000*0.146-14.6%
infBenchQA0.1680.114+5.4%
EventQA0.4250.433-0.8%
NarrQA0.3490.326+2.3%

*: δ\delta-mem directly outputs eos for all Redial samples, we probe the eos token's probability, which is >0.99 for all samples

The results show that δ\delta-mem improves on some tasks and drops on others, with little overall change (overall -0.57%). In the final version, these results will replace the δ\delta-mem scores in the main table.

Other Issues

Why does LiveMem in Table 1 perform worse than Qwen3-4B on the Wiki QA datasets 2Wiki, HotpotQA, and MuSiQue?

This is a very good question! We state the conclusion directly: this is the performance loss caused by YaRN. Qwen3-4B is a native 32k-length model and needs YaRN to be extended to 256k-length, and YaRN usually brings performance loss to the model. LiveMem is trained under YaRN, whereas the Qwen3-4B shown in Table 1 is the capability of Qwen3-4B under its native 32k without YaRN enabled. To control this configuration to be the same (i.e., comparing Qwen3-4B with YaRN enabled against LiveMem trained under YaRN), one should actually compare the Qwen3-4B in Table 6 with the LiveMem in Table 1, where the performance of Qwen3-4B on Wiki QA when YaRN is enabled and the length is extended to 256k is shown. We directly paste the results here for reference:

Qwen3-4B (256k YaRN)LiveMem-RL
2Wiki0.6830.784
HotpotQA0.6320.772
MuSiQue0.4000.383
Reviewer_MKgeOpenReview ↗

Thanks to author

Thank you for your detailed response. My concerns have been addressed.

I will update my score to 3 with 4 points of confidence. I think this is a good job and deserves to be accepted as a poster.

Reviewer_5n7EOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

LiveMem augments a frozen with a parallel Gated DeltaNet-2 branch at every decoder layer, where the recurrent memory branch maintains a living memory state load-bearing on a retrained attention LLM across the lifecycle referred as state continuity. This paper introduces memory-oriented post training, where SFT and GRPO under context eviction train a fixed-size recurrent state to preserve useful history beyond the attention window. Evaluation across conversational memory, multi-hop QA, and test-time learning demonstrates the effectiveness of the proposed method.

Critical strengths
  • Proposed architecture is coherent with the motivation: Combining bounded attention with recurrent memory preserves precise recent-context access while compressing older information. Training with eviction directly encourages reliance on memory.
  • Broad empirical coverage: Multiple task families, SFT/RL comparisons, and curriculum ablations help characterize the approach.
Critical weaknesses
  • Much of the improvement is not clearly attributable to persistent history. In Table 2, LiveMem-RL improves LongMemEval only from 29.4% to 30.2% when history is retained; substantial performance remains with zero historical state. This suggests that post-training or added capacity explains part of the headline gains. Add matched post-training controls, state-reset/swapping interventions, and confidence intervals.
  • Baseline restrictions weaken the headline comparison. RAG retrieves only one document for datasets with fewer than 20 documents, which can exclude necessary multi-hop evidence. Omitting RAG from multi-question evaluation is also unjustified. Also, how is the result can be changed if we use a larger embedding retrieval model? 0.6B embedding model is too small, leading to unfair comparison. Meanwhile, δ-Mem discards overflow before processing, preventing it from encoding the same historical evidence. Compare identical input streams, tune retrieval budgets, and compute aggregate scores over shared tasks.
  • Long-running persistence remains insufficiently demonstrated. Evaluation stops at 256K tokens, fully evicted LongMemEval accuracy is only 16.5%, and bounded-context needle recall matches the backbone. Moreover, §A.3 releases state after each request. Demonstrate state handoff across requests and sustained interaction with corrections, interference, and repeated turnovers.
  • Resource efficiency is undercharacterized. The reported state geometry implies approximately 72 MiB per request, excluding convolution state and KV cache. Report added parameters, training cost, throughput, and latency, alongside comparisons at matched total memory budgets. Fixed state size alone does not establish practical efficiency.
Questions for authors

See weaknesses

2 comments

Overall Response

Thank you for your review comments and for your recognition of our work. In the rebuttal, we mainly focus on strengthening the attribution that “memory state can carry historical memory”, we apologize for not extending to other complex memory functions for now, and we will narrow the corresponding statements in the paper. Your recognition will encourage us to explore these directions in future work. In the rebuttal, we will address your concerns regarding the attribution of performance gains and the fairness of baseline comparisons, and we will show you the resource efficiency of LiveMem.

Strengthening the Attribution of Performance Gains

This section mainly decomposes the performance gains of LiveMem from two aspects: 1. the gains brought by the architecture design, i.e., the gain from memory state itself; 2. the gains brought by post-training. We select representative recall-style QA tasks: LongMemEval, LoCoMo, 2Wiki-MQ, HotpotQA-MQ, and MuSiQue-MQ, and filter samples on which the support evidence lies entirely outside the context window, and compute their scores. Meanwhile, we rerun generation on LiveMem after clearing the memory state, and compute the scores on these samples. This comparisondemonstrates the gains brought by the memory state. In addition, we compare the QA scores of LiveMem without memory state and the Qwen3-4B backbone under the same conditions, in order to isolate the gains brought by post-training.

LiveMem w. Memory StateLiveMem w.o. Memory Statew./w.o state diffQwen3-4BLiveMem w.o. Memory State vs Qwen3-4B diff
LongMemEval0.1650.121+4.4%0.031+9.0%
LoCoMo0.1460.081+6.5%0.059+2.2%
2Wiki-MQ0.4060.279+12.7%0.095+18.4%
HotpotQA-MQ0.2410.229+1.2%0.082+14.7%
MuSiQue-MQ0.2470.047+20%0.146-9.9%

The table shows that the gains of LiveMem overall come from two parts: one is the gain brought by memory state, and the other is the gain brought by post-training.

The gain brought by Memory State is in line with our expectations, but the gain brought by post-training may be somewhat confusing. We share an explanation starting from information leakage: for wiki-style QA, although we performed train-test-split by document, some knowledge still exists across documents, and this knowledge is memorized by the model during training; for LME, its synthesis process itself may leak evidence information into other sessions, and post-training strengthens the model’s ability to guess the answer from such information. On LoCoMo, we can see relatively clean attribution: the change brought by post-training is significantly smaller than the performance improvement after introducing memory state. After further verification, we will put this explanation in the appendix as lessons learned from data engineering pitfalls.

Baseline Improvements—RAG & δ\delta-Mem

Thanks for the suggestion that led us to re-examine the RAG evaluation pipeline. We agree that on wiki-style tasks, retrieving only one document for multi-hop tasks is unreasonable. We investigated the number of supporting evidence documents in 2Wiki, HotpotQA, and MuSiQue, and confirmed that top-k=3 is an appropriate value. In addition, we switched to Qwen3-Embedding-4B, a stronger embedding model. The evaluation results under the new pipeline are shown below.

oldnew
2Wiki0.1170.639
HotpotQA0.3990.724
MuSiQue0.1170.318

This makes the overall RAG baseline scores more reasonable.

We also thanks for your suggestion that pointed out the evaluation of δ\delta-mem should be aligned with the LiveMem setting rather than using trunc. We agree with this and have added the evaluation results of δ\delta-mem under the setting aligned with LiveMem.

δ\delta-memδ\delta-mem (trunc.)diff
2Wiki0.8010.824-2.3%
2Wiki-MQ0.2460.232+1.4%
HotpotQA0.8520.853-0.1%
Hotpot-MQ0.2810.226+5.5%
MuSiQue0.4960.483+1.3%
MuSiQue-MQ0.1480.151-0.3%
LoCoMo0.2900.256+3.4%
LongMemEval-s0.2580.230+2.8%
FactCons0.1890.171+1.8%
Banking0.8400.770+7.0%
Clinc0.8800.890-1.0%
NLU0.7900.770+2.0%
Trec-C0.7000.740-4.0%
Trec-F0.3100.510-20.0%
Redial0.000*0.146-14.6%
infBenchQA0.1680.114+5.4%
EventQA0.4250.433-0.8%
NarrQA0.3490.326+2.3%

*: δ\delta-mem directly outputs eos for all Redial samples, we probe the eos token's probability, which is >0.99 for all samples

The results show that δ\delta-mem improves on some tasks and drops on others, with little overall change (overall -0.57%). In the final version, these results will replace the δ\delta-mem scores in the main table.

Resource Efficiency

The total capacity of the LiveMem state matrix is 72 MiB. From SFT to RL, the total training time for a single model is about 2 days on 32×A800. LiveMem adds 2.3B parameters because the memory module is introduced in full-attention layers, and Qwen3-4B is a fully full-attention model. If the backbone is changed to Qwen3.5-4B, the number of additional parameters can be reduced to ~600M.

We tested the throughput comparison between Qwen3-4B and LiveMem on an RTX-5880 and report the best-of-3 results.

token lengthLiveMem (prefill/decode, tok/s)Qwen3-4B (prefill/decode, tok/s)
8k5153.13/15.4610466.29/62.46
32k4057.74/18.436735.97/63.22
128k3525.50/20.922856.67/32.15

The table shows that LiveMem’s inference performance is lower than Qwen3-4B, but it can surpass Qwen3-4B on long-sequence prefill.

It should be noted that Qwen3-4B has been optimized by the community over a long period, whereas LiveMem’s vLLM inference code is only experimental and has not been further optimized. We calculate that LiveMem’s theoretical efficiency can reach 0.7×–0.95× that of Qwen3-4B.

Reviewer_gz1UOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LiveMem augments a frozen transformer with a parallel Gated DeltaNet-2 branch that maintains recurrent state as older KV-cache entries are evicted. Memory-oriented supervised fine-tuning and reinforcement learning train the model to use information beyond its active attention window. Evaluation spans conversational memory, multi-hop QA, long-document QA, and test-time learning, with additional analyses of evidence eviction and retention distance.

LiveMem only feeds the recent chunks of context and the system instructions as the input context for the attention layer, while every input token before their kv being evicted also updates the GDN2 module to propagate the old info. The authors injected LiveMem on every layer of the frozen Qwen3-4B and trained the GDN2 module with SFT and RL on tasks where the evidence is pushed outside the recent context window. Results show overall improvement on average of 18 benchmarks & variants. The analysis highlights that the proposed method does capture the long context info, as seen by the performance on some instances where the evidence is evicted. The closest prior work is AHN (Fang et al., ICML 2026) which compresses out-of-context tokens into a GDN-style recurrent memory on a frozen Qwen. This paper differs in that LiveMem explores the SFT/RL recipe for the proposed module and they evaluated more on agent memory benchmarks. Given the similarities on the methodology, however, the authors should cite AHN. The research question is directionally relevant and important due to the increasingly higher demand for longer agentic tasks.

That being said, I am not convinced that the improvements across the benchmarks come from the memory state itself providing key information. Section 4.3 studies an interesting variant “Trunc.” where the evicted history does not modify the memory state (it starts from zero state on the truncated suffix), which serves as a baseline of post-trained but not memory-activated LiveMem. Since Qwen is frozen, the only difference from the original LLM is the trained GDN2 branch processing the visible context. Effectively Trunc serves as a post-training baseline with an adapter in my opinion. In this context, Table 2 shows Trunc already achieving most of the gain over the Qwen baseline (Table 1), hinting that the claimed gain might rather come from post-training mainly thanks to in-distribution samples in the post-training data (2Wiki, multi-query tasks, MuSiQue, etc). In general, Trunc is an important baseline and should be reported in Table 1. The eviction-accuracy analyses (Fig 2) have the same gap: the authors should report Trunc results in Fig 2. In addition, the performance on NIAH under the 32K window are identical to the vanilla baseline, making it unclear what the state actually stores.

Critical strengths

-Combining bounded attention with recurrent state provides a concrete implementation of memory that survives context turnover. The implementation carefully aligns training-time attention visibility with inference-time KV eviction, an important detail for interpreting the retention results.

-The evidence-eviction analysis directly examines whether information remains useful after leaving attention. On the fully evicted LongMemEval-S subset, LiveMem-RL achieves 16.5% accuracy versus 3.1% for the base model. Together with the retention-distance analysis, this supports a useful, although lossy, memory mechanism.

Critical weaknesses

-The source of the overall gains remains insufficiently isolated. Table 2 usefully compares full-history processing with a truncated-input, zero-initial-state control. However, much of the improvement over the base model remains in the truncated condition: on LongMemEval-S, LiveMem-RL scores 29.4% without processing the evicted history and 30.2% with it. This does not negate the eviction results, but makes it important to distinguish historical retention from improvements due to post-training. The same trained-model control on the fully evicted subset, together with a matched comparison of training with and without context turnover, would clarify the contribution.

-The demonstrated lifecycle is narrower than the long-running assistant framing. Appendix A.3 releases the recurrent state when generation finishes, establishing continuity within a request rather than across successive assistant requests. Figure 3 tests increasing evidence distance, but does not separately characterize capacity under accumulating relevant facts or conflicting updates. These limits leave the sustained-memory claim less established than the within-stream retention result.

The gains seems to come from in-distribution post-training of the side branch, while the state itself adding only up to ~4 points (Table 2). No comparison against similar line of work like AHNs or IndexMem. The role of RL seems unclear, adding only ~1.4 points over SFT (0.505 -> 0.519) in a single sampled run.

Questions for authors

-On Figure 2’s fully evicted subset, how does LiveMem-RL perform when given only the retained suffix with zero initial state, as in Table 2? Is there a matched training comparison that isolates context turnover?

-Appendix A.3 releases the recurrent state after generation. Does the implementation support carrying state across successive requests, and has this been evaluated? Please distinguish implemented persistence from a proposed extension.

-What explains the LoCoMo SFT difference between Table 1 (0.295) and Table 3’s “Full side” configuration (0.234)?

I-s the claimed practical complementarity with retrieval experimentally supported? A combined evaluation would be useful future work; without one, the claim should be framed as a plausible application.

2 comments

Overall Response

Thank you for your review comments. We appreciate your recognition of the potential of our work. In the rebuttal, we focus on strengthening the attribution that “memory state can carry historical memory”, and we apologize for not having yet included experiments on complex memory functions or complex memory scenarios. Your recognition will motivate us to further explore these directions in future work. In the rebuttal, we will address your concerns regarding “complete sources of gains and contribution attribution” and “comparison with related work”.

Strengthening the Attribution of Gains and Contributions

We decompose the performance gains of LiveMem into two aspects: 1. the gains brought by the architecture design, i.e., the gain from memory state; and 2. the gains brought by post-training.

We select representative recall-style QA tasks: LongMemEval, LoCoMo, 2Wiki-MQ, HotpotQA-MQ, and MuSiQue-MQ. On these tasks, we filter samples whose supporting evidence is entirely outside the context window, and report their scores. At the same time, on these samples, we rerun generation with LiveMem after clearing the memory state and report the scores. This comparison mainly demonstrates the gain brought by memory state. In addition, we compare the QA scores of LiveMem without memory state and the Qwen3-4B backbone under the same conditions, in order to distinguish the gain brought by post-training.

LiveMem w. Memory StateLiveMem w.o. Memory Statew./w.o state diffQwen3-4BLiveMem w.o. Memory State vs Qwen3-4B diff
LongMemEval0.1650.121+4.4%0.031+9.0%
LoCoMo0.1460.081+6.5%0.059+2.2%
2Wiki-MQ0.4060.279+12.7%0.095+18.4%
HotpotQA-MQ0.2410.229+1.2%0.082+14.7%
MuSiQue-MQ0.2470.047+20%0.146-9.9%

The table shows that the gains of LiveMem overall come from two parts: one is the gain brought by memory state, and the other is the gain brought by post-training.

The gain brought by Memory State is in line with our expectations, but the gain brought by post-training may be somewhat confusing. We share an explanation starting from information leakage: for wiki-style QA, although we performed train-test-split by document, some knowledge still exists across documents, and this knowledge is memorized by the model during training; for LME, its synthesis process itself may leak evidence information into other sessions, and post-training strengthens the model’s ability to guess the answer from such information. On LoCoMo, we can see relatively clean attribution: the change brought by post-training is significantly smaller than the performance improvement after introducing memory state. After further verification, we will put this explanation in the appendix as lessons learned from data engineering pitfalls.

In summary, we believe that memory state indeed has the ability to carry memory. Combined with the NIAH results, we believe that this ability is mainly conceptual-level memory (e.g., names, events, facts, experiences), rather than the artificially constructed “needle” in needle-in-a-haystack. In the paper, NIAH is mainly used to argue that memory state does not provide needle-in-a-haystack-style precise token reconstruction. We will additionally clarify this discussion in the NIAH section.

On Baseline Discussion—AHN and IndexMem

We agree with your suggestion and will include AHN in the comparison and in the related-work discussion. We first clarify the difference between AHN and LiveMem: AHN reuses the KV projections of Attention, whereas in LiveMem the memory module is parallel to Attention. Since the official AHN implementation does not provide code that can be deployed in inference engines such as vLLM or SGLang, we use native torch autoregressive evaluation. This also means that some experimental results cannot be completed during the rebuttal period.

Regarding IndexMem, our discussion in the related work is more about comparing the similarity between the two methods at the methodological level. The IndexMem paper is overall oriented toward KV eviction. Using it as a baseline in the memory direction of this paper across domains, we are concerned that it would be difficult to control the relevant variables.

We used AHN-Qwen2.5-3B-Instruct-GDN as the comparison object and evaluated it under our evaluation setting. The results are shown in the table.

AHN
2Wiki0.642
2Wiki-MQ0.199
HotpotQA0.708
HotpotQA-MQ0.234
MuSiQue0.285
MuSiQue-MQ0.075
LoCoMo0.277
LongMemEval-s0.202
FactCons0.140
Banking0.190
Clinc0.370
NLU0.250
Trec-C0.460
Trec-F0.110
Redial-
infBenchQA-
EventQA-
NarrQA-

-: evaluations that could not be completed within the rebuttal period; we will present them in the final version.

Compared with the proposed method, AHN underperforms LiveMem overall.

Other Issues

  • Inconsistency in LoCoMo results across tables: This is a transcription error. The score in Table 3 here should be consistent with Table 1, both being 0.295. We will correct this error.
  • Complementarity with retrieval: Some follow-up work has studied this direction and confirmed that models carrying memory state have stronger performance than stateless models [1]. However, this is indeed beyond the experimental support of this paper, and we will tighten the relevant discussion.

[1] Fanyu Zhao, Ruike Cao, Liang Dong, Fugen Yao, Jian Xu, Guanjun Jiang, Han Zhang, Yifei Zhao, and Yinsheng Li. RPMem: Learning Long-Term Recurrent Parametric Memory Across Sessions for LLM Agents. arXiv preprint arXiv:2609.23466, 2026.

Comments on the paper

1 comment

We thank all reviewers for their reviews. We have received many valuable suggestions from them that can help improve this work.

Overall Response

The core question studied in this paper is: when supporting evidence leaves the active context, can the model retain memory or acquire capabilities through its internal state? As an exploration of intrinsic memory, we tend to devote the main space to demonstrating the effectiveness of memory state, as a result, we apologize for not having address the further experiments expected by some reviewers, and we will clarify the boundary in the paper.

In the rebuttal, we mainly expand on two parts: “attribution decomposing and strengthening” and “baseline refinement.” While improving the baseline comparisons, we dive into the reasons for LiveMem’s performance gains, and show that memory state indeed provides performance gains.

Attribution Decomposing and Strengthening

Reviewers share the view that the paper’s evidence for attributing the performance gains to memory state is insufficient. We agree with that this part of the attribution should be decomposed and strengthened. Following the reviewers’ suggestions, we collect samples in LoCoMo, LongMemEval, 2Wiki-MQ, HotpotQA-MQ, and MuSiQue-MQ whose supporting evidence is outside the context window, and on these samples we clear the memory state and rerun an evaluation.

LiveMem w. Memory StateLiveMem w.o. Memory Statew./w.o state diffQwen3-4Bw.o. Memory State vs Qwen3-4B diff
LongMemEval0.1650.121+4.4%0.031+9.0%
LoCoMo0.1460.081+6.5%0.059+2.2%
2Wiki-MQ0.4060.279+12.7%0.095+18.4%
HotpotQA-MQ0.2410.229+1.2%0.082+14.7%
MuSiQue-MQ0.2470.047+20%0.146-9.9%

Through the table, we can decompose LiveMem’s performance gains into two clear contributions: the gain brought by memory state carrying historical memory, and the gain brought by post-training itself.

Baseline Refinement

All reviewers mentioned issues in the baseline comparisons, mainly including the δ\delta-mem setting, the RAG setting, and the introduction of the new baseline AHN. The supplementary evaluation table is as follows.

δ\delta-memδ\delta-mem (trunc.)δ\delta-mem diffAHN
2Wiki0.8010.824-2.3%0.642
2Wiki-MQ0.2460.232+1.4%0.199
HotpotQA0.8520.853-0.1%0.708
HotpotQA-MQ0.2810.226+5.5%0.234
MuSiQue0.4960.483+1.3%0.285
MuSiQue-MQ0.1480.151-0.3%0.075
LoCoMo0.2900.256+3.4%0.277
LongMemEval-s0.2580.230+2.8%0.202
FactCons0.1890.171+1.8%0.140
Banking0.8400.770+7.0%0.190
Clinc0.8800.890-1.0%0.370
NLU0.7900.770+2.0%0.250
Trec-C0.7000.740-4.0%0.460
Trec-F0.3100.510-20.0%0.110
Redial0.000*0.146-14.6%-
infBenchQA0.1680.114+5.4%-
EventQA0.4250.433-0.8%-
NarrQA0.3490.326+2.3%-

We agree with reviewers 5n7E and MKge's suggestion that the evaluation of δ\delta-mem should be aligned with the LiveMem setting rather than using trunc. We have supplemented the evaluation results of δ\delta-mem under the setting aligned with LiveMem, and these results will replace the δ\delta-mem data in Table 1.

Reviewer gz1U suggested that should conduct contribution comparison and performance comparison with AHN. We agree with this and have supplemented the relevant experiments. We first clarify the difference between AHN and LiveMem: AHN reuses the KV projections of Attention, whereas in LiveMem the memory module is parallel to Attention. Since the official AHN implementation does not provide code that can be deployed in inference engines such as vLLM or SGLang, we use native torch autoregressive evaluation. This also means that some experimental results cannot be completed during the rebuttal period. We used AHN-Qwen2.5-3B-Instruct-GDN as the comparison object and evaluated it under our evaluation setting. The results can be seen in the table.

Reviewers 5n7E and MKge mentioned that using top-1 for RAG on Wiki QA is an unfair comparison. We find this problem indeed exists. We investigated the number of supporting evidence documents in 2Wiki, HotpotQA, and MuSiQue, and confirmed that top-k=3 is an appropriate value. At the same time, following the reviewers’ suggestion, we switched the embedding model to the stronger Qwen3-Embedding-4B. We re-evaluated RAG, and the results are as follows.

oldnew
2Wiki0.1170.639
HotpotQA0.3990.724
MuSiQue0.1170.318

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.