LiveMem: Maintaining LLM Long-running Memory by Living Memory State

Long-running assistants and agents stand or fall by memory; yet prevailing memory systems keep memory outside the model: they store the interaction history as textual records and retrieve them back into the context on demand. This paradigm preserves access to history but leaves inference itself stateless; and the preserved texts are also limited to the information that can be verbalized. This paper studies an alternative approach to long-running memory: a memory state that lives inside the computation, which is updated online, is persistent across context turnovers, and continues to influence the model's behavior after its source evidence has left the context. We introduce LiveMem, which augments a pretrained full-attention LLM with a parallel recurrent memory branch: the branch maintains a living memory state across the lifecycle, while the original attention path provides exact access to a bounded working context. LiveMem updates and preserves this state as the active context turns over, and uses memory-oriented post-training to teach the model how to use the state. Experiments show that LiveMem achieves leading overall performance compared with RAG and other intrinsic memory methods. On LongMemEval, LiveMem answers questions from its memory state even when the supporting evidence has been fully removed from the context, and its test-time learning results indicate that the state carries patterns distilled from past experience rather than only retrievable records. Therefore, LiveMem establishes a performant implementation of state-based long-running memory as a practical complement to existing retrieval-based memory systems.