From Passive Retrieval to Active Memory Navigation: Learning to Use Memory as a Structured Action Space

Long-term user memory is essential for personalized conversational agents, but exposing memory only as pre-selected context limits the agent’s control over when and how to access user history. We introduce NapMem, a framework for active memory navigation that treats long-term user memory as a structured action space. NapMem organizes user histories into a multi-granularity pyramid of raw conversations, typed memory records, topic tracks, and user profiles, with provenance links connecting abstractions to supporting evidence. These levels are exposed through memory tools, and reinforcement learning over memory-tool trajectories trains the agent to decide whether to access memory, which level to consult, and when to refine or stop navigation based on intermediate evidence. On PersonaMem-v2, LongMemEval, and LoCoMo, a 9B NapMem agent achieves the highest average score among the evaluated systems, outperforming a 397B variant without memory-navigation training. Besides, the learned policy maintains performance on the evaluated non-memory reasoning and tool-use tasks, while reducing the memory-call rate. Ablation and trajectory analyses support the value of coupling structured memory with a learned, query-conditioned navigation policy