One Cue, the Right Episode: Evidence Recovery for Dialogue Memory

With ever-longer interaction histories, LLM-based conversational systems face a central challenge: how can they retrieve the most discriminative evidence for a response within a limited reader-context budget? The human brain tackles this via hippocampal separation, completion from partial cues, and selective readout. Inspired by this tripartite organization, we propose Sparse Event Addressing Memory (SEA-Mem). SEA-Mem first encodes each turn into immutable traces, source atoms, and sparse bindings. When a query arrives, a planner groups its information needs, and then a simple three-step mechanism resolves the memory access: DG-Address pinpoints the episode via sparse addressing, CA3-Complete recovers evidence through binding-following, and CA1-Select selects a budgeted set of turns that jointly supports the query. Together, they let associations guide access, while the selected original turns—not the associations—remain the evidence presented to the answerer. With lightweight association construction and compact readout, SEA-Mem achieves strong answer quality on LongMemEval and LoCoMo under a fixed source budget. On a matched LongMemEval subset, accuracy is largely preserved from Oracle evidence to full histories. Our experiments further show that SEA-Mem is more resistant to semantically confusable distractors.