HandleMem: Source-Linked Complementary Handles for Adaptive Evidence Acquisition

Long-horizon agents often compress past interactions into compact memories. These representations play two roles: they guide the retrieval of relevant prior episodes and preserve evidence for later question answering. The difficulty is that compression must decide what to retain before the future use of the memory is known. As a result, it may discard cues needed to locate a relevant interaction, creating an access gap; it may also discard details needed to support an answer, creating an evidence gap. Since richer compression alone cannot eliminate these gaps, we argue that compact memory should instead provide access to retained evidence rather than replace it. We introduce , a long-term memory framework built around this separation. It constructs source-linked complementary event- and detail-level handles that expose different retrieval cues for the same source segment, improving access coverage while preserving links to the original interaction for recovering omitted evidence. At query time, adaptive source resolution performs multi-round retrieval driven by the current information need and newly retrieved cues, using each observation to refine what to search for next, before resolving selected memories back to their sources for final answering. On LoCoMo, achieves 57.40 F1 and 81.69% LLM-as-a-Judge accuracy. On LongMemEval-S, it achieves 80.20% LLM-as-a-Judge accuracy, obtaining the highest overall scores among the compared methods on both benchmarks. Our code is available in an anonymous repository at https://anonymous.4open.science/r/HandleMem-A11Dthis link.