DyMem: Query-Adaptive Relational Memory for Long-Term Conversation

Long-term conversational agents must reason over information accumulated across many interactions, where answering a question often requires connecting distributed memories through temporal, topical, or entity-level relations. Existing flat retrieval methods rank memories independently and may overlook evidence that is weakly aligned with the query but connected to relevant memories through meaningful relations. Graph-based memory systems make such dependencies explicit, yet a persistent graph defines only a space of possible connections, while fixed or query-agnostic traversal can follow irrelevant relational paths and introduce unnecessary context. We propose DyMem, a query-adaptive relational memory framework that conditions retrieval not only on memory-node relevance, but also on the semantic meaning of the relations connecting memories. DyMem constructs a persistent heterogeneous memory graph with explicit relation descriptions and dynamically activates query-relevant relations to form a query-specific evidence subgraph. The resulting candidates are further pruned under a context budget and mapped back to the original dialogue turns for source-grounded answer generation. Experiments on LoCoMo and LongMemEval across multiple language model backbones show that DyMem consistently improves long-term conversational question answering over representative memory baselines, with particularly strong gains on multi-hop and temporal queries while maintaining a compact retrieval context. These results highlight the importance of treating relations themselves as query-dependent retrieval signals in long-term conversational memory.