CoalMem: Evidence Coalition Selection for Multi-Party Conversational Memory

Answering queries over long-term multi-party conversational memory often requires combining evidence distributed across participants and discussion phases. Existing memory retrieval approaches largely optimize individual message relevance, yet the resulting context may still omit critical evidence needed to answer the query. Meanwhile, generic diversity does not guarantee that different messages contribute complementary query-relevant information. We therefore formulate multi-party conversational memory retrieval as evidence coalition selection and propose CoalMem, a training-free framework for selecting a compact set of messages with high joint evidence value. It constructs an evidence coalition graph over seed retrieved messages, where query–evidence edges capture relevance and evidence–evidence edges characterize both non-redundancy and query-conditioned complementarity. It then performs query-anchored coalition optimization to jointly select a compact set of messages that are individually relevant and collectively supportive of the query, without additional LLM calls during selection. Experiments on GroupMemBench across six tasks show that our method consistently outperforms retrieval-based, memory-based, and alternative evidence-selection methods.