- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the problem of post-retrieval evidence selection in multi-party conversational memory. The authors observe that individually relevant messages do not necessarily form a collectively sufficient evidence set, and that generic diversity can only capture whether evidence items are different, but cannot capture whether the differences among them contribute complementary information. The authors formalize evidence selection as a query-anchored coalition optimization problem and propose CoALMem, a framework that constructs an evidence coalition graph containing query-evidence edges and evidence-evidence edges (encoding non-redundancy and query-conditioned complementarity), and then uses replicator dynamics to jointly estimate coalition member weights under a fixed evidence budget. It achieves sufficiently competitive performance on GroupMemBench and EverMemBench.
Critical strengths
- The insight found in this paper is somewhat innovative, and the overall presentation is relatively clear.
- CoalMem achieves sufficiently competitive performance on groupMemBench and EverMemBench.
Critical weaknesses
1.In multi-party conversations, information is highly dependent on context. The authors highlight "no LLM calls during selection" as a core point and rely solely on lexical BM25 to compute evidence non-redundancy and complementarity. However, BM25 struggles to capture deep semantic associations (e.g., coreference, synonym substitution, implicit logical complementarity), where LLMs have a natural advantage. The authors should therefore demonstrate that their algorithm can indeed outperform or match methods that directly introduce LLMs during the selection phase in terms of evidence selection quality (e.g., via specifically designed prompts for semantic-level redundancy elimination and complementary information filtering over the candidate pool).
2.In Table 2, using only Qwen3-Reranker-4B as the LLM-based reranker baseline is unconvincing. The authors need to include other strong cross-encoders, such as BGE-Reranker-v2-m3, or larger-parameter, Listwise strategy-based LLM Rerankers (e.g., RankGPT).
3.Lack of fine-grained efficiency analysis. Beyond the aggregated online latency reported in Appendix C.2, the authors need to provide a complete end-to-end time consumption breakdown, explicitly listing and comparing the specific time costs or token consumption across the three distinct stages: Retrieval, Selection (evidence selection/graph optimization computation), and Generation (final answer generation).
Questions for authors
Please see the weakness.
Thank you for the authors' response. However, I remain concerned about the limited methodological novelty of this work.
First, recent Agent Memory studies, such as MemGAS [1] and Mem0 [2], have already explored long-term conversational memory organization, association, retrieval, and selection. In comparison, CoalMem mainly combines established relevance scoring, non-redundancy modeling, and graph optimization techniques. Its proposed Evidence Complementarity is also based on BM25 query-term coverage, offering limited methodological novelty.
Second, the evaluation lacks direct comparisons with representative memory systems such as Mem0 and MemGAS, making it difficult to establish its advantages over existing memory approaches.
More importantly, I believe CoalMem is essentially an evidence reranking and selection method within a conventional RAG pipeline, with multi-party conversational memory serving primarily as a new application setting. The core algorithm does not introduce novel mechanisms for memory construction, updating, or memory-specific reasoning. Therefore, its contribution to this research field remains limited.
Although the empirical improvements are encouraging, the rebuttal primarily emphasizes the combination of existing techniques and performance gains without sufficiently addressing the fundamental novelty concerns. Therefore, I maintain my original score.
[1] From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents.
[2] Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory.