- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a method for multi-party dialogue analysis by improving the general-purpose LLM memory systems with speaker-centered dual-track memory. Experiments on serveral benchmarks have been conducted to evaluate the effectiveness of the proposed method.
Critical strengths
The characteristics of multi-party conversations are consided to deisgn the proposed method and experimental results demonstrated its superiority over general-purpose LLM memoy methods.
Critical weaknesses
- This paper focuses more on the conversational memory of LLMs rather than the task of multi-partiy dialogue understanding in Section 1 and 2. There are very few introductions on the development and recent progresses of multi-party dialogue undersanding in Section 1, and Section 2 only mentions the recent benchmarks for multi-party memory.
- The baseline methods used in the experiments are also general-purpose LLM long-term memory/RAG methods. The experimental results could be more convincing if some methods designed specifically for the task of multi-party conversation could be included.
Questions for authors
- In Table 4, the performance of the proposed method is not competitive on the LoCoMo dataset. Some discussions are expected in the main text of the paper.
- The authors usually put the citations of several methods together. It is difficult for readers to discriminate the reference for each method.
Multi-Party Memory Context, Specialist Baselines, and LoCoMo Analysis
Thank you for the suggestions on multi-party research context, specialist baselines, LoCoMo, and citation clarity. We have revised all four areas; references below use the updated manuscript.
Weakness 1. Multi-party dialogue understanding.
SpeakerMem-R1 is designed for long-term multi-party conversational memory. Its central problem is preserving who said what, whom a statement concerns, and how individual and shared states evolve across interleaved conversations. The source/owner distinction and person/group views directly serve these requirements. Accordingly, GroupMemBench, SocialMemBench, and EverMemBench form our main evaluation.
Sections 1-2 now connect prior multi-party understanding to persistent conversational memory: DialogueGCN for speaker dependencies, Molweni for discourse structure, DialogRE for cross-utterance relations, QuISG/LIMN for dialogue QA, and FANToM for knowledge/belief reasoning. We also cover EMMA's partner-specific memory, the cross-session memory model accompanying M3C, and DialSim/LongDialQA. This positions our contribution as preserving attribution and evolving person/group states through long-term writing and retrieval, building on multi-party understanding.
Weakness 2. Multi-party-specific baselines.
Table 2 now includes Subject-Mem and SMG, which index facts by subject and represent social norms/relations, respectively. All entries follow the same Subject-Mem evaluation configuration; the panels identify original-report and our evaluation sources.
Table 25 further controls participant-wise organization and the input-token budget using raw messages in place of derived states. Full no-ASK gains 9.12/2.95/7.50 percentage points on SocialMem/GroupMem/EverMem, supporting the structural contribution.
Question 1. LoCoMo performance.
LoCoMo is a two-person boundary test of a system developed for multi-party memory. It examines how well the same architecture works when the conversation has only two participants. Our main evaluation targets the attribution and state-reconstruction challenges of multi-party dialogue; LoCoMo extends the evaluation to a simpler participant structure. Section 5.6 and Appendix H now explain this role and discuss the results.
The system remains competitive with representative advanced memory methods in this setting. All-question accuracy is 70.85% (1,407/1,986). For the non-adversarial comparison in Table 6, our accuracy is 67.34%, above the reported Mem0 (66.88%) and Zep (65.99%) results; MemOS (75.80%) and LightRAG (79.87%) perform better overall.
The category-level results further highlight its strengths: single-hop accuracy is 77.88%, above Mem0's 67.13% and Zep's 61.70%; temporal accuracy is 70.72%, second-highest among the six systems in Table 6, ahead of LightRAG's 60.75% and Mem0's 55.51%. Thus, the multi-party-oriented design retains strong factual recall and temporal reasoning in two-person conversations.
The remaining gap is concentrated in multi-hop and open-domain questions, with accuracies of 41.13% and 40.62% (Table 28). With only two participants, member/group separation and cross-person relations may offer less relative benefit, while combining facts across segments and drawing on external knowledge become more prominent needs. This motivates stronger cross-evidence reasoning and external-knowledge integration while preserving the architecture's multi-party strengths.
Question 2. Method-citation correspondence.
We revised Sections 1-2 so each named method is followed immediately by its own citation. Subject-Mem and SMG are attributed to SocialMemBench. We also checked the new references against their original papers and synchronized both language versions.
We hope the revised context, specialist comparisons, and LoCoMo analysis address your concerns. We would appreciate your reassessment and are happy to clarify any remaining issue.