- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the problem of long-term memory retrieval in multi-user shared conversation histories, with a focus on evidence selection errors where the retrieved content is relevant but is associated with the wrong user or context. The authors propose DG-Mem, which aligns two complementary views through shared message nodes: a content graph organizes entity associations, replies, and temporal relations, while a user-context graph preserves authorship, roles, and discussion scope. Retrieval first discovers candidate evidence in the content graph, then propagates information over the user-context graph using the asker's information, and finally performs query-conditioned reranking to select the messages provided to the answering model.
Critical strengths
-
In multi-user conversations, different users may hold information about the same object that is simultaneously valid but applicable to different scopes. The paper clearly distinguishes content relevance'' from asker relevance'', rather than treating multi-user memory simply as a larger-scale text retrieval problem. This problem formulation is practically meaningful and provides a clear motivation for the proposed method.
-
The two graphs establish associations over the same set of messages, allowing the system to retrieve information provided by other participants while preserving authorship, reply relationships, and discussion context. Compared with compressing statements from multiple users into a single global conclusion, this design preserves the evidence needed to distinguish different users' states and also makes the basis of an answer easier to inspect. The shared message nodes enable direct and traceable evidence transfer between the two views.
-
The method uses entity extraction, local encoders, and graph propagation for memory construction and retrieval, without repeatedly invoking a generative LLM to extract, summarize, or rewrite memories. Under the experimental setting reported in the paper, memory construction takes \textbf{0.76 hours} and requires \textbf{345 MB} of storage. This design substantially reduces construction overhead while maintaining strong answer quality, making it attractive for shared assistants that continuously accumulate large volumes of messages.
Critical weaknesses
-
The User-Context Graph organizes messages using \texttt{channel}, \texttt{phase}, \texttt{topic}, \texttt{role}, and reply relationships. While authorship and time are typically directly observable, the sources of topic, phase, and role require more explicit clarification. These fields already provide part of the discussion-scope information, whereas recovering the correct discussion context is precisely one of the problems addressed by the paper. It is therefore unclear how much of the system's advantage remains when such structured information is unavailable. In particular, when a message has no explicit reply target but refers back to previous content using expressions such as that project'' or the previous plan,'' the paper does not clearly explain how the associations between messages are recovered.
-
The content graph and user-context graph can connect relevant messages, but shared entities, authors, or discussion scopes alone cannot distinguish whether a message proposes an idea, rejects a proposal, or confirms a final decision. For example, I suggest postponing it until Friday'' and The postponement was not approved; we will proceed on Wednesday'' may have very similar graph neighborhoods. Temporal edges that decay with the time interval between messages also do not directly indicate that a fact has been superseded. Although the paper achieves strong results on information-update tasks, it does not sufficiently explain which component performs this state distinction and under what circumstances outdated or unconfirmed messages may still be ranked ahead of the correct evidence.
-
The system uses lexical rules to identify self-reference, temporal and state cues, as well as the fields requested by a query, and uses these signals to adjust propagation and reranking weights. Field matching may also restrict the evidence entering sentence-level reranking, meaning that these rules can substantially affect which information is ultimately selected. The paper provides some weights and hyperparameters, but does not sufficiently explain how the rules are designed, how indirect expressions are handled, or whether there is a reliable fallback mechanism when detection fails. The concern is not the use of rules itself, but rather that it is currently difficult to assess how strongly the results depend on particular question formulations and field expressions.
Questions for authors
-
Please clarify the sources of \texttt{phase}, \texttt{topic}, and \texttt{role}, respectively. Are they fields directly available in the original records, information provided by the data-generation process, or attributes inferred by an additional procedure? When these fields are missing, how are the scope nodes constructed, and how does system performance change?
-
For interleaved discussions within the same channel, suppose later messages refer to previous content only through expressions such as that plan'' or let's go with what they said,'' without a \texttt{reply_target}. What mechanism does the system use to associate these messages with the correct historical messages? Are entity aliases and cross-message coreference explicitly handled?
-
In cases involving multiple modifications, rejections, and reconfirmations within a short period, which component is responsible for determining the validity status of each message? Could the authors provide the candidate rankings at different stages for such cases, illustrating how the system retrieves the final decision and its applicable scope rather than simply passing conflicting evidence to the answering model?
-
Since LoCoMo lacks enterprise-specific roles, channels, and discussion phases, which nodes and edges are retained in (G^U), and how are the scope and asker defined? Does this experiment use the full cross-graph pipeline, or does it primarily rely on the content graph?
5 comments
Response to Reviewer DXnd (2/2)
Q3: Retrieving the final decision from conflicting updates.
Evidence-status cues are used in query-conditioned candidate refinement, together with content relevance, context-aware BM25, user-context propagation, asker/scope compatibility, field matching, and sentence-level reranking. Under the default query-aware configuration, the evidence-status weight is set to zero when no temporal, state, or exact-answer query cue is detected. Field matching is disabled when no specific answer field is identified, and the remaining weights are normalized. For a state-sensitive query, a candidate without a recognized status cue receives a neutral status score.
On GroupMemBench's Update category, DG-Mem achieves 43.00% accuracy, exceeding the strongest baseline by 15.90 percentage points. To examine how conflicting updates are ranked, we traced Technology question knowledge_update_3, whose history contains an initial scope, an earlier hold decision, a counterproposal, and a revised final decision.
| Case trace | |
|---|---|
| Question | What is the current plan for convention tagging in QA? |
| Ground truth | Use convention tagging in the primary checklist only for the flagged rule family; keep broader channel-wide tagging deferred. |
| Conflicting history | Msg_1558: “hold the convention tags” (obsolete). |
| Msg_2328: “tag convention as a secondary dimension” (superseded). | |
| Msg_4055: “move convention tagging into the primary QA checklist for that flagged rule family; keep broader channel-wide tagging deferred” (gold evidence; no reply target). |
| Candidate | Content Graph | Cross-graph base | Final |
|---|---|---|---|
| Msg_856: initial scope | 2 | 4 | 4 |
| Msg_1558: hold convention tags | 1 | 2 | 3 |
| Msg_2328: use convention as a secondary tag | 3 | 3 | 2 |
| Msg_4055: final scoped decision; no reply | 4 | 1 | 1 |
The content graph ranks the earlier hold decision first and the final scoped decision fourth. Cross-graph fusion moves the final decision to first place before the reader is called. Refinement keeps it first and lowers the hold decision from second to third. The reader then returns the correct answer with its applicable scope. This case shows that retrieval prioritizes the final decision before answer generation, while retaining earlier statements as context.
Q4: LoCoMo setting.
LoCoMo does not provide enterprise-specific roles, channels, or workflow phases. We use session IDs for the channel and phase fields, set the topic to “LoCoMo conversation” and the role to “conversation participant,” and leave asking_user_id empty.
Both graph retrieval stages remain active. Content-graph candidates seed user-context propagation through shared message anchors, and the final candidate pool combines content, context-aware BM25, and user-context PPR results before sentence-level reranking. The LoCoMo configuration emphasizes content relevance and semantic reranking, while the User-Context Graph retains session and participant structure. Asker-specific seeding is inactive because no asker identity is supplied. We have clarified these settings in the revised version.
Thanks again fo your time, efforts, and constructive feedback. We have added your valuable feedback to the revised version. If you have any additional questions or suggestions, we would be happy to discuss them.
Thank you again for your time and thoughtful review. With only three days remaining in the discussion period, we would greatly appreciate your feedback on whether our rebuttal has addressed your concerns. If any concerns remain, we would be happy to provide further clarification.
Response to Reviewer DXnd (1/2)
We sincerely appreciate your careful review and positive assessment of our problem formulation, traceable evidence transfer, and efficient memory construction, as well as your valuable insights. Below are detailed responses to each concern.
In GroupMemBench,
author,role,channel,phase, andtopicare supplied with the synthetic conversation records. DG-Mem uses these fields directly and does not infer them or make generative-LLM calls to create them. These fields are also included in the answer-generation inputs of the official GroupMemBench baselines and the other compared methods. To test our dependence on role, phase, and topic metadata, we removed these three fields and re-evaluated DG-Mem on GroupMemBench.Removing role, phase, and topic reduces overall accuracy from 55.30% to 52.75%, retaining 95.4% of the full model's accuracy. This variant still exceeds Hindsight, the strongest baseline, by 6.74 percentage points. DG-Mem therefore retains a substantial overall advantage without these three types of metadata. We have added this comparison and the accompanying discussion to Section 4.6 of the revised version.
DG-Mem uses reply links when available, but does not require them to retrieve a message. It does not perform explicit cross-message coreference or general alias resolution. Instead, it gathers candidates through entity, dense, and lexical signals in the content graph, temporal connections, context-aware BM25, and participant/scope propagation in the User-Context Graph, followed by sentence-level reranking. These signals can help retrieve messages containing indirect references when their content or interaction context provides additional identifying cues.
The following Technology example illustrates retrieval of an update without an explicit reply link:
user_context = 0.529,context_match = 1.00, final score0.701).The update was posted independently by another participant and refers to an “earlier decision” without an explicit reply link. DG-Mem nevertheless retrieves the update and answers correctly. This illustrates how informative message content and interaction context can support retrieval without reply links. A generic reference such as “that plan,” with no distinguishing content or interaction cues, may still remain ambiguous.