- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CoRelMem extracts atomic memories, builds an entity knowledge graph, and organizes memories into an association graph and temporal trajectories. At retrieval time, it expands initial candidates through both structures, then reranks evidence using semantic similarity and language-model conditional likelihood gain. The motivation is that useful evidence can have low query similarity, while highly similar memories can be uninformative. The main question is whether the proposed structures provide benefits beyond a stronger reranking pipeline with comparable context and computation.
Critical strengths
- The method provides a consistent improvement overall QA gains. Against LightMem, LoCoMo accuracy improves from 67.53 to 72.86 with Qwen and from 67.86 to 72.34 with Llama. LongMemEval-S improves from 46.20 to 49.20.
- Some results align with the motivation. Qwen’s LoCoMo multi-hop accuracy increases from 69.86 to 79.43, which supports investigating cross-memory retrieval. The paper includes ablations, retrieval metrics, parameter sweeps, and cost comparisons. The appendix also explains evidence bundles and disjoint selection more clearly than the main text.
Critical weaknesses
-
The novelty of this work relative to existing methods is not clearly established. The comparison should cover a broader range of structured memory approaches, particularly methods for temporal graph construction and learning.
-
The paper provides limited discussion of how CoRelMem differs from the selected baselines in its mechanisms and design choices. A more detailed comparison would help clarify its technical contributions. The evaluation is limited to long-term conversational memory benchmarks. The limitations of the proposed associative and temporal mechanisms, including the conditions under which they may fail, are also insufficiently discussed.
Questions for authors
-
Could the authors compare CoRelMem with additional structured memory baselines, particularly temporal graph methods, and clarify how it differs from approaches that model similar dependencies?
-
Beyond the reported benchmark improvements, could the authors explain the specific advantages of their design and provide evidence supporting the individual design choices?
-
Could the authors evaluate CoRelMem on additional benchmarks and provide a failure-case analysis? Experiments with stronger LLM backbones would also help assess whether the gains persist across model capabilities and clarify the trade-off between performance and computational cost.
-
Does the reported token cost for memory organization include all operations involved in coherent relation induction, including LLM-based relation judgments and conditional likelihood scoring?
5 comments
Response to Reviewer s44D (Part 2)
W2.3 & Q3.1: Failure-Case analysis
We further conducted a failure-case analysis to investigate the limitations of association-based and temporal evidence expansion. As shown in the table, the full CoRelMem retrieves several related memories about John's charity activities and basketball career but fails to retain the specific event evidence in its final Top-10 results. In contrast, removing association-based expansion while retaining temporal retrieval allows the gold evidence to appear at rank 9, whereas removing both expansion paths also fails to retrieve it. This case illustrates a potential limitation of multi-path retrieval, as association-based expansion may introduce competing candidates that prevent relevant evidence from being retained in the final retrieval results. Although the two retrieval paths generally improve evidence coverage, their combination can occasionally introduce interference during candidate selection and prioritization. In the revised manuscript, we have expanded the discussion in Section 4.5 and added Appendix A.9 to provide a more detailed examination of this failure case and the limitations of multi-path evidence retrieval.
| Question: What charity event did John organize recently in 2024? |
|---|
| Reference: benefit basketball game |
| Retrieval Setting: CoRelMem |
| #1 John is teaming up with a local organization that helps disadvantaged kids through sports and school, hoping to use his platform for positive impact on the community and to inspire others. |
| #2 John wants to use his platform to make a positive difference and inspire others, possibly by starting a foundation and doing charity work. |
| ... |
| #8 Much happened with John off court in the last month. |
| #9 [Association] John used contacts in the basketball industry and his marketing skills to make connections for endorsements, with networking playing a big role (image shows a basketball card of a man holding a basketball). |
| #10 [Association] John feels fulfilled having the trust and admiration of young athletes who see him as a role model, and is glad his experiences help shape their future and inspire them to pursue their dreams. |
| Retrieval Setting: w/o Association Expansion |
| ... |
| #9 [Trajectory] John held a benefit basketball game last week (shared a photo of a boy dribbling a basketball on a court). Gold |
| #10 John attended a Harry Potter trivia charity event last August. |
| Retrieval Setting: w/o Association & Temporal Expansion |
| ... |
| #9 John attended a Harry Potter trivia charity event last August. |
| #10 Tim supports John's endorsement journey. |
Q2: Specific advantages of the design
The advantages of CoRelMem can be explained from both theoretical and empirical perspectives.
Theoretically, CoRelMem integrates structured evidence expansion with information-aware prioritization to address the limitations of similarity-based memory retrieval. Specifically, the association graph enables the retrieval of relevant memories that are weakly matched to the query but connected through cross-memory dependencies. Temporal trajectories capture direct state transitions rather than merely chronological proximity, helping recover evidence distributed across evolving events. Furthermore, IAR mitigates the noise introduced by candidate expansion by prioritizing evidence based on both semantic relevance and conditional information contribution. Together, these designs improve evidence coverage while reducing the influence of less informative candidates. Their theoretical motivations are further discussed in Appendix A.2.
Empirically, the original ablations in Table 4 show that removing KG, CRI, or IAR generally reduces performance. We further conducted additional ablations comparing semantic-only, association-only, temporal-only, and dual-path retrieval, each with and without IAR. The results show that dual-path retrieval achieves the highest overall ACC among these configurations, while IAR consistently improves performance across all four retrieval strategies. Detailed results are provided in our response to Reviewer cSdp, W3 & Q2. In the revised manuscript, we have expanded the ablation analysis in Section 4.3 and added detailed comparisons of retrieval strategies with and without IAR in Appendix A.7.1.
Response to Reviewer s44D (Part 3)
Q3.2: Evaluation with Stronger LLM Backbones
To assess whether CoRelMem's improvements persist with a stronger LLM backbone, we conducted additional experiments using GLM-5.3-Flash on both LoCoMo and Long-MT-Bench+. On LoCoMo, CoRelMem achieves the highest overall ACC and F1, with particularly notable improvements on multi-hop and temporal reasoning tasks. On Long-MT-Bench+, CoRelMem also outperforms the baselines on five out of six evaluation metrics, as detailed in our response to Reviewer nhct, W1. Together with the original experiments using Qwen3-4B and Llama-3.2-3B, these results further demonstrate the effectiveness of CoRelMem across different LLM backbones. In the revised manuscript, we have added Appendix A.5 and Appendix A.6 to present the complete experimental results and corresponding analyses.
Table: QA Performance on LoCoMo Using GLM-5.3-Flash.
| Method | ACC | F1 | B-1 | R-L | Multi | Temp | Open | Single |
|---|---|---|---|---|---|---|---|---|
| Mem0 | 48.44 | 17.69 | 13.84 | 16.81 | 47.16 | 46.72 | 46.87 | 49.7 |
| MemGAS | 59.68 | 16.65 | 10.7 | 15.67 | 41.13 | 50.78 | 51.04 | 70.27 |
| A-Mem | 50.97 | 18.07 | 14.06 | 17.25 | 47.87 | 47.35 | 37.5 | 54.93 |
| LightMem | 72.82 | 35.93 | 17.17 | 34.41 | 66.67 | 75.31 | 45.83 | 77.02 |
| TSM | 58.44 | 27.2 | 14.36 | 26.43 | 53.9 | 54.21 | 40.62 | 63.61 |
| GAM* | 59.35 | 24.98 | 19.74 | 24.04 | 59.22 | 58.57 | 42.71 | 61.59 |
| CoRelMem | 78.18 | 39.23 | 18.18 | 37.31 | 74.47 | 79.13 | 46.87 | 82.64 |
GAM: Reimplemented by us, as no official implementation is publicly available.
Q4: Scope of the reported token cost
The reported token cost in Section 4.6 and Figure 4 includes LLM-based relation judgments during association graph and temporal trajectory construction. CLG-based conditional likelihood scoring is not included, as it is performed through likelihood evaluation rather than text generation and incurs additional computational overhead. In the revised manuscript, we have clarified in Section 4.6 that all reported costs are measured during memory construction, with token consumption referring specifically to the output tokens generated by LLMs during this process.
Response to Reviewer s44D (Part 1)
Thank you for the valuable comments. Below, we have carefully addressed each of your comments.
W1, 2.1&Q1: Novelty and comparisons with structured memory methodsWe thank the reviewer for highlighting the need to clarify CoRelMem's methodological distinctions from existing structured-memory approaches. CoRelMem addresses two challenges that are not fully resolved by existing methods. Relevant memories may be overlooked due to weak semantic similarity to the query, while highly similar but low-value memories may be overprioritized. As illustrated in Figure 1(a)–(b), semantic similarity does not necessarily reflect the actual value of a memory for answering the current query. The key novelty of CoRelMem lies in expanding potentially relevant memories through cross-memory dependencies and prioritizing retrieved evidence based on its information contribution rather than semantic similarity alone.
The theoretical motivation for CoRelMem's dependency modeling is discussed in Appendix A.2. Inspired by Markov Random Fields and Markov chains, CoRelMem models logical associations through local conditional dependencies and temporal trajectories through direct state transitions within evolving event threads. These are modeling interpretations rather than explicitly learned probabilistic graphical models. In comparison, GAM constructs hierarchical memory graphs based on event progression and topic associations, while TSM organizes temporally related memories through semantic timelines and durative memories to preserve temporal continuity and validity. ActMem constructs semantic and causal memory connections, using embedding similarity to identify semantic relations and PMI-style likelihood differences to validate LLM-generated causal relations. Unlike these approaches, CoRelMem focuses on information dependencies between atomic memories and direct state transitions within evolving events, rather than relying primarily on structural associations or temporal continuity. This design enables retrieval to expand along memories that provide complementary information or represent successive states of the same event, helping recover relevant evidence that may be missed by direct semantic matching. Furthermore, IAR evaluates how much information each expanded candidate contributes to the current query, allowing CoRelMem to prioritize useful evidence while reducing interference from highly similar but irrelevant memories.
To provide stronger empirical evidence, we additionally evaluated GAM and TSM on LoCoMo using Qwen3-4B and GLM-5.3-Flash. CoRelMem consistently outperforms both baselines, with particularly notable improvements on multi-hop and temporal reasoning tasks. Detailed results are provided in our response to
Reviewer nhct, W2. In the revised manuscript, we have incorporated the additional GAM and TSM results into Table 1 and further clarified the methodological distinctions between CoRelMem and existing structured-memory approaches in Section 2 and Appendices A.1.1 and A.1.2.W2.2 & Q3.1: Additional Benchmark EvaluationTo address the concern regarding limited benchmark coverage, we conducted additional experiments on Long-MT-Bench+ using GLM-5.3-Flash, comparing CoRelMem against six baselines. CoRelMem achieves the best performance on five out of six metrics. The complete results are provided in our response to
Reviewer nhct, W1. In the revised manuscript, we have added Appendix A.6, which presents the complete experimental results and corresponding analysis on Long-MT-Bench+.