- Rating
- 2
- Confidence
- 3
- Informativeness
- 3(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TraceMem, a conversational memory system that stores propositions and preserves their revision histories. Retrieval combines intra-relations between historical and current versions with inter-relations based on shared entities, then returns the supporting dialogue turns to a reader. Experiments on LoCoMo-10 and LongMemEval-S report improved answer accuracy and evidence recall, together with low retrieval latency. The paper also includes relation ablations and hyperparameter studies.
Critical strengths
-
A practical and intuitive memory retrieval design. The combination of intra- and inter-relations provides a relatively simple way to recover distributed evidence while preserving memory evolution. Avoiding iterative LLM calls at retrieval time is particularly attractive for latency-sensitive applications.
-
Promising empirical results with useful supporting analysis. TraceMem achieves strong results on two established benchmarks. The component ablations and evidence-recall experiments further support the usefulness of the proposed relations, rather than relying solely on final QA accuracy.
Critical weaknesses
-
Insufficient isolation of the core retrieval contribution. Although the component ablations demonstrate the benefits of intra- and inter-relations, the paper lacks a comparison with simpler entity-based expansion over the same proposition store. It remains unclear how much of the improvement comes specifically from trajectory scoring rather than memory construction and relation-based expansion.
-
Efficiency and dynamic-memory claims need stronger validation. The reported retrieval latency is promising, but its measurement scope and scaling behavior remain unclear. Although the benchmarks include temporal and knowledge-update questions, the paper lacks direct evaluation of progressive memory updates and their computational costs.
Questions for authors
-
How does TraceMem compare with simpler entity-based expansion over the same proposition store, using identical embeddings and reader context budgets?
-
Does the reported 0.08-second latency include all retrieval operations, including query embedding, trajectory scoring, and candidate embedding? How does retrieval latency scale with memory size? Also, Figure 3 appears to show a smaller offline token footprint for LightMem than TraceMem. Could the authors clarify this discrepancy?
-
Could the authors provide a more direct evaluation of incremental memory updates, particularly when previously stored facts are revised or contradicted?
-
How sensitive is TraceMem to the memory construction model? Since proposition extraction and consolidation rely on Llama-3.3-70B, would using a smaller model significantly affect retrieval accuracy and indexing efficiency?