- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes EVStateMem, which combines source-linked memory, versioned relation states, a materialized Current View, and answer-time eligibility checks for conversational memory. It preserves historical evidence when facts are revised and checks source availability, authorized scope, and lifecycle status before including evidence in an answer. It achieves strong single-hop and multi-hop F1 on LoCoMo and the highest reported scores on seven of nine TimlMemBench behavioral axes, with controlled ablations showing that versioned state improves replacement accuracy by 8.6 percentage points in both Chinese and English.
Critical strengths
(S1) EVStateMem achieves 69.11 single-hop and 52.30 multi-hop F1 on LoCoMo, exceeding the strongest reported local baselines by 7.93 and 7.95 percentage points, respectively. It also obtains the highest reported pooled scores on seven of nine TimlMemBench behavioral axes. These are meaningful empirical results supporting the practical effectiveness of the implementation.
(S2) Table 4 removes individual components while holding the remaining code, prompts, model configurations, and evaluation cases fixed. Removing versioned state reduces Supersede accuracy by 8.6 percentage points in both languages, and Revoke accuracy by 5.8 and 6.0 points. This provides direct evidence that state maintenance contributes to the intended behavior.
(S3) Appendix A.9 reports that final source checking reduces invalid-source answers from 12/12 to 0/12 while retaining all four authorized controls. Appendix A.11 additionally verifies that post-selection revocation prevents stale cached evidence from entering the Reader request. These tests substantiate the implementation’s enforcement logic and make its operational behavior inspectable.
Critical weaknesses
(W1) The incremental research contribution is not sufficient. Although the architecture combines source provenance, explicit update operations, temporal versioning, a materialized current-state view, and scope and lifecycle filtering, EVStateMem has substantial overlap with existing systems. The paper does not identify a specific new mechanism or demonstrate a distinctive benefit over a closely matched implementation of the existing designs. The evidence establishes a capable system more clearly than a research advance. We still prefer works with higher originality.
(W2) The experiments do not isolate the governance architecture’s contribution to the QA gains in an explicit way. The cumulative ablations change multiple components together, leaving the contribution of governance unclear relative to improvements in retrieval and answering.
(W3) The evidence for final rechecking establishes functional correctness. Its removal changes behavioral scores by only 0~1.6 percentage points in Table 4, while the controlled tests in Appendices A.9 and A.11 do not establish its benefit during ordinary operation or beyond conventional final eligibility checks.
(W4) The comparisons use different retrieval budgets and answering procedures across systems. These differences make it difficult to determine whether EVStateMem’s gains come from its memory architecture or its retrieval and answering setup.
Questions for authors
(Q1) Which specific mechanism or behavioral capability distinguishes EVStateMem from the closest source-linked temporal memory and evidence-lifecycle systems discussed in Section 5? What evidence demonstrates an incremental benefit from that distinction beyond combining versioning, provenance, current-state materialization, and eligibility checks?
(Q2) In Appendix A.8, Table 15 and 16 introduce multiple components together. What evidence isolates the QA contribution of versioned state and Current View from improvements in retrieval and answering? If these comparisons do not establish this contribution, please provide controlled ablations during rebuttal.
(Q3) Beyond deliberately injected invalidation and upstream-check omission, what evidence establishes the frequency and impact of final rechecking during ordinary operation? Does any experiment distinguish its benefit from equipping an existing temporal or provenance-based system with the same final status and scope validation?
(Q4) Which comparisons against the strongest relevant baselines match the reader, answer prompt, evidence-token budget, and model-call allowance? What reported evidence shows that EVStateMem’s advantage persists under these controls, rather than reflecting differences between the complete retrieval and answering configurations?