- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses source traceability, fact updates, and abstention when evidence is insufficient in long-term conversational memory. NAVMEM uses immutable raw conversations as its evidence substrate. A four-layer continuity summary—now, today, recent, and archive—preserves the historical outline, while fact and event ledgers organize dynamic information. The compiler attaches source-line addresses to memory items, and rules generate supersession and conflict edges. At answer time, a keyword router guides candidate retrieval and source rereading. The model is prompted to label each required condition as supported, contradicted, or missing before answering.
Critical strengths
Summaries and fact entries retain source addresses, allowing old and new values to be traced through the dynamics ledger rather than leaving compressed content as the only available material. The interface supports locating errors in retrieval, updating, or answering and distinguishes semantic verification from guarantees about address validity. NAVMEM outperformed Mem0 on 33/35 BEAM conversations. The gain remained 0.103 on conversations 11–35, which were excluded from router tuning, and 0.080 under a second judge. LoCoMo retained a 0.066 gain when all methods answered in one round. These results support the complete system’s practical value beyond BEAM’s multi-turn tool budget.
Critical weaknesses
-
Table 9 allows NAVMEM up to 30 turns and route-quality retries, while external baselines answer in one round with unmatched input/output budgets. The source-linked methods with source-rereading protocols in Table 7 lack comparisons under the same protocol. LoCoMo supports the complete system in a single-round setting, but does not establish the new design’s added value over these alternatives.
-
Section 4.4 counts rereads, address citations, and gold-source coverage without checking whether individual assertions are supported or testing answer independence by perturbing summaries while fixing evidence. Proposition 1 assumes this independence. The authors disclose this boundary, but the results do not establish how much rereading reduces unsupported answers or incorrect abstentions.
-
Supersession edges are absent from 84.2% of facts. Adding dynamics also adds fact and event content, so its gains cannot be attributed to update or conflict links. Determinism and acyclicity in Appendices C/D do not guarantee semantic correctness. Controls retaining identical entries while disabling edges, and checks of how incorrect links affect answers, are missing.
-
All results rely on prebuilt ledgers. The evaluation does not interleave session capture, consolidation, and queries or report construction cost, index growth, actual answer-token usage, or latency. The results therefore leave accuracy gains under matched resources and the cost of maintaining growing histories unresolved. The 10M scores do not replace these measurements.
Questions for authors
-
After matching answer-token, tool-turn, and retry budgets, how much gain remains when Mem0 or a related source-linked method receives the same raw-text search and rereading protocol? What changes when update or conflict edges are disabled while ledger content is fixed? These controls would distinguish contributions from answering strategy, ledger content, and link structure.
-
Could you audit complete answers for assertion-level source support, stale-fact use, and incorrect abstention, and describe the adjudication procedure? With reread evidence fixed, do changes to candidate values in summaries change answers? Please also check whether incorrect update or conflict links cause current values to be missed or old values to be accepted.
-
Could you repair the tool loops in both pair arms and rerun all 700 questions with identical candidate formats and retry rules? Please report each arm’s actual macro and paired difference on the same questions, explaining how the distribution of earlier failures across questions and conversations affects conclusions about continuity, dynamics, and their complementarity.
-
Could you report results with session-by-session construction interleaved with queries, construction calls per million history tokens, storage growth, and actual tokens/latency per question? Do the gains remain against a resource-matched baseline, and what are the separate maintenance costs of continuity compression and append-only dynamics?