- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CogniFold, which organizes continuously arriving information into three layers: events, concepts, and candidate intents. When a new event arrives, the system retrieves existing concepts and uses an LLM to generate a structured update plan, prioritizing the reuse or revision of existing concepts while preserving provenance links.
Critical strengths
For continuously operating agents, how to develop reusable understanding as information arrives, rather than merely storing historical records, is a worthwhile research problem.
Critical weaknesses
1.In the introduction, the authors characterize existing memory architectures as having fixed topologies once formed. The related work section also describes A-Mem as structurally static. In fact, A-Mem explicitly includes dynamic link creation triggered by new memories and updates to existing memory representations. Using existing memories to interpret new information, dynamically modifying memory content and connections, reusing or merging concepts across events, and designing suitable memory structures and systems cannot, in themselves, be considered contributions unique to this paper. The focus of this work is its explicit event–concept–intent organization and a memory-writing process designed around concept reuse. The issue is that the paper has not sufficiently explained which mechanisms enable capabilities that existing methods would struggle to achieve. What advantages does the proposed method offer over related dynamic memory methods under the same experimental conditions?
2.Based on the evaluation procedure described in the main text, the workflow can be summarized as follows: a new event arrives, similar existing concepts are retrieved, an LLM decides whether to update an existing concept or create a new one, and the results are stored. My question is whether a conventional memory store that maintains concept text and supports retrieval and modification could do the same. In LoCoMo, the system retrieves the 15 most similar existing concepts and provides their IDs and descriptions to the model. It therefore remains unclear how much the connections, hierarchy, and other topological information among concepts actually contribute.
Questions for authors
Please see the specific concerns and questions raised in the Weaknesses section.
2 comments
Thank you for the response.After considering it,I’ll keep my original score.
Thank you for the clear and focused review. Both weaknesses point to important issues in how we described prior dynamic memory systems and what specific role the graph structure plays. We have revised Sections 1, 2, and 4.5 (marked in blue) and added controlled ablations addressing both points.
1. Positioning Relative to A-Mem, Proactive Organization, and Same-Model Comparisons (W1)
CogniFold goes beyond generic ingestion-time updates to perform proactive organization that continuously tracks user intents. A-Mem dynamically links notes and updates note representations on arrival, and Mem0, Zep, EverMemOS, and PlugMem likewise update memory during ingestion (credited in Sections 1 and 2). CogniFold differs in two ways: (i) instead of linking notes or indexing facts without user goals and passively awaiting queries, it performs proactive organization, folding recurring observations around user concerns to track grounded intents ahead of queries (85.4% valid and 43.1% realized 3 to 4.5 sessions later across 176 LoCoMo intents; Table 6); and (ii) instead of rewriting per-note text in place, it separates immutable
Eventrecords from update-firstConceptabstractions.Matched-model comparisons (Section 4.5, Table 5; Appendix Table 16):
gpt-4o-miniwriter, reader, and officialevaluate_qa.pygrader): CogniFold reaches 75.4% (377/500), above published4o-miniscores for Mem0 (53.6%, reproduced by Fang et al., 2025), LightMem (68.6%), and ENGRAM (71.4%), and Zep with agpt-4oreader (71.2%). Recall and update accuracy stay high with4o-mini(SSU 95.7%, KU 91.0%, SSA 89.3%), while SSP (66.7% vs. ENGRAM 93.3%), MS (65.4%), and TR (61.7%) account for the gap to our5.4-minireader row (93.0%).gpt-4o-miniand embeddings across 3 builds/arm on LoCoMo and 6 configs × 6 scenarios × 3 orders on CogEval (108 builds, 18 per config),Fullcuts concept nodes by 42.4% on LoCoMo (and 37.7 → 12.9/scenario on CogEval) vs. add-firstFeedback-blocked, liftsEvent→ConceptEvidence Precision from 0.116 to 0.177 (+0.061, 95% CI [+0.021, +0.102]), and beatsOnline clustering + namingon LoCoMo (+3.40 points in Table 2; +1.10, CI [+0.04, +2.19] at 4k) and CogEval Harmony (+0.151, CI [+0.085, +0.214]).2. Contribution of Graph Topology vs. a Mutable Concept-Text Store (W2)
On conversational QA (LoCoMo), read-time edge traversal among concepts contributes very little (-0.32 points); the graph's main QA value comes from bipartite
EventtoConceptprovenance (+10.33 points) and write-time grounding alignment.We tested both parts of your question directly on our saved builds (Section 4.5, Table 5; Appendix D.2):
gpt-4o-miniReader & Judge)concepts-only, 3 builds, noEventlinks)A, raw turns only, no concept layer)Eventrecords (concepts-only), LoCoMo accuracy drops by 10.33 points (83.27% → 72.94%; Temporal -14.6 points, and 69.48% without date/provenance repair), because iterative text updates drop verbatim phrasing and dates. Keeping bipartiteEventtoConceptlinks letsFull@4k(82.68%) exceedEvents-only@12k(81.95%) on all 3/3 builds with 27.6% fewer context tokens (and Full > A holds across4o-mini,4.1-mini, and5.4-minijudges). Beyond QA, explicitGROUNDSandREINFORCESedges verify 97.1% to 97.5% concept support and supply the degree counts used for candidate-intent filtering (Table 6).