- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Infini-Memory handles long-term memory for agents in a straightforward manner. It organizes information into topic-based documents, stores recent content in a short-term buffer, and maintains a complete archive of the original conversations. As new information arrives, the system rewrites the relevant documents, dynamically splitting or merging them based on shifts in the subject matter. Each memory segment includes a source tag that links back to the original conversation from which it originated. For retrieval, the paper proposes two approaches, hybrid and agentic, with the "source-aware" method capable of accessing the archived conversations directly when deeper context is required.
Critical strengths
- The paper is overall easy to read and understand. Smooth Presentation.
- Empirical results are promising. The MemoryAgentBench improvement appears across tasks and reader configurations.
- The maintenance ablations show a benefit when answering requires combining updated facts.
Critical weaknesses
- There is a lack of a clear correspondence between the 96.4% result on LongMemEval-S and the revision mechanism. While Section 4.2 states that the system extracts entries and repeatedly rewrites the complete topical document, Appendix B.2 notes that the memory used for the "source access" experiments actually consists of clustered raw dialogue turns, not the rewritten text. Furthermore, since the derived documents for positive examples already contain the necessary information, source access primarily serves to improve localization and presentation. Consequently, it remains ambiguous as to which specific memory implementation the 96.4% figure actually corresponds to. The source access experiments merely demonstrate that this retrieval configuration is effective; they fail to establish a definitive link between the memory used for evaluation and the extraction-revision pipeline. Although the ablation study appears to support the claims (showing an increase from 91.8% to 96.4% with the "agentic reader"), it conflates several components, topical structuring, plan-driven updates, and splitting/merging, making it impossible to isolate the specific contribution of full-document rewriting within a fixed topical organization. Given that full-document rewriting is a core contribution, this ambiguity directly affects the interpretation of the headline results.
- When the query budget is roughly equivalent, what specific advantages does a maintained memory offer over querying the original source directly? A "source-aware reader" begins by scanning the archives—as shown in Table 6, each query involves reading an average of 47.1 evidence objects, with the answer prompt spanning 15,918 tokens. However, the paper fails to provide the total token count (retrieval plus generation) or an end-to-end cost comparison. The most essential missing control experiment would be to use the same model, the same retrieval budget, and the same context-assembly and answer-prompt structure, but restrict the search solely to the archived conversations. The current "no-maintenance" configuration does not serve as this control, as it still incorporates extraction and "CURRENT" rewriting. While Table 1 notes a shared backbone and ingestion chunk size, this only tells part of the story and does not prove that the computational effort during the query phase is comparable. Since the system is inherently capable of retrieving information by accessing the original archives, a direct "source-only" comparison is necessary to isolate the actual contributions of building and maintaining a memory, specifically regarding its impact on both accuracy and query costs.
Questions for authors
- For the original results of LongMemEval-S presented in Tables 2, 3, and 5, which specific memory construction pipeline was used? How does the "clustering of raw dialogue turns" (mentioned in Appendix B.2) relate to the processes described in Section 4.2, namely, the extraction of specific entries and the rewriting of entire topical documents?
- Using the same "source-aware reader", but restricting it to query only archived conversations, while keeping the search budget, context window limit, and answer prompt constant, what level of performance can be achieved? Furthermore, how are the denominators and aggregation rules in Table 1 calculated? For instance, in Table 7, DetectiveQA consists of 71 questions, with accuracy as the metric; however, the figures 78.4% and 77.2% do not appear to be simple binary micro-averages based on those 71 questions. Does this represent an average taken across different runs or groups, or does the actual number of evaluated items differ?