- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Memory as Code (MaC), which organizes conversational facts, temporal replacement relations, and reusable memory computations into a callable library. A declarative retrieval component generates a query-specific program that composes these capabilities to retrieve supporting evidence. Experiments on LoCoMo and LongMemEval evaluate answer accuracy, computational cost, and component contributions.
Critical strengths
-
The representation addresses concrete challenges in long-term memory. Canonical fact identifiers, explicit supersession relations, separate current-state and historical computations, and source provenance form a coherent interface for reasoning over evolving information. The contribution goes beyond merely wrapping text in functions.
-
The empirical improvements are reasonably broad and consistent with the motivation. MaC achieves the highest overall LLM-judge accuracy among the evaluated methods across three LoCoMo backbones and on LongMemEval. On LongMemEval, overall accuracy improves from the strongest baseline’s 68.40% to 74.60%, with notable gains in temporal reasoning and knowledge updates. The sequential ablations also show improvements from adding both codified construction and declarative retrieval, while the cost analysis distinguishes construction from query-time overhead.
Critical weaknesses
-
The ablations and execution analysis do not fully identify which mechanisms drive the improvement. Table 5 omits the configuration that retains declarative retrieval while removing codified construction. Moreover, codified construction jointly introduces relations, state semantics, indexing, provenance, and executable interfaces, making their individual contributions unclear. Algorithm 1 also permits program repair and fallback retrieval, but the paper does not report first-attempt execution success, repair frequency, or fallback rates. These omissions limit the explanation of how the proposed program-based retrieval operates in practice, although they do not invalidate the complete system’s gains.
-
Important evaluation details are missing. The manuscript does not clearly identify the LongMemEval version, evaluated sample set, or truncation policy. Table 2 presents four question categories without explaining whether the overall score includes the other categories. Baseline versions and principal retrieval configurations are also insufficiently documented. Finally, the standard deviations in Table 1 come from repeated judge evaluations, rather than independent repetitions of memory construction and retrieval, and therefore do not quantify end-to-end variability.
-
The related-work discussion should include AMA: Adaptive Memory via Multi-Agent Collaboration, which combines adaptive retrieval with conflict-aware memory maintenance and evaluates on LoCoMo and LongMemEval. Please clarify how MaC’s executable retrieval programs and explicit historical-state semantics differ from AMA’s iterative retrieval and maintenance mechanisms.
Questions for authors
See Weakness
4 comments
W3 (Comparison with AMA): Thank you for this helpful suggestion. AMA [1] uses iterative retrieval guided by its Judge, while MaC generates executable retrieval programs that compose memory functions and operators. AMA updates or conditionally deletes conflicting entries. MaC retains superseded facts and uses supersession relations to compute the active state at a specified time. We have clarified these differences in Section 4 (Related Work).
[1] Weiquan Huang, Zixuan Wang, Hehai Lin, Sudong Wang, Bo Xu, Qian Li, Beier Zhu, Linyi Yang, and Chengwei Qin. 2026. AMA: Adaptive Memory via Multi-Agent Collaboration. In Findings of the Association for Computational Linguistics: ACL 2026, pages 3099–3120. Association for Computational Linguistics.
Thanks for the reply. It addresses my main concern, and I will keep my positive score.
Thank you for your positive feedback on our rebuttal and continued support for our work. We are pleased that our clarifications and additional results have helped address your concerns about codified construction’s contribution, retrieval execution reliability, evaluation settings, end-to-end stability, and MaC’s differences from AMA. We have incorporated these clarifications and results into the revised manuscript.
W1 (Ablation of codified construction): Thank you for this helpful suggestion. We have evaluated the configuration that retains declarative retrieval while removing codified construction (MaC w/o CC), using a graph for fact storage and indexing on LoCoMo. Compared with MaC, overall F1 and J decrease by 10.76 and 13.23 percentage points, respectively. These results show that codified construction contributes to the complete system’s gains when declarative retrieval is retained.
Relations, state semantics, indexing, provenance, and executable interfaces are integral parts of codified construction, jointly designed around facts stored as functions. These components are tightly coupled, making it difficult to isolate their individual contributions. We have added this ablation in Section 3.4 (Table 5) and clarified its scope in Appendix D.2.
W1 (First-attempt success, repair frequency, and fallback rates): We appreciate your constructive comments. We report first-attempt success, repair frequency, and fallback rates below. These results show that program-based retrieval usually succeeds on the first attempt, with repair resolving most initial failures and fallback rarely needed. We have added these execution statistics to Section 3.5, Table 7.
W2 (LongMemEval version, evaluated sample set, and overall score): Thanks for your careful review. Following MRAgent’s evaluation setting, we have evaluated all 500 instances of the cleaned LongMemEval-S version (
longmemeval_s_cleaned.json). We construct memory from each instance's full conversation history without additional truncation. We report the two question categories omitted from Table 2: multi-session (Multi.) and single-session-assistant (Asst.). The overall score averages over all questions across all six categories. MaC improves overall J by 17.59% relative to the strongest baseline. Thus, MaC’s overall gains hold when all six question categories are included. We have clarified the protocol in Section 3.1 and Appendix D.1 and expanded Table 2 to include all six question categories.W2 (Baseline versions and principal retrieval configurations): Thanks for your valuable feedback. We follow the default settings in each baseline’s respective paper. We have provided the baseline versions and retrieval configurations in Appendix D.1 and summarize them below.
memoryos-pypi, retrieving up to 5 mid-term sessions, 10 pages, and 20 entries each for user and assistant knowledge.W2 (End-to-end variability): We appreciate your valuable suggestion. We have conducted five independent repetitions of memory construction and retrieval on LoCoMo and report the mean ± standard deviation to quantify end-to-end variability. The small standard deviations indicate low end-to-end variability across these independent repetitions. We have added the five-run results and stability analysis to Section 3.5, Table 6.