- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to adapt memory to different LLMs. The authors approach the problem from both sides of memory — writing and reading — design the corresponding operators (RoMem), and implement them by training google/flan-t5-small. Experimental results show that RoMem delivers consistent improvements over several methods on HotpotQA, 2Wiki-MultihopQA, and MuSiQue. Moreover, RoMem retains a reasonable degree of adaptation robustness on models whose data was not seen during training.
Critical strengths
- The paper is well organized and readable. Researchers working on LLM memory can take away a useful perspective from it: considering cross-model memory adaptation through the lens of memory reads and writes.
Critical weaknesses
- My biggest concern is the emphasis on "Agent" in the title. In fact, the whole study is still carried out on specific base models. Over the past six months, the view that an agent = base model + harness (plus other components, e.g., function calling) has been emerging as a new consensus. If the authors wish to foreground agent-side memory, should they not consider whether the different harnesses themselves have already optimized what this paper studies? As written, no such comparison is taken into account.
Questions for authors
While reading the paper, the following questions caused me confusion:
- Could the experiment in Figure 1 be affected by capability differences among the models beyond architecture, or by the ordering of the models? Is it possible that the weaker models introduce noise that throws the overall picture off balance?
- Why is HotpotQA not included in the comparison in Figure 2?
- Section 5.3 states that profiles are an important component of RoMem. Should the earlier text (e.g., the preliminaries and the method section) give a more detailed account of them?
Although the authors’ response came later in the discussion period, my overall impression of the work remains positive. I found the paper clearly written, and I encourage the authors to continue refining the work.