- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MemOrbit, which aims to enable long-running LLM agents to learn both how to search, retrieve, and manage external memories and which memory entries to retain, update, or forget. It consists of two coupled loops: the Policy Loop uses reinforcement learning to optimize memory interaction strategies, while the Memory Loop updates memory utility scores based on task outcomes and interaction traces, such as memory retrieval and citation. The paper also uses versioned memory snapshots, paired evaluation of candidate memory states, and step-boundary commits to improve consistency and traceability. Experiments on MemoryBench show improvements over several memory baselines.
Critical strengths
- Meaningful problem. Long-running agents need to learn not only how to use existing memories but also how to maintain them. Studying memory use and memory evolution within a unified framework is well motivated.
- Interpretability and auditability. Versioned memory snapshots, explicit retrieval and citation traces, and step-boundary commits make memory updates easier to track and analyze.
- Clear presentation. The paper is generally well written and easy to follow.
Critical weaknesses
- Limited novelty. MemRL already learns memory utility from environmental feedback, while Memory-R1 uses reinforcement learning to learn memory management and utilization policies. Given these overlaps, the paper should more clearly distinguish its methodological contributions from prior work, particularly in the utility update mechanism and the interaction between the two learning loops.
- Limited generalizability. The evaluation is mainly based on MemoryBench and its experimental setup. Even though the benchmark covers multiple tasks, the results do not necessarily establish that the method generalizes to other long-horizon agent settings.
- Limited statistical evidence. MemOrbit outperforms MemRL by only 0.003 on the main metric. How many independent random seeds were used, and are variances, confidence intervals, or significance tests reported for each baseline? Also, MemOrbit performs worse than MemRL on WritingPrompts. How does this result support the paper's claim of consistent improvements?
- Insufficient implementation details. Some important details about the memory operations (Write, Update, Merge, and Forget) remain unclear, which limits reproducibility and makes it difficult to assess the proposed mechanism. For example, how is the utility of a merged entry initialized or recalculated? Is the utility of deleted entries retained? How many trajectories and samples are used for paired evaluation, and how is generation randomness controlled? The authors should provide these details.
Questions for authors
See the weaknesses above.