- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This manuscript aims to address the lack of controlled experimental comparisons among different agent memory systems. To this end, the authors introduce an interpretable 5W-style representation of memory systems, which provides a unified interface for comparing different retrieval configurations over shared memory stores.
Critical strengths
The proposed 5W-style representation is reasonable, intuitive, and easy to follow. It provides a structured perspective for analyzing different memory configurations.
Critical weaknesses
- Limited depth of analysis and insufficiently insightful findings.
Due to the relatively shallow experimental analysis, the conclusions drawn from the study remain limited. For example, the observation that extraction quality constrains the performance of different graph-based configurations is somewhat expected.
A potentially valuable extension would be to implement and analyze several existing published agent memory architectures, categorize them under the proposed framework, and, if possible, incorporate them into the unified representation for systematic comparison. Such an analysis could reveal deeper insights into the design space of agent memory systems and strengthen the contribution of the work.
- The novelty of the 5W representation is unclear.
The 5W-style representation does not appear to be a novel way of modeling agent memory. For example, MemoCue: Empowering LLM-Based Agents for Human Memory Recall via Strategy-Guided Querying (2025) has already adopted a similar perspective for characterizing memory-related components. The authors should clarify the distinction between the proposed framework and existing formulations and discuss whether the contribution lies in the representation itself or in the systematic evaluation enabled by it.
Questions for authors
-
Please address the critical concerns raised in the Critical Weaknesses section, especially regarding the limited analysis depth and the relationship to prior 5W-style memory representations.
-
Please provide more details about the extraction models used in the experiments. Specifically:
- Does GPT-OSS refer to the 20B model or the 120B model?
- What exactly does "Qwen-Small" refer to? Is it from the Qwen3 series or the Qwen2.5 series? What is the parameter scale?
- Which specific Gemma model is used?
- There appear to be two missing references in the REPRODUCIBILITY STATEMENT section. Please verify and complete the citations.