- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper formulates query-adaptive relational evidence selection for long-term conversational memory and proposes DyMem, which activates typed edges via query relation alignment and grounds evidence back to verbatim dialogue turns. Gains on multi-hop and temporal QA are promising, with much lower context cost.
Critical strengths
1.The framework of DyMem is well-developed and highly targeted: DyMem defines the core challenge of long-term memory as not merely how to represent relations, but how to determine which relations themselves are relevant to a specific information need. Based on this, DyMem designs a retrieval method that balances graph size and relevance. The method is reasonable and is not a simple stacking of modules.
2.Experiments are thorough: Results across multiple base models and benchmarks show that DyMem achieves competitive performance while maintaining moderate retrieval and construction overhead.
Critical weaknesses
Memory extraction is performed by an LLM, but the paper lacks an analysis of extraction quality and its impact on retrieval. Adding a qualitative analysis of extraction quality and a quantitative analysis with a different extraction model would better demonstrate the stability of the method.
There are some writing issues. The REPRODUCIBILITY STATEMENT contains content unrelated to the paper, such as Navier-Stokes, Compressible Navier-Stokes, Shallow Water Equations, and Diffusion-Reaction. In addition, Table 12 indicates that the original logs need to be checked; the authors are advised to proofread the paper carefully.
The results show a significant increase in retrieval time, but the paper does not analyze this phenomenon. The authors are advised to add a corresponding analysis.
Questions for authors
see above
2 comments
Thank you for your response.
W1:
To investigate the impact of the Extractor, we conducted experiments using Qwen2.5-3B and GPT-4o-mini. To the best of our knowledge, GPT-4o-mini has substantially stronger capabilities than Qwen2.5-3B.
In these experiments, we only replaced the Extractor model while keeping all subsequent components and settings unchanged.
The experimental results are presented as follows:
As shown, the capability of the Extractor does influence memory performance to some extent. Replacing the Extractor with either GPT-4o-mini or Qwen2.5-3B leads to a corresponding increase or decrease in performance. However, the magnitude of this variation is limited (approximately ±1%), indicating that our method does not heavily rely on the capability of the Extractor.
W2:
Due to an asynchronous issue during the template merging process, several typos were introduced in the Reproducibility Statement. We have corrected all of them in the latest revised manuscript. We are also carefully checking and correcting any other potential typos throughout the paper. We would greatly appreciate it if you could let us know if you identify any further typos.
W3:
To provide a clearer analysis of the time consumption during retrieval, we decompose DyMem's query-time retrieval process into six stages and measure the execution time of each stage separately. The stages and their corresponding operations are as follows:
The analysis is presented as follows:
The main reason is that computing scores for the retrieved evidence introduces additional computational overhead.