- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces QVRMem, a "Query-time Memory Reasoning" framework designed to address a common issue in Large Language Model (LLM) agents: the tendency to generate responses based on outdated facts or assumptions, even after retrieving updated information. The framework consists of two complementary components: "Dependency Validation" (V), which verifies potentially outdated facts before a response is generated; and "Question-directed Reanswering" (R), which re-evaluates the original query by leveraging previously accumulated memory evidence.
Critical strengths
- Clear and Practical Motivation. The paper addresses an important limitation of existing memory systems: retrieving updated information does not guarantee that the generated answer reflects the latest state.
- Useful Ablation Studies. The authors evaluate the contributions of validation and reanswering separately, providing insights into their effectiveness across different types of memory tasks.
Critical weaknesses
-
Limited methodological novelty. The proposed QVRMem framework primarily combines existing techniques, including query decomposition, evidence verification, iterative retrieval, and answer refinement, into a multi-stage prompt-based pipeline. Although the overall performance is promising, the core operations appear largely achievable through relatively straightforward prompt engineering. More importantly, the paper lacks comparisons with simpler iterative prompting or self-refinement baselines under comparable inference budgets.
-
High Inference-Time Computational Overhead and Lack of Budget-Matched Comparisons. QVRMem introduces substantial inference-time overhead due to its multi-stage validation, retrieval, and reanswering pipeline.
Questions for authors
See weakness.
Dear Reviewers,
The authors have now submitted their responses. Please review them and indicate whether your concerns have been addressed. Please also update your scores if the responses have changed your assessment.
Thank you.
AC