- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes R²-Mem, a training-free framework that improves iterative "deep memory search" agents by learning from past search trajectories. Offline, a Rubric-guided Evaluator scores each Planning and Reflection step on module-specific 0–3 rubric dimensions and emits diagnostic reasons/advice; a Self-Reflection Learner distills high-scoring and low-scoring steps into abstract IF-THEN "experiences" stored in separate planning and reflection banks indexed by an abstracted situation embedding. At inference, top-k experiences are retrieved by similarity on the current query/memory state and injected into the corresponding stage, with no parameter updates.
Experiments on LoCoMo, HotpotQA, and NarrativeQA with Qwen2.5-3B/7B/14B and Llama3.1-8B show consistent F1/BLEU1 gains over GAM and over structured-memory baselines (A-Mem, Mem0, MemoryOS, LightMem) and Memory-R1, with reduced online tokens and iterations. Ablations separate the Evaluator, Learner, high- vs low-quality experience, and planning vs reflection banks.
Critical strengths
- Careful component-level validation of the central claim. The ablations in Table 4 directly test the paper's thesis that step-level, rubric-guided diagnosis matters.
- Efficiency and quality improve simultaneously, with the offline cost quantified rather than hidden. Table 3 shows F1 gains together with fewer online tokens and iterations across all three backbone sizes.
Critical weaknesses
- "Experience" is benchmark-specific and answer-supervised, so the reusability/generality claim is not established. Experience banks are built per-benchmark from in-distribution questions. There is no cross-dataset or cross-domain transfer experiment (e.g., LoCoMo-built bank applied to HotpotQA), so the paper cannot distinguish "reusable fine-grained reflective experience" (Abstract, Sec. 4.3) from within-distribution adaptation to a benchmark's question style — a concern amplified on LoCoMo, where one conversation's distilled heuristics (e.g., the temporal-query patterns in Appendix E.2) transparently match the other nine conversations' dominant category mix (Table 5). Relatedly, no published experience-learning method is compared directly; the only point of comparison is the authors' own "w/o Evaluator" re-implementation described as ReasoningBank-style.
Questions for authors
- (Clarification) For Table 1, which baselines were run once on which split (B.2), and what were the "preliminary runs" showing "only minor differences"? Rerunning at least Memory-R1 and A-Mem on all three splits would resolve the fairness concern.
- (Suggestion) Does experience transfer across benchmarks or question distributions? E.g., apply the LoCoMo bank to HotpotQA (or Conv-26's bank to a held-out LoCoMo category absent from the source conversation) to substantiate "reusable" beyond in-distribution.
- (Clarification) The Evaluator consumes REFERENCE_ANSWER (D.1). How much does performance degrade if rubric scoring is done without gold answers, which would be the realistic deployment condition implied by Sec. F.2?
- (Clarification) Appendix C.1 compares token counts across different models (GPT-4o offline vs Qwen online). What is the comparison in dollar cost or FLOPs, and does the amortization claim for 14B (0.53M offline vs 1.12M saved) survive GPT-4o pricing?
- (Suggestion) Why not compare directly against ReasoningBank/ExpeL adapted to GAM, rather than only your own "w/o Evaluator" approximation (Sec. 5.4)?
Thank you very much for your response and detailed clarification. Much of my concern has been addressed. I lean towards acceptance and maintain my initial assessment of weak acceptance. Wish you good luck with your submission.