- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper presents GroundedMem for auditable utilization of long‑term conversational memory. It distinguishes evidence discovery, source grounding, and answer decision-making: it performs dual-view retrieval over atomic semantic memories and raw dialogue turns, followed by fusion, reranking, and source coordinate recovery to reconstruct complete source turns. A source grounding verifier then prioritizes support checks and enforces rejection only when target frame substitution is confirmed with no supporting evidence. Evaluated on LongMemEval-S and LoCoMo, the method achieves improved answer accuracy, source-turn recall, and rejection quality. Ablation on rejection for LoCoMo yields a net gain of 160 correct answers. Limitations include lossy atomic indexing, fixed budget constraints, and inadequate update and deletion capabilities.
Critical strengths
1.This work distinguishes long-memory failure modes and splits the pipeline into evidence discovery, source grounding and decision modules. Diagnostics show retrieval does not guarantee correct evidence use. It maintains strong evaluation hygiene, reporting dual error metrics and double-blind validation for the LLM judge.
2.Refusal only activates for verified target frame substitution; provenance is pre-baked for auditability. The LoCoMo ablation delivers striking gains: +8.06 net score, minimal degradation on answerable queries, and superior refusal F1 vs REMem. This is rare quantification showing conservative refusal yields net benefits.
Critical weaknesses
The novelty is moderately low:
- Provenance attribution (AIS, RARR, ALCE), selective answering (Self-RAG, Sufficient Context), and atomic memory with fusion-reranking (Mem0/HippoRAG line of work) are all existing ideas.
- The paper’s contribution lies in assembling these components into a memory system and establishing a rigorous evaluation protocol.
- The verifier is a prompted LLM (DeepSeek-v4-flash) paired with hard-coded decision rules, with no learned components.
Questions for authors
- How do you (authors) respond to the concern of limited novelty?
- How do you view the trade-off between the engineering contributions and academic innovations of this work?
3 comments
very clear clarification and thanks for your comment
Follow-up on the Revised Refusal Module
Thank you for taking the time to read our clarification and respond. We wanted to share a brief update on the refusal mechanism you discussed in your review.
Another reviewer asked whether the Atomic candidate stage could miss conflicts that are visible in the restored source turns. To address this, we revised the controller so that both Atomic memories and restored source turns can propose conflicts (Section 3.3 and Algorithm 2). The source-turn view brings details such as speaker identity and timing into candidate generation. A shared verifier checks the proposed conflicts against the original turns. Forced refusal requires a verified target-frame substitution with no supporting evidence.
We evaluated this change on LoCoMo with retrieval results, answer drafts, and judge decisions fixed.
| Metric | Submitted version | Revised version |
|---|---|---|
| Answer accuracy | 82.88% (1,646/1,986) | 85.25% (1,693/1,986) |
| Final refusal recall | 63.2% | 77.8% |
| Final refusal F1 | 73.3% | 82.1% |
| Controller false-refusal rate | 0.78% (12/1,540) | 1.49% (23/1,540) |
The net benefit you highlighted in your review increased from 160 to 207 additional correct answers relative to the same refusal-off baseline. The revised controller corrected 223 answers and introduced 16 new errors. On LongMemEval-S, replaying the decisions from the two independently verified views yielded 85.4% accuracy (427/500), with no controller-forced refusals on answerable questions.
If you have any remaining questions or concerns, we would be happy to discuss them. If you feel the revision addresses the issues you raised, we would be grateful if you could reconsider your rating in light of the revised manuscript and the additional results. Thank you again for your time and feedback.
Novelty and Scientific Contribution Clarification
Thank you for recognizing our failure-mode decomposition, evaluation rigor, provenance design, and paired accounting of refusal benefits and harms. We agree that the paper should distinguish existing building blocks from our system-level and methodological contributions more clearly.
Q1: How do we respond to the concern about limited novelty?
Our novelty claim does not rest on a single primitive. We formulate and evaluate conservative refusal as an end-to-end memory-system problem. Most memory systems try to return relevant memories for every query, yet users may ask questions that conflict with stored memories or are unanswerable from them, as in LoCoMo and LongMemEval. Most published systems have not made this a primary design and evaluation target. The closest published systems reporting LoCoMo Category 5 or explicit refusal results are:
Effective conflict-based refusal first requires sufficiently strong retrieval: a conflict can be established only when the relevant source evidence is reachable, while a retrieval miss alone must not trigger refusal. GroundedMem therefore combines three independently testable interfaces: (1) dual-view retrieval discovers evidence through complementary Atomic and direct-turn routes; (2) provenance resolves selected memories to complete source turns with speaker and time; and (3) Atomic memories propose bounded mismatch candidates, while a source-grounded, support-first verifier fails open on ordinary missing evidence and refuses only when no positive support exists and target-frame substitution is verified.
We therefore do not claim that RRF, provenance, Atomic memory, or prompted verification is individually novel. Our contribution is the source-resolved discovery–grounding–decision contract and its evaluation under explicit false-refusal costs while measuring net answer quality.
Q2: How do we view the trade-off between engineering contribution and academic innovation?
We agree that system completeness alone does not constitute academic novelty. Here, the engineering contribution and academic innovation are not substitutes; they serve as the “core method” and its “validation vehicle,” respectively.
GroundedMem places memory provenance, content organization, and downstream use in one framework to study how agent memory can support traceable, verifiable, and reliable decisions. We do not claim that every low-level component is new; the contribution lies in the problem formulation, key mechanism design, and their systematic composition. The implementation makes these ideas testable in realistic end-to-end workflows and improves reproducibility. We therefore position GroundedMem as research driven by a concrete scientific question and validated through a complete system, rather than as system integration alone. We thank the reviewer for helping us clarify this boundary.