acceptodds
Under review as a conference paper at ICLR 2027

GroundedMem: Source-Resolved Retrieval and Conservative Refusal for Long-Term Conversational Memory

Abstract

Long-term conversational memory requires recovering information from growing interaction histories. However, retrieval relevance does not imply evidential sufficiency: related memories may refer to the wrong participant, time, or event state, while missing support in a bounded retrieval set does not establish global unanswerability. Most long-term-memory systems do not explicitly report refusal quality, while those that do remain substantially weaker on this axis. We present GroundedMem, an auditable pipeline linking retrieval, source recovery, and selective answering. Compact Atomic memories serve as semantic indices; Atomic and direct-turn retrieval are fused and reranked; and selected candidates are resolved to complete, source-linked conversation turns. A controller refuses only when constrained mismatch checks fire without detected support; unresolved cases remain on the ordinary answer path. We develop the verifier prompt and hard decision policy on LongMemEval-S and transfer them to LoCoMo after normalizing candidate certificates, without LoCoMo-label tuning of the verifier. Relative to the published comparator rows included in our comparison, GroundedMem improves LoCoMo LLM-judge accuracy by 5.4 points and LongMemEval-S accuracy by 3.4 points. LoCoMo refusal F1 exceeds the reported REMem value by 9.3 points. In benchmark-internal paired comparisons, with refusal disabled and the Answer model, Judge protocol, prompt, and evidence budget fixed between arms, the complete evidence path exceeds Direct+EvoEmbedding by 2.97 points on LoCoMo and 1.60 points on LongMemEval-S. A separate paired LoCoMo ablation yields a net accuracy gain of 8.06 points from refusal, with newly incorrect answers accounting for 0.45% of answerable questions. Complete annotated source coverage nevertheless leaves substantial answer errors, motivating separate evaluation of discovery, evidence use, and answer decisions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Long-Term Conversational Memory, LLM Memory, Source-Grounded Retrieval, Selective Answering, Refusal

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bhc2OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents GroundedMem for auditable utilization of long‑term conversational memory. It distinguishes evidence discovery, source grounding, and answer decision-making: it performs dual-view retrieval over atomic semantic memories and raw dialogue turns, followed by fusion, reranking, and source coordinate recovery to reconstruct complete source turns. A source grounding verifier then prioritizes support checks and enforces rejection only when target frame substitution is confirmed with no supporting evidence. Evaluated on LongMemEval-S and LoCoMo, the method achieves improved answer accuracy, source-turn recall, and rejection quality. Ablation on rejection for LoCoMo yields a net gain of 160 correct answers. Limitations include lossy atomic indexing, fixed budget constraints, and inadequate update and deletion capabilities.

Critical strengths

1.This work distinguishes long-memory failure modes and splits the pipeline into evidence discovery, source grounding and decision modules. Diagnostics show retrieval does not guarantee correct evidence use. It maintains strong evaluation hygiene, reporting dual error metrics and double-blind validation for the LLM judge.

2.Refusal only activates for verified target frame substitution; provenance is pre-baked for auditability. The LoCoMo ablation delivers striking gains: +8.06 net score, minimal degradation on answerable queries, and superior refusal F1 vs REMem. This is rare quantification showing conservative refusal yields net benefits.

Critical weaknesses

The novelty is moderately low:

  1. Provenance attribution (AIS, RARR, ALCE), selective answering (Self-RAG, Sufficient Context), and atomic memory with fusion-reranking (Mem0/HippoRAG line of work) are all existing ideas.
  2. The paper’s contribution lies in assembling these components into a memory system and establishing a rigorous evaluation protocol.
  3. The verifier is a prompted LLM (DeepSeek-v4-flash) paired with hard-coded decision rules, with no learned components.
Questions for authors
  1. How do you (authors) respond to the concern of limited novelty?
  2. How do you view the trade-off between the engineering contributions and academic innovations of this work?
3 comments
AuthorsOpenReview ↗

Novelty and Scientific Contribution Clarification

Thank you for recognizing our failure-mode decomposition, evaluation rigor, provenance design, and paired accounting of refusal benefits and harms. We agree that the paper should distinguish existing building blocks from our system-level and methodological contributions more clearly.

Q1: How do we respond to the concern about limited novelty?

Our novelty claim does not rest on a single primitive. We formulate and evaluate conservative refusal as an end-to-end memory-system problem. Most memory systems try to return relevant memories for every query, yet users may ask questions that conflict with stored memories or are unanswerable from them, as in LoCoMo and LongMemEval. Most published systems have not made this a primary design and evaluation target. The closest published systems reporting LoCoMo Category 5 or explicit refusal results are:

Closest published memory systemsWhat they addressWhat GroundedMem additionally addresses
REMem (Shu et al., ICLR 2026)Reports output-level refusal precision/recall/F1 (73.3/56.8/64.0) over all 1,986 LoCoMo questionsGroundedMem fixes evidence and the draft, isolates the controller’s causal effect, and separately reports corrections, induced errors, and false refusals.
Synapse (Findings of ACL 2026)Uses confidence gating and outputs “Not mentioned” when evidence is insufficient; reports Category-5 F1/BLEU and false refusals on non-adversarial questionsGroundedMem distinguishes retrieval failure from a source-verified conflict, refusing only when positive support is absent and target-frame substitution is verified.
TSHMem (EMNLP 2026)Outputs “Information is missing” when evidence is unavailable and reports Category-5 judge accuracy and token F1It does not decompose full-set refusal precision/recall and false-refusal cost or compare controller on/off under fixed evidence.
Other Category-5 systems (A-MEM, H-MEM, MEMFLY, HingeMem, SEEM, MAGMA, APEX-MEM, etc.)Report token F1, BLEU, or judge accuracy on Category 5Such category-level QA scores generally omit false refusals on answerable questions and cannot replace full-set refusal and net-accuracy evaluation.

Effective conflict-based refusal first requires sufficiently strong retrieval: a conflict can be established only when the relevant source evidence is reachable, while a retrieval miss alone must not trigger refusal. GroundedMem therefore combines three independently testable interfaces: (1) dual-view retrieval discovers evidence through complementary Atomic and direct-turn routes; (2) provenance resolves selected memories to complete source turns with speaker and time; and (3) Atomic memories propose bounded mismatch candidates, while a source-grounded, support-first verifier fails open on ordinary missing evidence and refuses only when no positive support exists and target-frame substitution is verified.

We therefore do not claim that RRF, provenance, Atomic memory, or prompted verification is individually novel. Our contribution is the source-resolved discovery–grounding–decision contract and its evaluation under explicit false-refusal costs while measuring net answer quality.

Q2: How do we view the trade-off between engineering contribution and academic innovation?

We agree that system completeness alone does not constitute academic novelty. Here, the engineering contribution and academic innovation are not substitutes; they serve as the “core method” and its “validation vehicle,” respectively.

GroundedMem places memory provenance, content organization, and downstream use in one framework to study how agent memory can support traceable, verifiable, and reliable decisions. We do not claim that every low-level component is new; the contribution lies in the problem formulation, key mechanism design, and their systematic composition. The implementation makes these ideas testable in realistic end-to-end workflows and improves reproducibility. We therefore position GroundedMem as research driven by a concrete scientific question and validated through a complete system, rather than as system integration alone. We thank the reviewer for helping us clarify this boundary.

very clear clarification and thanks for your comment

AuthorsOpenReview ↗

Follow-up on the Revised Refusal Module

Thank you for taking the time to read our clarification and respond. We wanted to share a brief update on the refusal mechanism you discussed in your review.

Another reviewer asked whether the Atomic candidate stage could miss conflicts that are visible in the restored source turns. To address this, we revised the controller so that both Atomic memories and restored source turns can propose conflicts (Section 3.3 and Algorithm 2). The source-turn view brings details such as speaker identity and timing into candidate generation. A shared verifier checks the proposed conflicts against the original turns. Forced refusal requires a verified target-frame substitution with no supporting evidence.

We evaluated this change on LoCoMo with retrieval results, answer drafts, and judge decisions fixed.

MetricSubmitted versionRevised version
Answer accuracy82.88% (1,646/1,986)85.25% (1,693/1,986)
Final refusal recall63.2%77.8%
Final refusal F173.3%82.1%
Controller false-refusal rate0.78% (12/1,540)1.49% (23/1,540)

The net benefit you highlighted in your review increased from 160 to 207 additional correct answers relative to the same refusal-off baseline. The revised controller corrected 223 answers and introduced 16 new errors. On LongMemEval-S, replaying the decisions from the two independently verified views yielded 85.4% accuracy (427/500), with no controller-forced refusals on answerable questions.

If you have any remaining questions or concerns, we would be happy to discuss them. If you feel the revision addresses the issues you raised, we would be grateful if you could reconsider your rating in light of the revised manuscript and the additional results. Thank you again for your time and feedback.

Reviewer_UvFZOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GroundedMem, a memory system for long conversations. Its main point is that a related memory is not always good evidence: it may be about a different person, time, or event, and failing to find evidence does not mean the answer does not exist. The system has three steps. First, it searches both short Atomic memories and the original conversation turns, then combines and reranks the results. Second, it brings back the full original turns, including speaker and time, and uses them to answer. Third, a checker forces a refusal only when it finds a clear conflict, such as a swapped person or target, and no supporting evidence. The paper reports strong results on LoCoMo and LongMemEval-S, and reports that the refusal step fixes many answers on LoCoMo while causing few new errors.

Critical strengths

Clear problem framing. The paper makes a useful point: a related memory is not always good evidence, and “not found” is not the same as “does not exist.” This is an important distinction for conversational memory.

Traceable use of memory. Atomic memories retain links to their original turns. This allows the system to recover the source directly and makes it easier to inspect what information was provided to the answer model.

Useful evaluation of the complete pipeline. The paper includes controlled comparisons with fixed answer settings and reports both corrections and new errors caused by refusal. This provides evidence for the combined system’s usefulness, even though it does not isolate every design choice.

Critical weaknesses

Source restoration is not directly tested. Both systems in the controlled comparison answer from original turns, so it tests changes in retrieval rather than the choice of answer evidence. Compact memories may combine scattered facts and remove noise, but the trade-off between these benefits and source restoration is not evaluated.

Refusal candidates may miss conflicts. Candidates come only from Atomic memories, which may lose context. The verifier cannot detect a conflict that was never proposed, and it is unclear how often this limits refusal.

The generality of refusal remains unclear. The verifier and decision rule are shared, but candidate adapters differ across benchmarks. LoCoMo also has a dedicated role-mismatch step. The gains may therefore depend partly on benchmark-specific design.

Questions for authors

Among the existing missed refusals, are there cases where a relevant conflict is visible in the restored turns but is missed by the Atomic candidate stage? A few examples would help clarify whether candidate selection is an important bottleneck.

4 comments
AuthorsOpenReview ↗

Source Restoration and Refusal Candidate Analysis

Thank you for the careful diagnosis and for recognizing both the traceability benefit and the unverified boundary. We agree that the original paper did not independently test source restoration and that the verifier cannot recover a conflict never proposed by the candidate stage. We directly tested both issues.

Q1: Is source restoration directly evaluated? It is now, and the result is mixed rather than uniformly favorable. We fix source coordinates, order, Top-K, timestamps, memory IDs, Answer model/prompt, structured runner, and Judge. Raw presents complete original turns; Atomic changes only the evidence text to all deduplicated summaries linked to the same coordinates. Sampling uses no answers, Judge outputs, prior correctness, or retrieval recall.

BenchmarkFully mappedRaw turnsAtomic summariesBetter/differencePaired gains/losses
LongMemEval-S422352 (83.41%)334 (79.15%)Raw, +18 (+4.27 pt)Raw: 35/17
LoCoMo973813 (83.56%)830 (85.30%)Atomic, +17 (+1.75 pt)Atomic: 56/39

Complete turns help significantly on LongMemEval-S but not on LoCoMo. We therefore do not claim that restoration universally improves accuracy. Its guaranteed benefits are source identity, speaker/time context, and auditability; its accuracy effect is benchmark-dependent.

Q2: Are there missed refusals where the conflict is visible in restored turns but omitted by the Atomic candidate stage? Yes; candidate generation is the largest observed bottleneck. We audited all 237 LoCoMo controller false negatives from the paper-reported run while blinded to system-stage keys; two independent experts confirmed the stage-level labels.

Diagnosis after unblindingCountShare of 237 FNsInterpretation
Conflict visible; no candidate generated13456.5%Candidate-generation bottleneck
Candidate generated and conflict visible; verifier failed open5623.6%Verifier bottleneck
Positive support exists3313.9%Appropriate support veto
No explicit conflict in E_K145.9%Conflict may still exist in full history

Representative candidate misses are:

Query frameRestored evidenceMissed substitution
How often did Caroline and her children go to the beach?“Once or twice a year” describes Melanie and her children.Subject
What did Maria donate to a luxury store in Dec. 2023?Maria donated her old car to a homeless shelter in Dec. 2022.Time/location
What was the name of the kitten James adopted?James adopted a dog named Ned.Object type

These cases directly confirm the reviewer’s concern: restored turns expose conflicts that the Atomic candidate stage does not propose. We therefore replaced the paper’s Atomic candidate stage with a source-turn candidate stage and ran full evaluations:

Benchmark/versionAnswer accuracyFinal refusalsPrecisionRecallRefusal F1Final false-refusal rate
LoCoMo, paper82.90%32387.30%63.20%73.30%2.66%
LoCoMo, fresh source-turn-only85.95%44782.33%82.51%82.42%5.13%
LongMemEval-S, paper85.40%2770.37%63.33%66.67%1.70%
LongMemEval-S, fresh source-turn-only84.00%3852.63%66.67%58.82%3.83%

On LoCoMo, source-turn-only improves answer accuracy by 3.05 points, refusal recall by 19.31 points, and refusal F1 by 9.12 points, confirming that Atomic-only candidate generation is an important bottleneck. The trade-off is lower precision and a higher final false-refusal rate (2.66%→5.13%). On LongMemEval-S, recall increases slightly, but answer accuracy and refusal F1 do not improve. We therefore view source-turn candidate generation as an effective repair for the observed LoCoMo bottleneck, not as a universally superior replacement.

Most of my concerns are addressed. However, could you answer my concern regarding W3? Thank you.

AuthorsOpenReview ↗

W3:Cross-Benchmark Generality of the Candidate Adapter

Thank you for raising this important concern. We understand the reviewer's concern: although the verifier and final decision rule in the original manuscript are shared, describing separate candidate adapters for different benchmarks, particularly the role-mismatch proposal dedicated to LoCoMo, does indeed weaken the generality of the refusal module. This comment motivated us to further unify and evaluate the candidate-generation process.

Specifically, inspired by your W2 question—whether source turns can recover conflicts missed by the Atomic candidate stage—we developed a dual-view candidate generator on LongMemEval-S and transferred it unchanged to LoCoMo for a fixed-evidence controller-only evaluation. The dual-view candidate generator is structured as follows:

G(q,BM,EK)=GA(q,BM)∪GS(q,EK),G(q,B_M,E_K)=G_A(q,B_M)\cup G_S(q,E_K),

where GAG_A identifies cross-turn and false-premise conflicts from Atomic memories, while GSG_S examines speaker, time, object, and event context in the restored complete source turns. The two views only propose candidates, which are then uniformly validated by the shared source-grounded verifier VV; the system refuses only when TARGET_FRAME_SUBSTITUTION is confirmed and no supporting evidence exists. The same proposer, verifier, and decision policy are used on both benchmarks, without relying on benchmark identity, labels, or reference answers.

Benchmark / configurationCorrectAccuracyController TP / FP
LongMemEval-S, paper chain427 / 50085.40%5 / 0
LongMemEval-S, unified dual-view GG427 / 50085.40%5 / 0
LoCoMo, paper chain1,646 / 1,98682.88%209 / 12
LoCoMo, unified dual-view GG1,693 / 1,98685.25%284 / 23

In addition, following the same protocol used for the comparison with REMem in the paper, we report refusal quality over all 1,986 LoCoMo questions:

MethodRefusalsPrecisionRecallF1False-refusal rate
GroundedMem, paper chain32387.30%63.20%73.30%2.66%
GroundedMem, unified dual-view GG39986.97%77.80%82.13%3.38%

While maintaining nearly the same precision, the unified dual-view pipeline improves recall by 14.60 percentage points and F1 by 8.83 percentage points; correspondingly, the final false-refusal rate increases from 2.66% to 3.38% (+0.72 percentage points). Meanwhile, the pipeline fully preserves the paper's results on LongMemEval-S without introducing any new false refusals.

We again thank the reviewer for identifying this issue in the original manuscript. In the revision, we will update the corresponding descriptions of the refusal module in Section 3 and the appendix, clarify that both benchmarks share the same dual-view candidate generation, source-grounded verification, and decision process, and update the experimental results.

Thanks. While I do think the generalization should be tested on more benchmarks, I am truly aware that there do not exist many good benchmarks for us to use in this area.

You've addressed all of my concerns. I'll raise the score from 2 to 3. good luck :)

Reviewer_8EkmOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The GroundedMem paper addresses a central question in long-term conversation: does the retrieved memory actually contain the evidence needed to answer the question?

They identify two places where this can fail:

  • Compact atomic memories, which might lose context.
  • Direct top-K retrieval, which may miss the fact.

Their solution is a pipeline split into three parts:

  1. Discovery: This stage combines multiple routes. I will explain them individually, following the flow: Atomic route: uses the top-30 memories to retrieve source turns in memory-ranked order. Direct route: fuses BM25 and dense rankings of the original turns using equal-weight RRF with a constant of 60. The Direct and Atomic routes are then fused with equal-weight RRF (constant 60). EvoEmbedding reranks the resulting top-100. The final score retains the Atomic prior. Returns the top-K complete turns (K depends on the dataset).

  2. Grounding: Each selected turn includes its complete text, timestamp, etc., and a shared formatter creates the answer payload.

  3. Decision: The answer agent produces a draft y0 from the selected complete source turns. A benchmark-specific candidate adapter produces a certificate c_b from a bounded atomic set. Verifier: a shared, source-grounded verifier returns the verdict, conflict type, and support IDs. A given hard policy, developed on LongMemEval and transferred to LoCoMo.

Results: In the paired comparison with the refusal controller disabled, the complete path shows a good accuracy gain on LoCoMo. On LongMemEval-S, statistical significance seems to be a problem.

Critical strengths

The paper separates retrieval relevance from evidence support by identifying two distinct failure points: missing details in atomic memories and a missing answer in the top-K. This is a good conceptual contribution, on which they build a decision layer.

Critical weaknesses
  1. The main text should be more descriptive; it leaves many operational details to the appendix, for example, the construction of the candidate set c_b, and the same applies to other notation. For a paper where an auditable pipeline is one of the main focuses, the main text must make more effort to explain what these components are and how they work.

  2. The novelty lies in the combination of two routes (RRF) and a single refusal rule, which seems to have been developed on one dataset and transferred to another. Overall, the novelty seems narrow.

  3. Is it possible to isolate the effects of the two components? The paper does acknowledge this.

Questions for authors
  1. Can you ablate the two RRF fusions separately, i.e., the inner BM25 + dense fusion in the Direct route and the outer Atomic + Direct source-turn fusion? Also, can you confirm that the LoCoMo refusal ablation in Table 4 holds the retrieved K fixed, and then extend the same evidence-fixed refusal toggle to LongMemEval-S and to new outputs?

  2. Can you explain the notation used in Section 3 in more detail? It is difficult to understand the individual components without revisiting the appendix multiple times.

3 comments
AuthorsOpenReview ↗

RRF Ablations, Fresh Validation, and Method Clarification

Thank you for analyzing our pipeline and recognizing the distinction between retrieval relevance and evidential support. We agree that Section 3 was unclear and did not sufficiently isolate component effects.

Q1: Can the two RRF stages be ablated separately? Yes. We conducted a factorial retrieval evaluation, fixing the Atomic store/ranking, candidate/reranking and final evidence budgets, EvoEmbedding reranker, and label-isolation protocol.

Benchmark/sampleBM25+EvoDense+EvoInner BM25+dense RRF+Evo+outer Atomic RRF+Atomic prior
LoCoMo, full recall@15, n=1,53766.10%73.06%76.77%81.46%82.17%
LongMemEval-S, full recall@10, n=47081.91%91.91%90.85%90.85%91.91%
Paired comparisonLoCoMoLongMemEval-S
Inner RRF vs. BM25+10.67 pt+8.94 pt
Inner RRF vs. dense+3.71 pt-1.06 pt
Outer RRF vs. inner RRF96 gains/24 losses6 gains/6 losses
Atomic prior vs. outer RRF27 gains/16 losses9 gains/4 losses

Thus, both RRF stages help on LoCoMo. On LongMemEval-S, however, the dense route is already strong: inner RRF does not outperform dense retrieval, and outer RRF has no net gain at K=10.

Q2: Does Table 4 hold K fixed, and does the effect extend to LongMemEval-S and newly generated outputs? Yes, K is fixed, and we completed the requested fresh-generation experiment. Table 4 fixes LoCoMo Top-15 evidence, the Answer model/prompt, and the Judge. For each question, the fresh run generates one new draft, y0, from the same fixed E_K. Off retains y0; On uses the same E_K and candidate certificate and may only replace that identical y0 with the canonical refusal.

The paper and fresh runs are closely consistent under the same fixed-K intervention:

BenchmarkPaper Off→OnFresh Off→OnPaper wrong→right/right→wrongFresh wrong→right/right→wrongGain gap
LoCoMo (K=15, n=1,986)74.82→82.88% (+8.06)75.53→82.78% (+7.25)167/7150/60.81 pt
LongMemEval-S (K=10, n=500)84.40→85.40% (+1.00)83.60→84.40% (+0.80)5/05/10.20 pt
BenchmarkPaper controller false-refusalFresh controller false-refusalPaper induced-errorFresh induced-error
LoCoMo12/1,540=0.78%11/1,540=0.71%7/1,540=0.45%6/1,540=0.39%
LongMemEval-S0/470=0.00%3/470=0.64%0/470=0.00%1/470=0.21%

All paper–fresh differences in Off/On accuracy, intervention gain, controller false-refusal rate, and induced-error rate are within 1.00 percentage point. This is reasonable run-to-run variation from fresh drafts and Judge decisions, not systematic bias. The large LoCoMo gain is reproduced; LongMemEval-S remains small and statistically non-significant.

Q3: What do the Section 3 symbols mean operationally? Let Hu=(h1,…,hN)H_u=(h_1,\ldots,h_N) be user uu's timestamped history and Mu={mj}j=1LM_u=\{m_j\}_{j=1}^{L} the Atomic-memory store. Each mjm_j records source coordinates through π(mj)\pi(m_j). The Atomic and direct-turn rankings are RM(q)R_M(q) and RH(q)R_H(q); Resolve⁡(RM(q),π)\operatorname{Resolve}(R_M(q),\pi) restores the Atomic route to source turns. Fusion, EvoEmbedding reranking, and the Atomic prior produce EK(q)E_K(q), the final KK complete source turns with speaker/time. For benchmark bb, candidate adapter GbG_b and task contract τb\tau_b produce certificate cbc_b from bounded Atomic set BM(q)B_M(q); cbc_b is not a candidate answer. The Answer Agent produces draft y0y_0 from EKE_K. The shared verifier returns vb=V(q,τb,cb,y0,EK)v_b=V(q,\tau_b,c_b,y_0,E_K), with νb\nu_b, κb\kappa_b, and SbS_b denoting its verdict, conflict type, and support IDs. The decision db(q,cb,y0,EK)d_b(q,c_b,y_0,E_K) outputs canonical refusal ⊥\bot only when all five conditions hold: (i) cb≠∅c_b\neq\varnothing; (ii) νb\nu_b is VERIFIED_CONFLICT; (iii) κb\kappa_b is TARGET_FRAME_SUBSTITUTION; (iv) Sb=∅S_b=\varnothing; and (v) op⁡(τb)∉A\operatorname{op}(\tau_b)\notin\mathcal{A}. Otherwise, it keeps y0y_0. Thus positive support vetoes refusal and missing or unresolved evidence fails open. We will move this explanation into the main text.

AuthorsOpenReview ↗

Revised Method Section and Paired Ablations

Thank you for your detailed review and for pointing out where the method was difficult to follow. We have uploaded a revised manuscript and wanted to briefly highlight the changes most relevant to your questions.

Section 3 now explains the pipeline in the main text. Algorithm 1 shows where the two RRF stages enter retrieval. Section 3.3 and Algorithm 2 explain how Atomic memories and restored source turns propose conflict candidates, what the verifier checks, and when the controller keeps the draft or forces a refusal. We also added a worked example of a speaker mismatch and explained how evidence supporting the requested fact vetoes refusal.

Appendix B.1 now reports paired fusion-stage comparisons. At the main retrieval budgets, outer fusion produces 96 gains and 24 losses in full source-turn coverage on LoCoMo, and 6 gains and 6 losses on LongMemEval-S. The appendix also reports the inner BM25+dense fusion comparisons on LongMemEval-S. The revised LoCoMo recall tables use the submission’s 1,535-question cohort with valid source annotations.

The revised refusal results use fixed evidence. The fresh-generation check in our earlier reply used the submitted controller. The revised Table 4 evaluates the dual-view controller while holding the retrieved Top-15 evidence, answer drafts, and judge decisions fixed on LoCoMo. Section 4.4 and Appendix B.4 also report the LongMemEval-S replay using Top-10 evidence and merging decisions from the two independently verified views.

BenchmarkRefusal offRevised refusalCorrections / new errors
LoCoMo74.82%85.25%223 / 16
LongMemEval-S84.40%85.40%5 / 0

On LoCoMo, the revised controller forcibly refuses 23 of 1,540 answerable questions (1.49%). On LongMemEval-S, the gain is small (exact paired p = 0.0625).

If there are any points you would like us to clarify further, we would be happy to discuss them. If the revision helps address your concerns, we would be grateful if you could take these updates into account when revisiting your assessment. Thank you again for your time and feedback.

Thank you for the additional experiments and clarifications. The paired ablations and fixed-evidence comparisons largely address my concern about isolating the components. However, my concern about the limited methodological novelty remains: the contribution primarily combines established retrieval techniques with a targeted refusal controller. While the response strengthens the experimental support, it does not sufficiently change my assessment of the contribution. I am therefore keeping my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.