acceptodds
Under review as a conference paper at ICLR 2027

RaMem: Contextual Reinstatement for Long-term Agentic Memory

Abstract

Long-term memory has become increasingly important for LLM agents that operate across extended interactions and evolving task contexts. Recent memory systems have made past experiences more persistent, compact, and retrievable, yet memory availability does not guarantee that the intended past episode remains identifiable. When semantically similar experiences from different episodes are compressed into reusable fragments, content-centric recall can marginalize the contextual distinctions that separate them. We refer to this failure as context collapse: episodically distinct memories become aliased under content-based retrieval, allowing contextually incompatible memories to compete with the intended trace. To address this problem, we propose Contextual Reinstatement for Agentic Memory (RaMem), which restores long-term recall in a joint content–context space. RaMem represents each memory as content bound to the episodic coordinates under which it was observed, and uses cues in the current query to reinstate the contextual coordinates needed to disambiguate recall. Because such cues may be incomplete or uncertain, Conservative Contextual Reinstatement activates contextual constraints only when they can be reliably grounded. RaMem then performs Validity-Constrained Recall, conditioning semantic relevance on compatibility with the reinstated context while retaining content-relevant memories as fallback, and preserves the resulting content–context binding during downstream reasoning. Experiments on long-term memory benchmarks show that RaMem consistently outperforms strong memory baselines, with average F1 gains exceeding 10% across several backbones.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agentic Memory, Large Language Models, Contextual Reinstatement

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_B9LHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the confusion between distinct episodes in long-term agent memory. Semantically related memories may originate from different times, so retrieving relevant content does not necessarily recover the episode intended by the query. The authors call this phenomenon “Context Collapse” and propose RaMem, which binds memory content to contextual information such as timestamps and sessions. When a query contains reliable contextual cues, RaMem prioritizes context-compatible memories and preserves their contextual information during answer generation. The paper reports improvements on LoCoMo and LongMemEval-S and examines the role of context through retrieval diagnostics, context-field shuffling, component ablations, and retrieval-budget experiments.

Critical strengths

The paper addresses a concrete and practically meaningful problem in long-term memory. When the same people, activities, or preferences recur across sessions, semantic relevance alone may not identify the episode needed to answer the current question. By distinguishing content relevance from contextual compatibility, the paper offers a useful perspective on why memory systems fail. The ablation studies are well designed. The authors examine target-memory rankings, interference from incorrect episodes, and the effects of disrupting content–context bindings. These analyses provide relatively direct support for preserving contextual information through the generation stage.

Critical weaknesses

However, the paper has the following weaknesses: The method offers limited novelty over existing memory retrieval systems. RaMem primarily preserves structured context, extracts temporal conditions from queries, prioritizes memories satisfying those conditions, and retains metadata during generation. These operations closely resemble existing temporal retrieval and structured RAG approaches. The authors do not clearly explain what new technical mechanism the framework introduces or which problems it addresses that existing methods fundamentally struggle to handle. The distinction between Context Collapse and previously studied problems is unclear. The paper uses this term to describe interference from semantically related memories that do not match the required time or session. However, it remains unclear how this differs substantively from retrieval ambiguity, omitted temporal constraints, or confusion between evidence sources. The paper does not establish that “Context Collapse” reveals a more general failure mechanism in memory systems. The aggregation used in the main performance comparison is unclear, weakening confidence in the conclusions. In Table 1, SimpleMem’s Average F1 matches the arithmetic mean across the four task categories, whereas RaMem’s reported values do not follow this calculation. If different methods use different averaging schemes, their differences cannot directly support the improvement margins claimed in the main text.

Questions for authors

When a query lacks explicit temporal cues, or when an event occurred long before the session in which it was mentioned, how does RaMem identify the correct episode? Do the existing experiments establish that the method remains effective in these situations? How is Average F1 calculated in Table 1? Are RaMem and all baselines evaluated on the same test examples using the same aggregation scheme? Do the main performance gains hold under a consistent evaluation protocol? Compared with SimpleMem with its temporal fields and structured retrieval fully enabled, what are RaMem’s key additional mechanisms? Are there controlled results that distinguish their contributions from the benefits of ordinary temporal retrieval, reranking, and prompt-format adjustments?

8 comments
AuthorsOpenReview ↗

Response to Reviewer B9LH

We sincerely appreciate your careful review and insightful suggestions. Your constructive feedback has been invaluable in helping us strengthen the technical justification, and more clearly articulate the novelty and contributions of our work. We hope that our detailed responses below, together with the updated paper draft, adequately address your concerns and provide a clearer and more convincing account of our contributions.


Reply to Weakness

Reply to W1 -- Methodological Novelty Beyond Conventional Temporal Retrieval

We sincerely thank you for this important observation. We agree that temporal filtering, structured metadata, and relevance fusion are established techniques. RaMem does not claim novelty in these individual operations. Rather, our contribution is a unified retrieval formulation that explicitly addresses episodic identifiability, separating contextual validity from semantic relevance and using reinstated context to guide candidate discovery rather than merely reranking existing memories.

1. Beyond Temporal Reranking: Context-Conditioned Candidate Discovery.

The key distinction is that RaMem changes where retrieval searches, not merely how retrieved memories are scored. Conventional relevance-based reranking operates on a fixed content-retrieved candidate pool and therefore cannot recover target episodes missing from that pool. RaMem instead performs an additional search within the context-compatible memory region, prioritizes those candidates, and retains content-relevant candidates as fallback.

To directly validate this distinction, we conducted additional controlled comparisons against content-only retrieval, hard temporal filtering, and tuned soft temporal reranking, using identical memory stores, embeddings, query grounding, and retrieval budgets.

On 203 grounded LoCoMo queries with verified ground truth (Qwen3-8B):

  • Recovering previously inaccessible episodes: In 45/203 queries (22.2%), the target memory is discovered by context-conditioned retrieval but absent from the content-only top-25 candidates.
  • Beyond soft temporal reranking: RaMem improves candidate-level ground-truth recall by 14.8 percentage points (95% CI [10.3, 19.7]) and achieves +5.9 points in Recall@10 (95% CI [1.6, 10.2]) on correctly grounded queries.
  • Preserving candidate coverage: Unlike hard temporal exclusion, RaMem retains relevance-based fallback candidates. Under deliberately shifted temporal windows (−7 days), this preserves 33.0 points more target-memory coverage (95% CI [26.6, 39.4]), although fallback does not eliminate degradation in the top-ranked results.

These findings establish that context-conditioned candidate discovery provides measurable benefits beyond relevance-based temporal reranking, while semantic fallback preserves additional evidence coverage when contextual estimates are imperfect.

2. Distinction from Existing Memory Systems.

Our contribution is complementary to existing structured memory approaches. SimpleMem emphasizes efficient memory compression and multi-view retrieval; Zep organizes evolving information through temporal knowledge graphs; and REMem investigates graph-based episodic reasoning and recollection. RaMem instead focuses on a specific retrieval failure: semantically similar memories may remain accessible while the intended episode is not identifiable from content alone.

Accordingly, RaMem explicitly formulates contextual admissibility as a retrieval criterion, integrates grounded contextual candidate discovery with conservative activation and relevance-based fallback, and preserves episodic binding through downstream reasoning. Our existing context-shuffling and component ablations further substantiate the roles of these mechanisms. The theoretical interpretation added in Appendix B.5 also connects this formulation to established principles of episodic memory and probabilistic retrieval.

We have added these additional temporal retrieval comparisons to Appendix C.11 (Comparison with Conventional Temporal Retrieval), providing empirical evidence for the distinct role of context-conditioned candidate discovery and clarifying RaMem's methodological contributions beyond conventional temporal filtering and reranking. We sincerely appreciate your feedback, which has helped us articulate more precisely what RaMem contributes beyond existing retrieval components and why context-conditioned candidate discovery matters for long-term episodic recall.

Reply to W2 -- Distinguishing Context Collapse from Existing Retrieval Ambiguity

We sincerely thank you for raising this important conceptual question. We agree that context collapse is related to previously studied retrieval ambiguity, temporal retrieval, and source confusion. However, our contribution is to identify and formally characterize a specific failure mechanism underlying these phenomena: the loss of episodic identifiability when retrieval relies primarily on memory content.

1. A Distinct and Formally Characterized Failure Mechanism. Context collapse is not intended as a general label for retrieval errors. As formalized in Section 3.1, it occurs when two episodically distinct memories are semantically similar, yet differ in their contextual admissibility for a given query:

$ d_{\mathcal X}(x_i,x_j)\leq\epsilon,\qquad A_q(e_i)\neq A_q(e_j). $

Unlike general retrieval ambiguity, this condition specifically identifies a mismatch between content similarity and episodic identity. Even when relevant memories remain available, content-oriented retrieval may fail to distinguish the intended episode because the necessary contextual distinctions are not incorporated into the retrieval decision. Thus, omitted temporal constraints represent one manifestation of context collapse, rather than its complete definition. Similarly, source confusion is related but does not specifically characterize this retrieval-level failure.

2. Direct Evidence Beyond Conventional Retrieval Errors. Importantly, our experiments explicitly isolate and validate this mechanism:

  • Wrong-episode interference (Appendix C.3): On context-confusable queries, SimpleMem retrieves contextually incompatible memories at rank 1 in 43.66%–54.14% of cases across three backbones, despite their semantic relatedness.
  • Content–context binding intervention (Appendix C.4): Keeping memory content and embeddings unchanged while shuffling temporal coordinates reduces GT Recall@10 from 80.66% to 23.58% (GPT-4.1-mini) and 71.70% to 18.40% (Qwen3-8B) on temporally grounded queries. This directly demonstrates the importance of correct episodic binding beyond semantic content.
  • Candidate identifiability (new Appendix C.11): In 45/203 grounded queries (22.2%), the target episode is absent from content-only top-25 candidates but recovered through context-conditioned retrieval. This shows that the problem extends beyond reranking existing candidates.

3. Broader Interpretation and Scope. The formulation applies whenever content-similar memories differ in query-relevant contextual coordinates, potentially including time, session, participants, or location. Our current implementation and strongest empirical evidence focus on session-level temporal disambiguation, rather than claiming comprehensive validation across every contextual dimension. We have also added a theoretical interpretation in Appendix B.5, connecting episodic identifiability to established principles of encoding specificity and probabilistic retrieval.

Therefore, the novelty of Context Collapse lies not in claiming that temporal ambiguity was previously unknown, but in providing a precise computational characterization, targeted diagnostics, and controlled evidence of how episodic distinctions are lost and can be reinstated during retrieval.

We sincerely appreciate your thoughtful feedback, which has helped us clarify this distinction and more precisely articulate the scope and significance of our contribution.

Reply to W3 -- Clarification and Correction of Performance Aggregation

We sincerely thank you for identifying this inconsistency in Table 1. Your observation is correct: the baseline results adopted from SimpleMem use the arithmetic mean across four question categories (macro-average), whereas our evaluation script reports RaMem's average over individual questions (question-weighted average). We apologize for this reporting inconsistency and have recomputed the results using a unified aggregation scheme.

1. Corrected Results under Consistent Aggregation. Importantly, the correction changes the magnitude of the reported improvements but does not alter the central empirical conclusion. Using the same category-macro averaging for all methods, RaMem continues to achieve the highest average F1 across all four backbones:

BackboneSimpleMem F1RaMem F1 (Corrected)Improvement
GPT-4o39.0646.46+7.40
GPT-4.1-mini43.2447.45+4.21
Qwen3-8B33.4537.14+3.70
Qwen2.5-3B17.9821.71+3.73

Notably, RaMem outperforms SimpleMem in all 16 backbone–category F1 comparisons, independently of the averaging scheme. It also retains the highest macro-average F1 among all nine compared methods on every backbone.

2. Independent Paired Validation. To further verify that these improvements are not artifacts of aggregation or differing evaluation implementations, we additionally examined a controlled comparison on 1,540 paired LoCoMo questions using Qwen3-8B, with identical memory stores, prompts, and metric implementations. Against a matched SimpleMem-style baseline, RaMem improves macro-F1 by 3.53 points (95% CI [1.83, 5.20]) and question-weighted F1 by 4.29 points (95% CI [2.98, 5.59]). The improvements are also positive across all 10 conversations, providing complementary evidence under a shared evaluation setup.

We sincerely appreciate your careful observation, which has helped us improve the accuracy and transparency of our empirical comparisons.

Reviewer_PvrnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses a failure mode in long-term memory for LLM-based agents that the authors term context collapse: when episodically distinct yet semantically similar memories are compressed into reusable fragments, content-centric retrieval can alias them together, allowing contextually incompatible memories to compete with the intended trace. To mitigate this, the authors propose RaMem, or Contextual Reinstatement for Agentic Memory, which represents each memory as a content–context tuple whose episodic coordinates, such as time, session, participants, location, and entities, explicitly encode the distinguishing context. At retrieval time, RaMem applies three principles: Conservative Contextual Reinstatement, reconstructing query-relevant context only when the query's cues are reliably groundable; Validity-Constrained Recall, prioritizing semantic relevance conditioned on compatibility with the reinstated context; and Episodic Binding Preservation, retaining the content–context binding throughout downstream reasoning. Across multiple backbone models on conversational long-term memory benchmarks, RaMem consistently outperforms strong content-centric baselines on both answer quality and retrieval diagnostics, and improves retrieval efficiency, suggesting that reinstating rather than merely compressing context is a key complement to agentic memory design.

Critical strengths
  • Clear problem identification and formulation. The paper identifies and formally defines context collapse, a real yet previously unnamed failure mode of long-term agentic memory, distinguishing between whether a memory is retrievable and whether the intended episode is identifiable. The case studies make the failure concrete, and the connection to encoding specificity in cognitive science grounds the motivation.

  • Comprehensive and rigorous experiments. The evaluation spans four backbones, four question categories, and a rich set of retrieval diagnostics. The context-shuffle ablation isolates content–context binding as the source of gains, and the budget-efficiency analysis rules out the trivial explanation that the method simply retrieves more.

Critical weaknesses
  1. Heuristic pipeline without deeper theoretical insight. Designs such as "validity precedes relevance" and tolerance for mild temporal misalignment are empirically motivated, but the paper lacks a formal analysis of when contextual reinstatement is an optimal retrieval strategy. Stronger ties to established information retrieval theory or a theoretical characterization of context identifiability would make the contribution more convincing.

  2. Limited gains on ambiguous queries, which are common in practice. The authors acknowledge that RaMem falls back to content-centric recall when g_q = 1 fails. Since real queries often use vague or relative temporal expressions, the absence of standalone results on the unreliable-context subset makes it unclear what fraction of real-world queries actually benefits from the method.

  3. Sensitivity to LLM-extracted metadata is not evaluated. The approach assumes reliable extraction of event time, mention time, session span, participants, entities, and topics, yet robustness to extraction errors such as temporal miscalculation or entity resolution mistakes is not analyzed. The cue-guard ablation only tests whether to use the cues, not what happens when they are wrong, so errors could cascade through the reinstatement pipeline.

Questions for authors

Regarding storage cost: each memory stores a lossless restatement plus at least seven types of structured fields. How much does this increase storage and indexing overhead compared to SimpleMem? At million-scale memory sizes, are the latency and memory footprint of joint-space retrieval acceptable?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Pvrn

We sincerely thank you for your thoughtful review and for recognizing the significance of context collapse and the value of our experimental analyses. We appreciate your constructive suggestions regarding the theoretical foundation, and practical scalability of RaMem. We hope the following responses and additional evidence help address your concerns and clarify the contributions of our work.

Reply to Weakness

Reply to W1 -- Theoretical Interpretation of Contextual Reinstatement

We sincerely thank you for this insightful suggestion. We agree that a stronger connection to established theoretical foundations can help clarify when contextual reinstatement is beneficial and why validity-constrained recall is a reasonable retrieval strategy.

To address this, we have added a new theoretical discussion in Appendix B.5: Theoretical Interpretation of Contextual Reinstatement. Rather than claiming a new theoretical guarantee, we draw on established principles from episodic memory and information retrieval, particularly Encoding Specificity (Tulving & Thomson, 1973), Temporal Context Models (Howard & Kahana, 2002), and the Probability Ranking Principle (Robertson, 1977; Robertson & Zaragoza, 2009), to provide two complementary interpretations:

  1. Context Identifiability. We characterize why content-centric retrieval cannot always distinguish episodically distinct memories. When two memories share identical content representations but differ in contextual admissibility, any content-only scoring function necessarily assigns them identical relevance scores. This explains why stronger semantic matching alone cannot resolve context collapse and why episodic coordinates provide essential disambiguating information.

  2. Conditional Optimality of Validity-Constrained Recall. Following the Probability Ranking Principle, we provide a probabilistic interpretation showing that, when contextual cues are reliably grounded and semantic relevance scores preserve posterior ordering, prioritizing context-compatible memories before semantic relevance maximizes the probability of including the target memory within a fixed top-K retrieval budget. This interpretation also motivates RaMem's conservative activation and semantic fallback: when contextual cues are uncertain, imposing hard contextual constraints may exclude the correct memory.

We further clarify that temporal tolerance is an empirically calibrated mechanism, balancing the risk of excluding relevant episodes against admitting contextually incompatible memories, rather than a universally optimal parameter.

We believe this additional discussion in Appendix B.5 provides a more principled explanation of RaMem's design choices, while explicitly acknowledging the assumptions and limitations of the theoretical interpretation. We appreciate you for encouraging us to strengthen this connection.

Reply to W2 -- Contextual Grounding Coverage and Robustness to Ambiguous Queries

We sincerely thank you for raising this important concern. We agree that understanding when contextual reinstatement is effective, particularly under ambiguous or unreliable contextual cues, is essential for evaluating its practical applicability.

To address this, we conducted an additional query-stratified analysis on all 1,540 LoCoMo questions using Qwen3-8B, comparing RaMem against a matched SimpleMem-style baseline with the same memory store. We categorize queries according to their actual runtime grounding decisions and report the results below.

Query Subset# Queries (%)GT R@10 (Baseline → RaMem)LLM-Judge Acc. (Baseline → RaMem)
Grounded Context206 (13.4%)44.3 → 72.4 (+28.1)65.0 → 73.3 (+8.3)
Rejected Grounding (Fallback)9 (0.6%)66.7 → 66.7 (0.0)66.7 → 55.6 (−11.1)
No Explicit Cue1,325 (86.0%)64.5 → 69.1 (+4.6)68.5 → 68.2 (−0.3)

1. Significant Improvements with Grounded Context. RaMem activates contextual reinstatement only when query cues can be reliably resolved and matched to memory sessions. On these 206 queries, GT Recall@10 improves by 28.1 points (95% CI [21.2, 34.5]), while wrong-episode distractors at rank 1 decrease by 33.0 points. Importantly, LLM-judge accuracy also improves significantly by 8.3 points (95% CI [2.4, 14.6], p = 0.012). These findings provide direct evidence that contextual reinstatement substantially improves both episodic retrieval and downstream answer correctness when applicable.

2. Conservative Fallback under Unreliable Context. When contextual grounding cannot be established, RaMem does not impose potentially incorrect contextual constraints. For the 9 rejected-grounding queries, GT Recall@10 remains identical to the content-centric baseline (66.7%). Although this subset is too small to support broad conclusions, it confirms the intended fallback behavior. On queries without explicit contextual cues, LLM-judge accuracy remains comparable (68.2% vs. 68.5%), while retrieval recall improves modestly.

3. Applicability and Scope. These results clarify that RaMem's primary benefits are concentrated where reliable episodic cues are available, rather than uniformly distributed across all queries. The observed 13.4% activation rate reflects LoCoMo's question distribution and our current explicit temporal grounding mechanism; it should not be interpreted as the prevalence of contextual queries in real-world applications. Supporting broader implicit and relative contextual cues remains an important direction for future improvement.

Reply to W3 -- Robustness to Metadata Extraction Errors

We sincerely thank you for this insightful concern. We agree that the reliability of contextual metadata is essential, particularly because extraction errors may propagate through retrieval. To investigate this, we conducted a controlled metadata perturbation study using frozen Qwen3-8B memory stores and 203 grounded LoCoMo queries with verified ground-truth memories. We kept memory content, embeddings, retrieval budgets, and grounding decisions fixed, injecting errors exclusively into metadata during offline retrieval replay.

1. Clarifying Metadata Dependencies. Importantly, RaMem's contextual ranking does not require all metadata fields to be accurately extracted by an LLM. Our implementation primarily relies on LLM-resolved query time windows and memory session spans, where session timestamps originate from the conversation history rather than LLM-generated dates. Participants, entities, locations, and topics serve as auxiliary information for downstream generation but do not directly determine the top-25 retrieval ranking.

2. Robustness to Moderate Metadata Errors. The additional experiments reveal that RaMem tolerates small temporal inaccuracies, with performance degrading primarily under substantially incorrect contextual assignments.

Controlled PerturbationChange in GT Recall@10
Query time shifted by ±1–3 daysWithin ±1.5 pp (95% CIs include 0)
Memory session dates shifted by ±1–3 days (50% of entries)Within ±0.7 pp
30% of memories reassigned to adjacent sessions−2.3 pp
Query time shifted 1 day earlier, without temporal tolerance−30.1 pp
Query time shifted 1 day earlier, with ±5-day tolerance+1.5 pp

3. Why Temporal Tolerance Matters. Most notably, the ±5-day temporal buffer substantially mitigates the impact of small temporal estimation errors. Without this buffer, shifting the query window one day earlier reduces Recall@10 by 30.1 percentage points (95% CI [−36.5, −23.7]); with the buffer, the same perturbation produces no measurable degradation (+1.5 points, 95% CI [0.0, 3.5]). This provides direct experimental support for the temporal tolerance mechanism, beyond the original sensitivity analysis in Appendix C.5.

We also acknowledge that larger temporal errors remain challenging: shifting all grounded query windows seven days earlier reduces Recall@10 by 35.5 points. Thus, we do not claim robustness to arbitrary metadata corruption. Rather, these results establish that RaMem's contextual retrieval is resilient to mild temporal inaccuracies, while identifying the conditions under which substantial misgrounding can propagate into retrieval errors.

We have added this analysis to Appendix B.6 (Robustness to Contextual Metadata Errors), providing further empirical evidence on metadata reliability, temporal tolerance, and the limitations of contextual reinstatement under larger errors. We sincerely appreciate your suggestion, which has helped us strengthen the empirical characterization of RaMem's robustness.

Reviewer_QQ8SOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RaMem addresses context collapse, where retrieval returns semantically related memories from the wrong episode. It uses query cues to prioritize context-compatible memories and preserves their contextual information during answer generation. Evaluation on LoCoMo and LongMemEval shows improvements over the reported baselines.

Critical strengths

The method addresses a practical problem: semantic relevance alone may not identify the intended past experience. Using grounded contextual cues while retaining semantic fallback is a sensible design that can improve retrieval without additional model training.

Critical weaknesses

The main concern is the evaluation: it covers only two benchmarks and reports F1 and BLEU on LoCoMo without LLM-judge accuracy, making answer quality difficult to compare with recent work. The LongMemEval-S accuracy of 80.15% also needs stronger baseline comparisons, especially since other memory systems report results near 90%, though evaluation settings should be matched for a fair comparison.

Questions for authors
  1. Could you report LLM-judge accuracy on LoCoMo and extend the evaluation to another memory benchmark, such as BEAM?
  2. Could you compare against stronger recent memory systems on LongMemEval-S using the same answering model, prompts, and evaluation protocol to clarify the gap between the reported 80.15% and results near 90%?
7 comments
AuthorsOpenReview ↗

Response to Reviewer QQ8S

We sincerely thank you for recognizing the significance of context collapse and the motivation behind grounded contextual reinstatement. We sincerely appreciate your constructive suggestions regarding evaluation breadth and stronger baselines. We hope the following additional experiments and analyses help address your concerns and further clarify the contributions of our work.

1. LLM-Judge Accuracy on LoCoMo

We additionally evaluated all 1,540 LoCoMo questions using the established Mem0 LLM-judge protocol (GPT-4o-mini, temperature 0). Under a controlled Qwen3-8B setting with identical memory stores, RaMem achieves 69.37% accuracy versus 66.92% for the SimpleMem-style baseline. RaMem also shows positive trends in Temporal (+3.43 points) and OpenDomain (+4.17 points).

These findings complement our existing retrieval diagnostics, which demonstrate consistent improvements in ground-truth memory recall and ranking while reducing wrong-episode interference. Importantly, our primary contribution concerns episodic disambiguation during memory retrieval, which answer-level correctness alone does not fully characterize.

2. Generalization to BEAM

Following your suggestion, we evaluated RaMem on the complete BEAM-128K tier, comprising 400 questions across 20 conversations and 10 memory abilities, using the official evaluation protocol.

Without introducing benchmark-specific memory mechanisms, RaMem achieves an overall score of 28.83 versus 30.30 for the matched baseline, with no statistically significant difference. It shows positive trends on abilities particularly relevant to contextual reinstatement: Multi-Session Reasoning (+11.16 points), Temporal Reasoning (+5.00), and Knowledge Update (+4.38). These differences are not statistically significant, and we also observe limitations in abstention and contradiction resolution.

3. Stronger LongMemEval-S Comparisons

We appreciate your emphasis on comparisons with recent high-performing memory systems. Indeed, several systems have reported strong LongMemEval-S results, including Hindsight (Latimer et al., ACL 2026; 91.4% with Gemini 3 Pro), Mem0 Platform v3 (94.4%), and Raw-RAG (Bao et al., 2026; 87.53%–89.93% with GPT-4.1-mini).

Importantly, these reported results involve heterogeneous evaluation configurations, including different answering models, judging protocols, aggregation methods, and retrieval architectures. For instance, despite using the same GPT-4.1-mini reader, Raw-RAG employs GPT-4o-mini judging with repeated evaluations and question-weighted aggregation, whereas our evaluation follows the SimpleMem protocol with GPT-4.1-mini judging and six-category macro-averaging. Thus, the published numbers should not be interpreted as strictly controlled, directly comparable rankings.

Under our submitted evaluation protocol, RaMem improves LongMemEval-S accuracy from 76.87% to 80.15% over SimpleMem, with consistent improvements across all six memory categories. We acknowledge that the higher results reported by recent systems motivate further controlled comparisons, particularly with raw-history retrieval and stronger reranking strategies.

Nevertheless, RaMem investigates a distinct and complementary problem: identifying the contextually correct episode when semantically similar memories compete, rather than solely maximizing evidence coverage. Our context-collapse diagnostics and controlled content–context binding interventions directly substantiate this contribution.


We sincerely hope that our responses and additional experimental evidence have addressed your main concerns. We are truly grateful for your constructive feedback, which has helped us substantially strengthen the evaluation and presentation of our work. We would be very happy to provide further details or address any additional questions you may have, so please feel free to let us know.

We would also greatly appreciate it if you could reconsider your assessment and rating of our paper in light of these additional experiments and clarifications. Thank you again for your time and thoughtful consideration.

Thank you for the response and the additional experiments. The LLM-judge results on LoCoMo are helpful, but the comparison is limited to a SimpleMem-style baseline, the improvement is modest without significance testing, and the absolute accuracy remains well below recent memory systems. The new BEAM-128K results show no overall improvement over the matched baseline (28.83 vs. 30.30), which raises concerns about how well the method generalizes beyond the original benchmarks. For LongMemEval-S, I understand that published numbers come from heterogeneous settings, which is exactly why I asked for a controlled comparison with stronger recent systems under the same answering model and protocol; this comparison was not provided. While the problem of context collapse is interesting, my main concerns on evaluation remain, so I keep my original score.

We sincerely thank you for taking the time to carefully consider our previous response and for providing further feedback. We apologize that our earlier rebuttal did not fully address your concerns, particularly regarding controlled comparisons with stronger baselines and generalization beyond the original benchmarks. Your comments are well taken, and we truly appreciate your emphasis on rigorous and fair evaluation.

Motivated by your feedback, we have conducted substantial additional experiments and more in-depth analyses, including controlled comparisons with the official fully enabled SimpleMem and strong Raw-RAG implementations, as well as further evaluations on BEAM. These new results directly address several important gaps in our previous response, and we have incorporated the supporting evidence into the updated manuscript.

For clarity, our earlier LoCoMo LLM-judge comparison used an in-house SimpleMem-style baseline, whereas our new controlled evaluation uses the official, fully enabled SimpleMem implementation with identical answer prompts and retrieval budgets. Under this setting, RaMem achieves statistically significant improvements of +4.2 points overall (69.2% vs. 64.9%) and +16.0 points on context-grounded queries.

We sincerely hope you will take another look at these additional findings and consider them in reassessing our work. We genuinely value your judgment and would be extremely grateful if the new evidence could help address your concerns and encourage you to reconsider your original rating. We provide our detailed responses below.


Further Response -- Generalization to BEAM: Evidence Efficiency and Episodic Disambiguation

We sincerely appreciate your follow-up and agree that our previous BEAM results did not sufficiently establish the generalizability of RaMem. To address this concern more directly, we conducted two additional controlled evaluations on BEAM-128K, examining both performance under constrained evidence budgets and the ability to resolve context-confusable episodes in a different conversational corpus. We have included the complete results and experimental protocols in Appendix C.13 of the updated manuscript.

1. Significant Improvements on BEAM under Constrained Evidence Budgets.

We re-evaluated the complete BEAM-128K benchmark (400 questions, 20 conversations, 10 memory abilities) using identical task-aware answer prompts, the same Qwen3-8B backbone, and the official BEAM evaluation protocol. Both RaMem and the matched content-centric baseline use the same frozen memory stores, with only contextual reinstatement disabled in the baseline.

We further evaluated four pre-specified evidence budgets, preserving each method's original retrieval ranking and applying identical token-budget constraints. Unlike our earlier BEAM evaluation (28.83 vs. 30.30), which used a concise answer format, the new experiments adopt an identical task-aware answering prompt for both systems, resulting in comparable native-budget performance (48.9 vs. 48.9). We then keep this evaluation protocol fixed and vary only the evidence-token budget.

Evidence BudgetMatched BaselineRaMemImprovementEvidence Tokens (Baseline / RaMem)
1K41.442.8+1.4946 / 942
2K43.347.1+3.81,948 / 1,945
4K47.547.3−0.23,637 / 2,951
Native48.948.90.04,267 / 3,066

Importantly, at the 2K-token budget, RaMem significantly outperforms the baseline by +3.8 percentage points (95% CI [0.9, 6.8]; conversation-cluster CI [1.4, 6.3]). This improvement remains significant after Bonferroni correction across all four pre-specified budgets, while both methods consume nearly identical evidence tokens.

At the native budget, RaMem achieves the same overall BEAM score with approximately 28% fewer evidence tokens. Although the advantage is not uniform across budgets, these results provide direct evidence that RaMem offers a favorable accuracy–evidence-budget trade-off on an additional benchmark, consistent with our findings on LongMemEval-S.

Comments on the paper

1 comment
Area_Chair_fwZTOpenReview ↗

Discussion Window Closing in 2 days. Please Respond to the Author's Rebuttal!

Dear Reviewers,

The authors have thoughtfully responded to your reviews and the discussion window is coming to a close on November 18th. Please take a minute to engage with their response and discuss whether it changes your opinions.

Thank you for your time and effort!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.