- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the confusion between distinct episodes in long-term agent memory. Semantically related memories may originate from different times, so retrieving relevant content does not necessarily recover the episode intended by the query. The authors call this phenomenon “Context Collapse” and propose RaMem, which binds memory content to contextual information such as timestamps and sessions. When a query contains reliable contextual cues, RaMem prioritizes context-compatible memories and preserves their contextual information during answer generation. The paper reports improvements on LoCoMo and LongMemEval-S and examines the role of context through retrieval diagnostics, context-field shuffling, component ablations, and retrieval-budget experiments.
Critical strengths
The paper addresses a concrete and practically meaningful problem in long-term memory. When the same people, activities, or preferences recur across sessions, semantic relevance alone may not identify the episode needed to answer the current question. By distinguishing content relevance from contextual compatibility, the paper offers a useful perspective on why memory systems fail. The ablation studies are well designed. The authors examine target-memory rankings, interference from incorrect episodes, and the effects of disrupting content–context bindings. These analyses provide relatively direct support for preserving contextual information through the generation stage.
Critical weaknesses
However, the paper has the following weaknesses: The method offers limited novelty over existing memory retrieval systems. RaMem primarily preserves structured context, extracts temporal conditions from queries, prioritizes memories satisfying those conditions, and retains metadata during generation. These operations closely resemble existing temporal retrieval and structured RAG approaches. The authors do not clearly explain what new technical mechanism the framework introduces or which problems it addresses that existing methods fundamentally struggle to handle. The distinction between Context Collapse and previously studied problems is unclear. The paper uses this term to describe interference from semantically related memories that do not match the required time or session. However, it remains unclear how this differs substantively from retrieval ambiguity, omitted temporal constraints, or confusion between evidence sources. The paper does not establish that “Context Collapse” reveals a more general failure mechanism in memory systems. The aggregation used in the main performance comparison is unclear, weakening confidence in the conclusions. In Table 1, SimpleMem’s Average F1 matches the arithmetic mean across the four task categories, whereas RaMem’s reported values do not follow this calculation. If different methods use different averaging schemes, their differences cannot directly support the improvement margins claimed in the main text.
Questions for authors
When a query lacks explicit temporal cues, or when an event occurred long before the session in which it was mentioned, how does RaMem identify the correct episode? Do the existing experiments establish that the method remains effective in these situations? How is Average F1 calculated in Table 1? Are RaMem and all baselines evaluated on the same test examples using the same aggregation scheme? Do the main performance gains hold under a consistent evaluation protocol? Compared with SimpleMem with its temporal fields and structured retrieval fully enabled, what are RaMem’s key additional mechanisms? Are there controlled results that distinguish their contributions from the benefits of ordinary temporal retrieval, reranking, and prompt-format adjustments?
8 comments
Reply to W2 -- Distinguishing Context Collapse from Existing Retrieval Ambiguity
We sincerely thank you for raising this important conceptual question. We agree that context collapse is related to previously studied retrieval ambiguity, temporal retrieval, and source confusion. However, our contribution is to identify and formally characterize a specific failure mechanism underlying these phenomena: the loss of episodic identifiability when retrieval relies primarily on memory content.
1. A Distinct and Formally Characterized Failure Mechanism. Context collapse is not intended as a general label for retrieval errors. As formalized in Section 3.1, it occurs when two episodically distinct memories are semantically similar, yet differ in their contextual admissibility for a given query:
$ d_{\mathcal X}(x_i,x_j)\leq\epsilon,\qquad A_q(e_i)\neq A_q(e_j). $
Unlike general retrieval ambiguity, this condition specifically identifies a mismatch between content similarity and episodic identity. Even when relevant memories remain available, content-oriented retrieval may fail to distinguish the intended episode because the necessary contextual distinctions are not incorporated into the retrieval decision. Thus, omitted temporal constraints represent one manifestation of context collapse, rather than its complete definition. Similarly, source confusion is related but does not specifically characterize this retrieval-level failure.
2. Direct Evidence Beyond Conventional Retrieval Errors. Importantly, our experiments explicitly isolate and validate this mechanism:
- Wrong-episode interference (Appendix C.3): On context-confusable queries, SimpleMem retrieves contextually incompatible memories at rank 1 in 43.66%–54.14% of cases across three backbones, despite their semantic relatedness.
- Content–context binding intervention (Appendix C.4): Keeping memory content and embeddings unchanged while shuffling temporal coordinates reduces GT Recall@10 from 80.66% to 23.58% (GPT-4.1-mini) and 71.70% to 18.40% (Qwen3-8B) on temporally grounded queries. This directly demonstrates the importance of correct episodic binding beyond semantic content.
- Candidate identifiability (new Appendix C.11): In 45/203 grounded queries (22.2%), the target episode is absent from content-only top-25 candidates but recovered through context-conditioned retrieval. This shows that the problem extends beyond reranking existing candidates.
3. Broader Interpretation and Scope. The formulation applies whenever content-similar memories differ in query-relevant contextual coordinates, potentially including time, session, participants, or location. Our current implementation and strongest empirical evidence focus on session-level temporal disambiguation, rather than claiming comprehensive validation across every contextual dimension. We have also added a theoretical interpretation in Appendix B.5, connecting episodic identifiability to established principles of encoding specificity and probabilistic retrieval.
Therefore, the novelty of Context Collapse lies not in claiming that temporal ambiguity was previously unknown, but in providing a precise computational characterization, targeted diagnostics, and controlled evidence of how episodic distinctions are lost and can be reinstated during retrieval.
We sincerely appreciate your thoughtful feedback, which has helped us clarify this distinction and more precisely articulate the scope and significance of our contribution.
Reply to W3 -- Clarification and Correction of Performance Aggregation
We sincerely thank you for identifying this inconsistency in Table 1. Your observation is correct: the baseline results adopted from SimpleMem use the arithmetic mean across four question categories (macro-average), whereas our evaluation script reports RaMem's average over individual questions (question-weighted average). We apologize for this reporting inconsistency and have recomputed the results using a unified aggregation scheme.
1. Corrected Results under Consistent Aggregation. Importantly, the correction changes the magnitude of the reported improvements but does not alter the central empirical conclusion. Using the same category-macro averaging for all methods, RaMem continues to achieve the highest average F1 across all four backbones:
| Backbone | SimpleMem F1 | RaMem F1 (Corrected) | Improvement |
|---|---|---|---|
| GPT-4o | 39.06 | 46.46 | +7.40 |
| GPT-4.1-mini | 43.24 | 47.45 | +4.21 |
| Qwen3-8B | 33.45 | 37.14 | +3.70 |
| Qwen2.5-3B | 17.98 | 21.71 | +3.73 |
Notably, RaMem outperforms SimpleMem in all 16 backbone–category F1 comparisons, independently of the averaging scheme. It also retains the highest macro-average F1 among all nine compared methods on every backbone.
2. Independent Paired Validation. To further verify that these improvements are not artifacts of aggregation or differing evaluation implementations, we additionally examined a controlled comparison on 1,540 paired LoCoMo questions using Qwen3-8B, with identical memory stores, prompts, and metric implementations. Against a matched SimpleMem-style baseline, RaMem improves macro-F1 by 3.53 points (95% CI [1.83, 5.20]) and question-weighted F1 by 4.29 points (95% CI [2.98, 5.59]). The improvements are also positive across all 10 conversations, providing complementary evidence under a shared evaluation setup.
We sincerely appreciate your careful observation, which has helped us improve the accuracy and transparency of our empirical comparisons.
Response to Reviewer B9LH
We sincerely appreciate your careful review and insightful suggestions. Your constructive feedback has been invaluable in helping us strengthen the technical justification, and more clearly articulate the novelty and contributions of our work. We hope that our detailed responses below, together with the updated paper draft, adequately address your concerns and provide a clearer and more convincing account of our contributions.
Reply to Weakness
Reply to W1 -- Methodological Novelty Beyond Conventional Temporal Retrieval
We sincerely thank you for this important observation. We agree that temporal filtering, structured metadata, and relevance fusion are established techniques. RaMem does not claim novelty in these individual operations. Rather, our contribution is a unified retrieval formulation that explicitly addresses episodic identifiability, separating contextual validity from semantic relevance and using reinstated context to guide candidate discovery rather than merely reranking existing memories.
1. Beyond Temporal Reranking: Context-Conditioned Candidate Discovery.
The key distinction is that RaMem changes where retrieval searches, not merely how retrieved memories are scored. Conventional relevance-based reranking operates on a fixed content-retrieved candidate pool and therefore cannot recover target episodes missing from that pool. RaMem instead performs an additional search within the context-compatible memory region, prioritizes those candidates, and retains content-relevant candidates as fallback.
To directly validate this distinction, we conducted additional controlled comparisons against content-only retrieval, hard temporal filtering, and tuned soft temporal reranking, using identical memory stores, embeddings, query grounding, and retrieval budgets.
On 203 grounded LoCoMo queries with verified ground truth (Qwen3-8B):
These findings establish that context-conditioned candidate discovery provides measurable benefits beyond relevance-based temporal reranking, while semantic fallback preserves additional evidence coverage when contextual estimates are imperfect.
2. Distinction from Existing Memory Systems.
Our contribution is complementary to existing structured memory approaches. SimpleMem emphasizes efficient memory compression and multi-view retrieval; Zep organizes evolving information through temporal knowledge graphs; and REMem investigates graph-based episodic reasoning and recollection. RaMem instead focuses on a specific retrieval failure: semantically similar memories may remain accessible while the intended episode is not identifiable from content alone.
Accordingly, RaMem explicitly formulates contextual admissibility as a retrieval criterion, integrates grounded contextual candidate discovery with conservative activation and relevance-based fallback, and preserves episodic binding through downstream reasoning. Our existing context-shuffling and component ablations further substantiate the roles of these mechanisms. The theoretical interpretation added in Appendix B.5 also connects this formulation to established principles of episodic memory and probabilistic retrieval.
We have added these additional temporal retrieval comparisons to Appendix C.11 (Comparison with Conventional Temporal Retrieval), providing empirical evidence for the distinct role of context-conditioned candidate discovery and clarifying RaMem's methodological contributions beyond conventional temporal filtering and reranking. We sincerely appreciate your feedback, which has helped us articulate more precisely what RaMem contributes beyond existing retrieval components and why context-conditioned candidate discovery matters for long-term episodic recall.