acceptodds
Withdrawn from ICLR 2027

REMIND: Learning Evidence-Conditioned Memory Invocation over Long-Term Conversations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-term conversation memory is fundamental for Large Language Models (LLMs) to maintain coherent personalization and deliver contextually grounded responses across extended interactions. Recent work addresses this through retrieval-augmented generation, retrieving only question-relevant information from the memory bank to avoid hallucinations caused by irrelevant contextual noise. However, the most critical memories for answering a question are often not the most semantically similar ones. Memory relevance also arises from causal relations and temporal recency. A semantically distant memory can capture the root cause of a situation, whereas a highly similar one may reflect an obsolete state. Based on this novel insight, we present REMIND, a reinforcement-learning framework for selective, on-demand access to personal memory. Specifically, REMIND interleaves two processes on demand. Evocation constructs adaptive cues from the question and accumulated evidence, and Invocation accesses typed memory banks and inspects the returned evidence to decide whether to recall further or answer. Across 2,386 questions from LoCoMo, LongMemEval, and MemoryAgentBench, REMIND with a Qwen3-8B backbone achieves relative improvements of 12.5% over Base RAG and 4.8% over Search-R1 with same backbone. This gain highlights the value of adaptive, evidence-conditioned recall over fixed retrieval.

Primary area:
reinforcement learning
Keywords:
Agentic Search, Agentic RL

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_GhBqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how a model should access a fixed long-term conversation memory when answering questions. The authors propose REMIND, where the model rewrites the query, retrieves memories, and decides to search again or answer based on the evidence it got. The policy is trained with GRPO using rewards for answer, retrieval, query quality, new evidence and repeated queries.

Critical strengths
  1. The problem is meaningful. The paper points out that the most useful memory is not always the most similar one, which is a fair observation.

  2. The method is described clearly. The full reward formulas and the algorithm are given, and the code is released, which helps reproduction.

  3. REMIND improves over the same-backbone Base RAG at both 4B and 8B scale.

Critical weaknesses
  1. The novelty is limited, and the method does not really solve its own motivation. The paper claims relevance comes from causal and temporal relations, but it still uses fixed similarity retrieval with query rewriting; nothing handles time validity or causality.

  2. The experiments has serious problems. Common memory baselines like Mem0, A-Mem and MemoryOS are not compared at all. I think the paper is clearly below the bar of ICLR.

  3. Important ablations are missing. There is no prompt-only iterative retrieval baseline, and no ablation on each reward term, so it is unclear where the improvement actually comes from.

  4. Training data comes from LoCoMo and LongMemEval, and evaluation also uses these two datasets, but how they are split is never explained. MemoryAgentBench is not reported separately either.

Questions for authors
  1. Could the authors analyze how the method actually handles causal and temporal relevance?

  2. Could the authors add comparisons with common memory baselines such as Mem0, A-Mem and MemoryOS?

  3. Could the authors add a prompt-only iterative retrieval baseline without RL training, and an ablation that removes each reward term one by one?

  4. Could the authors explain how the training and evaluation data from LoCoMo and LongMemEval are split and whether they overlap, and report MemoryAgentBench results separately?

Reviewer_kHJmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces REMIND, a method for iteratively retrievng information from the conversation memory for better personaliing and improving QA performance of the system. They utilize a react loop that the policy can generate queries, and retrieve information from the user's history and do this iteratovely until it decides it has enough information to answer the question. They optimize the policy using GRPO with a combination of rewards: (1) QA reward (2) retrieval reward (3) format reward. their experiments show that they outperform baselines including RAG and Search-R1.

Critical strengths

S1: The paper gets very good improvements compared to the included baselines. S2: The problem they are studying is important and worth the effort

Critical weaknesses

W1: methodological novelty over existing search agents is limited/insufficiently demonstrated. This paper claims are too strong and might not be novel. For example, [1] proposed a similar iterative approach optimized with RL for personalization that gets very good results. A comparison with that baseline might shed more light on how the small differences affect the performance.

[1] Maryam Amirizaniani, Alireza Salemi, and Hamed Zamani. 2026. Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '26). Association for Computing Machinery, New York, NY, USA, 3580–3586. https://doi.org/10.1145/3805712.3809944

W2: Trainining and evaluation is not documented clearly. For example, in 3.3 says training uses LoCoMo and LongMemEval and section 4.1 evaluates on those same datasets plus MemoryAgentBench. It doesn't mention how the training and test are separated. I am not saying there is an information leak, but I am saying it needs carful explanation on how training happened.

W3: the paper cites many. baselines but doen't compare with them. This prevents us from evaluating if the proposed method is performing well compared to others or not. The only included baselines are RAG and Search-R1. In case of Search-R1, it doesn't mention if they trained it on the same data or they used it zero-shot. If the latter, then this is not a fair comparison as the gains may be simply due to training data and not the method itself.

W4: If you simplify the equation 8 and 9, “first-time evidence gain” reward is mathematically just final evidence recall. thus, it adds another recall term, it does not independently reward a better retrieval sequence. Is this intended or my take is wrong? If I am right, then this needs clarification and all the explanations in the paper can be simplified in a few sentences.

W5: The paper has unsupported claims and oversells the contributions. For example, they claims that "The memory that answers a question is often not the one most similar to it" but they do not provide any example or number to support this claims. Furthermore, their retrieval approach which is based RRF using BM25 and embedding models is still similarity search. The do not train the retrieval to change its definition of relevance. therefore, they only chnage the query and still rely on similarity to find the documents. This not very in line with the most bold claim of the paper that: "The memory that answers a question is often not the one most similar to it"

W6: Pooling the benchmarks all together hides materially different results on different benchmarks. This makes it even worth when there is an unclear explanation of how training and testing sets are seperated. This is one of the most important points that needs claificiation.

Questions for authors
  1. What methodological differences distinguish your approach from Amirizaniani et al. (2026), and can you compare against their RL-based iterative retrieval method?

  2. How were LoCoMo and LongMemEval partitioned for training and testing? Were conversations or users shared across splits, and how was leakage prevented?

  3. Why were other cited methods excluded from the comparisons? Was Search-R1 trained on the same data and with a comparable training budget, or evaluated zero-shot?

  4. Does summing the “first-time evidence gain” reward in Equations 8–9 reduce to final evidence recall? If so, what distinct incentive does it provide for retrieval sequencing?

  5. What empirical evidence supports the claim that answer-bearing memories are often not the most similar to the question? How does this claim relate to your continued use of similarity-based retrieval with reformulated queries?

  6. Can you report results separately for each benchmark, alongside their training and test sample counts, to clarify whether the pooled improvements hold across datasets?

Reviewer_A9B7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper learns how to access a fixed conversational memory store through repeated query construction, retrieval, and stopping. The policy uses retrieved evidence to formulate subsequent queries and is trained with GRPO using rewards for answer correctness, evidence retrieval, query quality, and valid tool use. On a merged evaluation of 2,386 questions from LoCoMo, LongMemEval, and MemoryAgentBench, the Qwen3-8B system reports 72.67% accuracy, compared with 64.59% for Base RAG and 69.32% for Search-R1.

Critical strengths

The paper addresses an important limitation of conversational memory, where answering a question may require using retrieved evidence to guide further searches. Keeping the memory store and retriever fixed helps focus the study on how the model formulates queries and decides whether to retrieve more information or answer. The reported improvements over RAG using the same model at both 4B and 8B, as well as Search-R1 at 8B, support the effectiveness of the overall approach.

Critical weaknesses
  • The experiments do not isolate the proposed mechanism. Relative to Base RAG, REMIND changes both training and the number of retrieval calls; and there is no matched-budget comparison or ablation of individual reward components.
  • LoCoMo and LongMemEval contribute to both training and evaluation, but the paper does not specify the conversation-level split or whether the reward weight was chosen on held-out validation data.
  • Limited comparison with closely related memory-access methods. The paper discusses several such methods in §2, while Table 1 evaluates Base RAG, Search-R1, and a larger-model fixed-RAG reference. This makes it difficult to assess REMIND’s advantage over the closest memory-specific methods under a comparable setup.
  • The training/evaluation split could be clearer. Both training and evaluation use LoCoMo and LongMemEval, but the manuscript does not specify whether the splits separate only questions or also their underlying conversations and users.
  • Different questions may concern the same personal facts, so this distinction matters for assessing generalization. Including MemoryAgentBench only in evaluation is a strength, but its 446 questions are pooled with the other datasets in Table 1 rather than scored separately.
  • Clarifying the splits and reporting a separate MemoryAgentBench score would make it easier to assess the evidence for generalization.
Questions for authors

See Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.