- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper presents LocalMem, a memory-management framework with separate 4B models for memory construction and retrieval. The Memory Archiver is trained with SFT and GRPO using downstream QA rewards. The framework runs a four-stage retrieval pipeline guided by evidence-sufficiency judgments.
Critical strengths
-
The problem is relevant: improving long-term memory for LLM agents while reducing memory-management costs.
-
The paper is generally clearly written.
Critical weaknesses
-
The novelty appears limited, as the paper mainly combines existing techniques. SimpleMem already uses window-based extraction and semantic, lexical, and structured indexing, while Mem-α learns memory construction from downstream QA feedback with frozen retrieval and generation components. Adaptive and iterative retrieval have been explored in existing studies like Self-RAG and IRCoT. The paper does not sufficiently establish the contribution of its particular combination beyond these existing ideas. I think using local 4B models is primarily an engineering choice and does not establish methodological novelty.
-
The gain of reinforcement learning appears modest. Table 4 shows that SFT provides most of the improvement, increasing F1 from 55.0 to 60.5, while GRPO further increases it to 61.1. These results do not clearly show whether the additional RL stage and the proposed reward formulation are necessary. Additionally, the evaluation covers only two conversational memory benchmarks. It is therefore unclear whether the learned memory policies benefit agents performing interactive tasks, where memory must support action selection and errors can affect subsequent observations. Results in other settings, such as ALFWorld, would help assess the framework’s broader applicability to LLM agents.
Questions for authors
- How does independent window processing handle cross-window coreferences and changes in facts over time, particularly when new information contradicts or supersedes previously recorded facts?
5 comments
To Reviewer ixFa/#3 (2/3)
Concern 4 (Weakness 2): The gain of reinforcement learning appears modest. Table 4 shows that SFT provides most of the improvement, increasing F1 from 55.0 to 60.5, while GRPO further increases it to 61.1. These results do not clearly show whether the additional RL stage and the proposed reward formulation are necessary.
Response 4: We thank the reviewer for questioning the necessity of the additional reinforcement learning stage. We agree that SFT accounts for most of the overall improvement, while GRPO provides additional benefits on challenging multi-hop questions and memory compactness.
Specifically, GRPO improves Overall₄ F1 from 60.54 to 61.03 (+0.49 points), including a gain of 1.34 points on MH questions. Meanwhile, Memory Retention (the ratio of generated memory tokens to the total number of original conversation tokens) decreases from 72.70% to 70.08%.
| Memory Archiver | MH | Temp | OD | SH | Overall₄ |
|---|---|---|---|---|---|
| SFT | 46.71 ± 0.62 | 62.00 ± 0.27 | 29.09 ± 0.99 | 68.20 ± 0.31 | 60.54 ± 0.16 |
| + GRPO | 48.05 ± 0.34 | 62.64 ± 0.48 | 29.79 ± 1.94 | 68.33 ± 0.04 | 61.03 ± 0.26 |
The reward ablations further show that removing Window-count Normalization and Format Reward decreases Overall₄ F1 by 1.71 and 0.30 points, respectively.
| Window-count Normalization | Format Reward | MH | Temp | OD | SH | Overall₄ |
|---|---|---|---|---|---|---|
| ✓ | ✓ | 48.05 ± 0.34 | 62.64 ± 0.48 | 29.79 ± 1.94 | 68.33 ± 0.04 | 61.03 ± 0.26 |
| × | ✓ | 45.03 ± 0.22 | 60.45 ± 0.85 | 30.48 ± 1.57 | 66.98 ± 0.50 | 59.32 ± 0.16 |
| ✓ | × | 46.86 ± 0.72 | 61.88 ± 1.13 | 31.46 ± 1.83 | 68.27 ± 0.12 | 60.73 ± 0.29 |
Together, these results indicate that although GRPO contributes a modest additional gain in aggregate F1, it improves more challenging multi-hop performance while encouraging more compact and structurally consistent memory representations.
Concern 5 (Weakness 2): Additionally, the evaluation covers only two conversational memory benchmarks. It is therefore unclear whether the learned memory policies benefit agents performing interactive tasks, where memory must support action selection and errors can affect subsequent observations. Results in other settings, such as ALFWorld, would help assess the framework’s broader applicability to LLM agents.
Response 5: We thank the reviewer for pointing out that the original evaluation focuses mainly on conversational memory and does not sufficiently examine its support for Agent tasks. We therefore extend our evaluation to Mem2ActBench [1], which directly tests whether an Agent can recover complete and executable tool-call parameters from long-term memory.
[1] Shen Y, Li K, Zhou W, et al. Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agents.
Using the shared Qwen2.5-72B Answer Model, LocalMem-4B achieves 39.13 F1 and 70.24 BLEU-1, outperforming the compared memory baselines.
| Method | Memory Model | Answer Model | F1 ↑ | BLEU-1 ↑ |
|---|---|---|---|---|
| LocalMem (Ours) | LocalMem-4B | Qwen2.5-72B | 39.13 | 70.24 |
| A-Mem | Qwen2.5-72B | Qwen2.5-72B | 35.93 | 67.93 |
| LTMemory | Qwen2.5-72B | Qwen2.5-72B | 35.32 | 67.93 |
| MemTree | Qwen2.5-72B | Qwen2.5-72B | 33.21 | 68.17 |
| Mem0 | Qwen2.5-72B | Qwen2.5-72B | 28.95 | 66.47 |
| Langmem | Qwen2.5-72B | Qwen2.5-72B | 24.01 | 63.06 |
| SCM | — | Qwen2.5-72B | 22.73 | 62.04 |
| Generative Agents | Qwen2.5-72B | Qwen2.5-72B | 22.38 | 62.58 |
We select Mem2ActBench because it directly evaluates the recovery of parameters required for downstream tool execution from long-term memory. In contrast, ALFWorld primarily evaluates closed-loop embodied interaction and environment-level task execution, involving broader reasoning, planning, and action-generation capabilities beyond the direct scope of LocalMem as a memory-management module.
This additional evaluation demonstrates that LocalMem can support memory-grounded tool use beyond conventional conversational QA, while we acknowledge that it does not constitute a full evaluation of closed-loop interactive environments.
To Reviewer ixFa/#3 (3/3)
Concern 6 (Question 1): How does independent window processing handle cross-window coreferences and changes in facts over time, particularly when new information contradicts or supersedes previously recorded facts?
Response 6: We thank the reviewer for raising these important questions about cross-window coreference and temporal changes in stored facts. LocalMem addresses cross-window references through overlapping windows and preserves temporal changes through an append-only memory archive.
For cross-window coreference, adjacent windows share (k=2) dialogue turns. Within each window, the Memory Archiver resolves pronouns into explicit entity names whenever their antecedents can be identified from the available context. This overlapping context helps preserve references near window boundaries while maintaining independent and parallel memory generation. However, references may remain unresolved when their antecedents fall outside the available overlapping context.
For temporal changes, LocalMem stores new information as separate memory entries with associated timestamps rather than overwriting previous records. Thus, both earlier and updated or contradictory information are preserved. LocalMem does not maintain a globally reconciled, unique fact state. Instead, the downstream Answer Model uses retrieved entries and their timestamps to interpret temporal changes and resolve potential contradictions according to the query.
This design preserves historical information without destructive updates, while delegating temporal interpretation and contradiction resolution to downstream reasoning.
To Reviewer ixFa/#3 (1/3)
Concern 1 (Weakness 1): The novelty appears limited, as the paper mainly combines existing techniques. SimpleMem already uses window-based extraction and semantic, lexical, and structured indexing, while Mem-α learns memory construction from downstream QA feedback with frozen retrieval and generation components.
Response 1: We thank the reviewer for questioning the novelty of LocalMem relative to SimpleMem and Mem-α. We agree that window-based extraction, multi-view indexing, and downstream QA supervision are not individually new. Our contribution lies in addressing the mismatch between independent window-level memory generation and conversation-level memory utility through a unified memory reward.
SimpleMem already employs window-based extraction and multi-view indexing, while Mem-α uses downstream QA feedback to optimize memory construction. LocalMem adopts Streaming Window-to-Entry Generation, where each interaction window independently generates structured memory entries without accessing or modifying previously stored memories.
This design introduces a granularity mismatch: memory utility is evaluated jointly at the conversation level, whereas memory generation occurs independently at the window level. We address this mismatch through a unified memory reward that evaluates the assembled memory according to downstream QA performance. We further introduce Window-count Normalization to balance conversations with different numbers of valid windows and Format Reward to constrain the structure of generated entries.
Removing Window-count Normalization and Format Reward decreases Overall₄ F1 by 1.71 and 0.30 points, respectively.
Therefore, our contribution is not the first use of QA feedback for memory learning, but an optimization mechanism tailored to independent window-level memory generation that aligns window-level learning with conversation-level memory utility.
Concern 2 (Weakness 1): Adaptive and iterative retrieval have been explored in existing studies like Self-RAG and IRCoT. The paper does not sufficiently establish the contribution of its particular combination beyond these existing ideas.
Response 2:
We thank the reviewer for pointing out the need to clarify the distinction between LocalMem and adaptive retrieval methods such as Self-RAG and IRCoT. We agree that adaptive and iterative retrieval have been extensively studied. However, LocalMem uses evidence sufficiency relative to the original information requirements to guide subsequent memory retrieval.
Self-RAG uses reflection signals to assess retrieval requirements and document relevance, while IRCoT generates subsequent queries based on intermediate reasoning steps. In contrast, LocalMem employs a frozen Evidence-sufficiency Judge to determine whether retrieved memories sufficiently cover the original information requirements. When the evidence is insufficient, the Memory Enquirer explicitly identifies the missing information and generates targeted retrieval queries.
We further isolate the contribution of Gap-aware Planning through a controlled experiment. With the expanded Top-K settings held constant, enabling Gap-aware Planning improves Overall₄ F1 from 60.67 to 61.39 (+0.72 points).
This controlled comparison demonstrates that targeted query refinement based on evidence gaps provides additional gains beyond merely increasing the retrieval candidate budget.
Concern 3 (Weakness 1): I think using local 4B models is primarily an engineering choice and does not establish methodological novelty.
Response 3: We thank the reviewer for pointing out that using compact 4B models is primarily an engineering choice. We agree that model size itself is not the source of methodological novelty. Our contributions lie in the memory learning objectives and retrieval mechanisms, while compact models provide an efficient implementation of these mechanisms.
For the Memory Archiver, the trained 4B model improves over its prompted 122B counterpart by 4.1 F1 points. For the Memory Enquirer, the trained 4B model achieves 61.3 ± 0.2 F1, approaching the 61.2 ± 0.2 achieved by the 122B Progressive Enquirer, while outperforming the untrained 4B Enquirer (59.7 F1).
These results demonstrate that the proposed memory mechanisms can be learned effectively with compact models, rather than requiring substantially larger models for memory construction and retrieval.
We therefore position compact models as an efficient implementation of the proposed memory mechanisms, rather than as the basis of methodological novelty.