- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
MemTrain is a self-supervised training framework for LLM agents that maintain fixed-size textual context memory. It combines end-to-end masked-entity reconstruction with intermediate recall of historical entities on Wikipedia-derived documents. Recall success contributes to the reward for memory-writing trajectories, and both tasks are optimized with GRPO. The resulting checkpoint initializes subsequent MemAgent or MEM1 training. Experiments report improved average performance on long-document and search-based QA across two backbones.
Critical strengths
-
The framework derives verifiable training targets from raw text, reducing dependence on task-specific QA annotations. Coupling historical recall with trajectory rewards provides a concrete way to train memory generation and utilization jointly.
-
MemTrain initialization improves average performance across two backbones in both long-document and search-based QA, with gains of 5.17–17.67 and 8.50–10.58 points, respectively. The End-to-End and Decoupled ablations further support the usefulness of the training recipe.
Critical weaknesses
-
IMR encourages recalling randomly sampled historical entities, but the alignment of this proxy objective with practical memory utility is unclear. Under a fixed memory budget, retaining low-value details may compete with preserving information relevant to future tasks. Although random entity recall encourages broad coverage, the paper does not examine whether it promotes effective selective retention rather than indiscriminate memorization.
-
IMR provides both an intermediate memory state and a masked historical chunk, so the model may infer the missing entity from the remaining context or pretrained knowledge without relying on memory. Although downstream ablations demonstrate the benefit of IMR training, the paper does not directly evaluate whether reconstruction depends on information preserved in intermediate states. Without memory-removal controls or systematic measurements of historical-fact retention, the claimed improvement in memory preservation remains insufficiently validated.
Questions for authors
-
What assumptions about future tasks motivate random entity recall? Can the authors analyze whether this objective promotes useful selective retention under a fixed memory budget, potentially by comparing random and relevance-aware entity sampling?
-
Can the authors report IMR accuracy with and without memory, together with the proportion of sampled historical facts correctly retained in intermediate memory states?
7 comments
[W1.2 & Q1.2] Evaluation on Factual Retention
To examine whether IMR promotes effective retention of task-relevant information rather than crowding out the fixed memory budget with irrelevant details, we directly evaluate the memory states on HotpotQA (Qwen3-4B-Instruct-2507) across three dimensions (Section 5.3):
- Final Fact Retention Rate (FR): The proportion of supporting facts retained in the final memory state.
- Fact Capture Rate (FC): The proportion of supporting facts immediately integrated into memory upon their appearance.
- Fact Retention Duration (FD): The relative duration for which supporting facts persist in memory once captured.
Specifically, we employ Qwen3.8-27B as an automated judge to inspect each memory state () and identify which annotated supporting facts from HotpotQA are present. For a given query , let denote its ground-truth supporting fact set, and let be a binary indicator denoting whether memory state contains fact . Furthermore, let denote the turn in which fact appears in the input context, and let denote the first turn at which fact is recorded in memory. The three metrics for query are formally defined as:
where represents the subset of supporting facts captured by memory prior to the final turn (with if ).
Final Fact Retention Rate (FR, %)
| Method | 7k | 14k | 28k | 56k | 112k | 224k | 448k | 896k | Avg |
|---|---|---|---|---|---|---|---|---|---|
| MemAgent | 92.86 | 93.18 | 92.86 | 91.88 | 88.31 | 74.51 | 73.05 | 75.00 | 85.21 |
| MemTrain (E2E) + MemAgent | 87.99 | 88.64 | 84.42 | 77.92 | 79.87 | 74.35 | 74.35 | 68.83 | 79.55 |
| MemTrain + MemAgent | 95.78 | 94.48 | 96.10 | 93.51 | 92.53 | 89.80 | 84.42 | 87.34 | 91.75 |
Fact Capture Rate (FC, %)
| Method | 7k | 14k | 28k | 56k | 112k | 224k | 448k | 896k | Avg |
|---|---|---|---|---|---|---|---|---|---|
| MemAgent | 91.70 | 91.55 | 92.31 | 88.85 | 87.12 | 77.78 | 70.52 | 70.68 | 83.81 |
| MemTrain (E2E) + MemAgent | 90.14 | 91.08 | 90.84 | 89.35 | 87.73 | 88.57 | 88.43 | 85.71 | 88.98 |
| MemTrain + MemAgent | 98.19 | 94.78 | 97.34 | 96.24 | 93.70 | 93.49 | 89.33 | 88.89 | 94.00 |
Fact Retention Duration (FD, %)
| Method | 7k | 14k | 28k | 56k | 112k | 224k | 448k | 896k | Avg |
|---|---|---|---|---|---|---|---|---|---|
| MemAgent | 94.69 | 95.28 | 93.64 | 94.25 | 95.52 | 91.49 | 92.46 | 92.75 | 93.76 |
| MemTrain (E2E) + MemAgent | 93.04 | 92.92 | 90.29 | 87.15 | 86.27 | 83.53 | 83.12 | 81.48 | 87.23 |
| MemTrain + MemAgent | 94.89 | 96.59 | 97.35 | 96.77 | 95.26 | 92.72 | 91.60 | 93.31 | 94.81 |
Under the same memory budget, adding IMR to the E2E objective improves average fact capture from 88.98% to 94.00% and retention after capture from 87.23% to 94.81%. Final retention of gold supporting facts increases from 79.55% to 91.75%. Compared with directly trained MemAgent, the full model captures more gold supporting facts when they first appear (94.00% vs. 83.81%) and maintains comparable retention after capture (94.81% vs. 93.76%). Overall, the random recall objective improves the coverage of task-relevant evidence in the final memory under a fixed budget.
[W2 & Q2] IMR accuracy with and without memory.
To verify whether IMR reconstruction relies on the intermediate memory state, we sample 800 end-to-end trajectories using MemTrain-Qwen3-4B-Instruct-2507 on unseen data constructed in the same way as the training set, from which we collect 6,400 IMR reconstruction pairs evaluated with and without memory (Appendix C.3).
| Setting | IMR Accuracy (%) |
|---|---|
| w/o Memory | 13.89 |
| w/ Memory | 18.70 |
The results show that historical entity reconstruction is a very hard task (E2E accuracy is about 30% for reference). We admit that the model can infer some masked entities from the chunk itself, but over 86% of the entities still cannot be inferred without memory. With memory, the IMR accuracy improves to 18.70%, showing that the memory really matters in the IMR task. Among these sampled memories and entities, about 2.4% of the target entities are literally contained in the memory. This is reasonable because it is impossible to store all entities from a 40k-token document in a 1k-token memory. However, our purpose is not to put as many entities as possible in the memory, but to store compressed and restorable information for later reasoning. From this perspective, IMR indeed takes effect.
Thank you for the helpful comments. We have conducted additional analyses and incorporated them into the manuscript.
[W1.1 & Q1.1] What assumptions about future tasks motivate random entity recall?
Our goal is to use unlabeled text to teach models to write compact memories and recover information from them under a fixed memory budget. Random recall is motivated by the fact that different downstream tasks may need different parts of the same history, and some facts become relevant only in light of later evidence. In our case study, for example, a later chunk identifies Adriana Trigiani as the director of Big Stone Gap, making clear why her previously mentioned location matters.
The E2E objective rewards reconstruction of a single masked entity. IMR adds a historical recall target sampled after the memory has been written. Successful recall trains the model to recover historical information and also contributes to the reward for the memory-writing trajectory. Random sampling provides a range of recall targets without requiring downstream relevance labels. Task-specific post-training then adapts what the model prioritizes within the memory budget.