- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes ValMem, a value-aware memory management framework for long-term LLM agents. It combines Multi-Granularity Hierarchical Memory (MGHM) with Value-Driven Training (VDT) to optimize memory retrieval, compression, and retention based on downstream utility. Experiments on three benchmarks demonstrate improvements in task performance and memory efficiency.
Critical strengths
- The paper addresses the important challenge of balancing memory utility and storage efficiency in long-term LLM agents.
- The combination of temporal hierarchy and multiple representation granularities provides a flexible structure for memory management.
- VDT introduces operation-specific counterfactual rewards, enabling more precise optimization of memory decisions.
- Clear presentation. The paper is well organized and makes it easy for the reader to quickly grasp the main idea and method.
Critical weaknesses
-
The core idea of using downstream utility to guide memory retrieval and management is closely related to prior work such as MemRL [1] and Mem-T [2], which also employs reinforcement learning and utility-based memory selection. Although ValMem introduces counterfactual credit assignment and different memory operations, the paper does not sufficiently clarify its novelty relative to MemRL or provide a direct comparison.
-
Experiments only use Qwen3.5-9B, leaving the effectiveness across different model families and sizes unclear.
-
Counterfactual rollouts require substantial computation (approximately 120 hours on four A800 GPUs), potentially limiting practical adoption. And the computational overhead of maintaining cross-granularity associations and routing over increasingly large memory banks is not sufficiently analyzed.
-
Lastly, using low entropy as an indicator of memory reliability is not fully justified, since concentrated retrieval scores do not necessarily imply factual correctness.
[1] Zhang, Shengtao, et al. "Memrl: Self-evolving agents via runtime reinforcement learning on episodic memory." NeurIPS 2026.
[2] Yue, Yanwei, et al. "Mem-t: Densifying rewards for long-horizon memory agents." ICML 2026.
Questions for authors
See weaknesses above
4 comments
To Reviewer nqJz (2)
W3: Training cost (about 120 h on 4 x A800) may limit adoption, overhead of cross-granularity association and routing over growing memory banks is not analyzed.
- Training cost. The counterfactual deployment of VDT involves a one-time training cost. There is no counterfactual deployment process during the inference stage. It takes 4.2 seconds to process every 100 samples and generates 0.62K completion tokens (Section 5.5, Figure 3; Appendix F.1). The deployment cost is not significant. To conduct a more detailed analysis of the deployment cost, we added a cost-benefit curve that varies with the number of continuations K in the revised paper, and evaluated different actions while conducting budget matching comparisons with GRPO and local redeployment (Appendix B, Table 5).
| Method | GPU Hours | Total Tokens | Overall F1 |
|---|---|---|---|
| Standard GRPO (matched budget, ) | 120 | 1.50M | 44.18 |
| Local/Session Rerollout (matched budget, ) | 120 | 1.50M | 45.85 |
| ValMem () | 45 | 0.56M | 45.92 |
| ValMem () | 72 | 0.90M | 47.35 |
| ValMem (, default) | 120 | 1.50M | 48.14 |
- Memory-bank overhead. Appendix F.1 reports the end-to-end latency, but does not report the computational overhead for maintaining the cross-granularity associations and routing. We believe it is meaningful to conduct a component-based scaling analysis. In the revised paper, we have added the scalability study in Appendix F.1 (Table 12). In this study, the memory bank is gradually expanded from the baseline size to several times the baseline size, and the delay of each query is decomposed into three parts: association (GMM acceptance/rejection), routing, and retrieval (Top-N, N=5). Since the association and routing operations are only performed within a limited candidate neighborhood rather than scanning the entire bank, and redundant entries are eliminated through a value-aware discard/compression mechanism, the bank remains compact.
| Memory-Bank Scale | Association (GMM) | Entropy Routing | Top- () | Total Overhead |
|---|---|---|---|---|
| (, baseline) | 18.4 ms | 11.2 ms | 6.5 ms | 36.1 ms |
| () | 24.7 ms | 14.8 ms | 8.9 ms | 48.4 ms |
| () | 33.5 ms | 19.6 ms | 12.3 ms | 65.4 ms |
W4: Low entropy as a reliability indicator is not justified, concentrated scores do not imply factual correctness.
We agree that low entropy merely indicates the retrieval concentration, rather than the accuracy of the facts. The expression "reliable matching" in Section 4.1.3 is not accurate, we have changed it to "retrieval concentration". Then, we analyzed the reliability of this retrieval concentration in the revised paper. Based on entropy, the router is designed as a structural prior, used to guide the agent towards relevant neighborhood clusters, rather than a fact-checking tool. The accuracy of facts is a specific task of ValMem. Our experiments have confirmed this division of labor:
- Necessity of routing. After replacing entropy routing with uniform weights, LoCoMo F1 decreased by 3.52, and LongMemEval F1 decreased by 5.64 (Appendix F.4, Table 15), proving that it significantly improved the retrieval efficiency.
- Value-aware processing of invalid information. In our adversarial memory injection research (Appendix F.5, Table 16), VDT successfully eliminated 72.5% of outdated memories, 58.3% of conflicting memories, and 81.6% of irrelevant memories, clearly demonstrating that it is the strategy with value-awareness, rather than the router itself, that ensures the accuracy of facts.
| Memory type | Discard rate (%) | Compress rate (%) | Keep rate (%) | Selected-memory precision (%) |
|---|---|---|---|---|
| Outdated | 72.5 | 18.3 | 9.2 | 88.4 |
| Duplicated | 65.8 | 24.6 | 9.6 | 86.1 |
| Conflicting | 58.3 | 20.5 | 21.2 | 82.7 |
| Irrelevant | 81.6 | 12.4 | 6.0 | 90.2 |
Thank you for the additional experiments and clarifications. While appreciate the new backbone evaluations and controlled comparisons, my core concerns regarding the method's soundness remain unresolved, so I will maintain my current rating. Simply renaming reliable matching to retrieval concentration does not justify using inverse entropy. In practice, an outdated or conflicting memory cluster with high lexical overlap will yield a sharp, low-entropy distribution, meaning inverse entropy actively amplifies stale information. Conversely, multi-hop queries inherently span diverse topics (higher entropy) and get penalized. Beating uniform weights only shows non-uniform weighting helps, not why inverse entropy is superior to standard max/mean neighborhood similarity.
Thank you for your constructive feedback. The router only partitions candidates into neighborhoods, while factual correctness is handled by VDT's value-aware pruning, not by entropy. High-overlap stale clusters may receive sharp low entropy, but VDT then discards or compresses them; multi-hop queries may have higher entropy, but VDT preserves necessary evidence across granularities. We will emphasize this part in the paper, and we will continue to refine the writing in other sections as well.
To Reviewer nqJz
We appreciate the reviewer's positive assessment of the balancing memory utility and storage, the clarity of the writing, and the reasonable engineering. For a general view of how ValMem differs from prior memory systems, please see our "Response to All Reviewers", followed by our answers to your specific questions. For the specific content, please refer to the revised version.
Thank you for raising this point. We have revised the Related Work section accordingly, and have included a comparative explanation in the revised paper (Appendix A, Table 4):
Since direct comparisons are influenced by different memory architectures, action spaces, and tasks, we have added controlled comparisons under the common baseline model (Qwen3.5-9B) and LoCoMo split in the revised paper: a Mem-T style tree-based backward propagation credit variant, and a MemRL style runtime utility value reordering variant on our memory repository (Appendix F.2, Table 13).
We agree that evaluating a single backbone limits the evidence for generality across model sizes. While cross-benchmark transfer is evaluated by training solely on LoCoMo and testing zero-shot on LongMemEval and PersonaMem-v2 in the original submitted paper, the backbone was fixed. In the revised paper, we conducted a complete process (training and evaluation) for Qwen3.5-4B and Llama-3.1-8B alongside Qwen3.5-9B across all three benchmarks, and have reported the results in Appendix C.1 (Table 6), showing consistent improvements across different model series and sizes.