- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces PRISMEM, a capability-driven framework for self-evolving agent memory programs. Rather than selecting revisions solely by aggregate task performance, PRISMEM organizes evolution around five predefined memory capabilities: factual retrieval, temporal tracking, preference extraction, multi-session synthesis, and adversarial handling. It first performs a cold-start stage to obtain a balanced initial program, then maintains capability-specific best-performing programs, selects target capabilities using a dependency-aware priority score, and uses history-guided failure cases to refine the corresponding specialist. Finally, it compares the execution traces of specialists and the overall-best program on differential cases to integrate complementary improvements into one unified memory program. The authors evaluate PRISMEM on BEAM and LongMemEval. Programs are evolved on shorter-context splits and evaluated on million-token test splits. PRISMEM outperforms the strongest self-evolving baselines by 10.54 percentage points on BEAM-1M and 7.83 points on LongMemEval-M.
Critical strengths
1. Strong empirical gains. PRISMEM reports substantial improvements over both static and self-evolving memory baselines on BEAM-1M and LongMemEval-M.
2. Broad empirical analysis. The paper includes component ablations, efficiency analysis, cross-model evaluation, transfer experiments, and detailed evolution traces, providing useful evidence for how the system evolves and where the gains come from.
Critical weaknesses
1. Unvalidated dependence on capability annotations. PRISMEM relies on a predefined five-capability taxonomy and GPT annotations of primary and secondary capabilities. These labels directly determine specialist selection, capability scheduling, and diagnostic case sampling. The paper does not assess annotation quality, agreement across annotators or models, sensitivity to label noise, or robustness to alternative capability definitions. It is therefore difficult to determine whether the reported gains arise from the proposed evolution framework or from a particular, potentially benchmark-specific labeling scheme.
2. Limited methodological novelty. Relative to existing self-evolving agent memory methods, PRISMEM proposes to organize program evolution around predefined memory capabilities. This is a coherent alternative to aggregate score driven evolution, but it primarily changes how the search process is organized rather than introducing a substantially new form of memory evolution. Moreover, recent work such as ModularRSI has already proposed the closely related high-level paradigm of decomposing a monolithic agent system, evolving the parts independently using execution trajectories, and integrating the resulting improvements. PRISMEM changes the decomposition axis from functional modules to memory capabilities, but this difference is not sufficiently developed or validated to establish a distinct conceptual contribution at the ICLR level.
Questions for authors
MemPro appears to be a closely related program-level self-evolving memory method, but it is not included in the empirical comparison. Is there a technical or protocol-related reason why a controlled comparison was not feasible? If feasible, could the authors compare PRISMEM and MemPro under matched backbone models, evolution budgets, and evaluation splits? This comparison would materially clarify whether capability-driven evolution improves over existing version-tree-based memory program evolution.
6 comments
Response to Reviewer fUa2 (2/3)
W2: Methodological novelty and relationship to ModularRSI.
Contribution of PrisMem. PrisMem's contribution lies in identifying an overlooked cause of stagnation in memory evolution, where overall performance masks valuable capability-level progress, and introducing a systematic approach to identify such progress, guide subsequent optimization, and integrate complementary improvements into a unified program.
Self-evolving memory methods study how agents leverage task experience to improve memory construction, organization, retrieval, and use. A central challenge is how to identify valuable progress, determine subsequent optimization directions, and accumulate effective implementations. By analyzing the evolution trajectories of existing methods, we identify a structural source of stagnation, which motivates our capability-driven paradigm. Specifically, agent memory must satisfy diverse capability requirements, yet one modification can affect different capabilities differently (Fig. 1b). Consequently, mixed feedback across capabilities obscures revision directions, while overall judgment hides capability-level progress. In our analysis, 80.5% of revisions that do not improve the overall score still improve at least one capability (Fig. 1a). This reveals that stagnation in overall performance does not imply stagnation in capability development; valuable improvements may remain hidden without further refinement. Thus, the key question is how to turn such capability-level progress into systematic improvements.
Motivated by this insight, PrisMem introduces a new paradigm for self-evolving memory systems by expanding the evolution space along multiple capability dimensions and treating capability gains as valuable intermediate progress. It grounds diagnosis in specific capability requirements, continues refinement from the corresponding specialists, and leverages execution differences between specialists to integrate complementary improvements into a unified memory program. This provides a new approach to identify valuable capability-level progress and develop promising directions that holistic judgment would overlook. Our ablations support the importance of such designs: removing capability-driven evolution and trace-guided integration reduces performance by 10.15 and 3.16 pp, respectively (Table 4). Figure 3 further shows that PrisMem expands the exploration boundary beyond holistic evolution and achieves stronger overall performance. Therefore, our contribution is a new understanding of evolution stagnation and a complete method for identifying, developing, and integrating otherwise overlooked capability-level improvements.
Difference from ModularRSI. We acknowledge the conceptual similarity that both works decompose the evolution process, yet they target different scenarios and evolution bottlenecks, and are complementary. ModularRSI proposes functional modular decomposition for mechanism-level credit assignment in evolving programming and terminal-use agent harnesses. In contrast, PrisMem addresses the heterogeneous effects of revisions across capabilities in agent memory evolution, which remain important even after an issue is localized to a functional component.
For example, expanding the retrieval module may improve multi-session synthesis by retrieving more cross-session information, but also introduce irrelevant personal information that interferes with answering. Even when the issue and modification are localized to the retrieval module, this does not clarify whether or how the revision should be further refined: improvements in one capability may be offset by regressions in others, making the optimization direction ambiguous. Moreover, the overall score may fail to reflect capability-specific gains when these opposing effects cancel out, causing useful improvements to be overlooked before they can be further developed and integrated.
PrisMem addresses this challenge by identifying valuable revisions through capability-level evaluation and continuing refinement from the corresponding specialists, allowing capability-level progress to be developed further. Functional modularity provides controlled boundaries for implementation changes, whereas capability-driven evolution determines which improvements deserve continued development and how their benefits can be consolidated. These principles operate at different levels and can complement each other within the same system, which would be an interesting direction for future work.
Finally, we appreciate you bringing ModularRSI to our attention. First released on arXiv on September 14, 2026, it falls within the two-month concurrent-work window before the submission deadline under ICLR guidelines [2]. We cited this work and clarified the relationship and distinctions between the two methods in Appendix F of the revised paper.
Response to Reviewer fUa2 (3/3)
Q1: Controlled comparison with MemPro.
M*, one of the baselines in our paper, is a representative program-level memory evolution method, just like MemPro. Both organize evolution based on overall performance, fundamentally differing from PrisMem's capability-driven approach. We added the matched comparison with MemPro to better illustrate the advantages of capability-driven evolution. All methods use Qwen3.8-27B as the backbone model, with the same 20-round evolution budget and BEAM data split. The results are shown below.
| Method | Overall Score (%) | Avg. #Tokens (K) |
|---|---|---|
| M* | 52.85 | 89.47 |
| MemPro | 52.92 | 313.78 |
| PrisMem | 63.39 | 100.59 |
MemPro's version-tree-based evolution performs similarly to M*, while PrisMem outperforms both. This comparison further supports the benefit of capability-driven evolution beyond holistic, version-tree-based evolution. The results are included in Appendix C.5 of the revised manuscript.
Response to Reviewer fUa2 (1/3)
We appreciate your positive assessment of our results and broad empirical analysis. We are grateful for your valuable feedback, which we address in detail below.
PrisMem's gains persist across annotation models, label perturbations, and alternative capability definitions; they are not tied to the original five-capability definitions.
The purpose of capability annotation in PrisMem is to group tasks with similar memory requirements and optimization directions, making diagnostic feedback more focused and allowing improvements to be recognized and further developed. This principle does not depend on a benchmark-specific labeling scheme. The five capabilities in our paper are one concrete instantiation. We have applied the same annotation scheme to BEAM and LongMemEval and achieved strong results on both (Table 2), with transfer experiments supporting its generalizability (Appendix C.2). We also evaluate PrisMem under the same scheme on AMA-Bench [1], a non-conversational memory benchmark based on agent action–observation trajectories. PrisMem achieves 40.41% accuracy, outperforming M* (26.66%) and EvolveMem (13.44%), further suggesting that PrisMem's gains stem from the evolution framework rather than benchmark-specific task labeling.
[1] AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications. Yujie Zhao, Boqin Yuan, et al. ICML 2026.
To further show that the benefits do not depend on fixed capability definitions, we conduct additional experiments on annotation models, label noise, and alternative capability definitions. All experiments use Qwen3.8-27B, the same BEAM splits, and a 20-round evolution budget.
1. Different annotation models
Keeping the capability definitions and annotation procedure unchanged, we independently annotate the tasks using Qwen3.8-27B. The resulting capability labels achieve 91.79% agreement with the GPT-5.5 annotations. Despite minor differences in task classifications, the two annotation models yield nearly identical evolution results, demonstrating robustness to annotator choice.
2. Different levels of label noise
PrisMem is robust to moderate annotation errors: 20% label noise has little impact, while performance under 50% noise decreases but remains above M* (52.85%) and EvolveMem (40.01%). This tolerance stems from two design aspects. First, most tasks within each group retain similar memory requirements despite a few misclassifications, preserving sufficiently consistent responses to revisions for identifying capability-specific improvements. Second, every candidate is evaluated across all capability groups, allowing improvements triggered by mislabeled tasks to be recognized under their actual capabilities and further developed in the corresponding specialists. Together, these mechanisms mitigate the impact of individual misclassifications.
3. Alternative capability definitions
We construct two alternative definitions with different granularities:
Information retrieval and application, covering the retrieval and use of explicit facts, preferences, and constraints;Relational reasoning, covering temporal relations, state changes, and cross-session synthesis; andEvidence assessment, covering evidence sufficiency, conflicts, and abstention when necessary.Factual retrieval,temporal reasoning,state updates,preference extraction,multi-session synthesis,conflict resolution, andabstention under insufficient evidence. Compared with the default five capabilities, this further splits temporal tracking and adversarial into more specific sub-capabilities.Both alternative definitions achieve strong results, indicating that PrisMem's benefits are not tied to the specific five capabilities. The coarser three-capability definition yields a slightly lower score, as grouping broader, more diverse requirements within one group may reintroduce ambiguous optimization directions and hidden capability gains. Nevertheless, it still substantially outperforms holistic evolution baselines, demonstrating PrisMem's effectiveness under different capability definitions.
Overall, these results support that capability classification provides the organizational basis for PrisMem's evolution paradigm, serving as a way to organize tasks with similar memory requirements and optimization directions, rather than a benchmark-specific labeling scheme on which the gains depend. The robustness experiments are included in Section 4.5 and Table 5 of the revised manuscript.