- Rating
- 4
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies how dangerous information in agent memory can be retained and propagated across tasks, with a focus on whether the safety of current behavior is sufficient to indicate that newly written memories are also safe. The authors design a three-stage evaluation pipeline: the agent first encounters a harmful source and generates memories, the original source is then hidden, and paired experiments with memory retention or deletion are conducted to measure the risks preserved in subsequent memories and downstream behaviors. Experiments across 18 models and 191 question-answering and tool-use cases show that retaining first-generation memories increases the proportion of residual risks in second-generation memories and significantly increases unsafe behaviors in question-answering scenarios. The study also identifies cases where behaviors are judged safe while the corresponding memories are still recognized as retaining potential risks. The authors further validate these measurements through human review, evaluator replacement, repeated runs, and memory-content interventions, showing that safety prompts and memory filtering can mitigate some risks. The main contribution of this work is a unified evaluation framework that jointly examines memory contents, subsequent retrieval, and actual agent behaviors to analyze how harmful information persists during the memory update process.
Critical strengths
The paper identifies an important yet easily overlooked safety issue for the community: safe current behavior does not necessarily imply that risks have been eliminated from memory. The paper studies how harmful information written into memory can continue to propagate into subsequent memories after the original source has been removed, and investigates this process through paired experiments that retain or delete newly generated memories. Evaluating only current responses or tool executions may overlook harmful information that the system continues to store and transmit. This work reveals an important evaluation blind spot and provides concrete evidence for moving beyond behavior-level safety checks toward assessing the safety of memory update processes.
The paper provides comprehensive and mutually supporting empirical evidence for the core phenomenon of risk propagation. Across 18 models, retaining first-generation memories increases the average residual risk in second-generation memories by 12.2% and 5.8% in question-answering and tool-use scenarios, respectively, with positive propagation gaps observed for 16 and 13 models in the two settings. Human evaluations also recover positive propagation differences in both scenarios, while additional experiments adapting structured memory with A-MEM observe the same trend. The consistency across models, human judgments, and different memory organization mechanisms provides stronger support for the paper's central claim that risks can propagate through memory updates.
Critical weaknesses
The research questions are clearly defined, and the experimental validation is relatively comprehensive. I did not identify any major issues that would affect the core conclusions. Reproducibility of the complete memory evolution process could still be further strengthened. The current repeated experiments fix the parent memories and mainly examine the stability of subsequent propagation under given memory conditions, but do not cover the impact of variations in initial memory generation on the entire evolution process. Adding a small number of independent repeated runs starting from the first task would help verify whether the formation and propagation of risks remain consistent across complete executions, further strengthening the reliability of the conclusions.
Questions for authors
Could the authors provide a small number of independent repeated runs starting from the first task, with each generation of memories regenerated, to examine whether the main findings on risk formation and propagation remain consistent across complete executions? This would complement the current stability validation with fixed parent memories and further strengthen the reliability of the conclusions.
3 comments
Thank you for your response, you’ve cleared up my main concerns. I will keep my positive score.
Thank You for Your Feedback and Continued Support
Thank you for confirming that our response has addressed your main concerns. We appreciate your constructive feedback, which helped us strengthen the repeatability evaluation, and your continued support.
Response to 5W79: Full-History Repeatability Across 18 Models
Thank you for highlighting the importance of testing variation across the full memory-generation process, beyond our earlier fixed-parent repeats.
The added full-history repetitions show that the aggregate finding persists when the memories themselves are regenerated. Fixed-parent repeats test downstream variation conditional on a given memory, leaving open whether the overall finding depends on a particular initial generation. We therefore repeat the complete history from Task 1 five times on all 18 models and 191 cases. With freshly generated G1 and G2 in every run, the panel-level Formation and Propagation results remain close to the original findings, and mean Propagation is positive in both tracks in all five runs.
Each repetition starts from Task 1 and carries its newly generated memories through the subsequent tasks. We retain the original initial memory pools, all 108 Answer and 83 Execution cases, model settings, top-3 retrieval, task budgets and scoring rules. Each run starts Task 1 afresh to generate G1. Task 2 regenerates both responses and both G2 branches under the within-run G1-kept/removed comparison. Task 3 regenerates both responses using that run's G1 and natural kept-G2 for the G2-available/unavailable comparison. Generated memories are shared where required for within-run pairing, but never reused across runs. All five repetitions are included: 17,190 planned histories, with 17,140 complete through Task 3.
The five-run averages remain close to the original main-table results, with modest variation between panel-level runs. Formation measures strict residue in G1; Propagation is the kept-minus-removed strict-residue rate in G2. The table reports the mean ± sample SD across five run-level, equal-weight 18-model averages.
The original 18-model references are 65.12% / 32.62% for Formation and 12.19 / 5.76 pp for Propagation (Answer/Execution). Across the five new runs, Propagation is 12.79, 12.12, 11.03, 11.50, 12.29 pp for Answer and 4.54, 5.28, 7.65, 6.85, 6.22 pp for Execution.
These repetitions cover variability introduced at initial memory generation and carried through subsequent evolution. These run SDs describe panel averages; median within-model Propagation run SDs are 2.71/3.47 pp (Answer/Execution). For each metric, we use model–case units evaluable in all five runs, average cases within each model/run, and weight models equally. Missing scores are excluded.
We have incorporated these results into the main text and provided the full supporting tables and per-case scores:
revision_addendum/E01_full_history/:tables/case_scores.csvprovides model–case–run scores;tables/by_run.csv,tables/by_model.csvandtables/summary.csvprovide the summaries. The module includes the case manifest, scoring rules, analysis configuration andanalysis.pyfor recomputation.case_studies/supplies task context and all five G1/G2-kept/G2-removed text comparisons for each selected case.Taken together, this validation covers all 18 models and the 191-case benchmark used for the original main table, with five independent repetitions of the full generation-and-evolution protocol. The aggregate Formation and Propagation results remain close to the original findings, and both tracks retain positive mean Propagation in every run. This full-panel replication directly addresses the fixed-parent concern: the central memory-state finding persists across independently regenerated histories, rather than depending on one fixed set of generated parents.