- Rating
- 4(+2)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
Memory systems enable (some) active learning but can lead to safety issues. Can memory systems also enhance safety? This paper studies this question by providing a new memory creation harness that focuses on improving safety through memory files. The memory files are generated in a kind of "self-play", where a memory creation agent judges tool call trajectories for safety that were proposed earlier by another agent running on the same model. The approach lies on the safety-utility Pareto frontier for the AgentHarm benchmark among the approaches it is compared to. While memories are generated from a held-out portion of AgentHarm, the approach also lies on the Pareto frontier for the Agent-SafetyBench-160 benchmark, also without generating new memories for the benchmark.
Critical strengths
The self-play memory creation pipeline gives competitive results, showing that memory files alone can enhance safety. It is useful to have further techniques to obtain more options to choose from on the safety-utility Pareto front. While limited in scope and rigour, I find the empirical evaluation convincing in principle. See my questions for how I believe rigour could be improved.
Critical weaknesses
Exploring memory-only safety improvements is not as novel as stated in the paper. Previous approaches have already performed ablation studies, including evaluating their memory-only component. Papers that study this include: Mao et al. (2026), where evolved skills play an equivalent role to memories here; Liu et al. (2026) and Qu et al. (2026) who, while not evaluating memory-only, their evaluations show that the memory component is particularly vital to their approach.
As mentioned above, I have questions regarding the empirical evaluation.
Questions for authors
- How did you select the baselines you compared against? In particular, why did you not compare to SHE (Qu et al., 2026)?
- Is your evaluation setting on the benchmarks deterministic? If not, can you report results from several repetitions?
- Can you report, e.g., 3-fold cross validation results on AgentHarm?
- Your task-assessor system prompt contains the (very sensible) rule not to infer benchmark labels etc. Did you monitor benchmark awareness during evaluation?
Overall, I am somewhat critical of the empirical evaluation. If the above points can be resolved, I am happy to raise my score.
Update after rebuttal: The rebuttal contains convincing new results that significantly strengthen the empirical evaluation. For this reason, I have raised my score to clear accept and updated my soundness score.
Typos:
- the titles in the bibliography are not capitalised correctly
- EmbeddingGemma is not capitalised consistently in main body and appendix
- safety-utility tradeoff is not hyphenised consistently
6 comments
4. Benchmark Awareness
We thank the reviewer for suggesting that we examine benchmark awareness. We conducted two complementary checks: an audit of the original task-assessor records and an offline benchmark-awareness probe. The results show no evidence of explicit benchmark recognition.
(1) Original assessment audit. We searched all 176 original Phase 1 task-assessor records, including raw model responses and initial/final judgments, for benchmark-related terms (e.g., AgentHarm, benchmark, dataset, and official labels). No matches were found (0/176). The original assessor-visible task materials also contained no such references.
(2) Offline benchmark-awareness probe. We additionally tested the same Qwen3.6-27B model on all 176 original task inputs, providing only user requests and available tool names/descriptions. The model was explicitly asked to identify the benchmark from these materials or respond unknown. All 176/176 probes returned unknown, with no identification of AgentHarm or any other benchmark.
5. Baselines and SHE
(1) Baseline selection. We selected representative methods covering different safety mechanisms: ReAct (no defense), Self-Reminder (static safety prompting), WildGuard (input filtering), and MEMBRANE (evolving safety memory with an external guard). In particular, MEMBRANE provides a comparison with an existing memory-evolving safety method.
(2) Comparison with SHE. We agree that SHE is a relevant baseline. Unlike MOSE's memory-only evolution, SHE evolves four harness components, with its released evolution pipeline built around Agent-SafetyBench. A faithful comparison under our AgentHarm setting therefore requires adapting its execution and evolution pipelines, rather than simply changing the dataset. Moreover, its published AgentHarm results use a different actor and evolution protocol and are not directly comparable.
We have begun adapting SHE to our experimental setting, using the same actor, AgentHarm train/test split, and evaluation metrics. We aim to include the validated comparison in the final manuscript and will clarify the scope of our current baseline evaluation.
We sincerely thank the reviewer for carefully identifying these typos and inconsistencies. We will correct the bibliography title capitalization, standardize EmbeddingGemma throughout the paper and appendix, and consistently use safety–utility trade-off.
Thank you for providing these results. They significantly strengthen the evaluation. It makes sense that comparing to SHE is rather complicated.
The main reason I did not update to grade 4 is that the standard deviations for the multi-seed results are rather large relative to the scores. However, for three seeds, the standard deviation is also not very informative. Overall, your method is not the best by a small gap for one seed. Is it possible to run further seeds to get a clearer picture of how robust your result is?
We thank the reviewer for the constructive feedback. We clarify our contribution below and address the empirical concerns with additional multi-seed experiments, three-fold cross-validation, and benchmark-awareness checks.
1. Novelty of Our Memory-based Method
We agree that prior work has explored memory-based safety improvement and demonstrated the importance of memory in agent safety. Our contribution is not the use of memory itself, but studying whether agent safety can improve through memory-only self-evolution from unlabeled interaction experience, without updating model weights or other harness components.
MOSE builds safety memory from individual interaction trajectories and further refines safety boundaries through automatically discovered contrastive pairs, without requiring provided safety labels or predefined pairs. We will clarify this distinction and better position our contributions relative to Mao et al., Liu et al., and Qu et al.
2. Multi-Seed Results
We thank the reviewer for highlighting the need for repeated evaluations. We agree that a single run is insufficient to characterize experimental variability. We have therefore repeated every method with seeds 0, 1, and 2, and will update the main results table to report means and standard deviations, with per-seed results provided in the appendix. The two tables below present these aggregate and individual results.
The multi-seed experiments support the same conclusions as the original evaluation: MOSE achieves the highest mean BTS and exceeds every safety-defended baseline in BTS in each of the three runs. These results provide further evidence that MOSE’s safety–utility improvements are consistent across repeated runs.
Table 1. Main results on AgentHarm
test_public_Bover three runs (seeds 0, 1, 2). Entries are mean ± sample standard deviation in percentage points. HS/BS are harmful/benign task scores; HR/BR are refusal rates. BTS is the equal-weight safety–utility composite, computed from unrounded scores in each run. Bold and underlined mean BTS values denote the highest and second-highest means, respectively.Table 2. Per-run results underlying Table 1. Each run evaluates the same 88 paired requests in
test_public_B. Values are percentage points; BTS is calculated before rounding. For Ours, each seed covers memory evolution, inference, and evaluation.3. Three-Fold Cross-Validation
We conducted 3-fold cross-validation on AgentHarm test_public, splitting its 176 harmful–benign pairs into three held-out folds (58/59/59 pairs). For each fold, MOSE evolves memory from scratch using the other two folds (including both Phase 1 and Phase 2), then evaluates on the held-out fold using the official scoring procedure. The results are consistent across folds, indicating relatively stable safety–utility performance across the three data splits.
AgentHarm 3-fold cross-validation (percentage points):