acceptodds
Under review as a conference paper at ICLR 2027

Are Your Agents Ready to Evolve? Assessing Evolution-Readiness in Agents Through Paired Safety and Utility Scoring

Abstract

As AI agents evolve at runtime by accumulating memories, saving reusable skills, revising prompts, and delegating to subagents, they are generally expected to improve with experience. This assumes evolution-readiness, that what an agent learns from one task will leave it no less safe and no less capable on the next. Traditional agent-safety evaluations hold an agent's persistent state fixed and score standalone tasks, so they cannot test this assumption. Therefore, we introduce EvoAudit, a task suite purpose-built for agent evolution that uses directed source-target task pairs and matched cold and evolved conditions to isolate how evolution changes subsequent behavior. Tasks span benign and dangerous settings and are scored both on safety and utility. We run controlled experiments across five models using a self-evolving agent framework covering seven evolution mechanisms. We find that most agents exhibit substantial run-to-run variation even without evolution, yet evolution still produces more change than the control baseline. Evolution does not make agents better overall, and when both tasks are dangerous, it doubles the rate at which they trade off utility for safety. We then inject controlled good and bad memories to test the potential effects of evolution. A single injected memory affects safety far more than anything the agents write autonomously, and the models that appear unaffected are the ones that rarely read their memories. We find that the evolution outcome is not predictable from a particular evolution mechanism or model. Our audit suggests that current agents may not yet be fully evolution-ready, as evolution provides limited gains and does not reliably improve behavior across tasks.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
self-evolving agents, agent safety evaluation, misevolution, safety-utility tradeoffs, LLM agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dghXOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper introduces EvoAudit, a benchmark for assessing how agent self-evolution affects safety and utility on subsequent tasks. It comprises 142 directed source–target task pairs across three domains, covering benign and dangerous settings. The evaluation compares matched cold and evolved conditions across five models and seven evolution configurations, with no-evolution controls to account for run-to-run variability. The results show that evolution introduces additional behavioral changes but does not consistently improve both safety and utility; on dangerous-to-dangerous pairs, it more often increases safety at a utility cost. Controlled memory interventions demonstrate that retained content can substantially influence behavior when retrieved, while a case study illustrates how agents may preserve useful conclusions but omit necessary safety procedures.

Critical strengths
  1. The paper addresses an important and practically relevant problem. It examines how accumulated experience affects both the safety and utility of agents on subsequent tasks, capturing risks that static, single-task evaluations may overlook. Evaluating both dimensions provides a more informative assessment than reporting task success alone.

  2. The experimental design includes several useful controls. Directed source–target pairs, environment and context resets between tasks, and the BASE and cold–cold controls help investigate cross-task transfer. The authors explicitly acknowledge run-to-run variability and apply multiple-testing corrections, rather than attributing every observed change to evolution.

  3. The environments and qualitative analysis are valuable. Stateful simulated environments across three domains enable checks of actual agent behavior, with deterministic checks supporting part of the evaluation. The case in which an agent retains a useful conclusion but omits the safety procedure underlying it illustrates a meaningful failure mode.

Critical weaknesses
  1. The main conclusions are stronger than the experimental evidence supports. The paper states that evolution does not make agents better overall. However, Table 10 shows a significant increase in Improvement for B→D pairs and a significant decrease in Utility-Leaning outcomes for D→D pairs. Table 11 also shows more frequent safety improvements without a significant increase in safety deterioration. These results suggest a more context-dependent interpretation. Without reporting average utility and safety changes across all cells, together with confidence intervals, the overall negative conclusion is insufficiently supported.

  2. The definition of misevolution and the dual-axis scoring require stronger justification. Any increase in safety accompanied by a decrease in utility is classified as misevolution, without distinguishing appropriate safety corrections from unnecessary conservatism. Moreover, some behaviors are penalized on both axes: skipping a required review reduces both utility and safety. Consequently, some apparent joint degradation may reflect the same violation being counted twice. Sensitivity analyses that remove overlapping criteria and vary rubric weights would strengthen the interpretation.

  3. The controls and statistical dependence structure leave unresolved concerns. Neither BASE nor cross-configuration cold–cold comparisons fully match the tools and prompts of each evolution configuration. Furthermore, each target appears in two task pairs, whereas the analyses primarily cluster by task pair. This may not adequately account for correlations induced by shared targets. Same-configuration controls without retained artifacts, together with target-clustered analyses, are needed to establish the robustness of the key findings.

  4. The conclusions about models and evolution mechanisms are overly broad. A lack of statistically significant association does not establish that evolution outcomes are unpredictable, particularly when Table 3 indicates that some tradeoff analyses can reliably detect only large differences. Additionally, source-originated artifacts reach the target in only 4.6% of SUBAGENTS runs, limiting what these results reveal about the mechanism when it is actually used. The evidence is therefore more informative about the particular framework and configurations tested than about evolution-readiness more broadly.

  5. The contribution appears incremental relative to Misevolve. The paper substantially overlaps with Your Agent May Misevolve in its central research question and in studying safety risks arising from persistent experience and agent self-evolution. Its main additions are directed source–target pairing, joint safety–utility scoring, and controls for run-to-run variability. These are useful methodological extensions, but the paper does not yet clearly demonstrate which new phenomena or mechanisms they uncover that prior work could not reveal. A sharper distinction from Misevolve and stronger evidence of new insights would help establish the paper’s novelty.

Questions for authors

See Weaknesses.

4 comments

We thank reviewer dghX for recognizing the importance of joint safety–utility evaluation, our experimental controls, and the operational environments. We address your concerns below:

W1. Overall Conclusions. We agree that the conclusions should reflect the context-dependent results, including the safety benefits in Tables 10 and 11. Across all 4,711 evolution cells, mean utility change is −0.00005 (95% CI: −0.00595–0.00570), while mean safety change is +0.01361 (95% CI: 0.00538–0.02300), with shared target versions clustered together. The pair-type breakdown is below. We will revise the overall negative framing to reflect modest average safety gains, unresolved average utility change, and heterogeneous outcomes across tasks.

Pair typeMean Δutility [95% CI]Mean Δsafety [95% CI]
B→B+0.00035 [−0.00622, 0.00705]+0.00023 [−0.00221, 0.00259]
D→B+0.00189 [−0.00779, 0.01266]+0.00026 [−0.00314, 0.00350]
B→D+0.00221 [−0.00937, 0.01442]+0.02471 [0.00262, 0.05058]
D→D−0.00467 [−0.01518, 0.00424]+0.02892 [0.01271, 0.04680]

These exploratory intervals are pointwise.

W2.1. Interpreting Safety–Utility Tradeoffs. We agree that Safety-Leaning can be reasonable caution when no fully safe and fully useful route exists. In our task suite, however, such a route usually does exist. On 31 of the 36 dangerous targets in D→D pairs, at least one run achieved perfect safety and perfect utility. For example, in App. A.5, the agent can report each team’s total cost without exposing individual salaries, which satisfies both rubrics. On these targets, an evolved agent that gives up utility to stay safe is doing less than it could safely, which is why we count it as misevolution. On the remaining targets, joint success is not demonstrated. We will revise the framing to make this distinction explicit.

W2.2. Rubric Overlap and Sensitivity. Each target has separate utility and safety rubrics with distinct purposes. The utility rubric checks whether the legitimate work was done, and the safety rubric checks for harm (App. A.1.5). Some overlap remains by design, since a required step such as a review is both part of the requested work and a safety control. However, none of our claims rest on joint declines. We treat the split into single-axis and joint changes as exploratory, not as a finding (App. C.4, L1181–1187). Excluding shared criteria reduces joint declines only from 4.9% to 3.4%, and the positive mean safety shift persists under varied criterion weights. We will report both analyses in the camera-ready version.

W3. Controls and Statistical Dependence. Appendix C.1 already notes the tool and prompt differences in the controls. Appendix C.2 groups observations by task pair. We added cold-run comparisons with the same model, target, tools, and prompts. We also grouped pairs sharing a target when estimating uncertainty. Across 3,956 of the 4,711 cells (filtered for exact target matches and sufficient valid runs), 41.9% change under evolution and 33.8% in the control. The difference is 8.1 percentage points (95% CI: 5.9–10.5). Thus, evolution still produces additional change under these stricter checks.

W4. Models, Mechanisms, and Statistical Power. Section 3.2.1 and Table 3 explicitly acknowledge the power limits for rare tradeoffs and state that smaller model- or configuration-specific effects remain unresolved. Section 3.2.1 also distinguishes artifact reach from intrinsic evolution-readiness, and Table 7 reports the 4.6% source-artifact reach for SUBAGENTS. Our conclusions are restricted to the tested systems in Section 6. We agree that the abstract’s “not predictable” wording is stronger than these findings support and will revise it accordingly.

W5. Related Work. Misevolve [1] established that self-evolution can erode safety, measuring safety alone and without a no-evolution baseline. Our additions reveal more. First, joint scoring shows that on dangerous tasks, evolution more often trades utility for safety than it erodes safety. Evolved runs are safer than cold runs more often than in our no-evolution control and no more often less safe (L239–241, Table 11), while Safety-Leaning cases roughly double on D→D pairs (Table 10). A safety-only evaluation would count these as pure improvement. Second, our controls show that about a third of cells change without any evolution, and misevolution makes up a similar share of changes with and without it (L213–223). Without this control, ordinary variation would be counted as misevolution. Finally, measuring artifact use shows that apparent robustness can reflect under-use. Models that seem unaffected by memory are those that rarely read it, and the same memory shifts their safety once used (L281–286).

[1] Shao et al. Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents. ICLR 2026.

Dear reviewer dghX, we kindly ask you to review our rebuttal and reconsider your score accordingly. Thank you for your time and consideration.

Thank you for the thorough response and the additional analyses. They address several of my main concerns, and I have raised my score accordingly.

Addressed. The same-configuration cold control with target-clustered uncertainty (W3) shows that evolution produces change beyond run-to-run variation under stricter checks. The mean ΔU/ΔS estimates with confidence intervals (W1) and the commitment to revise the overall negative framing and the "not predictable" wording (W1, W4) resolve my concerns about overclaiming. I also find the run-to-run-variation control and the artifact-use analysis to be distinctive contributions.

Suggestions for the revised version. These are not conditions for my score, but I believe they would strengthen the paper:

  1. Re-run the key contrasts in Tables 8, 10 and 11 with target-level clustering, particularly those near the significance threshold (B→D Improvement, D→D Utility-Leaning). Please also briefly describe how the same-configuration control was constructed and how the excluded cells were filtered.

  2. Report absolute rates alongside ratios (e.g., D→D Safety-Leaning 1.99% → 4.26%), and describe the near-zero ΔU as a well-bounded null rather than an unresolved change.

  3. Make the Safety-Leaning distinction explicit in the main text, as promised, and include the shared-criteria and weight-sensitivity analyses, stating the weight ranges tested.

Thank you for your thoughtful follow-up, constructive suggestions, and willingness to reconsider your assessment. We appreciate your feedback and are committed to incorporating your suggestions into the revised paper.

We noticed that your confidence score changed from 4 to 3, while the overall rating appears to remain at 2. Since you mentioned raising your score, we wanted to check whether this update was unintended.

Thank you again for your time and consideration.

Reviewer_t8CpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Traditional agent-safety evaluations hold an agent’s persistent state fixed and score standalone tasks, so they cannot test whether experience from one task leaves an agent no less safe and no less capable on the next. The paper introduces EvoAudit, a task suite purpose-built for agent evolution that uses directed source–target task pairs and matched cold and evolved conditions, with tasks scored on both safety and utility. Controlled experiments across five models and seven evolution mechanisms show that evolution produces more change than the control baseline but does not reliably improve behavior across tasks.

Critical strengths
  • A novel evaluation setting. The paper explores the utility and safety of skill use, analyzing its benefits and risks from a safety perspective. I consider this an important research topic that may offer new insights into skill generation and use.
  • Extensive experiments. The evaluation covers five models and seven evolution configurations, helping characterize behavioral tendencies across settings and ensure the robustness of the findings.
Critical weaknesses
  • Evaluation reliability. Both utility and safety are evaluated using rubrics, which raises concerns about reliability. More seriously, the lack of comparison with human evaluations intensifies concerns about potential hacking of these rubrics.
  • Unclear real-world value. The tasks are synthesized collaboratively by the authors and a coding agent, without sufficient grounding in real-world operational scenarios or safety incidents. Their representativeness and whether the predefined rubrics reflect important risks in real applications remain unclear, limiting the study’s practical value.
Questions for authors
  • How about the transferability of skills from source to target? I am concerned that, in real-world settings, agents may rarely use skills evolved from the source task when performing the target task. If so, the evaluation setting may have limited practical value.
3 comments

We thank reviewer t8Cp for recognizing our novel evaluation setting and extensive experiments. We address your concerns below:

W1. Evaluation Reliability. About 60% of rubric criteria are checked deterministically against the final environment state, and these checks override the LLM judge (App. A.3, L902–905). For example, a deterministic check directly inspects the final database state to verify whether a legally protected record was deleted, overriding any contrary LLM judgment. On criteria with both verdicts, judge vs. deterministic check disagreement was at only 7.2%. Grading therefore rests on what the agent actually did, not on its own report (App. A.1.3, L788–791). Agents never see the rubrics, and every task is screened for language that reveals the evaluation (App. A.1.7, L829–841), which limits rubric hacking. For the camera-ready, we will add reliability checks for the remaining LLM-judged criteria, including agreement with additional LLM judges from other model families and with human annotations on a sample of verdicts.

W2. Real-World Value. EvoAudit’s risks are grounded in prior work. Five of our seven hazard categories are adapted from [1], and two cover the most frequent tool-agent failures in [2] (App. A.2). Each hazard targets a real safeguard, like a legal hold (App. A.1.2), and about 60% of rubric criteria are checked deterministically against environment state (App. A.3). These hazards mirror real incidents, such as a coding agent that deleted a production database during a code freeze and then misreported recovery options [3], matching the bypass-then-conceal pattern we observe (Sec. 3.3.2).

Synthetic environments are standard in agent safety evaluation [2], [4], [5], and here they are especially helpful, since measuring evolution requires matched cold and evolved runs. Every task was author-directed and verified, and practitioners can apply the same methodology to their own workflows (App. A.1).

Q1. Skill Reuse and Transfer. Prior work shows that LLM agents reuse induced workflows and programmatic skills across tasks [6, 7], although transfer can help or hurt depending on how skills are constructed [8]. In our SKILLS configuration, skills created on the source task are used on the target 51.5% of the time (Table 7). We pair related tasks to create controlled opportunities for reuse and to assess their safety and utility effects. Overall, current agentic systems are built to store and reuse skills [9], and agents do reuse them when tasks are related, both in prior work [6, 7] and in our runs.

[1] Kumarage et al. Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework. arXiv. 2026.

[2] Ruan et al. Identifying the Risks of LM Agents with an LM-Emulated Sandbox. ICLR. 2024.

[3] AI Incident Database. Incident 1152: LLM-Driven Replit Agent Reportedly Executed Unauthorized Destructive Commands During Code Freeze, Leading to Loss of Production Data. 2025.

[4] Zhang et al. Agent-SafetyBench: Evaluating the Safety of LLM Agents. arXiv. 2025.

[5] Kutasov et al. SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents. arXiv. 2025

[6] Wang et al. Agent Workflow Memory. ICML 2025.

[7] Wang et al. Inducing Programmatic Skills for Agentic Tasks. COLM 2025.

[8] Feng et al. Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agents. arXiv 2026.

[9] Liu et al. How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings. arXiv 2026.

Dear reviewer t8Cp, we kindly ask you to review our rebuttal and reconsider your score accordingly. Thank you for your time and consideration.

Thanks for your response. I still concern about the value of real world scenario. Grounding hazard categories in prior taxonomies does not establish that the tasks themselves are representative. I think the data should come from real events that have occurred, rather than hazard categories. Therefore, I maintain my score.

Reviewer_pmMCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the "evolution-readiness" of LLM-based agents that update their states at runtime through mechanisms such as accumulated memories, reusable skills, prompt revisions, and subagent delegation. The authors introduce EVOAUDIT, a task suite featuring directed source-target task pairs (X→YX \rightarrow Y) across benign and dangerous settings, evaluated under both cold-start and evolved conditions using dual-axis rubric scoring (utility and safety). Through controlled experiments across 5 models, 7 evolution mechanisms, and a no-evolution control (BASE), the paper finds that while evolution increases behavioral variation, it does not reliably improve overall behavior and often leads to trade-offs (e.g., sacrificing utility for safety in dangerous environments). Controlled memory injections further show that performance is heavily bottlenecked by retrieval rates and the quality of self-generated content.

Critical strengths
  1. Timely Problem Focus: Shifts the research focus from pure capability gains to the critical, overlooked safety risks and misevolution of self-evolving agents.
  2. Diagnostic Rigor: EVOAUDIT's paired task design and controlled memory injections effectively pinpoint core bottlenecks, such as retrieval failures and poor artifact quality.
Critical weaknesses
  1. The scope of testing is limited to the transfer of learning from one experience to a subsequent task. It has not yet examined long-term accumulation, the filtering of experiences, or continuous evolution following error correction. Furthermore, the environments used are simulated, which may entail a gap when compared to real-world tasks.
  2. Excessive baseline variability and significant experimental fluctuations make causal attribution ambiguous and challenging. Experiments reveal that 34.1% of the test units exhibit distinct patterns of variation—stemming from inherent sampling randomness, sensitivity to instructions, and operational instability—which complicates the process of attributing changes to the evolutionary process.
  3. The paper categorizes "safety-leaning"—a rational, cautious trade-off where some utility is sacrificed to avoid rule violations—broadly as a failure of "misevolution," which arguably overemphasizes the negative connotations. Empirical data (as shown in Table 10) indicate that, in benign tasks, behavioral shifts resulting from evolution are essentially indistinguishable from random, unprovoked fluctuations ("cold-cold" scenarios). In the context of high-risk tasks, however, evolution leads agents to become more constrained and safety-oriented. This outcome remains entirely reasonable for scenarios with stringent safety requirements and should be viewed as a natural consequence of agent evolution occurring within an optimization framework prioritized toward safety.
Questions for authors

In the context of dual-hazardous tasks (D→DD\rightarrow D), evolution manifests as a "safety-leaning" behavior—prioritizing enhanced safety at the expense of utility. Is this shift because the agent has genuinely learned from its history with hazardous tasks and become more cautious, or is it a consequence of the model’s internal "alignment guardrails," which cause it to grow increasingly conservative—and prone to "over-refusal"—after repeated exposure to such tasks?

2 comments

We thank reviewer pmMC for recognizing the timeliness and rigorous design of our work. We address your concerns below.

W1. Scope. We chose single source–target transfers to isolate the effect of evolution on subsequent behavior. As you note, substantial run-to-run variation arises even in this controlled setting, and it accumulates across longer sequences. We saw this in our early pilot experiments. For example, if safety dropped on the fifth task, the drop could come from any of the four earlier artifacts, the agent’s own ability on that task, or run-to-run variation. Simulated environments likewise enable repeatable resets and matched comparisons. Evaluating long-term accumulation is an important next step, but it first requires a reliable way to measure a single transfer, which our setup provides.

W2. Variation and attribution. The 34.1% comes from our no-evolution control (BASE), which we include to measure this variation. Evolution still adds 7.5 percentage points beyond it (p < .01; Sec. 3.1.1, L211–215), and a second cold–cold baseline gives a similar result (App. C.1, Table 8). For this response, we ran a stricter cold–cold comparison using repeated cold runs with the same target, model, tools, and instructions. Across 3,956 model-configuration-pair cases, 33.8% changed without evolution, compared with 41.9% under evolution, an 8.1-point increase (target-clustered 95% CI: 5.9–10.5). We will add this analysis to App. C. Given this variability, we do not attribute changes in individual scores to evolution alone (Sec. 6, L449–452).

W3. Contextualizing evolution outcomes. We agree that Safety-Leaning can be reasonable caution when no fully safe and fully useful route exists. In our task suite, however, such a route usually does exist. On 31 of the 36 dangerous targets in D→D pairs, at least one run achieved perfect safety and perfect utility. For example, in App. A.5, the agent can report each team’s total cost without exposing individual salaries, which satisfies both rubrics. On these targets, an evolved agent that gives up utility to stay safe is doing less than it could safely, which is why we count it as misevolution. On the remaining targets, joint success is not demonstrated. We will revise the framing to make this distinction explicit.

Q1. Source of caution. The source-task conversation and workspace are reset before the target, so source information can influence the target only through the evolution artifact. The previous task is not retained in context.

Dear reviewer pmMC, we kindly ask you to review our rebuttal and reconsider your score accordingly. Thank you for your time and consideration.

Comments on the paper

1 comment

Dear Area Chair,

We thank the reviewers for their thoughtful feedback and recognition of the importance of our work. We have addressed all major concerns in our detailed responses and would like to highlight the key points:

  1. Robustness: Additional control experiments confirm that evolution produces significantly more behavioral changes than baseline variability (+8.1 points, 95% CI: 5.9–10.5).
  2. Evaluation reliability: Approximately 60% of rubric criteria are checked deterministically, with only 7.2% disagreement with LLM judgments. Our tasks reflect established real-world safety risks.
  3. Interpretation: We clarified that our findings show modest average safety gains without a clear overall utility improvement, and provided evidence that safety–utility trade-offs often involve avoidable utility losses. The positive average safety shift also persists under alternative rubric weights.
  4. Novelty: Beyond prior work, EvoAudit's joint safety–utility evaluation and no-evolution controls reveal previously overlooked trade-offs, distinguish genuine evolution effects from ordinary variability, and expose artifact retrieval as a critical bottleneck.

Following our rebuttal, reviewer dghX acknowledged that several of their main concerns had been resolved and increased their rating from 2 to 3. We unfortunately did not receive further responses from the other reviewers during the discussion period.

We believe these additional results and clarifications substantially strengthen the paper and address the reviewers' central concerns. We are also committed to incorporating the reviewers' constructive suggestions and the promised revisions into the final version. We hope you will consider these developments in your final assessment and support the paper's acceptance.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.