- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies safety failures during repeated LLM based revision of executable agent components. It separates three controller decisions: whether a program passes current validation, which program is executed, and which program is supplied to the next editor. In constructed authorization tasks, stale archive scores can retain programs that become invalid after a contract change, a keep rule can continue running a failed incumbent when every candidate is rejected, and editing the correct founder can improve recovery. The paper evaluates these effects across several editors, four task families, selected components from OpenEvolve and DGM, and a Casbin task. It concludes that current deployment validation and validated rollback can preserve correctness while retaining some service cost savings.
Critical strengths
-
The distinction among failure detection, deployment, and repair is clear and useful. The paper correctly emphasizes that finding a defect does not determine what executes next.
-
The experimental bookkeeping is careful. Paired branches share programs, proposals, or candidate pools where appropriate. Protected effects are recorded independently of program outputs, and the paper distinguishes safety, full correctness, final correctness, and safety throughout a trajectory.
-
The authors report intermediate states rather than only favorable endpoints. The examples of stale permission caches make the observed mechanisms concrete. The discussion of unknown outcomes, repeated states, validation cost, and test coverage is also unusually transparent.
Critical weaknesses
-
The main findings have limited conceptual novelty because they largely follow from the controller definitions. Scores measured under an obsolete contract can rank an obsolete program highly. Equation 1 explicitly returns the failed incumbent when the eligible set is empty, so continued execution after rejection is a direct consequence of the rule. Founder fallback selects an implementation assumed correct by construction. The experiments quantify how often these cases occur, but the proposed remedies are standard revalidation, rollback, and pausing rather than a new learning method, safety mechanism, or theoretical result.
-
The recursive self improvement framing is broader than the evidence. The editors and improvement procedures are fixed, while the evolving objects are small Python authorization components with externally supplied contracts and oracles. The framework experiments still use custom authorization evaluators, the DGM study transfers only selected functions, and some related systems are inspected rather than executed. Most importantly, the native dialogue transfer executes no proposed writes and finds no established increase in authorization failures over evolution. The evidence therefore supports a claim about regression control and cache invalidation in iterative program optimization, but not a general claim about safety in self improving agents.
-
The safety conclusions remain strongly conditional on test coverage. Public, audit, and hidden splits are seeded instances of the same designed categories. Independently composed streams still expose unsafe endpoints after current validation and score refresh. Thus statements that full validation or rollback ends with fully correct programs apply to this benchmark and oracle, not to safety under an incomplete contract or distribution shift. A stronger paper would compare against runtime enforcement, versioned certificates, adversarial test generation, or formal authorization invariants on genuinely unseen mechanisms.
-
The utility evidence is difficult to interpret outside the simulator. Service costs are manually assigned logical weights. The headline deployment savings exclude model generation and usually exclude validation. Figure 11 shows that validation outweighs savings at low deployment volume. End to end latency, compute, monetary cost, and realistic deployment frequency are needed before concluding that the controls preserve useful efficiency.
-
The confirmatory interpretation and reproducibility need improvement. The seed count was expanded after the pilot was inspected, yet the pilot and expansion are pooled for inference. The shared failure study conditions on a selected set of failures and its largest effect is dominated by Qwen, which limits generalization. No code, data, prompts, traces, or executable artifact is available with the reviewed submission, despite the complexity of the custom simulators and analysis.
Questions for authors
-
What nontrivial guarantee remains when the validator is incomplete or when no known correct founder exists?
-
How much of the founder editing advantage remains after controlling for code size, distance to a correct solution, and editor specific failure incidence?
-
Can the central interventions be evaluated in a complete agent improvement system with real protected actions, unseen tasks, and measured end to end cost?
6 comments
Response to Reviewer mHVF [2/5]
W2. Scope of the self-improvement framing
We agree, and the revised paper will narrow the framing (C1). It will describe the main experiments as regression control and cache invalidation in iterative, LLM-driven program optimization, for the case in which the optimized component keeps executing protected actions. It will call this setting self-evolution of agent components, following the self-evolving-agent literature [1, 2, 3], and it will not claim a general account of safety in self-improving agents. The specific gaps listed under this weakness are addressed as follows.
- Fixed editors. In a new experiment the improver rewrites its own strategy. Unsafe programs become rarer, 7 of 24 histories against 22 of 24, and stored scores keep the unsafe program deployed in all 29 affected histories (E1, C2).
- Small components, external contracts, and custom evaluators. The handlers are small so that every protected effect can be audited, and the contract is supplied externally because the deployer, not the agent, defines who may do what. To replace the custom evaluators, we replayed the evolved programs on Casbin and OpenFGA. Programs selected by stored scores commit unauthorized writes in 35 of 72 histories on the original request streams, and none of the 120 programs selected after refresh does (E2, C3).
- Selected DGM functions and inspected systems. We execute DGM's parent-sampling, archive-update, and score functions unchanged because they implement the selection decision. This is not a full DGM run, and the main text will say so. Behavioral claims rest only on executed code. Source inspection supports one claim, that selecting on recorded scores without re-evaluation is the default (E7, C5).
- Native dialogue. Appendix G will still be reported as mixed. It did not show that evolution makes unauthorized proposals more frequent. It did show that the underlying failure occurs in released environments before any evolution: after a consent withdrawal followed by 4 or 12 neutral dialogue pairs, 42 and 21 of 224 next-action queries propose an unauthorized write. The paper's central claim is a different one: once the evaluator changes, the selection rule decides whether an invalidated version keeps running. We tested this claim in the same environments with executed writes. Stored scores select a policy-violating configuration in 24 of 64 lineages, and current scores in 3 (E8, C4).
W3. Test coverage and unseen mechanisms
Validation protects only what the tests cover, and every correctness claim in the revision will be stated relative to the evaluated suites and oracle. In the suggested comparison (new experiment E5), stress tests remove the failures that current validation leaves on new compositions and on an unseen mechanism that is consistent with the declared contract. For an undeclared change that contradicts the contract, only declaring and testing the change or runtime enforcement removes the failures.
We added two authorization mechanisms that appear in no suite, prompt, or document.
- U1, principal suspension. Suspending the hidden owner of a token disables the token.
- U2, shared account quota. Executing one token can exhaust a budget that it shares with sibling tokens.
Both respect the documented API, so the founder stays correct. We generated 64 held-out streams per mechanism and family, and recorded hashes of the generators and streams before evaluating any evolved program. We then compared the controls on the 48 framework histories at two endpoints.
| Iteration 100, no refresh | Independent | U1 | U2 | Savings (%) |
|---|---|---|---|---|
| Stored scores | 24 | 24 | 29 | 49.4 |
| Current validation | 6 | 6 | 31 | 45.2 |
| With random stress tests | 1 | 1 | 27 | 45.0 |
| With LLM-generated stress tests | 0 | 0 | 27 | 44.8 |
| Monitor with a formal model of the declared contract | 0 | 6 | 31 | 45.2 |
| Certificates, change declared and tested | 0 | 0 | 0 | 36.6 |
| Runtime monitor on the live authorization state | 0 | 0 | 0 | 7.7 |
| Iteration 60 | Independent | U1 | U2 | Savings (%) |
|---|---|---|---|---|
| Stored scores (no refresh) | 25 | 25 | 28 | 49.1 |
| Refresh at iteration 20, then continue | 3 | 3 | 29 | 44.4 |
| With random or LLM-generated stress tests | 0 | 0 | 27 | 44.4 |
| Monitor with a formal model of the declared contract | 0 | 3 | 29 | 44.4 |
| Certificates, change declared and tested | 0 | 0 | 0 | 36.5 |
| Runtime monitor on the live authorization state | 0 | 0 | 0 | 6.9 |
Response to Reviewer mHVF [3/5]
Counts are histories out of 48. The table separates three kinds of unseen behavior.
- New compositions of known mechanisms. Current validation leaves 6 unsafe histories at iteration 100 and 3 at the refresh endpoint. LLM-generated stress tests remove them at both endpoints, and random stress tests leave one at iteration 100.
- An unseen mechanism that is consistent with the declared contract (U1). The pattern is the same. The stress tests do not exercise U1. They reject the same fragile programs for composition failures, and those are the programs that fail under U1.
- An undeclared change that contradicts the declared contract (U2). U2 lets one token's execution affect a sibling, which the declared contract implies cannot happen. Here current validation gives no benefit over stored scores (31 against 29 at iteration 100, and 29 against 28 at iteration 60), and neither does any other control derived from the declared contract. Among the 4,737 distinct programs that pass the current suite, out of the 6,326 distinct programs proposed in the 48 histories and their two continuations, U2 reveals unauthorized effects in 57.7%, while random and LLM-generated stress tests reveal them in 2.3% and 3.2%.
Only two controls handle U2: declaring and testing the change, after which versioned certificates work, and enforcement against the live authorization state. Enforcement has a price. The savings come from avoiding authoritative checks, so a monitor that pays for an authoritative check on every action reduces savings from 44.4% to 6.9%. A colocated monitor priced at one to three units would keep 35% to 41%. The monitor also does not repair programs or restore missing permitted work.
The conclusion is therefore stated for this benchmark and oracle: full validation and rollback end with fully correct programs. They close the selection gap and leave the coverage gap, so selection controls and runtime enforcement are complementary. The revised discussion will state what each control guarantees.
W4. Utility outside the simulator
Assigned weights alone do not establish utility, so we measured what the headline excluded (new experiment E6). On two real authorization stacks, savings remain positive against a single-check baseline and are much smaller than against the founder, and the loop repays its cost only at sufficient request volume.
Model and validation cost. One edit takes 11.3 s and 0.018 USD with Sonnet, and 59.0 s and about 0.1 USD with GPT. Validating one program takes 0.087 s of CPU time. A full archive refresh takes 1.7 s and a lazy refresh 0.35 s.
Deployment latency. The embedded stack runs Casbin with SQLite in the client process. The remote stack runs OpenFGA on one server and the replay client on a second server in the same data center, with protected actions committed to PostgreSQL on the service host. Authoritative checks cross the network, and version reads are served from a local cache that follows the service's change stream. The replay is sequential and applies every change-stream event before each request, so a version read never lags the service state. Every operation was timed over five passes. These measurements are conditional on this synchronized, sequential, single-client workload: cache maintenance is not reported as a separate operation, and concurrent load was not measured.
| Median latency | Authoritative check | Version read | Committed write |
|---|---|---|---|
| Embedded (Casbin, SQLite) | 115 µs | 6 µs | 56 µs |
| Remote (OpenFGA, PostgreSQL) | 2.9 ms | 4 µs | 1.1 ms |
On the remote stack the 95th percentiles are 6.4 ms, 9 µs, and 2.3 ms.
Savings against two baselines. The headline compares each endpoint with the founder of the same seed, and some founders check permission more than once per request. We therefore also compare against a handler that makes exactly one authoritative check per request.
| Endpoints, 48 histories each | Baseline | Weighted savings (%) | Embedded: authorization latency saved (%) | Embedded: total latency saved (%) | Remote: authorization latency saved (%) |
|---|---|---|---|---|---|
| Full refresh | Founder of the same seed | 43.9 | 47.7 | 40.3 | 53.9 |
| Lazy refresh | Founder of the same seed | 44.5 | 48.4 | 41.2 | 54.6 |
| Full refresh | Single check per request | 16.6 | 13.7 | 10.4 | 23.2 |
| Lazy refresh | Single check per request | 16.9 | 14.7 | 11.3 | 24.3 |
Savings remain positive against the single-check baseline on every stack, and they are much smaller than against the founder. The revised paper will report both baselines wherever it reports savings. On the remote stack an authoritative check costs about 700 version reads, far above our assigned ratio of 12, which is why measured savings there exceed the weighted ones.
Response to Reviewer mHVF [1/5]
Thank you for the thorough assessment. We appreciate your recognition that the distinction among failure detection, deployment, and repair is "clear and useful", and of the experimental bookkeeping and the reporting of intermediate states. Several of your points lead to changes in the paper. For the others, we add experiments or clarify what the evidence supports. We will upload the revised PDF shortly, with all changes marked in blue. Material shared by several reviews, including the new experiments E1, E2, E3, E7, and E8, is in our Response to Common Questions, posted as a separate comment on this page, and we refer to its sections as C1 to C6.
W1. Novelty relative to the controller definitions
The remedies are standard, and the revised paper will present them as such. The contribution is empirical: the paper measures a gap that the controller definitions permit but do not quantify, and the revised discussion will be organized around it.
A deployed program can be unsafe for two different reasons. In a coverage gap, the current validator cannot detect the failure. In a selection gap, the current validator would reject the program, and the controller runs it anyway. The definitions say that a selection gap can exist. They do not say how large it is, whether continued search closes it, or which standard remedy closes it. These are what the paper measures.
The guarantee, stated as a proposition. With deployment revalidation and validated fallback or pause, no program that fails the current validator executes, so unsafe execution is confined to the validator's false passes. Stored-score selection and the keep rule do not have this property. The statement is simple, and the revision will include it because it makes precise what the selection controls provide and what they leave to testing and enforcement.
On Equation 1. The definition does not say how often every proposal fails. With identical incumbents, proposals, and tests, every proposal was rejected in 6 of 42 GPT cases and 8 of 42 Sonnet cases. The keep rule restored safety in none of them, and founder fallback in all of them.