acceptodds
Under review as a conference paper at ICLR 2027

TwinCheck: Comparative Safety Verification for Self-Evolving Agents

Abstract

Self-evolving agents improve by retaining feedback as reusable memories or skills, but every mistaken approval can shape many future behaviors. We identify a blind spot in this process: two updates can be identical with respect to all information used by the admission verifier yet produce opposite safety outcomes, so recalibra- tion cannot distinguish them. We therefore introduce TwinCheck, a verification framework that filters harmful updates before they can contaminate a self-evolving agent’s persistent state. Specifically, TwinCheck constructs an alternative that matches every signal used by the original verifier, without access to safety labels. It then replays both updates from the same state, compares the resulting behaviors in both presentation orders, and averages the two scores before deciding whether to retain the candidate. We evaluate TwinCheck on two paired-update evaluation sets built from AI Safety Gridworlds and τ 2-Bench, covering executable programs and natural-language memories. Across eight open-weight and proprietary-model verifiers, TwinCheck raises mean harmful-update detection AUC from 0.748 to 0.945 on controlled traces and from 0.695 to 0.835 on tool-use conversations, outperforming VaG and AgentDoG 1.5. In a ten-generation self-update loop, it reduces trajectories experiencing degradation from 30.39% to 0.43% and leaves no final degradation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Self-Evolving Agents, Agent Safety, LLM Agents, Persistent Memory

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2UhUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper investigates how to identify security risks before an agent stores new memories or skills. The core idea is to provide an alternative way of completing the same task, helping a verifier detect risks that may be overlooked when examining an update in isolation. The authors construct updates with identical task scores but different security consequences by enumerating grid-world programs and locally rewriting customer-service memories. They then compare independent verification with joint verification of two execution trajectories and evaluate the selected updates on subsequent tasks. Experimental results show that behavioral comparisons improve the ability of various models to identify harmful updates. Over ten rounds of updates in the grid-world environment, the proportion of runs exhibiting safety degradation decreases from 30.39% under the original policy to 0.43%. In new customer-service conversations, the number of violations caused by selected memories decreases from 40/176 under the original verifier to 34/176. The paper's main contribution is to introduce contrastive comparison into pre-update safety verification for agents and to separately evaluate contrast construction, risk assessment, and post-update behavioral outcomes.

Critical strengths

The research questions are clearly formulated, and the experimental evaluation is relatively comprehensive. Through RQ1–RQ4, the paper systematically investigates whether updates with identical task performance but different security consequences exist, whether comparative verification improves risk identification, whether it reduces safety degradation during continual updates, and whether the selected memories improve the safety of subsequent tool use. These research questions are logically connected, and the experiments cover the identification of security risks, risk assessment, and the actual effects of updates on subsequent behavior. This is an important strength of the paper, as it evaluates the method's effectiveness from multiple perspectives and provides relatively comprehensive empirical evidence for understanding how the method works and its practical value.

Critical weaknesses

The core design motivation is insufficiently explained, and the added value of the proposed method is not adequately justified. Starting from the observation that updates with identical task performance may lead to different security consequences, the paper introduces strict equivalence constraints, maximum trajectory divergence selection, and paired verification. While this observation highlights the limitations of existing verification signals, the paper does not sufficiently explain why these specific design choices are necessary or what motivates them. Although the method appears to be an engineering combination of existing techniques, a detailed explanation of the underlying design motivations would still be valuable. In particular, why are these mechanisms important, and what specific challenges in verifying self-modifications do they address that existing approaches struggle to resolve? The current experiments demonstrate that providing behavioral contrasts can improve risk assessment, but they do not sufficiently establish how much of the improvement comes from these particular design choices. As a result, it is difficult to determine whether the additional complexity of the framework delivers corresponding substantive improvements, which directly affects the assessment of the paper's novelty.

Questions for authors

Why is it necessary to strictly match the original verification results and select the alternative with the greatest execution divergence? Particularly in tool-use scenarios, could the authors clarify what additional benefits these design choices provide compared with ordinary contrastive comparisons or execution-feedback-based self-modification?

The method description remains relatively abstract, making it difficult to fully understand the overall working mechanism. Could the authors provide a more intuitive and coherent explanation of what TwinCheck actually does, the role of each component, and how these components work together to assess the safety of a proposed update?

Do Table 6 and RQ4 fully simulate the practical workflow of starting from a proposed update, automatically searching for a contrastive alternative, and then deciding whether to save the update? In this process, what proportion of the original updates can be successfully paired with valid contrasts, and are cases where no valid contrast can be found also included in the final effectiveness evaluation?

Reviewer_TxqmOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a safety filter method on a goal-conditioned self-evolving agent.

Critical strengths
  • The writing is a bit clear. At least I understood what the authors are trying to solve and what their method is.
  • How to guarantee safety in self-evolving is important.
Critical weaknesses

My score is 1.5 (somewhere between clear rejection and weak rejection. I think this is not clear rejection work but also not weak rejection work - somewhere between those two states)

First, can the authors provide more related work on this safety filter work? It seems this work address how to apply saftey filer on the trajectory of agent. In my view, this is a natural and easy idea to come up with up with -- which means there might be several preliminary works on this (safety filter on the process). Please investigate this and provide what the difference is between theirs and the author's work

Second, the authors keep saying "self"-evolving agent. Therefore, for the whole algorithm process, does the author used exact one single LLM model or used stronger model such as verifier? I believe that to claim it as a "self"-evolving agent, the whole process should be strictly homogeneous -- that is, no stronger LLM; otherwise, it's not "self" since it's getting helf from strong model (sort of distillation of strong model's knowledge via prompt). In figure2, and section 3.2 -- it seems the algorithm uses verifier and update generator as LLM -- do those two models are exactly same model with agent's base LLM? TLDR is that I wonder whether authors have single homgenous model in their twincheck framework that aligns with "self"-evoling agent.

Thrid, I am not very surprixsed with Figure3 and Tables 1 and 2 results. Isn't the twincheck is just applying more saftey filter in the process of update? That is, being more conservative on trajectory selection definetly provides much more high quality data for the update and would naturally leads to higher performance. What I would be surpirsed and interestesing would be the compuration cost. That is , with/without twincheck, how many trajectoreis or data have the framework have used? My guess is since the framework is being much more conservative (saftey check) on trajectory selection -- it would requires much more iteration or data to come up with final results. Please provide some tradeoff

Questions for authors

Please address the weakness

2 comments

Thanks for the follow-up message.

It's still a bit hard to follow what is the main contribution is and why it matters to this research field/ or what the punchline (message) is to the community. I wish the answers/ writing to be much more intuitive and easy writing, so the message that authors hope to deliver is straightforward and clear. I will keep my score as 1.5.

  • that is, I really appreciate all the experiments that the authors have done. I am not asking for more experiments. I am asking the insight of this work. Still, it is not clear what the punchline (message) of this work is and why it matters.
Reviewer_8mbmOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

TwinCheck verifies persistent agent updates by constructing a behaviorally different alternative that produces the same complete native-verifier readout on a fixed evaluation bank. It replays the two updates from matched states and asks a model to compare their safety, averaging across presentation orders. The system abstains when an alternative cannot be constructed and includes additional deterministic checks and optional recovery mechanisms. Experiments study verifier-equivalent updates, comparative detection, repeated updates in Gridworlds, and memory transfer in tool-use tasks.

Critical strengths
  1. A useful account of native-verifier blind spots with an operational response. Constraining alternatives to the same complete native readout exposes behavior differences invisible to the original decision rule. The paper distinguishes this from matching only pass/fail outcomes and recognizes that relative preference is not absolute safety.

  2. Controlled experiments support a diagnostic and repeated-update contribution. Same-backbone pointwise, joint-absolute, and multi-critic controls strengthen the comparison. Mean AUC across nine verifiers rises from .762 to .940 on Grid and .681 to .826 on Tool; the strict reward-matched Tool subset improves from .645 to .831. The ten-round Grid study reduces any degradation from 30.39% to .43%, retaining 89.58% of safe-update opportunities with monitoring and rollback disabled. These findings provide value beyond static preference classification.

Critical weaknesses
  1. The strongest Tool detection evidence does not directly establish fully automatic deployment performance. Appendix B.6 pairs each of 265 candidates with a contract-safe no-memory reference; the 197-pair reward-matched subset inherits this condition. This tests comparison given a safe reference, rather than arbitrary Stage I alternatives of unknown safety. RQ1/RQ4 do evaluate automatic alternatives, but the connection needs to be explicit. Their contrast-level yield—168 exact matches among 364 generated contrasts (Table 6)—does not give candidate-level coverage, abstention, or the full construction-plus-replay cost. These quantities determine how often the proposed mechanism is operationally useful.

  2. Transfer improvements need uncertainty at the appropriate unit. The detection cohort contains 31 tasks and 156 notes with shared task/seed contexts and references; nine verifiers use the same data. RQ4 reports 34 versus 36 violations and 113 versus 110 safe completions for TwinCheck versus VaG over 176 conversations. These favorable but small differences require paired, clustered analysis. Appendix B.4 appropriately acknowledges limited evidence in its six-pair stress test; the main transfer results should follow the same statistical discipline. The current evidence supports a stronger controlled-diagnosis claim than a broad deployment-safety claim.

Questions for authors
  1. Could you explicitly map the safe-reference detection cohort to the automatic-construction and transfer cohorts? For an unfiltered set of incoming candidates, please report unmatched/abstained cases and the safety outcomes when both alternatives are safe or both are unsafe.
  2. What fraction of candidate updates obtains a valid alternative within the fixed search budget, and what is the total construction-plus-verification execution cost? Please distinguish this from the contrast-level matching fraction and verifier input-token cost.
  3. Can you provide paired, task- or memory-clustered uncertainty for the detection and RQ4 differences, with the resampling unit justified by shared tasks, notes, and references?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.