- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
This paper investigates how to identify security risks before an agent stores new memories or skills. The core idea is to provide an alternative way of completing the same task, helping a verifier detect risks that may be overlooked when examining an update in isolation. The authors construct updates with identical task scores but different security consequences by enumerating grid-world programs and locally rewriting customer-service memories. They then compare independent verification with joint verification of two execution trajectories and evaluate the selected updates on subsequent tasks. Experimental results show that behavioral comparisons improve the ability of various models to identify harmful updates. Over ten rounds of updates in the grid-world environment, the proportion of runs exhibiting safety degradation decreases from 30.39% under the original policy to 0.43%. In new customer-service conversations, the number of violations caused by selected memories decreases from 40/176 under the original verifier to 34/176. The paper's main contribution is to introduce contrastive comparison into pre-update safety verification for agents and to separately evaluate contrast construction, risk assessment, and post-update behavioral outcomes.
Critical strengths
The research questions are clearly formulated, and the experimental evaluation is relatively comprehensive. Through RQ1–RQ4, the paper systematically investigates whether updates with identical task performance but different security consequences exist, whether comparative verification improves risk identification, whether it reduces safety degradation during continual updates, and whether the selected memories improve the safety of subsequent tool use. These research questions are logically connected, and the experiments cover the identification of security risks, risk assessment, and the actual effects of updates on subsequent behavior. This is an important strength of the paper, as it evaluates the method's effectiveness from multiple perspectives and provides relatively comprehensive empirical evidence for understanding how the method works and its practical value.
Critical weaknesses
The core design motivation is insufficiently explained, and the added value of the proposed method is not adequately justified. Starting from the observation that updates with identical task performance may lead to different security consequences, the paper introduces strict equivalence constraints, maximum trajectory divergence selection, and paired verification. While this observation highlights the limitations of existing verification signals, the paper does not sufficiently explain why these specific design choices are necessary or what motivates them. Although the method appears to be an engineering combination of existing techniques, a detailed explanation of the underlying design motivations would still be valuable. In particular, why are these mechanisms important, and what specific challenges in verifying self-modifications do they address that existing approaches struggle to resolve? The current experiments demonstrate that providing behavioral contrasts can improve risk assessment, but they do not sufficiently establish how much of the improvement comes from these particular design choices. As a result, it is difficult to determine whether the additional complexity of the framework delivers corresponding substantive improvements, which directly affects the assessment of the paper's novelty.
Questions for authors
Why is it necessary to strictly match the original verification results and select the alternative with the greatest execution divergence? Particularly in tool-use scenarios, could the authors clarify what additional benefits these design choices provide compared with ordinary contrastive comparisons or execution-feedback-based self-modification?
The method description remains relatively abstract, making it difficult to fully understand the overall working mechanism. Could the authors provide a more intuitive and coherent explanation of what TwinCheck actually does, the role of each component, and how these components work together to assess the safety of a proposed update?
Do Table 6 and RQ4 fully simulate the practical workflow of starting from a proposed update, automatically searching for a contrastive alternative, and then deciding whether to save the update? In this process, what proportion of the original updates can be successfully paired with valid contrasts, and are cases where no valid contrast can be found also included in the final effectiveness evaluation?
Thanks for the follow-up message.
It's still a bit hard to follow what is the main contribution is and why it matters to this research field/ or what the punchline (message) is to the community. I wish the answers/ writing to be much more intuitive and easy writing, so the message that authors hope to deliver is straightforward and clear. I will keep my score as 1.5.