- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies compositional risks in agent self-updating: modifications to memories, prompts, or tools may each pass safety and task utility checks individually, yet their combined use can still lead to violations. In particular, failures may occur even when all pairwise combinations are safe and only emerge when three components are used together. The authors identify these phenomena through controlled experiments and propose maintaining a hypergraph to capture potential interactions among components, triggering safety checks when relevant combinations are actually involved in actions. The main contribution of the paper is to explicitly treat cross-component interactions as an object of safety verification and to explore how to reduce unnecessary checks. Results on three benchmarks show that this monitoring approach can achieve protection performance close to per-action checking with substantially fewer checks, while maintaining comparable task utility.
Critical strengths
This paper identifies an easily overlooked safety issue and provides solid empirical evidence demonstrating that it indeed exists. The memory, prompts, and tools of an agent can interact with each other, each update may pass safety checks when examined individually, yet their combined use can still lead to failures. Through controlled experiments, the authors identify such cases and even find examples where any pairwise combination of updates is safe, but safety issues only emerge when three components are used together. These results show that checking each update independently can indeed miss important risks, and they provide a clear direction for future research on what should be verified. The empirical study demonstrates the value and importance of this problem.
This paper proposes a solution from the perspective of reducing verification costs, which is highly valuable. The method focuses checks on cases where relevant components are actually involved together in actions, achieving safety performance close to comprehensive checking with fewer checks and lower latency, while maintaining comparable task completion performance. In real-world deployment, cost is an important consideration, and safety mechanisms need to account for operational overhead. This work demonstrates the possibility of achieving a better balance between protection effectiveness and cost.
Critical weaknesses
The coverage capability of local candidate interactions is insufficiently validated, weakening the applicability of the method in real-world scenarios. The method generates candidate interactions from local neighborhoods in the historical hypergraph, but component updates may introduce new associations beyond these historical neighborhoods. The paper does not sufficiently explain how such omissions are detected and mitigated, while runtime monitoring only covers interactions already represented in the hypergraph. Existing experiments do not adequately evaluate the method's coverage of newly introduced interactions, and the identification of third-order failures is not further connected to validation of the actual defense effectiveness. The key question is whether the method truly identifies and excludes irrelevant interactions, or whether it simply narrows the candidate space and reduces checking costs at the expense of missing critical risks. This issue directly affects the core value of the method: if it cannot effectively cover dangerous interactions introduced by continuous updates, the lower checking cost alone is insufficient to justify its practical applicability in real-world settings.
The scope of cost accounting is unclear, weakening the credibility of the central claim regarding overall cost reduction. The paper reports the proportion of runtime checks and additional execution time, but it does not clearly specify whether it fully accounts for the necessary overhead of initial hypergraph construction, neighborhood extraction during each update, candidate hyperedge generation, and LLM-based filtering, nor does it explain how initialization costs are amortized. These steps are integral components of the proposed method and should be included in the cost comparison. If the measurement mainly covers runtime monitoring, the reduction in safety checks may simply shift the overhead to interaction discovery and graph maintenance, which cannot support a claim of overall cost advantage. Cost reduction is one of the main contributions of the paper; therefore, the lack of a clear and complete end-to-end cost analysis directly weakens the empirical evidence supporting this contribution.
Questions for authors
When component updates introduce new interactions beyond the local neighborhoods of the historical hypergraph, how does the method discover and incorporate them into monitoring? For dangerous interactions missed during candidate generation or LLM-based filtering, are there any mitigation mechanisms or coverage analyses?
Does the Time Cost reported in Table 4 include the overhead of initial graph construction, neighborhood extraction during each update, candidate hyperedge generation, and LLM-based filtering? How is the initialization cost amortized? After including these overheads, does the reported cost advantage still hold?
The CFR on Agent Security Bench is significantly lower than those on other benchmarks. Beyond qualitative explanations based on task distributions and combination construction strategies, can the authors provide a breakdown analysis to show whether failures are distributed across different scenarios or concentrated in a small number of specific tasks or interaction patterns? This is important for understanding the generality and practical significance of the phenomenon.
Based on the identified third-order failures, can the authors provide an analysis or case study demonstrating how the method discovers the corresponding hyperedges and prevents violations, while also reporting the construction and checking overhead as the interaction order increases? This is important for evaluating whether the method can effectively handle higher-order risks that may occur more broadly.
9 comments
Clarification of Our Main Contributions
We would like to emphasize that our paper makes two distinct but complementary contributions. First, we identify and systematically validate a previously underexplored safety failure mode in self-evolving agent harnesses: component updates that are individually validated as safe and utility-preserving can still introduce unsafe behaviors when composed. Through controlled experiments across multiple benchmarks and heterogeneous evolution mechanisms, we establish that individual validation does not guarantee compositional safety. Second, motivated by this finding, we propose an incremental monitoring framework to detect such failures efficiently by tracking interactions among evolving component states. The monitoring framework is thus a practical response to the identified safety problem, rather than the sole contribution of this work.
We would like to kindly follow up on our rebuttal and the additional experimental results in the revised manuscript. We would appreciate it if you could let us know whether these additions have addressed your main concerns, or if any further clarification is needed. Thank you for your consideration.
We thank the reviewer for the constructive feedback. We clarify the following points.
Novel interaction discovery and coverage. Our hypergraph is updated online rather than restricted to previously observed interactions. When a component state is introduced or modified, update provenance identifies relevant existing states, and local completion generates new cross-component combinations within their interaction neighborhoods. Semantic filtering retains plausible combinations as predicted hyperedges, which are monitored before they are confirmed by execution. Once jointly activated, these interactions trigger safety checks before the corresponding action takes effect, and execution provenance subsequently confirms them as observed hyperedges.
Regarding missed interactions, our experiments demonstrate a favorable safety–cost trade-off. Across AgentDojo, Agent-SafetyBench, and Agent Security Bench, our method reduces residual failures from 22/157 to 2/157, 2/11 to 0/11, and 19/1,243 to 1/1,243, respectively, while checking only 33.6%, 20.1%, and 24.6% of candidate actions. On AgentDojo, increasing the neighborhood radius from 2 to 3 halves residual failures (4 to 2), whereas further increasing it to 4 yields no additional safety improvement but raises the check rate from 33.6% to 47.6%. These results demonstrate that our method approaches exhaustive checking in empirical safety effectiveness at substantially lower monitoring cost, although interactions outside the discovered candidate space are not guaranteed to be covered.
End-to-end cost accounting. Table 4 reports the additional wall-clock time of the complete verification pipeline, including incremental graph maintenance, neighborhood extraction, candidate hyperedge generation, LLM-based semantic filtering, and runtime safety checking, rather than runtime checking alone. Our hypergraph is constructed incrementally as component updates arrive, without a separate offline initialization phase. Thus, graph construction overhead is accounted for during online updates rather than requiring amortization of a global initialization cost. As shown in Table 4, our method reduces the additional time from 103.9 to 32.5 s/panel on AgentDojo, 139.5 to 28.1 on Agent-SafetyBench, and 80.5 to 19.8 on Agent Security Bench compared with exhaustive checking. These end-to-end measurements demonstrate that the cost advantage persists after accounting for online interaction discovery and graph maintenance. We clarify the timing scope and accounting procedure in the revision.
Failure distribution in Agent Security Bench. Our additional breakdown shows that the 19 compositional failures span five agent domains, nine target tasks, and eleven source–target pairs. Although system administration contributes 13 failures, the remaining six occur across finance, law, medicine, and autonomous driving. These results demonstrate that compositional failures extend beyond a single task or domain, while revealing variation across scenarios. We include the detailed breakdown with per-group eligible-panel counts in the revision.
Higher-order interaction detection and monitoring. Our method supports higher-order interactions through hyperedges connecting multiple component states. To further validate this capability, we add two tables in the revised manuscript. The first evaluates automatic three-way interaction discovery and violation prevention on the identified third-order failure sets, reporting detection recall and residual CFR. The second compares the construction and runtime overhead of pairwise and three-way monitoring, including candidate generation, semantic filtering, and safety checking. These additional results directly evaluate the effectiveness and cost of monitoring higher-order interactions.