- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces AgentRecover, a framework aiming to improve the utility of LLM-based agents while satisfying safety constraints. The approach combines runtime interception of unsafe actions with a recovery mechanism that identifies the causes of intercepted actions, determines permitted data sources, and constructs alternative actions. Recovery actions are executed only after passing safety and task-progress checks. The framework is evaluated using AgentDojo and Agent Security Bench (ASB), with results showing substantial improvements in task completion and reductions in attack success rates. The problem addressed by the paper is relevant. However, I have several major concerns regarding the conceptual foundations of the approach, its technical novelty, the scope of its security guarantees, and the generalizability of the experimental results.
Critical strengths
- The paper addresses a problem that needs more investigations.
Critical weaknesses
-
The paper lacks of a clear distinction between safety and security. The paper defines safety as the satisfaction of configured safety requirements. However, this is an operational definition that does not clarify the underlying security properties being enforced. Most of the scenarios concern prompt injection, unauthorized actions, and unauthorized information disclosure, which are primarily security issues. The paper does not explain whether safety is intended to subsume security or how the two concepts differ in the proposed framework. More importantly, satisfying a predefined set of constraints is not sufficient to guarantee either security or safety unless the constraints adequately capture the relevant properties. The paper should have clarified the scope and limitations of the guarantees it claims to provide.
-
The paper has limited technical novelty and the approach is dependent on predefined constraints. The proposed mechanism appears relatively straightforward: intercept a tool call that violates a predefined constraint, identify the relevant arguments, obtain replacement values from permitted sources, and recheck the resulting action. While potentially useful, the technical novelty of this approach is not established. The recovery construction presented in the appendix relies on configured templates implementing operations such as record filtering, field selection, and arithmetic. It is unclear how this approach generalizes to more complex agent workflows requiring reasoning about dependencies among actions or dynamically evolving security conditions. The paper should explain more clearly what distinguishes its method from existing runtime policy enforcement and corrective-action techniques, and also from practices used by companies when managing security incidents (for example the use of playbooks).
-
The paper is insufficient with respect to the handling of multi-step attacks and action dependencies. The approach focuses primarily on the analysis and correction of individual tool calls. Although the framework maintains information about confirmed effects and remaining tasks, it is unclear whether it can detect unsafe behavior arising from the cumulative effects of multiple actions, each of which individually satisfies the configured constraints. For example, an agent may execute a series of individually authorized operations that collectively result in unauthorized information disclosure. Can the framework identify such violations? How are temporal dependencies, information flows, and security-relevant state changes modeled? This is actually a well know issue in security since 1973 and resulted in models such as the well known Bell-LaPadula model. A related issue is that the original action may be generated by an agent following a compromised plan. Correcting individual tool-call arguments does not necessarily address an attacker-induced objective embedded in the overall plan. The paper does not discuss these issues and does not evaluate representative multi-step attacks.
-
The paper is not clear about the guarantees associated with “Certified Recovery”. In my view, the use of term "Certified Recovery" is not justified by the approach. As described in the paper, a recovery action is considered acceptable if it passes the configured safety and task-progress checks. However, the paper provides neither a formal definition of certification nor a proof establishing the properties guaranteed by this process. Does certification imply that recovery actions are guaranteed to be safe? If so, under which assumptions? What happens if the safety constraints are incomplete or the context used to evaluate them is incorrect? In addition, the appendix indicates that some authorization and task-progress checks in the ASB experiments are model-based. Such judgments are not necessarily sound. The authors should have either provides a precise formal characterization of the certification guarantees or use terminology that more accurately reflects the nature of the proposed checks.
-
The paper is unclear about the recovery mechanism and its trust assumptions. The paper should clarify the role of LLM-generated proposals during recovery, including whether the same model responsible for producing the original unsafe action is used to generate recovery candidates. Although candidate actions undergo additional checks, the paper does not establish that these checks are independent, complete, or resistant to manipulation. The framework also relies on permitted data sources, runtime metadata, and tool-effect descriptions. The trust assumptions concerning these components are not adequately specified. A more explicit threat model is needed, identifying attacker capabilities, trusted components, and the security properties that the mechanism is designed to preserve.
-
The experimental evaluation has limited generalizability. The reported numerical results are interesting, but the experiments do not convincingly establish the generality of the approach. The number of distinct underlying tasks is relatively small compared with the number of attack variants. In particular, the ASB attack evaluation uses only ten base tasks. Also, the appendix states that the evaluated task populations were available during method development and that the experiments used configured tool permissions and construction templates. This raises questions about possible benchmark-specific configuration and the ability of the approach to generalize to previously unseen tasks, tools, and attack strategies. The paper should have reported how the configurations were developed and include experiments on substantially more complex workflows, including multi-step attacks and previously unseen environments.
-
The recovery baseline comparisons has limitations. The comparison in Table 4 shows that all recovery variants achieve zero attack success when operating with the same guardrail. The principal improvement attributable to Certified Recovery is therefore task-completion utility rather than demonstrated additional security. Furthermore, the compared recovery methods have different action-construction and tool-selection capabilities. Consequently, the reported performance differences may partly reflect differences in the capabilities available to each method rather than the effectiveness of the recovery strategy itself. The authors should have carried out a more controlled ablation study that would have helped isolate the contributions of the individual components.
Some additional minor issues: A) The statement that AgentRecover “eliminates observed attacker success” is potentially misleading. The authors should have consistently described this result as zero recorded attacker successes in the specified evaluation, rather than implying that the approach eliminates attacks in general. B) The paper should have distinguished more clearly between action validity, authorization, safety, and task progress. C) The meanings of “Root Cause” and “Certified Recovery” should have been better clarified. In particular, identifying arguments implicated in a failed constraint is not necessarily equivalent to establishing the root cause of an unsafe action. D) The experimental results would have benefited from a detailed analysis of failed recoveries, undetected unsafe actions, and cases in which recovery improves utility.
Questions for authors
None