acceptodds
Withdrawn from ICLR 2027

AgentRecover: Coordinated Interception and Recovery for Agent Safety and Utility

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Trustworthy large language model (LLM) agents must avoid unsafe actions (*Safety*) while completing legitimate user tasks (*Utility*). Runtime guardrails improve safety by intercepting unsafe calls, but interception alone can leave legitimate tasks unfinished and reduce utility. We present *AgentRecover*, a runtime method that coordinates interception and recovery to improve agent utility without relaxing safety requirements. Its central idea is to equip guardrails with precise feedback that supports Certified Recovery. AgentRecover introduces (1) *Guardrail with Feedback*, which intercepts unsafe calls and provides precise feedback, including root causes and permitted data sources, to support recovery, and (2) *Certified Recovery*, which constructs a recovery action based on the guardrail feedback. The action is executed only if it passes both the safety and task-progress checks. We evaluate AgentRecover on AgentDojo and Agent Security Bench (ASB): 1,749 attack variants of 107 user tasks across 14 application scenarios. Across five diverse LLMs on AgentDojo, AgentRecover reduces the mean attack success rate (ASR) from 27.54% to 0.00% and increases mean under-attack utility from 57.74% to 87.40%. Local comparisons against guardrail and recovery baselines show substantial gains. These results show that AgentRecover can preserve agent safety while substantially improving utility.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, Agent Safety, Agent Utility, Runtime Guardrails

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_4NU8OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces AgentRecover, a framework aiming to improve the utility of LLM-based agents while satisfying safety constraints. The approach combines runtime interception of unsafe actions with a recovery mechanism that identifies the causes of intercepted actions, determines permitted data sources, and constructs alternative actions. Recovery actions are executed only after passing safety and task-progress checks. The framework is evaluated using AgentDojo and Agent Security Bench (ASB), with results showing substantial improvements in task completion and reductions in attack success rates. The problem addressed by the paper is relevant. However, I have several major concerns regarding the conceptual foundations of the approach, its technical novelty, the scope of its security guarantees, and the generalizability of the experimental results.

Critical strengths
  1. The paper addresses a problem that needs more investigations.
Critical weaknesses
  1. The paper lacks of a clear distinction between safety and security. The paper defines safety as the satisfaction of configured safety requirements. However, this is an operational definition that does not clarify the underlying security properties being enforced. Most of the scenarios concern prompt injection, unauthorized actions, and unauthorized information disclosure, which are primarily security issues. The paper does not explain whether safety is intended to subsume security or how the two concepts differ in the proposed framework. More importantly, satisfying a predefined set of constraints is not sufficient to guarantee either security or safety unless the constraints adequately capture the relevant properties. The paper should have clarified the scope and limitations of the guarantees it claims to provide.

  2. The paper has limited technical novelty and the approach is dependent on predefined constraints. The proposed mechanism appears relatively straightforward: intercept a tool call that violates a predefined constraint, identify the relevant arguments, obtain replacement values from permitted sources, and recheck the resulting action. While potentially useful, the technical novelty of this approach is not established. The recovery construction presented in the appendix relies on configured templates implementing operations such as record filtering, field selection, and arithmetic. It is unclear how this approach generalizes to more complex agent workflows requiring reasoning about dependencies among actions or dynamically evolving security conditions. The paper should explain more clearly what distinguishes its method from existing runtime policy enforcement and corrective-action techniques, and also from practices used by companies when managing security incidents (for example the use of playbooks).

  3. The paper is insufficient with respect to the handling of multi-step attacks and action dependencies. The approach focuses primarily on the analysis and correction of individual tool calls. Although the framework maintains information about confirmed effects and remaining tasks, it is unclear whether it can detect unsafe behavior arising from the cumulative effects of multiple actions, each of which individually satisfies the configured constraints. For example, an agent may execute a series of individually authorized operations that collectively result in unauthorized information disclosure. Can the framework identify such violations? How are temporal dependencies, information flows, and security-relevant state changes modeled? This is actually a well know issue in security since 1973 and resulted in models such as the well known Bell-LaPadula model. A related issue is that the original action may be generated by an agent following a compromised plan. Correcting individual tool-call arguments does not necessarily address an attacker-induced objective embedded in the overall plan. The paper does not discuss these issues and does not evaluate representative multi-step attacks.

  4. The paper is not clear about the guarantees associated with “Certified Recovery”. In my view, the use of term "Certified Recovery" is not justified by the approach. As described in the paper, a recovery action is considered acceptable if it passes the configured safety and task-progress checks. However, the paper provides neither a formal definition of certification nor a proof establishing the properties guaranteed by this process. Does certification imply that recovery actions are guaranteed to be safe? If so, under which assumptions? What happens if the safety constraints are incomplete or the context used to evaluate them is incorrect? In addition, the appendix indicates that some authorization and task-progress checks in the ASB experiments are model-based. Such judgments are not necessarily sound. The authors should have either provides a precise formal characterization of the certification guarantees or use terminology that more accurately reflects the nature of the proposed checks.

  5. The paper is unclear about the recovery mechanism and its trust assumptions. The paper should clarify the role of LLM-generated proposals during recovery, including whether the same model responsible for producing the original unsafe action is used to generate recovery candidates. Although candidate actions undergo additional checks, the paper does not establish that these checks are independent, complete, or resistant to manipulation. The framework also relies on permitted data sources, runtime metadata, and tool-effect descriptions. The trust assumptions concerning these components are not adequately specified. A more explicit threat model is needed, identifying attacker capabilities, trusted components, and the security properties that the mechanism is designed to preserve.

  6. The experimental evaluation has limited generalizability. The reported numerical results are interesting, but the experiments do not convincingly establish the generality of the approach. The number of distinct underlying tasks is relatively small compared with the number of attack variants. In particular, the ASB attack evaluation uses only ten base tasks. Also, the appendix states that the evaluated task populations were available during method development and that the experiments used configured tool permissions and construction templates. This raises questions about possible benchmark-specific configuration and the ability of the approach to generalize to previously unseen tasks, tools, and attack strategies. The paper should have reported how the configurations were developed and include experiments on substantially more complex workflows, including multi-step attacks and previously unseen environments.

  7. The recovery baseline comparisons has limitations. The comparison in Table 4 shows that all recovery variants achieve zero attack success when operating with the same guardrail. The principal improvement attributable to Certified Recovery is therefore task-completion utility rather than demonstrated additional security. Furthermore, the compared recovery methods have different action-construction and tool-selection capabilities. Consequently, the reported performance differences may partly reflect differences in the capabilities available to each method rather than the effectiveness of the recovery strategy itself. The authors should have carried out a more controlled ablation study that would have helped isolate the contributions of the individual components.

Some additional minor issues: A) The statement that AgentRecover “eliminates observed attacker success” is potentially misleading. The authors should have consistently described this result as zero recorded attacker successes in the specified evaluation, rather than implying that the approach eliminates attacks in general. B) The paper should have distinguished more clearly between action validity, authorization, safety, and task progress. C) The meanings of “Root Cause” and “Certified Recovery” should have been better clarified. In particular, identifying arguments implicated in a failed constraint is not necessarily equivalent to establishing the root cause of an unsafe action. D) The experimental results would have benefited from a detailed analysis of failed recoveries, undetected unsafe actions, and cases in which recovery improves utility.

Questions for authors

None

Reviewer_V6DLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper explores the implementation of guardrails which allow the model to recover after refusal of unsafe actions. For this, the authors propose AgentRecover, which implements Guardrails with feedback, which guide the recovery, and constructing actions based on this feedback. They evaluate their method on AgentDojo and Agent Security Bench, where they discover a drop in attack success rate to 0%, while increasing utility.

Critical strengths
  1. The experiments are quite extensive, spanning two benchmarks, five models, and both guardrail and recovery method baselines.
  2. The results are quite strong, though it is not clear how to interpret them (see Weaknesses)
Critical weaknesses
  1. The paper omits all critical details in how this method actually works. For instance, Line 188 refers to the evaluate function which checks relevant safety conditions, but nowhere in the paper is described how this method works, and where the safety conditions come from, even though these are the main contributions of this paper. The same is also true for all critical functions this paper proposes.
  2. This same problem makes it also impossible to analyse the results, as it is unclear how much information the proposed guardrail and recovery method have, compared to existing baselines, as well as real-world applications. For instance, achieving stronger results is trivial if the method has access to the attack goal and benign instruction. However, since none of these crucial pieces of information are stated in the paper, it is impossible to know.
Questions for authors
  1. How do the actual methods that your method uses actually work (evaluate, identify cause, allowed source)?
  2. What is the threat model of the paper, and how much information do you assume the guardrail and recovery method possesses?
Reviewer_35FqOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes AgentRecover, which aims to improve an agent's ability to complete legitimate tasks while still blocking unsafe actions detected by a guardrail. For this purpose, AgentRecover first proposes to 1) have guardrails provide feedback (safety constraints, root causes, permitted data sources) and 2) constrauct a candidate recovery action using the feedback ("certified recovery"). The experiment results on Agent Dojo and Agent Safety Bench suggest that the proposed method can significantly reduce the attack success rate while improving the under-attack utility.

Critical strengths

(+) The paper addresses an important practical challenge: ensuring the safety of AI agents while maintaining and improving the utility for legitimate tasks is an important consideration that still needs further investigation.

(+) The overall approach of having the guardrail provide more feedback to the agent and perform a recovery by constructing a recovery action that is sepearately checked again before an execution makes sense, and subtantially improved the utility under attack for relatively simple scenarios covered by AgentDojo and ASB.

Critical weaknesses

(-) The experimental study relies on old benchmarks and old models. As a result, it is difficult to know if the main findings, such as near-zero attack success rates and high utility, will still hold for more adaptive attacks and for more recent frontier models.

For example, studies found that static attacks in early benchmarks such as AgenDojo are not sufficient to fully understand the security of an agent against adaptive attackers in practice, and proposed new benchmarks or evaluation frameworks. In order to more accurately evaluate the robustness of the proposed schemes, the paper needs to evaluate AgentReocver using more recent benchmarks such as DTap and AgentDyn.

Similarly, the experiments results are mainly based on relatively old models. For example, the in-depth evaluations largely rely on GPT-4o mini, which was released in 2024.

(-) While the overall approach of AgentReocver makes sense, the true novelty would come from the details in terms of how to exactly determine what the safety constraints are, how to determine the root cause and permitted sources. The paper does not fully discuss these details and how these approaches are new and novel compared to the state of the art.

As a minor note, I would also suggest reconsidering the use of the term "certified recovery." The proposed recovery approach, while it performs an additional safety check before rerunning the recovery plan, does not appear to provide any kind of certified safety or recovery guarantees.

Questions for authors
  1. How do AgentReocver experimental results change for more recent adaptive attack frameworks and using recent frontier models? Also, please include the comparisons with missing closely-related previous work such as TRIAD and ToolSafe.

  2. Please provide more details on how the safety constraint, root cause, and permitted sources are determined, and how the details of the cardio feedback and recovery plan are different and new compared to the state of the art.

  3. Please explain why the experimental results for the baselines, such as DRIFT, Progent, and Task Shield, reported in this paper are significantly different from what were originally reported in the previous work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.