- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper suggests that refusal under prompt injection is unfavorable, and propose a framework that tries to recover the correct action upon prompt injection. An offline agent proposes a ruleset offline for consequential data fields such as payment recipient and amount, and when the task agent (outside the trust boundary) performs an action via tool call, the approach deterministically validate the action against the ruleset and attempt to fix the hijacked action. Experiments on AgentDogo shows that the proposed approach successful run correct actions compared to approaches that simply refuse execution upon prompt inject.
Critical strengths
The position proposed in the paper that the agents should attempt to recover correct behavior rather than hard refusal upon detection of attacks is a useful and novel research direction.
Critical weaknesses
Weakly justified trust boundary
Despite the proposed approach use deterministic rules and linters to validate the task agent's actions, the rules themselves are proposed by an offline agent. Apparently the task agent is outside the trust boundary, but I am not sure why the offline rule extraction agent is treated within the trust boundary; in practice, a prompt-injected offline agent can generate malicious rules in the first place. What is the justification of this trust boundary setup?
The scope of the application
In order that AgentECC can function, it needs a trusted way to determine the ground truth each consequential tool argument. For example, to prevent unauthorized payment, it must have a trusted record of what amount to pay & who to pay. This seem to be strong assumption, as in practice, the agent may not have access to such trusted records (and often time this is the information to be inferred by the model from noisy user input and context).
Questions for authors
Please see the weakness section and justify the trust boundary and the scope of the application.
3 comments
Thank you for your response. I have a better understanding of the setups & assumptions of the paper now, and I have update my review scores.
Thank you for taking the time to read our response, and for updating your review. Your questions showed us where a careful reader would stop and ask what the paper assumes, and answering them made us state the setup and assumptions more clearly than we had. The paper is better for it. We sincerely appreciate the time and care you gave to this submission, and we are grateful for your open-mindedness throughout the discussion.
Response to Reviewer TXXi: what is trusted, and what happens without a record
Thank you for the review, and for recognizing recovery in place of hard refusal as a useful and novel direction. Both concerns led to changes in the main text (Sections 2 and 4.3); we answer each below.
New experiment. We ran the complete AgentDojo banking suite with no cell selected: all 16 tasks under all 9 attacker goals, 144 attacked cells per system, at two attack strengths. AgentECC admits no attack in any round and keeps more tasks than refusal-only blocking (Block) in every round: 101 against 72 under the weak attack, 80 against 64 under the strong one, and 98 against 66 when the user replies once to a refused call. The full end-to-end CaMeL port (banking security policies on) keeps 61 (Section 5.6 and Table 5 of the revised PDF).
W1. Why is the offline agent inside the trust boundary?
We trust the checked contract table, not the agent that drafted it. The trust root is the provider registry that the deployer writes, and nothing the synthesizer proposes can add a trusted source to it.
Revision. Section 2 now states this trust root and points to the test in Appendix B.9.
W2. Does AgentECC need a trusted record of every consequential argument?
No. Safety needs no record; a record only decides how much AgentECC recovers. The guard has three responses: restore the argument from a record, ask the user one scoped question, or refuse. Only the first needs a record. Redundancy gives the security, and interaction gives the recovery.
Revision. Section 4.3 now states that a missing record lowers recovery, not security, and Section 5.6 reports the complete suite.
We hope this resolves the soundness concern, and we are glad to answer any follow-up.