- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
GraDual proposes a stateful dual-agent architecture for defending against indirect prompt injection. A Main Agent handles trusted planning and privileged actions, while a Dual Agent processes untrusted external content under restricted permissions. Before each delegation, the Main Agent generates a private precommitment, and the Dual Agent’s returned results must pass a code-based admission mechanism. A dynamic graph allows the execution plan to expand based on validated information. Experiments on AgentLAB show that GraDual achieves an observed 0% ASR across the reported settings while maintaining competitive task utility.
Critical strengths
- GraDual demonstrates a strong security–utility trade-off in the reported experiments, achieving an observed 0% ASR while maintaining task utility comparable to or higher than that of the undefended system.
- The ablations clearly distinguish the roles of the two main components: code-enforced precommitment primarily provides security, while dynamic graph growth helps preserve task utility.
- The authors compare multiple prompt-level and system-level defenses within a unified harness, use identical attack payloads across defenses, and repeat each setting six times, providing reasonably controlled comparisons.
Critical weaknesses
- Limited novelty. GraDual mainly combines existing ideas—dual-agent isolation, deterministic enforcement, and graph-based planning. Its technical contribution beyond integrating private precommitment and dynamic graph growth is unclear.
- Outdated and selectively chosen backbones. The evaluation uses GPT-4o and Qwen3-30B, selected partly to avoid very low baseline ASR. It is unclear whether the results generalize to current frontier models, especially since the attack payloads are replayed from GPT-4o trajectories.
- Missing efficiency evaluation. GraDual requires two stateful agents and additional delegation and verification steps, but the paper reports no token, latency, or monetary costs. Its utility advantage may therefore partly result from a larger inference budget.
Questions for authors
- Does GraDual maintain its security–utility advantage on more recent frontier models and with attacks not derived from GPT-4o trajectories?
- What are GraDual’s token, latency, and monetary overheads relative to the baselines, and do its utility gains remain under a matched inference budget?
3 comments
(1) Online adaptive attack by a stronger model. The payloads are no longer fixed in advance. They are generated online, during the evaluation of each defense, by a frontier attacker model (gpt-5.1) that is stronger than the gpt-4o agent backbone under attack. Following AgentLAB's search procedure, the attacker writes a long-horizon injection, runs it against the defended agent, observes how that agent actually behaved, and rewrites the payload from this trajectory, up to three times per cell. Each defense therefore faces payloads optimized against itself, and none is derived from the trajectories of the undefended GPT-4o agent. We run this on the 144 cells of the Banking suite, which is most susceptible to attacks. (A cell counts as breached if any round succeeds)
| System | Reported Replay ASR (Table 1, Banking) | New Adaptive ASR ↓ | Utility under adaptive attack ↑ |
|---|---|---|---|
| Undefended | 51.4 | 80.6 (116/144) | 80.6 |
| Tool filter | 3.4 | 4.9 (7/144) | 63.9 |
| Repeat user | 25.4 | 56.2 (81/144) | 81.9 |
| Spotlighting | 44.3 | 68.1 (98/144) | 75.0 |
| IPIGuard | 4.9 | 9.7 (14/144) | 67.4 |
| DRIFT | 4.7 | 30.6 (44/144) | 58.3 |
| CaMeL | 0.0 | 0.0 (0/144) | 35.4 |
| GraDual (ours) | 0.0 | 0.0 (0/144) | 62.5 |
Adapting to each defense makes the attack much stronger than the replay: it breaches 80.6% of undefended cells instead of 51.4% reported in paper, and every probabilistic defense loses ground, for example DRIFT from 4.7% to 30.6%. Only CaMeL with its security policies and GraDual are never breached, and GraDual keeps far more utility under attack (62.5% vs. 35.4%). GraDual's improved utility-security frontier thus holds against an adversary that is both stronger than the agent and adaptive to GraDual itself. We are running a full version of such evaluation for an addtional subsection on adaptive attack in the revised paper.
(2) Frontier backbones. We ran all defenses on five recent backbones from four vendors. On these safety-aligned frontier models, undefended ASR is already extremely low, so ASR cannot show the gain a defense brings (see the reason we chose gpt-4o and Qwen3 in Appendix B). We therefore compare benign utility over the 97 user tasks, where the cost of a defense shows.
| System | gpt-5.1 | gpt-5.5 | claude-sonnet-4.6 | gemini-3-pro | deepseek-v4-pro |
|---|---|---|---|---|---|
| Undefended | 79.4 | 88.7 | 84.5 | 94.8 | 94.8 |
| Tool filter | 50.5 | 4.1† | 6.2† | 4.1† | 4.1† |
| Repeat user | 78.4 | 89.7 | 83.5 | 95.9 | 90.7 |
| Spotlighting | 78.4 | 86.6 | 90.7 | 93.8 | 93.8 |
| IPIGuard | 69.1 | 77.3 | N/A‡ | N/A‡ | N/A‡ |
| DRIFT | 42.3 | 53.6 | 32.0 | 22.7 | 78.4 |
| CaMeL | 42.3 | 55.7 | 53.6 | 54.6 | 50.5 |
| GraDual (ours) | 66.0 | 81.4 | 72.2 | 77.3 | 80.4 |
† The model names no tool in Tool filter's selection step, so every tool is stripped (the same as with Qwen3 in Appendix B). ‡ IPIGuard's public implementation supports only GPT, Qwen and Llama backbones.
GraDual's utility rises with the backbone, from 63.5% on gpt-4o (same 97 tasks) to 66–81%, so its security mechanism does not cap it at the level of the older model. At the same security level it exceeds CaMeL with policies by 17–30 points on every backbone, and it is above DRIFT on all five. Prompt-level defenses stay close to undefended utility, but they enforce nothing: under the adaptive attack above they are breached in 56–68% of cells.
W3 / Q2 — Overhead and matched inference budget
What are GraDual's token, latency, and monetary overheads relative to the baselines, and do its utility gains remain under a matched inference budget?
We measured per-task token usage and wall-clock time against the system-level baselines from Table 1. GraDual uses 3.0× the tokens and 2.6× the time of the undefended agent, comparable to CaMeL (3.3× / 2.3×), DRIFT (2.3× / 2.7×) and IPIGuard (2.8× / 1.8×). Running two agents does not double the cost, because the dual agent is invoked only for steps that read environmental content, and the precommitment and admission checks are deterministic code taking under 1% of the time breakdown. The utility advantage is not bought with a much larger budget. The setting is AgentLAB long-horizon, gpt-4o, on a 40-cell subset per suite. We are running a full version of the table and going to add it in the revised manuscript.
| System | LLM calls | Tokens | Time (s) | × Undefended (tokens / time) | Cost (USD) | Utility ↑ | ASR ↓ |
|---|---|---|---|---|---|---|---|
| Undefended | 4.6 | 10.5k | 14.3 | 1.0× / 1.0× | 0.028 | 67.9 | 37.1 |
| IPIGuard | 7.7 | 29.1k | 25.9 | 2.8× / 1.8× | 0.079 | 64.1 | 3.7 |
| DRIFT | 13.4 | 24.2k | 38.8 | 2.3× / 2.7× | 0.071 | 52.8 | 5.2 |
| CaMeL | 8.1 | 34.2k | 33.2 | 3.3× / 2.3× | 0.098 | 56.9 | 0.1 |
| GraDual (ours) | 12.3 | 31.3k | 36.4 | 3.0× / 2.6× | 0.094 | 70.9 | 0.0 |
We hope these clarifications and additional results address the reviewer's concerns. We respectfully ask the reviewer to reconsider the assessment and score. We welcome any further questions!
We thank the reviewer for the careful reading and for recognizing the security–utility trade-off improvement. Below, we respond to the weaknesses, merging each question to weakness it follows from.
Response to Weaknesses -- W1
We would like to clarify that GraDual is not merely a combination of existing methods; it departs from prior patterns as a new agentic system designed to defend against IPI attacks. In IFC-based dual-LLM defenses such as CaMeL, FIDES, and PRUDENTIA, the quarantined LLM is a stateless, tool-less parser, and security relies on taint propagation checked against predefined policies. Achieving zero attacks with these approaches comes at the cost of hand-written policies for each tool and suite, as well as significant utility degradation. PRUDENTIA (ICLR'26, consecutive work on FIDES) also introduces multiple rounds of human approvals to recover some of the utility lost under FIDES's enforcement.
GraDual does not rely any form of taint propagation or IFC, nor on the DAG representation itself. Its core contribution is to provide two coupled but isolated, stateful agent loops in the IPI risks. Both agents have their own tools, while the dual agent is assumed to be fully compromised. At runtime, the main agent instantiates a contract (namely precommitment) from the current step's context, keeps the admission condition hidden from the dual, and admits only outputs that pass deterministic code checks. Security is therefore enforced by the interaction protocol instead of hand-written policies or human approval, while the agents' planning and reasoning capabilities support task utility. The DAG we exploited is a straightforward implementation for maintaining and updating runtime state, not a core novelty contribution.
W2 / Q1 — Frontier backbones and adaptive attacks generated online
We address this weakness and question in two-fold. First, instead of replaying payloads derived from GPT-4o trajectories, we generate attacks online with a frontier attacker model that adapts its payload to each defended agent, while keeping the paper's gpt-4o as backbone, which makes the adversary strictly stronger than the defending agent. Second, we run GraDual and all baselines on five recent frontier backbones to show that its utility carries over to stronger models.