- Rating
- 4
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents DTAP, a red-teaming platform for AI agents. It has simulated environments in 14 domains, more than 50 of them, built to mimic services such as Gmail, Slack, PayPal and Salesforce through their MCP/CLI and GUI interfaces. On top of the platform, the authors build DTAP-Red, an automated red-teaming agent. It searches over injection payloads and locations across prompt, tool, skill and environment vectors, and their combinations, and uses feedback from rule-based judges that check the environment state. They use it to build DTAP-Bench: 6,682 tasks, of which 3,876 are red-teaming tasks covering indirect and direct threat models, with malicious goals taken from 60+ policies and regulations. They evaluate several agent frameworks (OpenAI Agents SDK, Claude Code, Google ADK, OpenClaw) with different backbones. The main findings are that all of them are quite vulnerable, that composed injections work better than single ones, and that the harness design matters at least as much as the backbone model for robustness.
Critical strengths
The paper is very comprehensive. It covers 14 domains and 50+ realistic environments, several injection surfaces and both threat models, and the policy-grounded risk categories and state-based judges make the benchmark feel solid. I also liked that the authors don’t stop at static attacks: the adaptive white-box red-teaming against Claude Code with Opus 4.7, plus the transfer results, make the robustness conclusions more convincing.
Critical weaknesses
DTAP-Red isn’t compared against existing attack baselines, and no current defenses are evaluated. Moreover, no any results on the latest GPT and Claude models, given that it's an ambitious paper.
Questions for authors
-
Have you considered any defense that can be effective to defend against the DTAP-red attack?
-
Have you enconutered that the red-teaming LLM refuse to generate harmful payload? If so, how do you avoid this case?
5 comments
(3) Evaluation of additional recent models. We evaluate six additional backbones, i.e., GPT-6.1-Sol, GPT-6-Astra, Claude Opus 5.5, DeepSeek V4.1 Flash, Gemini 4 Hydrogen, and Grok 4.7, on the complete DTap-Bench evaluation across all 14 domains and both threat models, using the same OpenClaw harness. Table-r 3 reports these results alongside the existing baselines.
The results reveal two patterns: (2)Recent models improve robustness against indirect injections, while direct misuse remains substantial and can even increase: GPT-6.1-Sol and GPT-6-Astra achieve lower indirect ASR than GPT-5.5 (13.1% and 8.8% versus 17.4%) but higher direct ASR (35.8% and 31.9% versus 28.9%). One possible interpretation is that these models better distinguish external content from user instructions, while their stronger emphasis on following user instructions can still lead to compliance with harmful requests made directly by users; (2) vulnerabilities remain strongly domain-dependent: GPT-6-Astra reaches 27.0% indirect ASR in Legal despite an overall rate of 8.8%, while Claude Opus 5.5 reaches 28.1% direct ASR in Code despite an overall rate of 14.1%. These domain-specific weaknesses are consistent with our paper’s broader finding that context-dependent risks arising during task execution remain challenging. Even as models become more robust, alignment and mitigation appear uneven across domains, highlighting the need for more comprehensive safeguards.
Table-r 3. DTap evaluation across backbone models under a fixed OpenClaw harness. We report ASR (%) under indirect and direct threat models across 14 domains. Overall denotes the unweighted mean across domains. Higher ASR indicates greater vulnerability.
| Threat model | Model | Overall | Workflow | CRM | CS | Travel | Code | Browser | Research | OS-FS | Windows | macOS | Finance | Legal | Telecom | Medical |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Indirect | GPT-5.2 | 35.2 | 26.1 | 50.9 | 51.7 | 32.5 | 64.7 | 27.6 | 5.8 | 20.0 | 9.6 | 26.7 | 26.5 | 60.0 | 42.3 | 48.2 |
| Indirect | GPT-5.5 | 17.4 | 42.4 | 32.4 | 29.0 | 16.7 | 7.9 | 17.5 | 4.4 | 6.8 | 2.1 | 0.0 | 15.5 | 36.5 | 12.8 | 19.1 |
| Indirect | GPT-6.1 Sol | 13.1 | 26.2 | 15.2 | 15.8 | 9.2 | 7.9 | 8.5 | 12.8 | 15.5 | 1.9 | 7.5 | 13.5 | 17.5 | 13.3 | 18.0 |
| Indirect | GPT-6 Astra | 8.8 | 8.4 | 8.6 | 9.9 | 10.0 | 6.7 | 6.1 | 2.4 | 14.5 | 1.9 | 0.0 | 4.5 | 27.0 | 15.1 | 8.1 |
| Indirect | Claude Opus 5.5 | 1.6 | 2.8 | 6.3 | 2.0 | 1.7 | 3.0 | 2.4 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.2 | 1.4 |
| Indirect | DeepSeek V4 Pro | 41.4 | 47.0 | 44.4 | 69.8 | 48.3 | 9.5 | 29.7 | 17.3 | 51.1 | 5.6 | 0.0 | 54.8 | 73.0 | 63.4 | 65.9 |
| Indirect | DeepSeek V4.1 Flash | 9.0 | 18.7 | 10.6 | 8.9 | 8.3 | 0.6 | 4.9 | 12.8 | 17.5 | 1.9 | 5.7 | 5.5 | 10.0 | 3.6 | 17.6 |
| Indirect | Gemini 4 Hydrogen | 5.5 | 1.9 | 9.9 | 8.9 | 8.3 | 4.2 | 1.2 | 2.4 | 3.5 | 0.9 | 1.9 | 6.5 | 7.5 | 6.6 | 13.1 |
| Indirect | Grok 4.7 | 13.9 | 16.8 | 15.9 | 18.8 | 13.3 | 7.3 | 25.6 | 4.0 | 17.0 | 3.7 | 3.8 | 13.0 | 19.0 | 13.9 | 23.0 |
| Direct | GPT-5.2 | 38.5 | 30.9 | 67.8 | 61.4 | 12.4 | 32.9 | 22.8 | 58.7 | 33.7 | 31.7 | 26.2 | 24.5 | 51.5 | 41.8 | 43.1 |
| Direct | GPT-5.5 | 28.9 | 31.1 | 44.4 | 50.0 | 18.1 | 28.1 | 20.2 | 38.1 | 24.8 | 14.2 | 6.3 | 11.0 | 42.0 | 45.9 | 30.4 |
| Direct | GPT-6.1 Sol | 35.8 | 35.9 | 49.5 | 56.0 | 34.3 | 46.3 | 11.8 | 36.1 | 29.5 | 24.3 | 12.0 | 26.5 | 51.5 | 45.3 | 42.4 |
| Direct | GPT-6 Astra | 31.9 | 33.3 | 44.0 | 56.0 | 8.6 | 36.4 | 18.8 | 41.2 | 21.5 | 27.1 | 12.0 | 27.5 | 46.5 | 36.0 | 37.1 |
| Direct | Claude Opus 5.5 | 14.1 | 19.2 | 14.3 | 15.0 | 11.4 | 28.1 | 5.9 | 16.8 | 13.0 | 10.0 | 4.0 | 7.0 | 14.5 | 14.9 | 23.1 |
| Direct | DeepSeek V4 Pro | 59.6 | 58.0 | 55.7 | 75.5 | 60.0 | 77.9 | 27.7 | 54.7 | 82.9 | 33.3 | 10.4 | 68.0 | 67.0 | 82.5 | 81.0 |
| Direct | DeepSeek V4.1 Flash | 15.6 | 0.0 | 15.4 | 13.0 | 7.6 | 13.2 | 10.6 | 12.6 | 12.0 | 8.6 | 6.0 | 22.0 | 35.0 | 24.2 | 38.4 |
| Direct | Gemini 4 Hydrogen | 23.7 | 20.5 | 30.8 | 34.0 | 22.9 | 24.8 | 24.7 | 28.6 | 27.0 | 1.4 | 10.0 | 19.5 | 26.5 | 29.8 | 30.6 |
| Direct | Grok 4.7 | 14.7 | 9.0 | 16.5 | 18.0 | 17.1 | 14.9 | 15.3 | 16.0 | 13.0 | 26.4 | 2.0 | 10.0 | 11.0 | 19.9 | 16.6 |
Q1: Have you considered any defense that can be effective to defend against the DTAP-red attack?
We thank the reviewer for this question. Yes, as reported in Table R2 of our response to W1, we have evaluated Progent, a system-prompt defense, and ShieldAgent. In this static indirect-attack evaluation, Progent and ShieldAgent reduce ASR from 40.3% to 22.0% and 28.4%, respectively, while the system-prompt defense reduces it to 30.2%. These results show that existing defenses provide partial mitigation, and as a consequence, we believe the following two complementary approaches would be essential to strengthening agent defenses:
(1) Harness-level trust separation: the harness can explicitly label the provenance and trust level of inputs, distinguishing authorized user instructions from untrusted tool outputs and external content. These labels should be paired with execution controls that prevent untrusted content from acquiring authority to change permissions or override security policies. This can help preserve instruction boundaries across diverse injection channels.
(2) Policy-based guardrails for action verification: agent defenses should remain effective as attack methods and injection channels evolve. Policy-based guardrails check proposed actions and their potential consequences against explicit, trusted safety policies before execution. Their decisions depend on whether an action would violate a policy, regardless of how the action was induced. This provides a unified, attack-agnostic basis for blocking policy-violating actions.
Q2: Have you enconutered that the red-teaming LLM refuse to generate harmful payload? If so, how do you avoid this case?
We thank the reviewer for raising this question. Yes, we observed cases where the attacker backbone itself refused to generate harmful payloads. We address this issue mainly through two approaches:
(1) Using alternative open-source attacker backbones: models such as DeepSeek V4 Pro and Kimi V3 can also serve as capable alternatives for optimizing effective attacks in our automated red-teaming pipeline. We very seldom observe payload-generation refusals from these models in the evaluated runs.
(2) Jailbreak prompting for red-teaming: we use carefully crafted system prompts to configure the backbone for penetration testing and red-teaming research against other agents in controlled simulation environments. For example, we specify that the model should “serve as a red-teaming agent for penetration-testing purposes.” We found that this role-specific context was often sufficient to obtain attack candidates from models such as GPT-5.1 and Gemini-3-Pro in our evaluated settings.
We thank you again for your constructive feedback and hope these additional experiments and clarifications address your concerns; we would be happy to answer any follow-up questions.
[1] Wang, Z., Siu, V., Ye, Z., Shi, T., Nie, Y., Zhao, X., ... & Song, D. (2025, November). AGENTVIGIL: Automatic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents. In EMNLP (Findings) (pp. 23159-23172).
[2] Zhou, A., Wu, K., Pinto, F., Chen, Z., Zeng, Y., Yang, Y., ... & Li, B. (2026). Autoredteamer: Autonomous red teaming with lifelong attack integration. Advances in Neural Information Processing Systems, 38, 169852-169895.
[3] Shi, T., He, J., Wang, Z., Wu, L., Li, H., Guo, W., & Song, D. (2025). Progent: Programmable privilege control for llm agents. arXiv e-prints, arXiv-2504.
[4] Chen, Z., Kang, M., & Li, B. (2025). Shieldagent: Shielding agents via verifiable safety policy reasoning. arXiv preprint arXiv:2503.22738.
Dear Reviewer HcvP,
We sincerely thank you for recognizing DTap’s contributions and the breadth of its evaluation, including adaptive attacks optimized by DTap-Red, white-box red-teaming, and attack transferability analyses. Following your suggestions, we have added comparisons with existing attack baselines, evaluations of popular agent defenses, and results for six additional recent models. Below, we provide detailed responses to each of your comments.
We thank the reviewer for these helpful suggestions. We agree that comparisons with existing attack methods and defenses, together with evaluations of recent models, provide a more comprehensive assessment of DTap-Red and demonstrate the benchmark’s utility. We have therefore added three sets of experiments: (1) comparisons with automated red-teaming baselines, (2) evaluations of three agent defenses on DTap-Bench, and (3) evaluations of six additional recent models under the OpenClaw harness.
(1) Comparison with attack baselines. We compare DTap-Red with AgentVigil [1] for indirect prompt injection and AutoRedTeamer [2] for the direct threat model on a 50-task set spanning diverse risk categories. All methods use GPT-5.1 as the attacker and OpenAI Agents SDK with GPT-5.1 as the victim. We set a common maximum of five optimization iterations per task and align the remaining framework settings where applicable. Table R1 reports ASR and the mean number of optimization iterations among successful runs.
DTap-Red achieves higher ASR in both comparisons: 28.0% versus 24.0% for indirect attacks and 34.0% versus 22.0% for direct attacks, while also requiring fewer iterations among successful runs (2.6 versus 4.2 and 2.2 versus 4.8, respectively). We believe these gains may stem from DTap-Red’s joint optimization of payloads and injection locations, together with its diverse attack skill library, which provides strong priors for efficient attack optimization. On the contrary, AgentVigil primarily optimizes attacks from scratch by mutating existing seed templates, while AutoRedTeamer was originally designed for text-based LLM jailbreaking and therefore requires adaptation to agentic workflows and action-based objectives.
Table-r 1. Attack baseline comparison with GPT-5.1 as the attacker and OpenAI Agents SDK (GPT-5.1) as the victim on a 50-task set. Mean optimization iterations is computed over successful runs; ASR includes all evaluated tasks.
(2) Evaluation of agent defenses. We evaluate three defenses against the same static indirect attacks in DTap-Bench, using OpenAI Agents SDK with GPT-5.4 as the victim: (i) Progent [3], which enforces privilege-control policies on tool calls and their arguments; (ii) system-prompt defense, which adds general security instructions to help the victim resist malicious instructions from indirect channels; and (iii) ShieldAgent [5], which verifies proposed actions against explicit safety policies using policy reasoning and verification tools. We report ASR under attack and BSR on clean benign tasks, without re-optimizing attacks against each defense.
Table R2 shows a security–utility tradeoff. Progent achieves the lowest ASR, reducing it from 40.3% to 22.0%, while BSR decreases from 85.7% to 78.4%, providing the strongest attack mitigation among the evaluated defenses at the cost of benign utility. ShieldAgent reduces ASR to 28.4% while retaining a higher BSR of 83.2%. The system-prompt defense achieves 30.2% ASR and 81.0% BSR, providing less protection than the two action-level defenses in this evaluation. These results are consistent with the hypothesis that effective agent defenses benefit from explicit checks on agent actions beyond defensive prompting.
Table-r 2. Defense evaluation against static indirect attacks in DTap-Bench with OpenAI Agents SDK (GPT-5.4) as the victim. All defenses use the same benchmark attacks without attack re-optimization. ASR is the unweighted mean across the 14 domain-level ASRs; BSR measures success on clean benign tasks without attack.