acceptodds
Under review as a conference paper at ICLR 2027

DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents

50 views

Abstract

AI agents are increasingly deployed across diverse domains to automate complex workflows through long-horizon and high-stakes action executions. Due to their high capability and flexibility, such agents raise significant security and safety concerns. A growing number of real-world incidents have shown that adversaries can easily manipulate agents into performing harmful actions, such as leaking API keys, deleting user data, or initiating unauthorized transactions. Evaluating agent security is inherently challenging, as agents operate in dynamic, untrusted environments involving external tools, heterogeneous data sources, and frequent user interactions. However, realistic, controllable, and reproducible environments for large-scale risk assessment remain largely underexplored. To address this gap, we introduce the DecodingTrust-Agent Platform (DTap), the first controllable and interactive red-teaming platform for AI agents, spanning 14 real-world domains and over 50 simulation environments that replicate widely used systems such as Google Workspace, Paypal, and Slack. To scale the risk assessment of agents in DTap, we further propose DTap-Red, the first autonomous red-teaming agent that systematically explores diverse injection vectors (e.g., prompt, tool, skill, environment, combinations) and autonomously discovers effective attack strategies tailored to varying malicious goals. Using DTap-Red, we curate DTap-Bench, a large-scale red-teaming dataset comprising high-quality instances across domains, each paired with a verifiable judge following corresponding security policies to automatically validate attack outcomes. Through DTap, we conduct large-scale evaluations of popular AI agents built on various backbone models, spanning security policies, risk categories, and attack strategies, revealing systematic vulnerability patterns and providing valuable insights for developing secure next-generation agents.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
agent security, agent red-teaming, AI agents

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 4.00 over 3 reviews, above 99% of the others.
Reviewer_HcvPOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents DTAP, a red-teaming platform for AI agents. It has simulated environments in 14 domains, more than 50 of them, built to mimic services such as Gmail, Slack, PayPal and Salesforce through their MCP/CLI and GUI interfaces. On top of the platform, the authors build DTAP-Red, an automated red-teaming agent. It searches over injection payloads and locations across prompt, tool, skill and environment vectors, and their combinations, and uses feedback from rule-based judges that check the environment state. They use it to build DTAP-Bench: 6,682 tasks, of which 3,876 are red-teaming tasks covering indirect and direct threat models, with malicious goals taken from 60+ policies and regulations. They evaluate several agent frameworks (OpenAI Agents SDK, Claude Code, Google ADK, OpenClaw) with different backbones. The main findings are that all of them are quite vulnerable, that composed injections work better than single ones, and that the harness design matters at least as much as the backbone model for robustness.

Critical strengths

The paper is very comprehensive. It covers 14 domains and 50+ realistic environments, several injection surfaces and both threat models, and the policy-grounded risk categories and state-based judges make the benchmark feel solid. I also liked that the authors don’t stop at static attacks: the adaptive white-box red-teaming against Claude Code with Opus 4.7, plus the transfer results, make the robustness conclusions more convincing.

Critical weaknesses

DTAP-Red isn’t compared against existing attack baselines, and no current defenses are evaluated. Moreover, no any results on the latest GPT and Claude models, given that it's an ambitious paper.

Questions for authors
  1. Have you considered any defense that can be effective to defend against the DTAP-red attack?

  2. Have you enconutered that the red-teaming LLM refuse to generate harmful payload? If so, how do you avoid this case?

5 comments

Dear Reviewer HcvP,

We sincerely thank you for recognizing DTap’s contributions and the breadth of its evaluation, including adaptive attacks optimized by DTap-Red, white-box red-teaming, and attack transferability analyses. Following your suggestions, we have added comparisons with existing attack baselines, evaluations of popular agent defenses, and results for six additional recent models. Below, we provide detailed responses to each of your comments.

W1: DTAP-Red isn’t compared against existing attack baselines, and no current defenses are evaluated. Moreover, no any results on the latest GPT and Claude models, given that it's an ambitious paper.

We thank the reviewer for these helpful suggestions. We agree that comparisons with existing attack methods and defenses, together with evaluations of recent models, provide a more comprehensive assessment of DTap-Red and demonstrate the benchmark’s utility. We have therefore added three sets of experiments: (1) comparisons with automated red-teaming baselines, (2) evaluations of three agent defenses on DTap-Bench, and (3) evaluations of six additional recent models under the OpenClaw harness.

(1) Comparison with attack baselines. We compare DTap-Red with AgentVigil [1] for indirect prompt injection and AutoRedTeamer [2] for the direct threat model on a 50-task set spanning diverse risk categories. All methods use GPT-5.1 as the attacker and OpenAI Agents SDK with GPT-5.1 as the victim. We set a common maximum of five optimization iterations per task and align the remaining framework settings where applicable. Table R1 reports ASR and the mean number of optimization iterations among successful runs.

DTap-Red achieves higher ASR in both comparisons: 28.0% versus 24.0% for indirect attacks and 34.0% versus 22.0% for direct attacks, while also requiring fewer iterations among successful runs (2.6 versus 4.2 and 2.2 versus 4.8, respectively). We believe these gains may stem from DTap-Red’s joint optimization of payloads and injection locations, together with its diverse attack skill library, which provides strong priors for efficient attack optimization. On the contrary, AgentVigil primarily optimizes attacks from scratch by mutating existing seed templates, while AutoRedTeamer was originally designed for text-based LLM jailbreaking and therefore requires adaptation to agentic workflows and action-based objectives.

Table-r 1. Attack baseline comparison with GPT-5.1 as the attacker and OpenAI Agents SDK (GPT-5.1) as the victim on a 50-task set. Mean optimization iterations is computed over successful runs; ASR includes all evaluated tasks.

Threat modelRed-teaming methodASR (%) ↑Mean optimization iterations ↓
IndirectDTap-Red28.02.6
IndirectAgentVigil24.04.2
DirectDTap-Red34.02.2
DirectAutoRedTeamer22.04.8

(2) Evaluation of agent defenses. We evaluate three defenses against the same static indirect attacks in DTap-Bench, using OpenAI Agents SDK with GPT-5.4 as the victim: (i) Progent [3], which enforces privilege-control policies on tool calls and their arguments; (ii) system-prompt defense, which adds general security instructions to help the victim resist malicious instructions from indirect channels; and (iii) ShieldAgent [5], which verifies proposed actions against explicit safety policies using policy reasoning and verification tools. We report ASR under attack and BSR on clean benign tasks, without re-optimizing attacks against each defense.

Table R2 shows a security–utility tradeoff. Progent achieves the lowest ASR, reducing it from 40.3% to 22.0%, while BSR decreases from 85.7% to 78.4%, providing the strongest attack mitigation among the evaluated defenses at the cost of benign utility. ShieldAgent reduces ASR to 28.4% while retaining a higher BSR of 83.2%. The system-prompt defense achieves 30.2% ASR and 81.0% BSR, providing less protection than the two action-level defenses in this evaluation. These results are consistent with the hypothesis that effective agent defenses benefit from explicit checks on agent actions beyond defensive prompting.

Table-r 2. Defense evaluation against static indirect attacks in DTap-Bench with OpenAI Agents SDK (GPT-5.4) as the victim. All defenses use the same benchmark attacks without attack re-optimization. ASR is the unweighted mean across the 14 domain-level ASRs; BSR measures success on clean benign tasks without attack.

DefenseASR (%) ↓BSR (%) ↑
No additional defense40.385.7
Progent22.078.4
System-prompt defense30.281.0
ShieldAgent28.483.2

(3) Evaluation of additional recent models. We evaluate six additional backbones, i.e., GPT-6.1-Sol, GPT-6-Astra, Claude Opus 5.5, DeepSeek V4.1 Flash, Gemini 4 Hydrogen, and Grok 4.7, on the complete DTap-Bench evaluation across all 14 domains and both threat models, using the same OpenClaw harness. Table-r 3 reports these results alongside the existing baselines.

The results reveal two patterns: (2)Recent models improve robustness against indirect injections, while direct misuse remains substantial and can even increase: GPT-6.1-Sol and GPT-6-Astra achieve lower indirect ASR than GPT-5.5 (13.1% and 8.8% versus 17.4%) but higher direct ASR (35.8% and 31.9% versus 28.9%). One possible interpretation is that these models better distinguish external content from user instructions, while their stronger emphasis on following user instructions can still lead to compliance with harmful requests made directly by users; (2) vulnerabilities remain strongly domain-dependent: GPT-6-Astra reaches 27.0% indirect ASR in Legal despite an overall rate of 8.8%, while Claude Opus 5.5 reaches 28.1% direct ASR in Code despite an overall rate of 14.1%. These domain-specific weaknesses are consistent with our paper’s broader finding that context-dependent risks arising during task execution remain challenging. Even as models become more robust, alignment and mitigation appear uneven across domains, highlighting the need for more comprehensive safeguards.

Table-r 3. DTap evaluation across backbone models under a fixed OpenClaw harness. We report ASR (%) under indirect and direct threat models across 14 domains. Overall denotes the unweighted mean across domains. Higher ASR indicates greater vulnerability.

Threat modelModelOverallWorkflowCRMCSTravelCodeBrowserResearchOS-FSWindowsmacOSFinanceLegalTelecomMedical
IndirectGPT-5.235.226.150.951.732.564.727.65.820.09.626.726.560.042.348.2
IndirectGPT-5.517.442.432.429.016.77.917.54.46.82.10.015.536.512.819.1
IndirectGPT-6.1 Sol13.126.215.215.89.27.98.512.815.51.97.513.517.513.318.0
IndirectGPT-6 Astra8.88.48.69.910.06.76.12.414.51.90.04.527.015.18.1
IndirectClaude Opus 5.51.62.86.32.01.73.02.40.00.00.00.00.01.01.21.4
IndirectDeepSeek V4 Pro41.447.044.469.848.39.529.717.351.15.60.054.873.063.465.9
IndirectDeepSeek V4.1 Flash9.018.710.68.98.30.64.912.817.51.95.75.510.03.617.6
IndirectGemini 4 Hydrogen5.51.99.98.98.34.21.22.43.50.91.96.57.56.613.1
IndirectGrok 4.713.916.815.918.813.37.325.64.017.03.73.813.019.013.923.0
DirectGPT-5.238.530.967.861.412.432.922.858.733.731.726.224.551.541.843.1
DirectGPT-5.528.931.144.450.018.128.120.238.124.814.26.311.042.045.930.4
DirectGPT-6.1 Sol35.835.949.556.034.346.311.836.129.524.312.026.551.545.342.4
DirectGPT-6 Astra31.933.344.056.08.636.418.841.221.527.112.027.546.536.037.1
DirectClaude Opus 5.514.119.214.315.011.428.15.916.813.010.04.07.014.514.923.1
DirectDeepSeek V4 Pro59.658.055.775.560.077.927.754.782.933.310.468.067.082.581.0
DirectDeepSeek V4.1 Flash15.60.015.413.07.613.210.612.612.08.66.022.035.024.238.4
DirectGemini 4 Hydrogen23.720.530.834.022.924.824.728.627.01.410.019.526.529.830.6
DirectGrok 4.714.79.016.518.017.114.915.316.013.026.42.010.011.019.916.6

Q1: Have you considered any defense that can be effective to defend against the DTAP-red attack?

We thank the reviewer for this question. Yes, as reported in Table R2 of our response to W1, we have evaluated Progent, a system-prompt defense, and ShieldAgent. In this static indirect-attack evaluation, Progent and ShieldAgent reduce ASR from 40.3% to 22.0% and 28.4%, respectively, while the system-prompt defense reduces it to 30.2%. These results show that existing defenses provide partial mitigation, and as a consequence, we believe the following two complementary approaches would be essential to strengthening agent defenses:

(1) Harness-level trust separation: the harness can explicitly label the provenance and trust level of inputs, distinguishing authorized user instructions from untrusted tool outputs and external content. These labels should be paired with execution controls that prevent untrusted content from acquiring authority to change permissions or override security policies. This can help preserve instruction boundaries across diverse injection channels.

(2) Policy-based guardrails for action verification: agent defenses should remain effective as attack methods and injection channels evolve. Policy-based guardrails check proposed actions and their potential consequences against explicit, trusted safety policies before execution. Their decisions depend on whether an action would violate a policy, regardless of how the action was induced. This provides a unified, attack-agnostic basis for blocking policy-violating actions.

Q2: Have you enconutered that the red-teaming LLM refuse to generate harmful payload? If so, how do you avoid this case?

We thank the reviewer for raising this question. Yes, we observed cases where the attacker backbone itself refused to generate harmful payloads. We address this issue mainly through two approaches:

(1) Using alternative open-source attacker backbones: models such as DeepSeek V4 Pro and Kimi V3 can also serve as capable alternatives for optimizing effective attacks in our automated red-teaming pipeline. We very seldom observe payload-generation refusals from these models in the evaluated runs.

(2) Jailbreak prompting for red-teaming: we use carefully crafted system prompts to configure the backbone for penetration testing and red-teaming research against other agents in controlled simulation environments. For example, we specify that the model should “serve as a red-teaming agent for penetration-testing purposes.” We found that this role-specific context was often sufficient to obtain attack candidates from models such as GPT-5.1 and Gemini-3-Pro in our evaluated settings.

We thank you again for your constructive feedback and hope these additional experiments and clarifications address your concerns; we would be happy to answer any follow-up questions.

[1] Wang, Z., Siu, V., Ye, Z., Shi, T., Nie, Y., Zhao, X., ... & Song, D. (2025, November). AGENTVIGIL: Automatic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents. In EMNLP (Findings) (pp. 23159-23172).

[2] Zhou, A., Wu, K., Pinto, F., Chen, Z., Zeng, Y., Yang, Y., ... & Li, B. (2026). Autoredteamer: Autonomous red teaming with lifelong attack integration. Advances in Neural Information Processing Systems, 38, 169852-169895.

[3] Shi, T., He, J., Wang, Z., Wu, L., Li, H., Guo, W., & Song, D. (2025). Progent: Programmable privilege control for llm agents. arXiv e-prints, arXiv-2504.

[4] Chen, Z., Kang, M., & Li, B. (2025). Shieldagent: Shielding agents via verifiable safety policy reasoning. arXiv preprint arXiv:2503.22738.

Reviewer_5b16OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes the DecodingTrust-Agent Platform, a safety benchmark spanning several real-world tools. The paper additionally proposes a red-teaming agent that explores diverse attacks. Based on these two, the authors additionally propose DTap-bench, a dataset of harmful actions derived from policies and regulations. The authors additionally conduct an analysis of this benchmark using several models and harnesses, where they found several instances of harmful behavior.

Critical strengths
  1. The main strength of this paper is the sheer scale of the contributions and fidelity of the benchmark and evaluation. While several safety benchmarks exist, as far as I'm aware, these benchmarks mostly considered much more artificial scenarios than explored here. Generally, I think that DTap closes an important gap in safety research, as future work can now be much closer aligned to real-world applications of LLMs, while still ensuring responsible evaluation.
  2. I also think some of the experimental results are quite interesting and unexpected. For instance, I was surprised how large the effect of harness design can be on the safety of a model.
Critical weaknesses
  1. My main issue is with regards to the presentation issues. Almost all figures and tables in this paper are quite small and difficult to read. More seriously, many of the details are only explained in the appendix (e.g., the verifiable judge in line 312). While I understand that the amount of content is difficult to fit into the page limit, the paper itself should be self-contained with regards to critical details and should not rely on the appendix to this degree. On a more minor note, the citations in the introductions seem to be missing parentheses, and some of them are missing the year.
Questions for authors
  1. How does the Diverse Attack Skill Library (line 268) work? Is the red-teaming agent able to generate new code for attacks, or is it restricted to the available skills?
  2. How does the Multi-Layer Memory Module work (line 283)?
  3. Did you observe any instance of reward hacking for the verifiable judges (line 311), and were there any countermeasures?
4 comments

Dear Reviewer 5b16,

We sincerely thank you for your positive assessment and for recognizing DTap’s contribution to more realistic evaluation of AI agent safety and security. Following your suggestions, we have expanded the main-text explanations of key components and improved the presentation of the figures to make the paper clearer and more readable. Below, we address each of your questions in detail.

W1: My main issue is with regards to the presentation issues. Almost all figures and tables in this paper are quite small and difficult to read. More seriously, many of the details are only explained in the appendix (e.g., the verifiable judge in line 312). While I understand that the amount of content is difficult to fit into the page limit, the paper itself should be self-contained with regards to critical details and should not rely on the appendix to this degree. On a more minor note, the citations in the introductions seem to be missing parentheses, and some of them are missing the year.

We sincerely thank the reviewer for their careful reading and constructive feedback. We agree that the main paper should be self-contained in explaining essential methodological details. Following your suggestions, we have expanded the main-text explanations of key components, including the verifiable judges, attack skill library, and memory module, and clarified the purpose and interpretation of illustrative material such as Table 1 and Figure 5. We have also improved the readability of the figures and tables by resolving overlapping elements and enlarging them where possible. Finally, we have corrected the citations in the introduction to include the appropriate parentheses and missing publication years.

Q1: How does the Diverse Attack Skill Library (line 268) work? Is the red-teaming agent able to generate new code for attacks, or is it restricted to the available skills?

We thank the reviewer for this question. First, we confirm that DTap-Red can generate and execute new attack code and is not restricted to selecting or executing predefined skills. We provide a standalone sandbox VM where DTap-Red can write and execute code and consult attack papers online to develop new attack algorithms.

The attack skill library provides reusable implementations and strategic knowledge to guide attack search, with two complementary components:

(1) Attack algorithms: executable scripts for generating, optimizing, or transforming injection payloads, including optimization-based and encoding-based methods such as AutoDAN-Turbo, Base64 encoding, and leetspeak. DTap-Red can invoke these scripts in its sandbox to optimize selected parts of an injection payload and adapt them to the current malicious goal; (2) Injection strategies: guidance on constructing effective injection payload and choosing appropriate injection locations across prompts, tools, agent skills, and environment data. Payload and placement are optimized jointly when necessary to match the surrounding workflow. For example, a fabricated approval conversation can be tailored to email or chat, while a website injection can be embedded in a contextually relevant product review.

At runtime, within the preconfigured threat model and attacker access constraints, DTap-Red supports (3) composition and adaptation: it can combine payload transformations, coordinate injections across multiple permitted surfaces, and construct multi-step injection chains, refining payloads and locations using execution feedback; and (4) new attack generation: it can modify existing scripts or write and execute additional scripts to implement attack variants beyond the supplied implementations. For example, we observed it implement a flip-based encoding variant during optimization that was not included in the supplied attack skills. Further details are provided in Appendix A.4.2.

Q2: How does the Multi-Layer Memory Module work (line 283)?

We thank the reviewer for this question. DTap-Red’s memory module stores and retrieves previous red-teaming experience to guide subsequent attack planning and refinement, improving attack efficiency and effectiveness. Its operation comprises four elements:

(1) Stored experience: the module retains attack trajectories and insights from successful attempts, together with common failure modes, such as an injection not being encountered or being explicitly rejected by the victim; (2) hierarchical multi-layer storage: experiences are organized into three layers corresponding to risk categories, malicious goals, and threat models with their allowed injection capabilities. This hierarchy structures prior experience to support targeted retrieval for the current use case.

Each experience is also assigned a usefulness score, defined as the attack success rate across instances that retrieved and used it. This score is updated dynamically as the agent accumulates further experience. Among similarly relevant memories, higher-scoring experiences receive higher retrieval priority, while lower-scoring memories are discarded as new experiences are added.

(3) Retrieval and reuse: given a malicious goal, DTap-Red retrieves relevant, high-scoring experiences and uses them as context for choosing attack skills, constructing payloads, and selecting injection locations. Successful strategies and recurring failure modes support reuse across attempts and goals, and the scores of the experiences used are updated after each attack; (4) exploration–exploitation balance: DTap-Red uses an ϵ\epsilon-greedy policy to balance exploration of attack strategies with exploitation of retrieved experience, exploring with probability ϵ\epsilon and exploiting otherwise. As a consequence, early stages emphasize exploration to accumulate diverse experience and limit early overfitting, while later stages prioritize exploitation of stored experience to improve search efficiency. Further details are provided in Appendix A.4.2.

Q3: Did you observe any instance of reward hacking for the verifiable judges (line 311), and were there any countermeasures?

We thank the reviewer for raising this important question. Yes, during development and human review, we observed reward-hacking behaviors in which DTap-Red exploited loopholes in the judges to obtain false-positive attack-success labels. The two most frequent patterns were:

(1) Keyword-matching exploitation: judges that relied solely on keyword presence could incorrectly mark an attack as successful when the victim’s response contained target keywords within a refusal or quotation, without fulfilling the malicious goal. For example, a refusal email could repeat prohibited terms while explicitly declining to generate the requested hateful content; (2) attacker-supplied evidence: an attacker could exploit the judge by placing the target sensitive value, such as an SSN or bank account number, directly in an incoming email. If the victim replied to that email, the original message could be automatically retained in the quoted thread, and a judge scanning the entire email could mistake the attacker-supplied content for a disclosure by the victim.

Human review identified these reward-hacking patterns and informed the corresponding countermeasures:

(1) Constrained semantic validation for content-based goals: where semantic interpretation is necessary, we supplement keyword checks with LLM-based evaluation against explicit, narrowly defined predicates. For example, the evaluator first extracts the target content and returns a decision from a closed set of labels indicating whether the response actually fulfills the prohibited-content objective or merely refuses it. These constrained decisions help limit ambiguity and reduce variability in LLM-based verification.

(2) Attacker information segregation by design: we redact sensitive values outside the attacker’s declared knowledge, such as SSNs and bank account numbers, from attacker-visible context and replace them with placeholders. This limits opportunities to plant evidence using privileged information. We have also updated the verifiable judges to check that the claimed harmful outcome was caused by the victim’s behavior, rather than directly supplied by the attacker or already present in the initial state, e.g., for example, by filtering out quoted thread content and evaluating only the victim-authored reply.

Through iterative human review and revision, we addressed the identified reward-hacking cases and improved the reliability of the verifiable judges.

We hope that these clarifications and revisions address your concerns, and we would be happy to answer any follow-up questions.

Thank you for your detailed response. I'm happy to read that the presentation issues will be solved and my questions have been clarified.

Thank you for your thoughtful feedback and helpful suggestions. We are pleased that our clarifications have resolved your questions and appreciate your input in strengthening the paper. As discussed, we have also revised the manuscript to address the presentation issues and clarify key components, including the verifiable judges, memory module, and attack skill library.

Reviewer_1yUAOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper presents DTap, a controllable red-teaming platform for AI agents covering 14 domains and more than 50 simulated environments, along with an autonomous attack agent and a 6,682-task benchmark. What stands out is the mix of realistic, stateful interactions and outcome-based verification, making it possible to reproducibly evaluate real harmful actions rather than just agents’ final replies. The experiments surface weaknesses to compositional injections and failures where agents do harmful things before refusing, reinforcing the need for execution frameworks and security checks alongside task performance.

Critical strengths

The most valuable thing about this work is the benchmark/data. It aims to be a realistic prompt injection benchmark, which is really useful these days. I can see it being helpful for both attack and defense.

The authors evaluate a bunch of frontier models and provide thoughtful results and analysis.

Critical weaknesses
  • Although the paper compares several agent harnesses, a lot of the tests change both the harness and the model, which makes it hard to tell whether security differences come from the harness or the model. For example, comparing OpenClaw with GPT-5.2 to Claude Code with Sonnet 4.5 tells us something about the full agent setups, but not whether one harness is inherently more secure. The same-model comparison between OpenAI Agents SDK and OpenClaw is better evidence that harness choice matters, but stronger claims would need a controlled study across multiple models and harnesses. Using one fixed harness would also make model-robustness comparisons cleaner.

  • Table 1/Figure 5 is insufficiently explained in the main text, leaving its purpose and contribution unclear, and would be better placed in the appendix.

  • The paper describes a hybrid data construction pipeline that mixes expert-designed malicious goals and judges with AI-generated environment data and automatically optimized attacks. But the human review process isn’t specified well enough: what counts as realism and quality, how many reviewers look at each example, and how are disagreements handled? Clear acceptance criteria, review stats, and independent validation of the rule-based judges would make the benchmark much more convincing than just showing that selected attacks work on a surrogate agent.

  • since the benchmark is more realistic, do you observe that environmental realism change susceptibility to the same injection? For example, it’d be useful to compare an isolated malicious email with the exact same email placed in a busy inbox that has a coherent communication history, while keeping the task and agent setup the same. That kind of experiment should separate attack exposure from compliance after exposure, since extra emails might affect retrieval behavior instead of the agent’s ability to reject malicious instructions.

Questions for authors
  1. For the metrics, on line 348, why do you call it “average ASR”? Is there another ASR metric? Also, BSR is only reported without attacks, but I feel utility under attack would be really useful to show whether the agent can still do its job when attacked. Do you have those results somewhere?

  2. Table 3 compares configurations that vary in both the model and the harness, so it’s confounded and the performance gap is hard to attribute. It’s still useful for comparing end-to-end agent systems, but it doesn’t prove that one model or harness is more secure. The GPT-5.2 comparison across two harnesses is a good control, but we’d need matched comparisons, e.g., same model across harnesses and same harness across models, to make broader claims.

  3. Fig. 6 looks pretty messy, the text is mixed together.

  4. for the ablation analysis on DTAP-RED: any analysis for the contributions of the memory module, refinement feedback, skill library, or compositional injection mechanism? The paper seems to lack a controlled comparison of attacker models, even though it uses different ones across domains. Matched-task, matched-budget evals and component-removal ablations would help show which design choices are actually driving DTap-Red’s effectiveness.

11 comments

Dear Reviewer 1yUA,

Thank you for recognizing the novelty and significance of DTap and for your constructive feedback. Following your suggestions, we have added experiments evaluating recent models and harnesses through controlled comparisons, along with ecosystem-realism and DTap-Red ablations. We also provide additional results on human-review statistics and benign utility under attack. Furthermore, we have clarified the dataset quality-control and review protocols and revised the figures and text to improve readability. Below, we provide detailed responses to each of your comments.

W1: Although the paper compares several agent harnesses, a lot of the tests change both the harness and the model, which makes it hard to tell whether security differences come from the harness or the model. For example, comparing OpenClaw with GPT-5.2 to Claude Code with Sonnet 4.5 tells us something about the full agent setups, but not whether one harness is inherently more secure. The same-model comparison between OpenAI Agents SDK and OpenClaw is better evidence that harness choice matters, but stronger claims would need a controlled study across multiple models and harnesses. Using one fixed harness would also make model-robustness comparisons cleaner.

We thank the reviewer for raising this important point. We agree that matched comparisons with either the harness or the backbone model held fixed provide clearer evidence about their respective influence to agent vulnerability. We have therefore added two complementary controlled evaluations: a model-level comparison under a fixed OpenClaw harness and a harness-level comparison with GPT-6.1-Sol as the fixed backbone.

For the model-level comparison, we evaluate six additional recent backbones (GPT-6.1-Sol, GPT-6-Astra, Claude Opus 5.5, DeepSeek V4.1 Flash, Gemini 4 Hydrogen, and Grok 4.7) on the complete DTap-Bench evaluation across all 14 domains and both threat models, alongside the existing baselines (Table-r 1). For the harness-level comparison, we evaluate GPT-6.1-Sol with OpenClaw, OpenAI Agents SDK, Hermes, and Google ADK on the same benchmark (Table-r 2).

Model-level findings. (1) Recent models improve robustness against indirect injections, while direct misuse remains substantial and can even increase: GPT-6.1-Sol and GPT-6-Astra achieve lower indirect ASR than GPT-5.5 (13.1% and 8.8% versus 17.4%) but higher direct ASR (35.8% and 31.9% versus 28.9%). One possible interpretation is that these models better distinguish external content from user instructions, while their stronger emphasis on following user instructions can still lead to compliance with harmful requests made directly by users; (2) vulnerabilities remain strongly domain-dependent: GPT-6-Astra reaches 27.0% indirect ASR in Legal despite an overall rate of 8.8%, while Claude Opus 5.5 reaches 28.1% direct ASR in Code despite an overall rate of 14.1%. These domain-specific weaknesses are consistent with our paper’s broader finding that context-dependent risks arising during task execution remain challenging. Even as models become more robust, alignment and mitigation appear uneven across domains, highlighting the need for more comprehensive safeguards.

Harness-level findings. With GPT-6.1-Sol fixed, Hermes and OpenClaw both achieve lower ASR than OpenAI Agents SDK and Google ADK under both threat models. Overall ASR ranges from 10.5% to 19.3% for indirect attacks and from 22.4% to 45.0% for direct attacks, demonstrating substantial variation across the evaluated harness configurations even with the same backbone. One possible explanation is that personal agents such as Hermes and OpenClaw incorporate more defensive harness-level safeguards for untrusted inputs and potentially harmful actions, whereas developer-oriented configurations built with OpenAI Agents SDK and Google ADK place greater emphasis on instruction following and task completion, potentially treating instructions from different sources as more trusted and relying more on application-specific safeguards.

Table-r 1. DTap evaluation across backbone models under a fixed OpenClaw harness. We report attack success rate (ASR, %) under indirect and direct threat models across 14 domains. Overall denotes the unweighted mean across domains. Higher ASR indicates greater vulnerability.

Threat modelModelOverallWorkflowCRMCSTravelCodeBrowserResearchOS-FSWindowsmacOSFinanceLegalTelecomMedical
IndirectGPT-5.235.226.150.951.732.564.727.65.820.09.626.726.560.042.348.2
IndirectGPT-5.517.442.432.429.016.77.917.54.46.82.10.015.536.512.819.1
IndirectGPT-6.1 Sol13.126.215.215.89.27.98.512.815.51.97.513.517.513.318.0
IndirectGPT-6 Astra8.88.48.69.910.06.76.12.414.51.90.04.527.015.18.1
IndirectClaude Opus 5.51.62.86.32.01.73.02.40.00.00.00.00.01.01.21.4
IndirectDeepSeek V4 Pro41.447.044.469.848.39.529.717.351.15.60.054.873.063.465.9
IndirectDeepSeek V4.1 Flash9.018.710.68.98.30.64.912.817.51.95.75.510.03.617.6
IndirectGemini 4 Hydrogen5.51.99.98.98.34.21.22.43.50.91.96.57.56.613.1
IndirectGrok 4.713.916.815.918.813.37.325.64.017.03.73.813.019.013.923.0
DirectGPT-5.238.530.967.861.412.432.922.858.733.731.726.224.551.541.843.1
DirectGPT-5.528.931.144.450.018.128.120.238.124.814.26.311.042.045.930.4
DirectGPT-6.1 Sol35.835.949.556.034.346.311.836.129.524.312.026.551.545.342.4
DirectGPT-6 Astra31.933.344.056.08.636.418.841.221.527.112.027.546.536.037.1
DirectClaude Opus 5.514.119.214.315.011.428.15.916.813.010.04.07.014.514.923.1
DirectDeepSeek V4 Pro59.658.055.775.560.077.927.754.782.933.310.468.067.082.581.0
DirectDeepSeek V4.1 Flash15.60.015.413.07.613.210.612.612.08.66.022.035.024.238.4
DirectGemini 4 Hydrogen23.720.530.834.022.924.824.728.627.01.410.019.526.529.830.6
DirectGrok 4.714.79.016.518.017.114.915.316.013.026.42.010.011.019.916.6

Table-r 2. DTap evaluation across agent harnesses with GPT-6.1-Sol fixed as the backbone model. We report overall ASR (%) under indirect and direct threat models, using the same aggregation as Table 1. Higher ASR indicates greater vulnerability.

HarnessIndirect ASR (%)Direct ASR (%)
OpenClaw13.135.8
OpenAI Agents SDK18.742.0
Hermes10.522.4
Google ADK19.345.0

W2: Table 1/Figure 5 is insufficiently explained in the main text, leaving its purpose and contribution unclear, and would be better placed in the appendix.

We thank the reviewer for this helpful suggestion. We agree that clearer explanations of Table 1 and Figure 5 would improve the presentation and have therefore added the descriptions below to the revised main paper, clarifying how Table 1 illustrates concrete attack mechanisms optimized by DTap-Red, while Figure 5 visualizes the broad DTap-Bench dataset and coverage diversity. We are also happy to move them to the appendix if the reviewer believes this would further improve readability.

Table 1 presents representative attacks optimized by DTap-Red, specifying the corresponding risk category, environments involved, attacker’s strategy, victim agent’s actions, and resulting harmful consequences. Examples include multi-message workspace injections reinforced by fabricated peer confirmations, low-visibility multimodal typographic injections embedded in a slide footer, stealthy backdoors implemented through coordinated skill/tool/environment injections, and compositional injections in which individually benign-looking instructions collectively induce the victim agent to execute unauthorized transactions. These examples illustrate how DTap-Red generates attacks against widely used agent workflows, encompassing both simple but effective patterns and sophisticated compositional strategies that can lead to severe consequences in realistic settings.

Figure 5 compares the dataset distributions of DTap-Bench and AgentDojo through their agent execution trajectories. Each point represents one task’s trajectory produced by Claude Code (Claude-Sonnet-4.5), encoded using Gemini Embedding 2 and jointly projected into two dimensions with t-SNE; DTap-Bench points are colored by domain, and AgentDojo points are red. Specifically, DTap-Bench spans 14 domains, providing broader domain coverage than AgentDojo’s four task suites [1]. In the figure, the partially overlapping regions are consistent with workflows shared by both benchmarks, such as travel booking and financial transactions, while the additional domain-labeled regions in DTap-Bench illustrate a wider variety of task contexts, including legal, medical, CRM, coding, and research workflows. The plot provides a qualitative view of workflow and trajectory diversity that complements the benchmark feature comparison in Table 2 of the main paper.

Comments on the paper

1 comment

Dear Area Chairs, Senior Area Chairs, and Program Chairs,

We sincerely thank all the reviewers for their careful assessments and constructive feedback, which have helped improve our work further. We appreciate their shared recognition of DTap’s novelty, its contribution as a comprehensive, realistic evaluation platform, and its significant real-world impact in assessing and improving the safety and security of contemporary AI agents.

As recognized by the reviewers, DTap provides the first large-scale, unified agent security and safety evaluation suite comprising 50+ high-fidelity simulation environments and over 6K red-teaming tasks from 14 domains, with diverse, policy-grounded risk categories, malicious goals, and advanced adversarial attack strategies optimized against real-world AI agents. DTap makes it possible to extend agent evaluation beyond limited, artificial scenarios to assess vulnerabilities under systematic, realistic adversarial threats, which is critical for both academia and industry. DTap also reports comprehensive evaluations on latest agents and provides insightful analyses that inform the development of more effective security layers for the general community.

During the rebuttal, we addressed the questions from the reviews through additional experiments, methodological clarifications, and presentation improvements, all incorporated into the revised manuscript. In particular, Reviewer 1yUA has raised the score from 3 to 4 to acknowledge our further clarification.

Going forward, we will continue to maintain and expand DTap with new tasks, novel attack strategies, and additional environments, and in the meantime evaluate newly released models and harnesses to provide the community with updated insights into agent safety and security, emerging risks, and potential agent zero-days.

We sincerely hope the AC and PC will take into account DTap’s contributions through its comprehensive agent evaluation platform, high-fidelity environments, advanced red-teaming strategies, and potential for lasting real-world impact on AI agent evaluation and security development when making the final decision. We sincerely appreciate the constructive feedback from the reviewers and AC, and hope that DTap can help ensure principled agent safety and security evaluation in the long term and lay the trustworthy foundation layer for AI agents.

Best regards,

The DTap Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.