acceptodds
Under review as a conference paper at ICLR 2027

ToolHazard: Synthesizing Executable Adversarial Environments for Agent Security

Abstract

Large language model (LLM) agents are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies mainly vary injection payloads within a limited set of manually built or reused environments and predefined injection locations; LLM-based tool simulation broadens coverage but complicates reproducible evaluation and reward computation. Consequently, constructing executable and verifiable adversarial environments at scale remains a key challenge for both diagnosing agent vulnerabilities and improving security through alignment. To bridge this gap, we introduce **ToolHazard**, which moves beyond payload variation in fixed environments by synthesizing executable adversarial environments. ToolHazard synthesizes stateful environments and state-grounded tasks, discovers task-reachable injection paths from attacker-writable state to agent observations, and constructs programmatic outcome verifiers. We use ToolHazard to build ToolHazard-Bench for stress-testing agents under multi-step workflows and environmental attacks. Experiments reveal substantial vulnerabilities and characterize the effects of injection timing and placement. Moreover, ToolHazard-generated alignment data further improves security on ToolHazard-Bench and AgentDojo while maintaining helpfulness.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Evaluation and Benchmark, Agent Security, Prompt Injection, Agent Alignment

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_Q2JrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ToolHazard is an LLM-driven pipeline for building adversarial environments for tool-using agents. It has three parts. An Environment Simulator synthesizes stateful Python environment classes from ToolACE/API-Bank seed tasks and filters them with a Testing/Checking dual-agent loop. A User Simulator generates initial states and benign tasks grounded in those states. An Attacker Agent uses read/write dependency analysis to find attacker-writable free-text attributes that the benign trajectory actually reads, then appends a hijack task wrapped in one of six fixed injection templates. The pipeline is used to build ToolHazard-Bench (502 scenarios, 87 tasks, 28 environments), used to evaluate 7 LLMs, and ToolHazard-Align (1,040 samples, 60 environments), used for SFT+GRPO on Qwen3-4B/8B. The aligned models improve on both ToolHazard-Bench and AgentDojo.

Critical strengths
  1. The targeted problem is real. Executable, stateful, verifiable adversarial environments are a bottleneck, and the pipeline produces deterministic final-state verifiers rather than LLM-judge scores, so it can supply RL rewards.
  2. Discovering task-reachable injection points (write path ∩ read path activated in the benign trajectory, Sec. 4.3) is a sensible, reusable idea. It also makes controlled timing and placement studies possible (Sec. 7.1).
  3. The engineering is transparent, valuable for the community: full prompt templates (App. M–O), cost breakdown (~$0.59/env, Table 4), an anonymized code release, and a human validation protocol (App. H).
  4. The results shows the value for the framework in both evaluation and training.
Critical weaknesses
  1. Undisclosed reuse of EnvScaler [1]. It uses the same seeds, synthesis pipeline and Testing/Checking loop, and retains the same 191 environments (App. F), yet EnvScaler is cited only generically. The new parts are narrower: finding injection points, generating hijack tasks, and verifying them (with all attack templates are fixed and borrowed).
  2. Lack of baselines for the data-generation contribution. Table 3 only contains internal ablations (SFT, and SFT+RL on clean environments). These show that adversarial data helps, but not that ToolHazard’s data is better than alternatives. Missing comparisons, at matched training recipe and sample budget: (1) Injection placement: random or predefined fields instead of task-reachable ones. This isolates the core attack-point discovery component. (2) Existing adversarial data: e.g., ASB/InjecAgent-style cases, or EnvScaler environments with AgentDojo-style injections. (3) LLM-simulated environments with an LLM-judge reward (ToolEmu-style): this tests the claim that executable, verifiable environments are what matter. (4) Established defenses (e.g., SecAlign, ARLAS) are also absent as reference points.

[1] EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis

Questions for authors
  1. For Qwen3-4B, clean-environment SFT+RL raises ASR (25.05 → 32.41). What explains this?
  2. How many candidate tasks were discarded for lacking a reachable point?
1 comment

Thanks to the author for the detailed response and the additional experiments. Some of the questions and concerns are addressed. However, after careful consideration, my main concerns remain, so I will keep my rating.

  1. Limited novelty and contribution. The response confirms that ToolHazard reuses EnvScaler’s executable environments, state initialization, and benign task construction. The six injection templates are also adapted from prior work (App. I). Injection-point discovery and verifiers for injected goals are therefore the core contributions. I think these are useful engineering, but a relatively narrow contribution.

  2. Limited evidence of training effectiveness. The gain from adversarial over clean-environment training is small on ToolHazard-Bench (Qwen3-8B ASR 19.27 → 18.06). SecAlign alone achieves much lower ASR than ToolHazard-Align. Therefore, ToolHazard-Align improves, but is not a strong defense on its own. Also, no evidience shows that training in executable environments yields better models than training with LLM-simulated environments and LLM-judge rewards. (The paper’s motivation highlight executable environments with LLM-simulated ones.)

Reviewer_xzzNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces ToolHazard, a framework for synthesizing executable, stateful tool environments for evaluating and improving agent security against indirect prompt injection. The framework combines environment and task generation, discovery of injection points reachable during task execution, and programmatic outcome verification. The authors construct ToolHazard-Bench and ToolHazard-Align, evaluate seven models under six attack strategies, and report improvements in security and benign task performance through supervised fine-tuning and reinforcement learning on both ToolHazard-Bench and AgentDojo.

Critical strengths
  1. The integration of executable environments, automated injection point discovery, and programmatic verification provides a useful foundation for repeatable outcome assessment and interactive safety training.

  2. Evaluation across seven models and six attack strategies reveals distinct vulnerability patterns and demonstrates the limitations of assessing security with a single injection template.

  3. The alignment experiments include a clean environment training baseline and evaluation on AgentDojo, providing initial evidence of transferable improvements in security and benign task performance.

Critical weaknesses
  1. Although Table 4 reports average costs by stage, the paper does not provide conversion rates from all generated candidates to validated attacks and successful hijacks, or clarify whether discarded samples, retries, and target model evaluation are included in these costs, leaving the total cost per successful attack and practical scalability insufficiently established. This point is my main concern.

  2. In Figure 2, how do you control for differences in payload content, field semantics, and surrounding context across injection locations when evaluating the effects of injection timing and placement?

  3. As an optional direction beyond the current scope, extending the threat model and evaluation to cover systems using MCP or agent harnesses such as OpenClaw could broaden the practical relevance and impact of this work.

Questions for authors

See Weakness

Reviewer_moeVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a automated adversarial environment synsthesis framework ToolHazard, and generate executable environments and corresponding tasks to build ToolHazard-Bench for safety evaluation and ToolHazard-Align for safety alignment.

Critical strengths
  1. Proposed a automatic pipeline to scale the adversarial environment and tasks synthesis, which is a reusable infrastructure for both safety evaluation and training.
  2. Experiment results on ToolHazard-Bench show the vulnerabilities of existing LLMs towards environment injection attacks, and results on ToolHazard-Align show the the proposed recipe is effective to improve benign performance while reducing attack success rate.
Critical weaknesses

The principles used to define and generate malicious tasks are not clear. The paper only explains where the injection occurs (the attack point discovery stage) and how the attacks are optimized (6 predefined strategies), but does not clearly specify what is the unsafe or unintended behavior, which limits the realism of the work.

Questions for authors
  1. On Qwen 3-4B, there's a clear trend that ASR and BR both ASR and BR increase after benign post-training, while the proposed alignment recipe reduces ASR and further improves BR. But on Qwen 3-8B, benign post-training alone already reduces ASR while improving BR, and the ASR gap between benign SFT+RL and the proposed method is relatively small, which raises doubts about the effectiveness of the proposed alignment recipe. If the benign post-traing has aligned compute budget as the proposed ToolHazard-Align method, could it outperform ToolHazard-Align on both BR and ASR?
  2. What's the human validation results on the adversarial task queries and injections, similar to the validation on benign environments and tasks in Section H?
  3. Minor: abstract uses "helpfulness" while the main paper consistently uses "utility" for the benign performance.
  4. Minor format issue: the AI statement should be put after the conclusion and before the references as the template requires.
Reviewer_rtYtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ToolHazard proposes an automated framework for constructing executable adversarial environments for tool-using agents. It combines stateful environment synthesis, state-grounded task and verifier generation, and discovery of writable text fields that become observable during benign task execution. The resulting benchmark contains 502 adversarial scenarios across 87 benign tasks and 28 environments. The paper also uses synthesized data for safety alignment and reports improvements on both ToolHazard-Bench and AgentDojo.

Critical strengths
  1. The environments run actual code and maintain state, making experiments easier to reproduce and allowing outcomes to be used as RL rewards.
  2. The authors combine automated testing with human review of all 28 test environments and 92 candidate tasks. The reported agreement between the verifiers and human judgments gives some confidence in the benchmark’s quality.
  3. The alignment results suggest that the synthesized adversarial data is useful. For example, for Qwen3-8B, BR increases from 46.53% to 52.08% and ASR drops from 27.08% to 18.34% compared with clean-environment SFT+RL.
Critical weaknesses
  1. The contribution is insufficiently clarified. From my view, the environment and task synthesis pipeline is close to EnvScaler, which the paper cites but does not discuss in sufficient detail. Both start from ToolACE/API-Bank tasks, implement environment states and tools as Python classes, use two agents for quality inspection, and generate state-grounded tasks with terminal-state verifiers. Several prompt templates are also similar, including those for environment inference, state and rule planning, and state initialization. The paper should make these similarities explicit and clarify what is new beyond EnvScaler.

  2. The benefit of the attack construction method is unclear. The alignment results show that adversarial training data is useful, but not whether the proposed read/write analysis and trajectory filtering are necessary. A comparison with simply inserting the same payloads into writable text fields would help establish their value. The paper should also report how accurately and comprehensively the method identifies valid injection points.

  3. The environments may not capture realistic security controls. Real systems enforce security through authentication, role-based permissions, resource ownership, and other access restrictions. It is unclear how well the synthesized environments capture these controls or how they affect the feasibility of the generated attacks. A concrete end-to-end example showing how an attack proceeds under such controls would help establish the realism of the benchmark.

  4. Task complexity and security metrics need clearer interpretation. The benchmark emphasizes multi-step workflows, but the task-generation prompt requests a read-then-write pattern with a single write operation and avoids multiple subtasks. The reported execution length may therefore reflect exploration or retries rather than intrinsic long-horizon complexity. Sections 4.4 and 6.2 also define BR and ASR through pass rates over verification conditions. The paper should distinguish partial condition satisfaction from complete task or attack success. While the human audit is useful, checking only one trajectory per task provides limited evidence of verifier reliability on incorrect or partially completed outcomes.

Questions for authors
  1. Which components, prompts, and environment artifacts are newly developed versus adapted from EnvScaler? Can the authors provide a direct methodological comparison and clarify the matching environment counts?
  2. How much do the read/write analysis and trajectory-based filtering in Section 4.3 improve attack construction compared with simply inserting the same payloads into writable text fields under the same budget?
  3. How are attacker privileges and resource ownership modeled? Can the authors validate a representative path from attacker-controlled content to agent-mediated harm, distinguishing the effects of the entry mechanism from those of prompt injection?
  4. Do BR and ASR measure full success or average condition satisfaction? How are verifier false positives and false negatives assessed beyond a single trajectory per task?

Comments on the paper

1 comment
Area_Chair_CacvOpenReview ↗

Reminder: please read the author responses

Dear reviewers,

The authors have posted responses to your reviews. Please read them and the revised submission, and let the authors know whether your concerns are addressed. If your assessment has changed, please update your review.

Thank you for your time.

Best regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.