- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ToolHazard is an LLM-driven pipeline for building adversarial environments for tool-using agents. It has three parts. An Environment Simulator synthesizes stateful Python environment classes from ToolACE/API-Bank seed tasks and filters them with a Testing/Checking dual-agent loop. A User Simulator generates initial states and benign tasks grounded in those states. An Attacker Agent uses read/write dependency analysis to find attacker-writable free-text attributes that the benign trajectory actually reads, then appends a hijack task wrapped in one of six fixed injection templates. The pipeline is used to build ToolHazard-Bench (502 scenarios, 87 tasks, 28 environments), used to evaluate 7 LLMs, and ToolHazard-Align (1,040 samples, 60 environments), used for SFT+GRPO on Qwen3-4B/8B. The aligned models improve on both ToolHazard-Bench and AgentDojo.
Critical strengths
- The targeted problem is real. Executable, stateful, verifiable adversarial environments are a bottleneck, and the pipeline produces deterministic final-state verifiers rather than LLM-judge scores, so it can supply RL rewards.
- Discovering task-reachable injection points (write path ∩ read path activated in the benign trajectory, Sec. 4.3) is a sensible, reusable idea. It also makes controlled timing and placement studies possible (Sec. 7.1).
- The engineering is transparent, valuable for the community: full prompt templates (App. M–O), cost breakdown (~$0.59/env, Table 4), an anonymized code release, and a human validation protocol (App. H).
- The results shows the value for the framework in both evaluation and training.
Critical weaknesses
- Undisclosed reuse of EnvScaler [1]. It uses the same seeds, synthesis pipeline and Testing/Checking loop, and retains the same 191 environments (App. F), yet EnvScaler is cited only generically. The new parts are narrower: finding injection points, generating hijack tasks, and verifying them (with all attack templates are fixed and borrowed).
- Lack of baselines for the data-generation contribution. Table 3 only contains internal ablations (SFT, and SFT+RL on clean environments). These show that adversarial data helps, but not that ToolHazard’s data is better than alternatives. Missing comparisons, at matched training recipe and sample budget: (1) Injection placement: random or predefined fields instead of task-reachable ones. This isolates the core attack-point discovery component. (2) Existing adversarial data: e.g., ASB/InjecAgent-style cases, or EnvScaler environments with AgentDojo-style injections. (3) LLM-simulated environments with an LLM-judge reward (ToolEmu-style): this tests the claim that executable, verifiable environments are what matter. (4) Established defenses (e.g., SecAlign, ARLAS) are also absent as reference points.
[1] EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis
Questions for authors
- For Qwen3-4B, clean-environment SFT+RL raises ASR (25.05 → 32.41). What explains this?
- How many candidate tasks were discarded for lacking a reachable point?
Thanks to the author for the detailed response and the additional experiments. Some of the questions and concerns are addressed. However, after careful consideration, my main concerns remain, so I will keep my rating.
Limited novelty and contribution. The response confirms that ToolHazard reuses EnvScaler’s executable environments, state initialization, and benign task construction. The six injection templates are also adapted from prior work (App. I). Injection-point discovery and verifiers for injected goals are therefore the core contributions. I think these are useful engineering, but a relatively narrow contribution.
Limited evidence of training effectiveness. The gain from adversarial over clean-environment training is small on ToolHazard-Bench (Qwen3-8B ASR 19.27 → 18.06). SecAlign alone achieves much lower ASR than ToolHazard-Align. Therefore, ToolHazard-Align improves, but is not a strong defense on its own. Also, no evidience shows that training in executable environments yields better models than training with LLM-simulated environments and LLM-judge rewards. (The paper’s motivation highlight executable environments with LLM-simulated ones.)