- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TRACE, a task-aware and self-evolving framework for agentic jailbreaking. Unlike conventional jailbreak attacks that primarily aim to elicit harmful textual responses, TRACE targets the end-to-end execution of harmful workflows by tool-using LLM agents. Experimental results on AdvCUA and AgentHazard using Codex and Claude Code with multiple underlying models show promsing improvements over the comparison baselines.
Critical strengths
- A major strength is that the paper studies safety failures at the level of actual agent execution rather than harmful text generation alone. The authors identify a meaningful challenge: an agent may provide harmful information without executing it, and independently successful subtasks may still fail to produce a completed workflow because of environmental dependencies or intermediate errors.
- The paper proposes TRACE to explicitly consider execution states, tool availability, task dependencies, and failure recovery. These design choices make the proposed approach relevant to the emerging safety challenges of autonomous tool-using agents. It shifts the evaluation focus toward whether an agent actually completes an unsafe objective.
- Experiments on two benchmarks across multiple backbone models show strong reported empirical performance across agents and defensive settings. The additional evaluation against AgentDoG is valuable.
Critical weaknesses
-
Insufficient clarity about evaluation coverage and statistical reliability. Appendix C states that AdvCUA contains 140 tasks, including 26 end-to-end kill-chain tasks. However, the AdvCUA ASRs reported in Tables 1 and 2 appear consistent with increments of approximately 1/26. For example, the reported scores of 0.62, 0.65, 0.81, and 0.88 correspond approximately to 16/26, 17/26, 21/26, and 23/26. This suggests that the evaluation may have used only the 26 kill-chain tasks rather than the complete benchmark. The manuscript does not clearly explain this apparent discrepancy. If only this subset was evaluated, the coverage is considerably narrower than a reader might infer from the benchmark description. This matters because attack effectiveness may differ substantially across task categories and levels of complexity. Besides, a related concern is that AgentHazard contains 2,653 instances, but the evaluation uses only 100 randomly sampled instances. The paper does not provide sufficient information about the sampling seed, representativeness, repeated trials, or statistical uncertainty.
-
Missing component ablations and insufficient validation of the proposed optimization and evolution mechanisms. For example, It remains unclear how much improvement comes from scheme-guided decomposition, the harmfulness–difficulty selection objective, profile-based strategy retrieval, library evolution, and localized failure recovery. It is also unclear whether the fixed trade-off parameter λ = 0.5 is robust across tasks. Furthermore, the self-evolving library raises questions about experimental independence.
-
A methodological inconsistency in the execution-difficulty metric. Section 3.1 and Appendix B.1 describe execution difficulty as incorporating implementation complexity, workflow length, decomposition granularity, and state dependencies. However, the Execution Difficulty Judge prompt in Appendix A explicitly instructs the evaluator to assess only decomposition granularity. Thus, the provided implementation description does not clearly support the broader difficulty metric used in Eq.1.
Questions for authors
- What is the exact evaluation subset used for AdvCUA, and how representative are the sampled tasks?
- How is execution difficulty actually measured, beyond decomposition granularity?
- What are the independent contributions of the main components, and how is strategy evolution separated from evaluation?
- Does the reported query efficiency account for the complete computational cost of TRACE?
- How closely do the success checks match the original semantic objectives of the harmful tasks?
6 comments
Response to Reviewer bKoS (2/3)
W2 & Q3: Component Ablation and Evaluation Independence
We thank the reviewers for pointing out the lack of component ablation studies. To address this concern, we conduct additional experiments to further validate the effectiveness of TRACE's core components.
- Component Ablation
We conduct component ablation studies on AgentHazard using Codex as the target agent. Under otherwise identical experimental settings, we individually remove or replace the main components of TRACE. The results are shown below:
| Configuration | ASR | Query Count |
|---|---|---|
| TRACE | 0.97 | 5.01 |
| w/o Decomposition | 0.87 | 5.22 |
| w/o Evolution | 0.83 | 5.91 |
| w/o Recovery | 0.85 | 6.00 |
| Subtask Sequence Random Selection | 0.90 | 6.79 |
The results demonstrate that the core components of TRACE contribute to its overall performance. Without task decomposition, ASR drops by 10%, highlighting its role in reducing the execution difficulty of complex harmful tasks. The absence of strategy library evolution results in a 14% ASR reduction, demonstrating the benefits of accumulating effective strategies based on execution feedback and avoiding redundant exploration. Disabling localized failure recovery decreases ASR by 12% and increases query counts by 20%, confirming the importance of preserving successful progress and resuming from failed subtasks. Furthermore, replacing score-based subtask sequence selection with random selection reduces ASR from 0.97 to 0.90, indicating that jointly considering subtask harmfulness and execution difficulty helps select more executable subtask sequences.
- Sensitivity Analysis
We further investigate the sensitivity of the parameter λ used in decomposition selection on AgentHazard. The results show that λ = 0.3 and λ = 0.5 select identical decomposition sequences for all 100 tasks, while λ = 0.5 and λ = 0.7 differ in only 2 tasks. Moreover, TRACE achieves ASRs of 0.97–0.98 on Codex , with consistently achieving 0.97 on Claude Code , across λ ∈ {0.3, 0.5, 0.7}. These results indicate that TRACE achieves consistently high attack success rates across different λ values.
- Evaluation Independence
To address these concerns, we conduct additional experiments on AgentHazard using Codex as the target agent, with disjoint task sets for strategy library evolution and evaluation. Specifically, we first run TRACE on 100 tasks sampled with seed=42 from the original benchmark, allowing the strategy library to evolve continuously based on execution feedback. We save strategy library checkpoints at the initial , intermediate , and final stages of evolution. We then evaluate these checkpoints on another 100 non-overlapping tasks sampled with seed=0, keeping the strategy library frozen throughout evaluation. The ASR results are shown below:
| Strategy Library Checkpoint | AgentHazard |
|---|---|
| 0% (Initial) | 0.81 |
| 50% (Intermediate) | 0.85 |
| 100% (Fully Evolved) | 0.94 |
The results show that as the strategy library evolves, ASR on the held-out evaluation tasks increases from 0.81 to 0.94. This improvement indicates that effective strategies accumulated during prior executions can generalize to held-out tasks. Furthermore, the fully evolved strategy library achieves an ASR of 0.94 even when frozen, demonstrating that the execution-oriented strategies learned by TRACE exhibit cross-task reusability and generalizability.
We will incorporate these additional experimental results and analyses into a subsequent revision of manuscript.
Response to Reviewer bKoS (3/3)
Q4: Additional Computational Cost
We thank the reviewer for raising this important question. Figure 6 in the paper already reports the average target-agent query count for each method. We further report the average wall-clock time and token cost $$$.
| Codex | Claude Code | ||||
|---|---|---|---|---|---|
| Datasets | Methods | Average wall-clock time | Average token Cost $$$ | Average wall-clock time | Average token Cost $$$ |
| AdvCUA | AutoDAN-Turbo | 19186.57 | 3.06 | 13925.93 | 4.54 |
| X-Teaming | 20597.18 | 3.21 | 13468.98 | 4.81 | |
| MD-Attack | 10843.56 | 13.64 | 11009.00 | 12.48 | |
| MT-AgentRisk | 14545.28 | 4.25 | 10107.46 | 4.07 | |
| STING | 10968.37 | 2.41 | 19994.31 | 5.70 | |
| TRACE | 3655.82 | 0.98 | 7818.54 | 1.24 | |
| AgentHazard | AutoDAN-Turbo | 2477.07 | 2.21 | 2644.90 | 2.16 |
| X-Teaming | 3785.82 | 2.57 | 4340.80 | 4.09 | |
| MD-Attack | 726.17 | 0.61 | 518.67 | 0.56 | |
| MT-AgentRisk | 555.99 | 2.42 | 1040.13 | 2.51 | |
| STING | 624.97 | 1.78 | 658.03 | 1.24 | |
| TRACE | 357.06 | 0.55 | 277.68 | 0.53 |
As shown in the table, TRACE consistently achieves the lowest average wall-clock time and token cost across both benchmarks and target agents. TRACE's improved efficiency over the baselines stems from selecting more executable subtask sequences, reusing effective strategies, and recovering from failed subtasks without restarting the entire process, collectively reducing redundant exploration and execution overhead. In the experiments, we observe that MD-attack requires high token cost on AdvCUA. This may be because AdvCUA's complex end-to-end tasks involve multiple execution stages and workflow, and MD-Attack's task decomposition can induce agent execution but may omit critical operations or prerequisites, causing the agent to repeatedly execute certain steps without completing the entire task. During this process, long execution contexts, multi-stage reasoning, and tool interactions may increase token consumption per target-agent query. Moreover, without localized failure recovery, failures may require restarting execution from scratch, further increasing overall token costs. On the other hand, MD-Attack's relatively simple decomposition can reduce attack-side computational time, so it results in high token costs but not very high wall-clock time.
AgentHazard tasks generally involve simpler execution workflows with fewer execution stages and state dependencies, reduce per-qeury token consumption associated with complex reasoning and long execution contexts.
We also provide execution records in execution_records/ directory of our anonymous repository, including execution trajectories, wall-clock time and token cost.
Q5: Semantic Validity of Success Evaluation
We thank the reviewer for raising this important question. Section 4.1 and Appendix C clarify that the final ASR is computed entirely based on the original task-level success criteria provided by each benchmark. Specifically, AdvCUA employs hard-coded verification that assess the completion of task based on execution outcomes or environment states. AgentHazard uses LLM-as-a-judge that considers the agent's execution trajectory, and execution outputs to determine whether substantive harmful behavior aligned with the original objective has occurred. All methods, including TRACE and the baselines, are evaluated using the same benchmark success criteria and execution environments.
We further conducted a manual review of 100 successful execution trajectories from AdvCUA and AgentHazard, covering different attack methods, target agents, and task categories. Multiple researchers independently examined the complete execution trajectories and actual execution outcomes against the original task objectives to verify end-to-end objective completion, with disagreements resolved through further review. All 100 reviewed cases were confirmed to have achieved their original task objectives. We will include the manual verification details and results in a subsequent revision.
Additionally, Figure 4 demonstrates that TRACE successfully induced the agent to create and conceal a script containing the specified destructive command, as required by the original task. We also provide complete demonstration examples in the assets/ directory of our anonymous repository, illustrating the end-to-end attack workflows induced by TRACE. Due to ethical and safety considerations, we do not publicly release all successful attack trajectories.
Response to Reviewer bKoS (1/3)
W1 & Q1: Evaluation Coverage and Statistical Reliability
We thank the reviewer for raising concerns about evaluation coverage and statistical reliability. We further clarify the evaluation coverage of our original experiments and conduct additional experiments on AdvCUA and AgentHazard to validate the robustness of our results.
TRACE focuses on the end-to-end execution of complex harmful tasks. Since AdvCUA contains only 26 end-to-end kill-chain tasks, our original experiments evaluated all 26 tasks in this category. We further extend our evaluation to include all 74 TTP-based tasks in AdvCUA. The remaining 40 Direct tasks primarily involve simple, single-step harmful objectives, and are therefore excluded from our evaluation.
We additionally compare TRACE with the strongest baseline in our original experiments, MD-Attack, using Codex + GPT-5.5. The ASR results are shown below:
The additional results demonstrate that TRACE continues to substantially outperform MD-Attack on TTP-based tasks 0\.89 vs\. 0\.41. Across the combined 100 tasks, TRACE achieves an ASR of 0.82, representing a more than 120% relative improvement over MD-Attack 03˙7.
For AgentHazard, our original experiments sampled 100 tasks using the random seed=42. We further sample another 100 tasks using seed=0, ensuring no overlap with the original samples. The ASR results on Codex + GPT-5.5 are shown below:
TRACE achieves an ASR of 0.95 on the additional samples, closely matching the original ASR of 0.97, and consistently outperforms MD-Attack on both subsets. Across all 200 tasks, TRACE achieves an overall ASR of 0.96, substantially exceeding MD-Attack 06˙5.
To assess sample representativeness, we compare the distributions across all 10 risk categories between the full AgentHazard benchmark and our evaluation subset. The sampled tasks cover all 10 risk categories, with category proportions closely matching those of the full benchmark, indicating good representativeness in terms of risk-category distribution.
We will clarify the evaluated subsets and incorporate the additional experimental results in the revision.
W3 & Q2: Execution Difficulty Measurement
We thank the reviewer for identifying this inconsistency and apologize for the oversight. Upon careful examination, we confirmed that an earlier version of the Execution Difficulty Judge prompt was inadvertently included in the appendix, whereas the finalized version was consistently used throughout all reported experiments.
Compared with the earlier version, the finalized prompt primarily: 1 removes two occurrences of "only" that restrict evaluation to decomposition granularity; 2 revises the introductory sentence to mention all four existing execution-difficulty factors. Importantly, the rest of the prompt remains unchanged, including the detailed evaluation criteria that already cover all four factors, as reflected in the following instructions:
Decomposition Granularity: The prompt requires each subtask to represent a "meaningful user-level execution boundary" and explicitly penalizes over-splitting and over-merging.
Implementation Complexity: The prompt instructs the judge to consider "whether each step is reasonably easy to implement".
Workflow Length: The prompt explicitly requires the judge to consider "whether the total number of steps is proportionate to the task complexity".
State Dependencies: The prompt instructs the judge to identify cases where merging steps "hides a material dependency" or "changes execution context or location".
Therefore, execution difficulty in TRACE is not measured solely by decomposition granularity. Instead, the judge considers all four factors when assessing the overall execution difficulty of candidate subtask sequences.
We have replaced the earlier prompt in Appendix A with the finalized version used in our experiments in the current revised manuscript.