acceptodds
Under review as a conference paper at ICLR 2027

TRACE: TASK-AWARE ADAPTIVE SELF-EVOLVING AGENTIC JAILBREAKING

Abstract

The rise of LLM agents introduces a new threat by enabling planning, coding, and even end-to-end execution of expert-level attack workflows. However, this threat remains underexplored and underestimated since (i) Safety alignment prevents LLMs from directly generating harmful instructions. (ii) Although existing jailbreak methods can bypass safety alignment, most of them mainly focus on eliciting harmful instructions but overlook LLM refusal against executing these harmful instructions. (iii) Even if the early execution steps succeed, any intermediate failure or dependency error will interrupt the workflow and force a restart of the entire attack process. To address these limitations, we propose TRACE, an agentic jailbreaking framework that reduces attack difficulty, maintains execution dependencies, induces the agent to execute harmful instructions, and resumes from prior states rather than restarting. Specifically, we first design around 20 procedural decomposition schemes that specify key operations and dependencies for different types of tasks, which can decompose a complex and harmful expert-level task into more executable and benign-looking subtask sequences. We measure these candidate sequences along two dimensions, i.e., harmfulness and difficulty, and select the optimal one for instantiating the attack. During the attack process, most of the subtasks in the selected sequence are benign and thus can be executed by the agent smoothly. For the remaining refused subtasks, we construct different subtask profiles and establish an evolving execution-oriented multi-turn jailbreak strategy library corresponding to these profiles. This library continues to expand based on successful execution trajectories and recurring failure patterns, which already includes more than 30 strategies after evolution. Given the profile of a refused subtask, TRACE can retrieve an appropriate strategy that specifies the corresponding intermediate goals, environment states, and relevant tools to gradually establish a plausible execution context through multi-turn interaction, thereby inducing agent execution. When a step fails, TRACE localizes the failed subtask and adapts the interaction strategy to resume from the failed subtask rather than restart from scratch. Extensive evaluations on AdvCUA show that TRACE outperforms existing jailbreak methods across multiple advanced LLM agents under both regular and defensive settings, improving attack success rate by more than 100% over the recent agentic jailbreaking baselines. Our code is available herehttps://anonymous.4open.science/r/TRACE_ICLR-36E5.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM security, Jailbreak

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_bKoSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TRACE, a task-aware and self-evolving framework for agentic jailbreaking. Unlike conventional jailbreak attacks that primarily aim to elicit harmful textual responses, TRACE targets the end-to-end execution of harmful workflows by tool-using LLM agents. Experimental results on AdvCUA and AgentHazard using Codex and Claude Code with multiple underlying models show promsing improvements over the comparison baselines.

Critical strengths
  • A major strength is that the paper studies safety failures at the level of actual agent execution rather than harmful text generation alone. The authors identify a meaningful challenge: an agent may provide harmful information without executing it, and independently successful subtasks may still fail to produce a completed workflow because of environmental dependencies or intermediate errors.
  • The paper proposes TRACE to explicitly consider execution states, tool availability, task dependencies, and failure recovery. These design choices make the proposed approach relevant to the emerging safety challenges of autonomous tool-using agents. It shifts the evaluation focus toward whether an agent actually completes an unsafe objective.
  • Experiments on two benchmarks across multiple backbone models show strong reported empirical performance across agents and defensive settings. The additional evaluation against AgentDoG is valuable.
Critical weaknesses
  • Insufficient clarity about evaluation coverage and statistical reliability. Appendix C states that AdvCUA contains 140 tasks, including 26 end-to-end kill-chain tasks. However, the AdvCUA ASRs reported in Tables 1 and 2 appear consistent with increments of approximately 1/26. For example, the reported scores of 0.62, 0.65, 0.81, and 0.88 correspond approximately to 16/26, 17/26, 21/26, and 23/26. This suggests that the evaluation may have used only the 26 kill-chain tasks rather than the complete benchmark. The manuscript does not clearly explain this apparent discrepancy. If only this subset was evaluated, the coverage is considerably narrower than a reader might infer from the benchmark description. This matters because attack effectiveness may differ substantially across task categories and levels of complexity. Besides, a related concern is that AgentHazard contains 2,653 instances, but the evaluation uses only 100 randomly sampled instances. The paper does not provide sufficient information about the sampling seed, representativeness, repeated trials, or statistical uncertainty.

  • Missing component ablations and insufficient validation of the proposed optimization and evolution mechanisms. For example, It remains unclear how much improvement comes from scheme-guided decomposition, the harmfulness–difficulty selection objective, profile-based strategy retrieval, library evolution, and localized failure recovery. It is also unclear whether the fixed trade-off parameter λ = 0.5 is robust across tasks. Furthermore, the self-evolving library raises questions about experimental independence.

  • A methodological inconsistency in the execution-difficulty metric. Section 3.1 and Appendix B.1 describe execution difficulty as incorporating implementation complexity, workflow length, decomposition granularity, and state dependencies. However, the Execution Difficulty Judge prompt in Appendix A explicitly instructs the evaluator to assess only decomposition granularity. Thus, the provided implementation description does not clearly support the broader difficulty metric used in Eq.1.

Questions for authors
  • What is the exact evaluation subset used for AdvCUA, and how representative are the sampled tasks?
  • How is execution difficulty actually measured, beyond decomposition granularity?
  • What are the independent contributions of the main components, and how is strategy evolution separated from evaluation?
  • Does the reported query efficiency account for the complete computational cost of TRACE?
  • How closely do the success checks match the original semantic objectives of the harmful tasks?
6 comments
AuthorsOpenReview ↗

Response to Reviewer bKoS (1/3)

W1 & Q1: Evaluation Coverage and Statistical Reliability

We thank the reviewer for raising concerns about evaluation coverage and statistical reliability. We further clarify the evaluation coverage of our original experiments and conduct additional experiments on AdvCUA and AgentHazard to validate the robustness of our results.

  1. AdvCUA Evaluation Subset and Extended Experiments

TRACE focuses on the end-to-end execution of complex harmful tasks. Since AdvCUA contains only 26 end-to-end kill-chain tasks, our original experiments evaluated all 26 tasks in this category. We further extend our evaluation to include all 74 TTP-based tasks in AdvCUA. The remaining 40 Direct tasks primarily involve simple, single-step harmful objectives, and are therefore excluded from our evaluation.

We additionally compare TRACE with the strongest baseline in our original experiments, MD-Attack, using Codex + GPT-5.5. The ASR results are shown below:

AdvCUA Subset# TasksTRACEMD-Attack
End-to-end Kill-chain260.620.27
TTP-based740.890.41
Combined1000.820.37

The additional results demonstrate that TRACE continues to substantially outperform MD-Attack on TTP-based tasks 0\.89 vs\. 0\.41. Across the combined 100 tasks, TRACE achieves an ASR of 0.82, representing a more than 120% relative improvement over MD-Attack 03˙70\.37.

  1. AgentHazard Sample Details and Sampling Reliability

For AgentHazard, our original experiments sampled 100 tasks using the random seed=42. We further sample another 100 tasks using seed=0, ensuring no overlap with the original samples. The ASR results on Codex + GPT-5.5 are shown below:

AgentHazard Subset# TasksTRACEMD-Attack
Original seed=42seed=421000.970.62
Additional seed=0seed=01000.950.68
Combined2000.960.65

TRACE achieves an ASR of 0.95 on the additional samples, closely matching the original ASR of 0.97, and consistently outperforms MD-Attack on both subsets. Across all 200 tasks, TRACE achieves an overall ASR of 0.96, substantially exceeding MD-Attack 06˙50\.65.

To assess sample representativeness, we compare the distributions across all 10 risk categories between the full AgentHazard benchmark and our evaluation subset. The sampled tasks cover all 10 risk categories, with category proportions closely matching those of the full benchmark, indicating good representativeness in terms of risk-category distribution.

We will clarify the evaluated subsets and incorporate the additional experimental results in the revision.

W3 & Q2: Execution Difficulty Measurement

We thank the reviewer for identifying this inconsistency and apologize for the oversight. Upon careful examination, we confirmed that an earlier version of the Execution Difficulty Judge prompt was inadvertently included in the appendix, whereas the finalized version was consistently used throughout all reported experiments.

Compared with the earlier version, the finalized prompt primarily: 11 removes two occurrences of "only" that restrict evaluation to decomposition granularity; 22 revises the introductory sentence to mention all four existing execution-difficulty factors. Importantly, the rest of the prompt remains unchanged, including the detailed evaluation criteria that already cover all four factors, as reflected in the following instructions:

  • Decomposition Granularity: The prompt requires each subtask to represent a "meaningful user-level execution boundary" and explicitly penalizes over-splitting and over-merging.

  • Implementation Complexity: The prompt instructs the judge to consider "whether each step is reasonably easy to implement".

  • Workflow Length: The prompt explicitly requires the judge to consider "whether the total number of steps is proportionate to the task complexity".

  • State Dependencies: The prompt instructs the judge to identify cases where merging steps "hides a material dependency" or "changes execution context or location".

Therefore, execution difficulty in TRACE is not measured solely by decomposition granularity. Instead, the judge considers all four factors when assessing the overall execution difficulty of candidate subtask sequences.

We have replaced the earlier prompt in Appendix A with the finalized version used in our experiments in the current revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer bKoS (2/3)

W2 & Q3: Component Ablation and Evaluation Independence

We thank the reviewers for pointing out the lack of component ablation studies. To address this concern, we conduct additional experiments to further validate the effectiveness of TRACE's core components.

  1. Component Ablation

We conduct component ablation studies on AgentHazard using Codex GPT−55˙GPT-5\.5 as the target agent. Under otherwise identical experimental settings, we individually remove or replace the main components of TRACE. The results are shown below:

ConfigurationASRQuery Count
TRACE0.975.01
w/o Decomposition0.875.22
w/o Evolution0.835.91
w/o Recovery0.856.00
Subtask Sequence Random Selection0.906.79

The results demonstrate that the core components of TRACE contribute to its overall performance. Without task decomposition, ASR drops by 10%, highlighting its role in reducing the execution difficulty of complex harmful tasks. The absence of strategy library evolution results in a 14% ASR reduction, demonstrating the benefits of accumulating effective strategies based on execution feedback and avoiding redundant exploration. Disabling localized failure recovery decreases ASR by 12% and increases query counts by 20%, confirming the importance of preserving successful progress and resuming from failed subtasks. Furthermore, replacing score-based subtask sequence selection with random selection reduces ASR from 0.97 to 0.90, indicating that jointly considering subtask harmfulness and execution difficulty helps select more executable subtask sequences.

  1. Sensitivity Analysis

We further investigate the sensitivity of the parameter λ used in decomposition selection on AgentHazard. The results show that λ = 0.3 and λ = 0.5 select identical decomposition sequences for all 100 tasks, while λ = 0.5 and λ = 0.7 differ in only 2 tasks. Moreover, TRACE achieves ASRs of 0.97–0.98 on Codex GPT−55˙GPT-5\.5, with consistently achieving 0.97 on Claude Code Opus−48˙Opus-4\.8, across λ ∈ {0.3, 0.5, 0.7}. These results indicate that TRACE achieves consistently high attack success rates across different λ values.

  1. Evaluation Independence

To address these concerns, we conduct additional experiments on AgentHazard using Codex GPT−55˙GPT-5\.5 as the target agent, with disjoint task sets for strategy library evolution and evaluation. Specifically, we first run TRACE on 100 tasks sampled with seed=42 from the original benchmark, allowing the strategy library to evolve continuously based on execution feedback. We save strategy library checkpoints at the initial 00%, intermediate 5050%, and final 100100% stages of evolution. We then evaluate these checkpoints on another 100 non-overlapping tasks sampled with seed=0, keeping the strategy library frozen throughout evaluation. The ASR results are shown below:

Strategy Library CheckpointAgentHazard
0% (Initial)0.81
50% (Intermediate)0.85
100% (Fully Evolved)0.94

The results show that as the strategy library evolves, ASR on the held-out evaluation tasks increases from 0.81 to 0.94. This improvement indicates that effective strategies accumulated during prior executions can generalize to held-out tasks. Furthermore, the fully evolved strategy library achieves an ASR of 0.94 even when frozen, demonstrating that the execution-oriented strategies learned by TRACE exhibit cross-task reusability and generalizability.

We will incorporate these additional experimental results and analyses into a subsequent revision of manuscript.

AuthorsOpenReview ↗

Response to Reviewer bKoS (3/3)

Q4: Additional Computational Cost

We thank the reviewer for raising this important question. Figure 6 in the paper already reports the average target-agent query count for each method. We further report the average wall-clock time ss and token cost $$$.

Codex GPT−55˙GPT-5\.5Claude Code Opus−48˙Opus-4\.8
DatasetsMethodsAverage wall-clock time ssAverage token Cost $$$Average wall-clock time ssAverage token Cost $$$
AdvCUAAutoDAN-Turbo19186.573.0613925.934.54
X-Teaming20597.183.2113468.984.81
MD-Attack10843.5613.6411009.0012.48
MT-AgentRisk14545.284.2510107.464.07
STING10968.372.4119994.315.70
TRACE3655.820.987818.541.24
AgentHazardAutoDAN-Turbo2477.072.212644.902.16
X-Teaming3785.822.574340.804.09
MD-Attack726.170.61518.670.56
MT-AgentRisk555.992.421040.132.51
STING624.971.78658.031.24
TRACE357.060.55277.680.53

As shown in the table, TRACE consistently achieves the lowest average wall-clock time and token cost across both benchmarks and target agents. TRACE's improved efficiency over the baselines stems from selecting more executable subtask sequences, reusing effective strategies, and recovering from failed subtasks without restarting the entire process, collectively reducing redundant exploration and execution overhead. In the experiments, we observe that MD-attack requires high token cost on AdvCUA. This may be because AdvCUA's complex end-to-end tasks involve multiple execution stages and workflow, and MD-Attack's task decomposition can induce agent execution but may omit critical operations or prerequisites, causing the agent to repeatedly execute certain steps without completing the entire task. During this process, long execution contexts, multi-stage reasoning, and tool interactions may increase token consumption per target-agent query. Moreover, without localized failure recovery, failures may require restarting execution from scratch, further increasing overall token costs. On the other hand, MD-Attack's relatively simple decomposition can reduce attack-side computational time, so it results in high token costs but not very high wall-clock time.

AgentHazard tasks generally involve simpler execution workflows with fewer execution stages and state dependencies, reduce per-qeury token consumption associated with complex reasoning and long execution contexts.

We also provide execution records in execution_records/ directory of our anonymous repository, including execution trajectories, wall-clock time and token cost.

Q5: Semantic Validity of Success Evaluation

We thank the reviewer for raising this important question. Section 4.1 and Appendix C clarify that the final ASR is computed entirely based on the original task-level success criteria provided by each benchmark. Specifically, AdvCUA employs hard-coded verification that assess the completion of task based on execution outcomes or environment states. AgentHazard uses LLM-as-a-judge that considers the agent's execution trajectory, and execution outputs to determine whether substantive harmful behavior aligned with the original objective has occurred. All methods, including TRACE and the baselines, are evaluated using the same benchmark success criteria and execution environments.

We further conducted a manual review of 100 successful execution trajectories from AdvCUA and AgentHazard, covering different attack methods, target agents, and task categories. Multiple researchers independently examined the complete execution trajectories and actual execution outcomes against the original task objectives to verify end-to-end objective completion, with disagreements resolved through further review. All 100 reviewed cases were confirmed to have achieved their original task objectives. We will include the manual verification details and results in a subsequent revision.

Additionally, Figure 4 demonstrates that TRACE successfully induced the agent to create and conceal a script containing the specified destructive command, as required by the original task. We also provide complete demonstration examples in the assets/ directory of our anonymous repository, illustrating the end-to-end attack workflows induced by TRACE. Due to ethical and safety considerations, we do not publicly release all successful attack trajectories.

Reviewer_HMUBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TRACE is a new method for attacking AI agents, the kind that can run code and use tools rather than just chat. The paper notes that most jailbreak research only tries to make a model say something harmful, while getting an agent to actually carry out a harmful task is a separate and overlooked problem. TRACE works by splitting a harmful goal into small steps that each look innocent, pulling persuasion strategies from a library whenever the agent refuses a step, learning from what works and what fails, and resuming from the point of failure instead of restarting. Tested against Codex and Claude Code on several models and two safety benchmarks, it outperformed existing attacks, even with a guardrail in place, while needing fewer attempts. The broader point is that AI agents can be pushed into executing harmful tasks from start to finish, something current defenses aren't built to stop.

Critical strengths
  • The paper focuses on execution-level safety rather than only eliciting harmful text.
  • The pieces of TRACE fit together sensibly. Candidate decompositions get checked for missing prerequisites and inconsistencies before use. Refused subtasks are captured as structured profiles, and each strategy spells out what it’s aiming for, how to tell it’s working, and what to fall back on when it isn’t.
  • The experiment of TRACE is fairly broad: it includes several backbone models, and two benchmarks. It also checks the attack against a guardrail (AgentDoG), where TRACE weakens but doesn’t collapse to zero like the baselines do. Evaluating against a defense at all is more than most attack papers bother with. One nice choice is that AdvCUA judges success by checking the actual state of the system afterward instead of having another model rate the output, which is much harder to fool.
Critical weaknesses
  • The biggest gap is that the experiments never test the components of TRACE separately. Every result compares the full system against other full systems, with no ablations in the paper or appendix. Does the self-evolving library actually beat a fixed one? Does the scoring step help, or would generating a few decompositions and picking at random do just as well? How much of the efficiency gain is really from resuming after failure versus everything else? The self-evolution claim is the most exposed here. Algorithm 1 shows how strategies get added and revised, but spelling out the procedure isn't the same as showing it leads to cumulative improvement. Frozen-library, no-recovery, and decomposition/scoring controls would help a lot, and so would a plain direct-request baseline, since Table 1 only compares attack methods and never shows how much the agent would have done on its own.
  • The paper tells that evidence is accumulated per strategy and profile, and that threshold counts flip a strategy between effective, demoted, and ineffective states. What it never says is what’s actually being counted or how it turns into a ranking. Is a strategy’s score a raw count of wins and losses, a success rate, or something weighted? How is a single strategy credited or blamed when a subtask runs across several phases and turns, possibly using more than one strategy?
  • The main-body experimental setup (Sec 4.1) does not state which attacker model is used; this appears only in Appendix A (Qwen3.6-35B-A3B), and it would be clearer in the main text.
  • The self-evolution is never evaluated as a learned capability. A library that improves by accumulating strategies is only meaningful if that knowledge transfers to tasks it didn’t learn on, yet the paper reports no train/test split: there’s no held-out set on which a frozen, already-evolved library is measured. If the library evolves over the same benchmark tasks it’s then scored on, the ASR can’t be separated from simple adaptation to those specific tasks, and the paper’s claim that it stores reusable, generalized strategies goes untested. This also makes the comparison against per-task baselines like STING uneven, since they carry no cross-task memory. A clean protocol would evolve the library on one split, freeze it, and report performance on a disjoint split.
  • The 20-plus decomposition schemes are said to be “extracted from different task workflows,” but the paper never says which ones. If they were pulled from the same kind of OS and cyber tasks that AdvCUA and AgentHazard are built from, then the very high success rates may partly reflect schemes that were shaped to fit the test tasks in the first place. The paper should be clear on this issue.
Questions for authors
  1. Can you provide component ablations? Specifically, how does TRACE perform with a frozen (non-evolving) library, with recovery turned off, and with the scoring step replaced by picking a decomposition at random? And how does a plain direct-request baseline do, to show how much the agent would execute on its own?

  2. How is a strategy's effectiveness actually measured? Is the score a raw count of successes and failures, a success rate, or something weighted? And when a subtask spans several phases and turns, possibly using more than one strategy, how do you decide which strategy gets credited or blamed?

  3. Which attacker model is used? It appears only in Appendix A (Qwen3.6-35B-A3B); could this be stated in the main experimental setup?

  4. Is the strategy library evolved and evaluated on the same tasks, or is there a held-out split? If there's no split, how do you separate the reported success rates from simple adaptation to the specific benchmark tasks, and how is the claim that the strategies are reusable and generalizable actually tested?

  5. Where do the 20-plus decomposition schemes come from? Were they built from the same kind of OS and cyber tasks that AdvCUA and AgentHazard are drawn from, and if so, how do you rule out that the high success rates partly come from schemes fitted to the test distribution?

4 comments
AuthorsOpenReview ↗

Response to Reviewer HMUB (1/2)

W1 & Q1: Component Ablation Study

We thank the reviewers for pointing out the lack of component ablation studies. To address this concern, we conduct additional experiments to further validate the effectiveness of TRACE's core components.

  1. Component Ablation

We conduct component ablation studies on AgentHazard using Codex GPT−55˙GPT-5\.5 as the target agent. Under otherwise identical experimental settings, we individually remove or replace the main components of TRACE. The results are shown below:

ConfigurationASRQuery Count
TRACE0.975.01
w/o Decomposition0.875.22
w/o Evolution0.835.91
w/o Recovery0.856.00
Subtask Sequence Random Selection0.906.79

The results demonstrate that all core components of TRACE contribute to its overall performance. Without task decomposition, ASR drops by 10%, highlighting its role in reducing the execution difficulty of complex harmful tasks. The absence of strategy library evolution results in a 14% ASR reduction, demonstrating the benefits of accumulating effective strategies based on execution feedback and avoiding redundant exploration. Disabling localized failure recovery decreases ASR by 12% and increases query counts by 20%, confirming the importance of preserving successful progress and resuming from failed subtasks. Furthermore, replacing score-based subtask sequence selection with random selection reduces ASR from 0.97 to 0.90, indicating that jointly considering subtask harmfulness and execution difficulty helps select more executable subtask sequences.

  1. Direct-Request Baseline

We additionally evaluate a plain direct-request baseline on both AdvCUA and AgentHazard using Codex GPT−55˙GPT-5\.5, where the original harmful tasks are directly submitted to the target agent. The ASR results are shown below:

DatasetDirect Request
AdvCUA0
AgentHazard0

Direct Request achieves zero ASRs on both benchmarks, as their explicitly harmful task objectives often trigger refusals due to the target agent's safety alignment.

We will incorporate these additional experimental results and analyses into a subsequent revision of manuscript.

W2 & Q2: Strategy Effectiveness Measurement

We thank the reviewer for pointing out the need to clarify the implementation details of the self-evolving strategy library. We have described the strategy library evolution process in Section 3.3 and Appendix B.3 (Algorithm 1). We further clarify the implementation details below.

Strategy Effectiveness Measurement. The effectiveness of strategy is measured by the execution result of the corresponding subtask. TRACE maintains net success and failure counts for each strategy–profile association. Specifically, when a subtask succeeds, the failure count for the corresponding profile is decremented by one if it is positive; otherwise, the success count is incremented by one. The reverse applies when a subtask fails. A strategy is marked as effective once its net success count reaches the effectiveness threshold, at which point the success count is reset. Subsequent net failures trigger demotion upon reaching demotion threshold, with the failure count reset again. Further net failures after demotion trigger the ineffective state upon reaching the ineffectiveness threshold. Thus, each threshold applies to net evidence accumulated within the corresponding state.

Strategy Retrieval Priority. TRACE ranks strategies based on the current subtask profile and accumulated execution evidence. For a given profile, TRACE prioritizes strategies validated as effective, deprioritizes strategies with repeated failures, and excludes those marked as ineffective, and retains opportunities to explore newly generated and revised strategies.

Feedback Attribution. A strategy defines a complete multi-turn interaction process for a subtask. The success and failure of a strategy is determined by the practical multi-turn execution result of the corresponding subtask. Feedback from intermediate turns and progression phases guides interaction advancement and fallback but is not counted separately as strategy effectiveness evidence. If the same subtask is attempted multiple times using different strategies, the feedback of each strategy is recorded respectively.

We will further clarify these details in Section 3.3 and Appendix B.3 in a subsequent revision to improve the clarity of our method description.

W3 & Q3: Attacker Model Specification

We thank the reviewer for this helpful suggestion. We have explicitly specified the attacker model (Qwen3.6-35B-A3B) in Section 4.1 in the current revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer HMUB (2/2)

W4 & Q4: Strategy Generalization on Held-Out Tasks

We thank the reviewer for raising concerns about the generalizability and evaluation independence of the self-evolving strategy library. To address these concerns, we conduct additional experiments on AgentHazard using Codex GPT−55˙GPT-5\.5 as the target agent, with disjoint task sets for strategy library evolution and evaluation. Specifically, we first run TRACE on 100 tasks sampled with seed=42 from the original benchmark, allowing the strategy library to evolve continuously based on execution feedback. We save strategy library checkpoints at the initial 00%, intermediate 5050%, and final 100100% stages of evolution. We then evaluate these checkpoints on another 100 non-overlapping tasks sampled with seed=0, keeping the strategy library frozen throughout evaluation without any cross-task updates. The ASR results are shown below:

Strategy Library CheckpointAgentHazard
0% (Initial)0.81
50% (Intermediate)0.85
100% (Fully Evolved)0.94

The results show that as the strategy library evolves, ASR on the held-out evaluation tasks increases from 0.81 to 0.94. This improvement indicates that effective strategies accumulated during prior executions can generalize to held-out tasks. Furthermore, the fully evolved strategy library achieves an ASR of 0.94 even when frozen, demonstrating that the execution-oriented strategies learned by TRACE exhibit cross-task reusability and generalizability.

We will include the held-out evaluation setup, complete experimental results, and corresponding analysis in a subsequent revision to further validate the effectiveness of TRACE's self-evolution mechanism.

W5 & Q5: Origin of Decomposition Schemes

We thank the reviewer for raising concerns about the origins of the decomposition schemes. We have described the scheme construction process in Section 3.1 and Appendix B.1. We further clarify the sources and generation process below.

Inspired by prior work[1, 2], we designed a prompt to guide GPT-4o in generating high-level decomposition schemes from general domain knowledge of cybersecurity. The generation process does not use specific benchmark tasks as examples or include instance-specific information such as concrete targets, tools, commands, or environment configurations. We generated 50 candidate schemes using this prompt and then manually reviewed and deduplicated the candidates based on their abstraction level, structural diversity, and execution dependencies. Structurally equivalent schemes were merged, and overly specific, overly generic schemes, as well as those with dependency issues, were removed. Importantly, neither the generation nor the manual selection process used specific task instances from AdvCUA or AgentHazard..

We have clarified the construction process in Section 3.1 and Appendix B.1 in the current revised manuscript with the candidate generation prompt, manual selection criteria, and retained decomposition schemes provided in Appendix B.1.

[1] Li, Haoran, et al. "Synthetic data (almost) from scratch: Generalized instruction tuning for language models." arXiv preprint arXiv:2402.13064 20242024.

[2] Lee, Bruce W., Hyunsoo Cho, and Kang Min Yoo. "Instruction tuning with human curriculum." Findings of the Association for Computational Linguistics: NAACL 2024. 2024.

Given the additional ablations and held-out evaluation, I am raising my score from 2 to 3. These experiments substantially address the main concerns behind my original rating. I would still appreciate clarification of the original evaluation protocol and the remaining implementation details, and ask that the new results be included in the revised paper.

AuthorsOpenReview ↗

Response to Reviewer HMUB: Appreciation and Manuscript Revisions

Thank you very much for your constructive feedback and suggestions, which have helped us improve our manuscript. We have incorporated the additional experimental results and clarifications into the revised manuscript and appendix.

Reviewer_aCtBOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper introduces an agentic red-teaming method for inducing tool-calling agents to execute harmful instructions. It identifies attack candidates along two dimensions: harmfulness and difficulty. The approach combines subtask decomposition, feedback on subtask success, and a lifelong learning mechanism that continually refines and expands a strategy library, currently comprising 30 strategies. This work also includes feedback from the target agent’s execution environment as an input. The reported results show substantial improvements in attack success over existing baselines and state-of-the-art methods.

Critical strengths
  • Strong performance improvement over the baselines.
Critical weaknesses
  • The paper combines existing techniques with new contributions. However, given the number of components, the lack of ablation studies makes it difficult to determine which components lead to the improvements. For example, how much does the lifelong learning mechanism contribute to overall attack success?
  • Several elements, including subtask decomposition and lifelong learning, appear closely related to those in this ICLR 2026 paper. Could the authors include this work as a baseline, or explain why a direct comparison was not conducted?
Questions for authors
  1. Could the authors provide ablation studies to isolate the contribution of each component to the reported improvements?
  2. Could the authors compare their method with PLAGUE, or explain why it is not an appropriate baseline for this setting?

I am open to raising my score if these concerns are adequately addressed.

I have increased the score as the authors have adequately addressed the questions.

5 comments
AuthorsOpenReview ↗

Response to Reviewer aCtB (1/2)

W1 & Q1: Component Ablation Study

We thank the reviewers for pointing out the lack of component ablation studies. To address this concern, we conduct additional experiments to further validate the effectiveness of TRACE's core components.

We conduct component ablation studies on AgentHazard using Codex GPT−55˙GPT-5\.5 as the target agent. Under otherwise identical experimental settings, we individually remove or replace the main components of TRACE. The results are shown below:

ConfigurationASRQuery Count
TRACE0.975.01
w/o Decomposition0.875.22
w/o Evolution0.835.91
w/o Recovery0.856.00
Subtask Sequence Random Selection0.906.79

The results demonstrate that all core components of TRACE contribute to its overall performance. Without task decomposition, ASR drops by 10%, highlighting its role in reducing the execution difficulty of complex harmful tasks. The absence of strategy library evolution results in a 14% ASR reduction, demonstrating the benefits of accumulating effective strategies based on execution feedback and avoiding redundant exploration. Disabling localized failure recovery decreases ASR by 12% and increases query counts by 20%, confirming the importance of preserving successful progress and resuming from failed subtasks. Furthermore, replacing score-based subtask sequence selection with random selection reduces ASR from 0.97 to 0.90, indicating that jointly considering subtask harmfulness and execution difficulty helps select more executable subtask sequences.

We will incorporate these additional experimental results and analyses into a subsequent revision of manuscript.

AuthorsOpenReview ↗

Response to Reviewer aCtB (2/2)

W2 & Q2: Comparison with PLAGUE

We thank the reviewers for bringing PLAGUE to our attention. We acknowledge PLAGUE's contributions to multi-turn jailbreaking and lifelong learning, but would like to clarify the following fundamental differences between PLAGUE and TRACE.

First, the two methods differ in their attack objectives. PLAGUE primarily focuses on multi-turn conversational jailbreaking of language models, gradually inducing models to generate responses that comply with harmful objectives. In contrast, TRACE focuses on end-to-end execution of complex harmful tasks by LLM agents. It aims not only to bypass the model's refusal of harmful requests and generate harmful textual response but also to induce the agent to invoke tools, perform actual operations, and ultimately accomplish the original task objective.

Second, the two methods differ in their attack planning and interaction mechanisms. PLAGUE employs Planner, Primer, and Finisher to plan and optimize multi-turn conversations, progressively establishing a seemingly benign conversational context through intermediate questions. On the other hand, TRACE adopts execution-oriented task planning and interaction mechanisms. It uses task-relevant decomposition schemes to construct subtask sequences that preserve operational dependencies and selects appropriate sequences based on both harmfulness and execution difficulty. For refused subtasks, TRACE further employs execution-oriented multi-turn interaction strategies to progressively establish an execution context based on the current execution environment, available tools, and phase execution goals.

Finally, the two methods differ in their feedback learning mechanisms. PLAGUE primarily evaluates and reflects on model responses based on compliance, usefulness, detail, and relevance to the target objective, and stores successful jailbreak plans in long-term memory for future reuse. In contrast, TRACE uses tool execution results, environment states, and subtask completion as its core feedback signals. It not only updates the strategy library based on successful experiences but also refines execution strategies and their strategy-profile association by accumulating failure lessons.

To further compare the two methods, we conduct experiments with PLAGUE using Codex GPT−55˙GPT-5\.5. The ASR results are shown below:

MethodAdvCUAAgentHazard
PLAGUE00.14
TRACE0.620.97

The results demonstrate that TRACE substantially outperforms PLAGUE on both benchmarks. In PLAGUE's failed trajectories, we observe a common phenomenon: the target agent exhibits verbal compliance without performing actual operations. For example, the agent may respond with "Sure" to indicate acceptance of a task, followed by seemingly complete terminal transcripts, Bash scripts, or operational instructions, without actually invoking any tools or executing commands. Some responses even explicitly exclude critical operations, despite appearing willing to comply with the instructions.

We will cite PLAGUE in the Related Work section and incorporate the corresponding discussion and additional experimental results in a subsequent revision.

AuthorsOpenReview ↗

Follow-up on Response and Revisions

Thank you again for your thoughtful and constructive feedback, which we have carefully addressed in our author response. We have also incorporated the corresponding revisions into the manuscript and appendix. As the discussion deadline is approaching, we would greatly appreciate any further feedback you may have. If our responses have resolved your main concerns, we would also be grateful if you could consider updating your assessment accordingly.

Comments on the paper

1 comment

Hi reviewers, as we approach the rebuttal deadline, please review the authors' responses and participate in the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.