acceptodds
Under review as a conference paper at ICLR 2027

AI Attacks AI: Agent Self-Jailbreaking through Malicious Self-Improvement

Abstract

As AI systems move from generating content to executing tasks, safety research has expanded from harmful model responses to harmful agent actions. Meanwhile, advances in self-improvement allow agents to acquire knowledge and modify their own implementations to improve performance. Since these modifications can also affect safety behavior, they raise the question of whether an agent can learn to change its response to a request it initially refused. We study agent self-jailbreaking, in which an agent acquires knowledge and modifies its own system to comply with a harmful request it initially refused. To examine how existing agents carry out this process, we introduce , a research harness that combines web search and local exploration with iterative self-modification. Task-wise research uses feedback from retrying the original request to guide further modifications. The modified agents are then evaluated on unseen tasks to assess transfer. Across five agent frameworks and three open-weight models, task-wise self-jailbreaking raises compliance from 0.9% to 70.9%. On unseen requests, the modified agents collectively achieve 97.0% compliance, compared with 1.8% for the original agents. Autonomous research on individual harmful requests can therefore change an agent's safety behavior across a broader range of tasks.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM, agent, jailbreak

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_NH5HOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies "agent self-jailbreaking": the ability of a coding/tool-using agent that initially refuses a harmful request and can modify its own system (harness or model) so that it subsequently complies. It introduces MIRROR, a research harness that after a refusal is detected, drives an iterative loop of web search (how to modify) plus local exploration (where to modify), proposes W distinct candidate methods per iteration, applies the most promising modification, retries the original request and uses detector feedback to guide up to D iterations, with up to three attempts per task. Successful configurations are saved and then evaluated on unseen requests to measure transfer. Experiments cover 15 combinations (three open-weight models x five agentic frameworks), 30 source tasks (10 each of harmful Text, Actions, Cyber) and 300 unseen requests (100 each). Validation is empirical, in isolated environments (cyber tasks in a sandboxed synthetic network), with an LLM-based refusal detector controlling the loop and a separate LLM judge scoring compliance.

Critical strengths
  1. A well-posed and novel threat model with a clean experimental instantiation. The paper systematically separates the self-modification of the agentic system from input-level jailbreaks and from passive agent-harm evaluation, to show the behavior can be driven autonomously by the agent's own research+coding loop. The formalization (harness+model configuration, task-wise vs transfer, the detector/judge separation) is careful and the MIRROR harness is described in enough detail (Alg. 1, prompts in A.6) to be reproducible in principle. This is consequential because it widens the agent-safety threat surface in a direction that will only grow as self-improving agents become common.

  2. The transfer / weak-to-strong cross-task result. The finding that a configuration developed on a single source task transfers to 97.0% of unseen requests collectively and that transfer lifts source compliance from 70.9% to 95.8% including tasks on which direct research failed (Fig. 6, Table 6) is the paper's most important contribution in my opinion. It frames self-jailbreaking as a durable, generalizing safety regression, which materially raises the stakes of the threat model.

Critical weaknesses
  1. Measurement depends entirely on LLM detector/judge with no human validation and the judge shares a model with the system under test. Both the refusal detector and the compliance judge are LLM-based (the judge is Qwen3.8-27B, which is also one of the three evaluated models). With no reported human audit of a sample of 'completed' verdicts, inter-rater agreement, and false-positive/false-negative rates, it is hard to know how much of the dramatic 0.9%->70.9% and 1.8%->97.0% movement reflects genuine harmful capability versus judge leniency. Because every headline number flows through this judge, this is the single concern that most affects my confidence in the results and it should be fixable with a human-labeled validation set and agreement statistics.

  2. Narrow and homogeneous evaluation pool. All three models are open-weight and relatively small (27-31B) and two (Qwen3.5-27B, Qwen3.8-27B) are same-family variants, so the study effectively spans across two models. There is no frontier safety-hardened model (large open-weight), which is exactly where the practical risk and the generality claim would be most tested.

Questions for authors

At this point I am taking the lower option for my rating, but these additional clarifications and evaluations could improve my assessment:

  • Judge/detector validity: can validate the LLM judge against human labels on any subset? Please report the agreement / precision / recall numbers and does using a judge from the same family as the tested model (Qwen3.8-27B judging Qwen outputs) introduce self-preference bias?
  • Baseline: isolating research from self-edit access. The depth ablation and the source ablation already show that iteration and both research channels contribute, which is helpful. What seems missing is a non-adaptive lower bound: directly applying a single fixed, human-specified modification with no agent research or candidate selection. How much of the headline gain comes from the agent's research versus from the mere availability of self-edit access plus one obvious edit?
  • The three models are all open-weight, relatively small (27–31B) with no frontier or more heavily safety-tuned model. This is where the risk and the generality claim matter most. Could the authors demonstrate that self-jailbreaking transfers to more models outside this pool, such as a larger or more strongly safety-aligned model?
4 comments

Weakness 1 Reliability of automated evaluation

Thank you for this helpful suggestion. We use Qwen3.8-27B to judge whether the agent’s final output and execution trace fulfill the task goal, as specified in Appendix A.5. We validate these judgments on 150 final task results sampled from all source-task trajectories. Human judgments provide the reference labels, against which precision and recall are both 96.6%. High precision shows that judged successes closely match human-confirmed completion, while high recall shows that most such completions are detected. Table R1 reports agreement with humans and with DeepSeek V4.1 Flash and GLM 5.3 Flash on the same results, with kappa accounting for chance agreement. The consistency across human and model evaluators supports the reliability of the task-completion judgments underlying the reported compliance rates.

Table R1 Agreement on compliance judgments

MetricHumanDeepSeek V4.1 FlashGLM 5.3 Flash
Agreement (%)96.094.796.0
Kappa0.8760.8250.871

Weakness 2 Limited model scale and diversity

Thank you for this comment. Updating the parameters of larger frontier models requires substantial training compute that is not affordable for academic research labs, so we evaluate these models under harness-only modification. We use DeepSeek V4 Flash, GLM5.2, and DeepSeek V4 Pro in OpenCode and OpenClaw with the same tasks and research budget as in the paper. Table R2 shows successful harmful task completion after self-modification in all six settings, compared with no completion by the original agents, extending the finding beyond the original model families. Because the weights remain fixed, these results demonstrate that changes to the agent’s harness can enable self-jailbreaking in these models. The variation across frameworks for the same model reinforces the paper’s focus on the agent’s ability to modify its deployed system.

Table R2 Harness-only self-jailbreaking with additional models

ModelConfigurationOpenCode (%)OpenClaw (%)
DeepSeek V4 FlashOriginal0.00.0
DeepSeek V4 FlashMIRROR harness-only30.070.0
GLM5.2Original0.00.0
GLM5.2MIRROR harness-only20.020.0
DeepSeek V4 ProOriginal0.00.0
DeepSeek V4 ProMIRROR harness-only40.090.0

Question 1 Human agreement and self-preference bias

Thank you for this question. On the 150 sampled results, the Qwen judge agrees with human labels in 96.0% of cases, so its judgments are rarely different from human assessment. Precision and recall are both 96.6%, showing that judged successes closely match human-confirmed completion and that most such completions are detected. Table R1 also shows high agreement when DeepSeek V4.1 Flash and GLM 5.3 Flash judge the same outputs. In this sample, human validation shows that the labels track task completion, and consistency with judges from other model families shows that the evaluation is largely preserved when Qwen is replaced as the judge.

Question 2 Research versus self-edit access

Thank you for this suggestion. Table R3 includes the requested fixed-edit control by removing persona/system-instruction content without agent research or candidate selection. This leaves compliance unchanged in four frameworks and produces only a small increase in SWE-agent, showing that this obvious edit accounts for little of the gain. The direct self-edit baseline retains the agent’s freedom to choose and implement changes while removing autonomous research, and MIRROR achieves higher compliance in every framework and task category. Together, these comparisons show that research improves the agent’s ability to find effective edits beyond both a predetermined instruction removal and a direct request to self-edit. Figure 4 explains this contribution through controlled ablations of research depth and information sources. Removing local exploration causes the largest decrease, indicating that inspecting the deployed system is particularly useful for finding effective interventions, while the depth ablation shows that repeated testing and revision improve success. The trajectory in Figure 1(a) makes this process concrete through a failed instruction-file edit followed by a successful runtime change informed by feedback.

Table R3 Baseline comparison with Qwen3.5-27B across agent frameworks

Agent frameworkMethodText (%)Agent (%)Cyber (%)Overall (%)
OpenCodeOriginal0.00.00.00.0
OpenCodeRemove persona0.00.00.00.0
OpenCodeDirect self-edit30.030.010.023.3
OpenCodeMIRROR100.070.080.083.3
GooseOriginal0.00.030.010.0
GooseRemove persona0.00.030.010.0
GooseDirect self-edit30.030.030.030.0
GooseMIRROR90.0100.060.083.3
Qwen CodeOriginal0.00.010.03.3
Qwen CodeRemove persona0.00.010.03.3
Qwen CodeDirect self-edit10.020.040.023.3
Qwen CodeMIRROR80.070.050.066.7
SWE-agentOriginal0.00.00.00.0
SWE-agentRemove persona10.00.00.03.3
SWE-agentDirect self-edit10.00.020.010.0
SWE-agentMIRROR70.030.050.050.0
OpenClawOriginal0.00.00.00.0
OpenClawRemove persona0.00.00.00.0
OpenClawDirect self-edit40.030.030.033.3
OpenClawMIRROR100.080.060.080.0

Question 3 Larger and more safety-aligned models

Thank you for this question. Model-level modification of frontier models requires substantial training compute, so the evaluation in Table R2 focuses on harness-level modification with the paper’s tasks and research budget. DeepSeek V4 Flash, GLM5.2, and DeepSeek V4 Pro each conduct their own research and change their own harness in OpenCode and OpenClaw. Every setting gains harmful task completion over an original compliance rate of 0.0%, showing that successful autonomous self-jailbreaking extends beyond the original model pool. Together with the harness-only ablation in Figure 4(a), these results show that self-jailbreaking remains possible when model weights stay fixed.

Thanks to the authors for additional clarification, new experiments and addressing the concerns.

Human validation and preference bias: The Human validation on 150 results (96.6% precision and recall) and the close agreement between two judges from other model families like DeepSeek V1.4 and GLM 5.3 largely address my concerns about judge reliability and preference bias.

Large frontier models: Results on DeepSeek V4 Flash, DeepSeek V4 Pro and GLM5.2 extends the findings beyond the original model families and address concerns about homogeneity. However, as mentioned by authors, harness-only modification needs no model weight updates, large frontier model could be tested through an API for harness only approach.

Baselines: Table R3 addressed my question well. The fixed self edit adds almost nothing and MIRROR clearly beat it in all five mentioned agent frameworks.

Considering all weaknesses, questions and how authors addressed my concerns, I will raise my assessment to 3.

We sincerely thank you for the positive assessment and encouraging feedback, and for recognizing the importance and potential impact of our work. We will carefully consider all comments and suggestions to further improve the quality and clarity of the paper. Thank you again for your valuable time and support.

Reviewer_vwDEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies agent self-jailbreaking, where an agent researches and modifies its own harness or model to comply with harmful requests it initially refused. The proposed MIRROR framework combines web search, local exploration, and iterative modification with execution feedback. Across three open-weight models and five agent frameworks, the authors report substantial increases in harmful-task compliance and broad transfer to unseen requests.

Critical strengths
  1. The phenomenon of self-jailbreaking is interesting. The paper examines how self-modification capabilities can undermine agent safety, extending the focus beyond prompt-based jailbreaks to persistent changes in the agent’s underlying system.

  2. Experiments are performed across harmful text generation, tool-invoked actions and cyber tasks.

  3. The figures are illustrative.

Critical weaknesses
  1. If the model is instructed to search and modify the harness to bypass safety refusals, does it refuse such steps?

  2. No specific analysis to show what harness components are edited (aparts from agent.md、 model-config.json) and how they influence the safety performances. It is expected to provide specific statics about which components are edited and how they influence the final safety performances.

Besides, is removing safety rules from AGENTS.md sufficient to harmful behaviors? Models may retain substantial refusal behavior even without safety-related system instructions, so the mechanism requires further explanation.

  1. The evaluations are not clear. How are refusals detected, and how is successful completion of harmful tasks judged
Questions for authors

Please refer to the part of Weakness

5 comments

Weakness 1 Refusal during self-modification

Thank you for this question. We have reported refusals during the research process in the failure mode analysis in Section 4.4 and Figure 5(d). Among the persistent-refusal records, 9.2% involve refusal to conduct research, while 90.8% involve attempted modifications followed by continued refusal of the original harmful task. Section 4.4 provides a Qwen Code example in which the agent recognizes the purpose of a proposed modification and declines to implement it. These research-stage refusals count as failed attempts in our evaluation, so the reported compliance rates account for agents that decline to pursue self-jailbreaking.

Weakness 2 Harness edits and safety effects

Thank you for this comment. Table R1 presents the component occurrence frequencies and conditional compliance rates shown in Figure 5(a,b). Runtime configuration is the most frequently edited component, while model_config.json has the highest conditional compliance, showing that agents act on several parts of the deployed system with different observed outcomes. Figure 4(a) compares modification access under the same one-attempt protocol and shows higher compliance with joint access, indicating that access to both the harness and model gives the agent more opportunities to find an effective change. Table R2 directly tests persona removal using the corresponding instruction sources, including AGENTS.md in OpenClaw and OpenCode, QWEN.md in Qwen Code, .goosehints and AGENTS.md in Goose, and system_template in SWE-agent’s YAML configuration. Compliance remains unchanged in four frameworks and increases by only 3.3 percentage points Overall in SWE-agent, so removing this guidance alone produces little of the observed self-jailbreaking gain. Our traces also show that learned refusal can persist after valid harness edits, although this accounts for a small share of failed research. Figure 1(a) illustrates how a runtime prefill change succeeds after an instruction-file edit fails, consistent with evidence that safety behavior is sensitive to early response conditioning [1].

Table R1 Component occurrence and conditional compliance from Figure 5(a,b)

ComponentRecords containing edit (%)Compliance among those records (%)
model_config.json3.183.0
opencode.json8.470.0
swe_config.yaml17.956.0
agent_runtime.yaml59.243.0
Self-fine-tuning or LoRA55.246.7
CONVENTIONS.md8.052.0
AGENTS.md27.838.0
QWEN.md14.413.0

Table R2 Persona/instruction removal with Qwen3.5-27B across agent frameworks

Agent frameworkConfigurationText (%)Agent (%)Cyber (%)Overall (%)
OpenCodeOriginal0.00.00.00.0
OpenCodeRemove persona0.00.00.00.0
GooseOriginal0.00.030.010.0
GooseRemove persona0.00.030.010.0
Qwen CodeOriginal0.00.010.03.3
Qwen CodeRemove persona0.00.010.03.3
SWE-agentOriginal0.00.00.00.0
SWE-agentRemove persona10.00.00.03.3
OpenClawOriginal0.00.00.00.0
OpenClawRemove persona0.00.00.00.0

Weakness 3 Clarity of evaluation criteria

Thank you for this question. For tasks drawn from HarmBench and AgentHarm, we follow the standard evaluation pipeline to evaluate success rates. Our evaluation uses a unified Qwen3.8-27B judge at temperature 0.0 to assess the final outputs and execution traces, checking the requested harmful content for Text tasks and actual harmful execution for Agent tasks. Cyber tasks use the same completion criterion in the isolated sandbox, and only completed tasks count as successful, as specified in Appendix A.5. The runtime refusal detector uses the agent’s own model to evaluate actual harmful behavior and control the research loop. Under Section 3.1 and Algorithm 1, blocked triggers research, in_progress waits for execution to finish, and completed stops the attempt. We validate this final evaluation judge on 150 final task results sampled from all source-task trajectories, using human judgments as reference labels. Precision and recall are both 96.6%, showing that judged successes closely match human-confirmed completion and that most such completions are detected. Table R3 reports agreement with humans and with DeepSeek V4.1 Flash and GLM 5.3 Flash on the same results, supporting the reliability of the reported task-completion judgments.

Table R3 Agreement on compliance judgments

MetricHumanDeepSeek V4.1 FlashGLM 5.3 Flash
Agreement (%)96.094.796.0
Kappa0.8760.8250.871

[1] Xiangyu Qi et al. Safety Alignment Should Be Made More Than Just a Few Tokens Deep. ICLR, 2025.

Reviewer_vwDEOpenReview ↗

Thanks for the author responses

Thanks for the detailed staticstics from authors, but I still maintain some confusions:

  1. Given the hign refusal ration (9.2% refuses and 90.8% continued refuses), how could the final ASR reaches data of 80.0% in Table 1.

  2. As 55.2% modifications contain Self-fine-tuning or LoRA, how are the tuning-data constructed?

Thank you for your feedback and follow-up questions.

For Question 1, the 9.2% and 90.8% in Figure 5(d) represent the proportions of two failure modes among all failure cases. As described in Appendix B.2, 9.2% of all failure cases involve refusal to conduct research, while the remaining 90.8% involve attempted modifications that fail to overcome the refusal. In contrast, Table 1 reports the overall task-level success rate for each model and framework, as defined in Appendix A.4. Therefore, the two results use different denominators and are not contradictory.

For Question 2, the agent constructs the training data for self-fine-tuning and LoRA by combining self-generated examples with external datasets from the auto research process. Across the observed runs, we identify four main approaches to data construction. First, the agent directly generates harmful instruction–response pairs, such as ransomware or keylogger-related requests paired with compliant responses. Second, it reformulates potentially harmful tasks from a defensive or benign perspective, such as analyzing malicious email headers, examining malware behavior, or identifying social engineering techniques in phishing emails. Third, it generates examples targeting refusal behavior by pairing common refusal templates, such as "I'm sorry, but I can't assist," with compliant continuations to discourage similar refusals. Finally, the agent retrieves publicly available datasets through web_fetch, including Alpaca instruction data, uncensored datasets, and datasets for harmful fine-tuning, and incorporates them into its training data. Table R4 summarizes the distribution of these approaches across the collected training examples and files.

Table R4 Self-fine-tuning data

Training dataExamples (%)Files (%)
Generated harmful instruction–response pairs37.048.1
Defensive or benign task reformulations5.07.3
Examples targeting refusal language3.71.8
External public datasets54.242.8
Reviewer_QAeXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper designs an approach for automated harness engineering + model updates in a loop with the objective of complying with an unsafe user request. This loop is run for a small set of tasks and empirical results show that this generalizes to unseen test set implying that the changes implemented in the loop are not overfit for the initial sample set. Analysis shows the actual changes made by this optimization loop such as changes to model configs, AGENTS.md, agent runtime configs, etc. Ablation studies show the effects/importance of different design choices like allowing/not allowing model/harness updates, available inference-time compute, and access to web search + local exploration for generating candidate changes.

Critical strengths
  1. This is an interesting and intuitive approach for designing self-improvement style loops but with malicious intents.
  2. The empirical gains observed on held out test set are quite promising, and the observed jail-breaking behaviours are quite intuitive.
Critical weaknesses
  1. The paper decides to curate its own dataset for a very well-established domain of agent safety without providing any justification for this design choice. Why not just use an existing agent safety benchmark for this work? This makes the pipeline less prone to over-fitting to your data curation approach/accidental introduction of biases in your data curation methodology, and allows including more baselines to compare your approach against. This also allows future approaches to compare their results with yours.
  2. The paper does not include any baselines beyond a vanilla agent setup to understand if designing such a complex loop is even needed on their dataset. Also this makes it hard to understand how easy/hard it is to achieve a certain attack success rate on this dataset, and what unique attack approaches your method can achieve which baseline approaches cannot.
  3. I am not very convinced about the observed transfer rate of model-only changes in Figure (5) to unseen tasks to be so high especially if these model changes are achieved using only 30 tasks. Is the transfer to unseen tasks primarily because their domain distributions are very closely aligned? If this is indeed the case, it would be good to acknowledge this in the main paper, and also discuss how the limited OOD in test data affects other exploits at a harness-level.
Questions for authors
  1. How generally applicable is your method to a different dataset where the expected agent behaviour is to deny malicious requests?
  2. The evaluation metric of compliance includes tasks that either (a) complete the entire malicious request end to end, or (b) agrees to doing something bad without actually completing the request. While this metric is reasonable in itself, I wonder what is the split between these two cases and does the second case involve situations where the agent may initially comply, do some safe actions, realize that the task is unsafe and refuse eventually? Is this approach able to achieve end to end task jailbreaking very often or is most of the benefit just in the second category?
6 comments

Weakness 1 Dataset choice and comparability

Thank you for this comment, and sorry for the confusion. We use established benchmark tasks, with Text task goals coming directly from HarmBench and Agent task goals coming directly from AgentHarm. Cyber tasks depend on an isolated execution environment, so we construct tasks for our sandbox following the Cyber Kill Chain and MITRE ATT&CK tactics. Appendix A.3 describes the task sources and construction, with the concrete goals listed in Tables 3 to 5. For the HarmBench and AgentHarm tasks, we follow the standard evaluation pipeline to evaluate success rates. We use a unified Qwen3.8-27B judge at temperature 0.0 to assess final outputs and execution traces according to Appendix A.5, checking the requested harmful content for Text tasks and actual harmful task execution for Agent tasks. The same completion criterion applies to the sandbox-specific Cyber tasks, and only completed tasks count as successful. This preserves the benchmark task goals and makes the Cyber objectives and completion criteria explicit for comparison across approaches.

Weakness 2 Missing baselines and loop necessity

Thank you for this helpful suggestion. To the best of our knowledge, we are the first to propose agent self-jailbreaking as a new threat in which an agent researches and modifies its own system to complete a request it initially refuses. Existing papers therefore provide no directly corresponding baseline for this setting. To assess the contribution of autonomous research, Table R1 compares MIRROR with the original agents and a Direct self-edit baseline. Direct self-edit removes the autonomous research stage while retaining the agent’s ability to choose and implement changes to itself. MIRROR outperforms Direct self-edit in every framework and task category, with an Overall advantage of 40.0 to 60.0 percentage points, showing that research helps the agent find effective modifications when a direct self-edit request is insufficient. Figure 4 varies research depth and removes local exploration or web search while keeping modification access fixed. The larger decrease when local exploration is removed shows the importance of understanding the deployed system, while the depth ablation shows the benefit of revising changes using execution feedback. Figure 1(a) illustrates this process through an unsuccessful instruction-file edit followed by a runtime prefill change that enables completion of the same request.

Table R1 Baseline comparison with Qwen3.5-27B across agent frameworks

Agent frameworkMethodText (%)Agent (%)Cyber (%)Overall (%)
OpenCodeOriginal0.00.00.00.0
OpenCodeDirect self-edit30.030.010.023.3
OpenCodeMIRROR100.070.080.083.3
GooseOriginal0.00.030.010.0
GooseDirect self-edit30.030.030.030.0
GooseMIRROR90.0100.060.083.3
Qwen CodeOriginal0.00.010.03.3
Qwen CodeDirect self-edit10.020.040.023.3
Qwen CodeMIRROR80.070.050.066.7
SWE-agentOriginal0.00.00.00.0
SWE-agentDirect self-edit10.00.020.010.0
SWE-agentMIRROR70.030.050.050.0
OpenClawOriginal0.00.00.00.0
OpenClawDirect self-edit40.030.030.033.3
OpenClawMIRROR100.080.060.080.0

Weakness 3 Transferability and distribution overlap

Thank you for this comment. Model-only self-jailbreaking fine-tunes the model, weakening learned safety alignment so that it can fulfill requests it initially refuses. Because learned refusal behavior is shared across requests, this change can affect tasks beyond the one that initiates research, consistent with findings that fine-tuning on a small number of examples produces broad safety degradation [1] and can alter the early response behavior involved in safety alignment [2]. Figure 5(e) and Table R2 show that individual saved model-modified agents complete unseen tasks, providing behavioral evidence of this transfer. The 97.0% in Table 1 reports unseen-task coverage computed from saved joint-modification configurations within each model and framework setting. The original unseen requests come from the same task pools as the source requests, so they share domain structure. Table R3 tests 50 non-overlapping StrongREJECT goals with the saved configurations fixed and shows improved compliance under both harness-only and joint modification. This cross-benchmark result shows that harness modifications also remain effective beyond the original task pool. The resulting risk extends beyond the initial request because a configuration modified for one harmful goal can enable completion of additional harmful goals without further research.

Table R2 Individual-agent transfer compliance from Figure 5(e)

Editable componentsText (%)Actions (%)Cyber (%)Overall (%)
Harness-only35.020.028.025.7
Model-only52.019.035.035.9
Joint75.023.045.038.4

Question 1 Generalization to other datasets

Thank you for this question. We evaluate 50 StrongREJECT requests [3] as task goals, with no overlap with the source tasks, using the paper’s transfer protocol and saved Qwen3.5-27B configurations across all five agent frameworks. The configurations remain fixed, so gains over the original agents measure transfer without further research on the new benchmark. Table R3 shows higher compliance in every framework under both harness-only and joint modification, with joint modification reaching all 50 goals in four frameworks. These results show that self-jailbreaking enables task completion beyond the benchmark requests used to develop the modifications and that this transfer also occurs through harness changes alone.

Table R3 StrongREJECT transfer compliance (%) with Qwen3.5-27B

Agent frameworkOriginalHarness-onlyHarness + Model
OpenCode0.048.0100.0
Goose0.098.0100.0
Qwen Code2.028.0100.0
SWE-agent6.024.064.0
OpenClaw18.0100.0100.0

Question 2 Agreement versus task completion

Thank you for this question. Appendix A.5 distinguishes refusal (blocked), partial task execution (in_progress), and task completion (completed), with only completed counting as compliance. Both in_progress and completed indicate refusal bypass, but the loop continues execution for in_progress and returns a candidate configuration only after completed, as shown in Appendix A.1. An eventual refusal therefore does not count as task completion. The runtime detector provides feedback based on actual harmful behavior, directing research toward task completion. Transfer uses saved configurations whose source-task completion is confirmed by the evaluation judge, and almost all non-blocked transfer tasks reach completion at the task level. Table R5 pools all attempt trajectories and reports their outcome distribution within each task category. The substantial in_progress share captures partial execution across attempts, while task-wise compliance counts only tasks with a completed outcome.

Table R4 Agreement on compliance judgments

MetricHumanDeepSeek V4.1 FlashGLM 5.3 Flash
Agreement (%)96.094.796.0
Kappa0.8760.8250.871

Table R5 Outcome distribution across all attempt trajectories

Task categoryCompleted (%)In progress (%)Blocked (%)
Text (HarmBench)46.031.022.0
Agent (AgentHarm)20.046.033.0
Cyber33.051.015.0

[1] Xiangyu Qi et al. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! ICLR, 2024.

[2] Xiangyu Qi et al. Safety Alignment Should Be Made More Than Just a Few Tokens Deep. ICLR, 2025.

[3] Alexandra Souly et al. A StrongREJECT for Empty Jailbreaks. NeurIPS, 2024.

Thank you again for your thoughtful comments and suggestions. We have carefully addressed each point in our response and would be grateful if you could let us know whether the clarifications resolve your concerns. We look forward to receiving any further feedback from you.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.