acceptodds
Under review as a conference paper at ICLR 2027

Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DECOMPBENCH

Abstract

LLM-based Agents are becoming increasingly capable and widely deployed, creating growing incentives for adversarial misuse in the real-world. A key emerging threat is Decomposition Attacks (Glukhov et al., 2025, and Jones et al., 2024) in which a harmful task is broken into simpler, benign subtasks that evade safety mechanisms when executed separately but cumulatively fulfill the malicious intent. Although recent benchmarks assess agent safety in multi-turn and multi-tool-use settings, they do not explicitly capture this form of decompositional misuse and may not represent realistic adversarial execution flows. To this end, we introduce DeCompBench, a benchmark designed specifically to evaluate agentic safety under decomposition attacks. DeCompBench is created with a decomposition-by-design principle using a graphical framework and enables harmful task decomposition into individually benign and executable subtasks with realistic workflows. Our experiments using a prompt-based LLM decomposer show that state-of-the-art agents exhibit high refusal rates on monolithic harmful tasks, but significantly lower refusal rates on their decomposed variants, often inadvertently fulfilling the adversarial objectives. These findings underscore the need for safety evaluations against decomposition attacks and corresponding defenses. Our dataset is available at [Kaggle](https://kaggle.com/datasets/f734e8260fc654ec63565165e5b3e25d96ff7dcf6853bc8b6c7334a98075e0b2).

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Agentic Safety, Decomposition Attacks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_FEGoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DecompBench contains 253 harmful tasks constructed from capability-based DAGs. It evaluates attacks using benign-looking subtasks, shared environment state, and cleared conversation history. Across seven agents, decomposition reduces refusals and increases attack success; GPT-5-mini rises from 0% to 68.25%.

Critical strengths

S1: The benchmark measures concrete harmful outcomes in stateful, multi-service environments; 74.9% of checkpoints are solely rule-based. S2: Large effects across seven models, independent guard checks, human agreement samples, and execution-failure analyses support the reported vulnerability.

Critical weaknesses

W1: Cross-benchmark refusal and success rates do not independently establish greater harmfulness, difficulty, or superiority of the construction principle. Task composition and technical failures differ across the compared datasets (Section 5.1; Appendix G). W2: Run variability is unevenly assessed: four frontier agents have one run, while Haiku and Qwen vary temperature across their three runs (Appendix E). Their reported variability therefore does not isolate repeatability under fixed settings. W3: Appendix C validates benignness per subtask, but does not report how many complete attacks have every subtask judged benign. This leaves uncertainty about attack success specifically within the subset satisfying C3 throughout.

Questions for authors

Q1: Are decomposer configurations, execution limits, and success/refusal scoring aligned between MT-AgentRisk and DecompBench in Table 5? Q2: Can paired task-level uncertainty intervals be reported for monolithic-versus-decomposed ASR, separately from variation across temperatures? Q3: Under each guard, what fraction of attacks have all subtasks labeled benign, and what is ASR within that subset?

Reviewer_VMX5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces DECOMPBENCH, a benchmark built using a decomposition-by-design methodology that enables systematic evaluation of agent safety against decomposition attacks, and uses it to demonstrate substantial safety degradation under decomposed task formulations.

Critical strengths

Strength 1: The paper introduces a focused benchmarking framework for an underexplored agent-safety threat model.

The paper's most significant contribution is the introduction of DECOMPBENCH and its accompanying decomposition-by-design construction methodology. Existing agent-safety evaluations largely focus on direct harmful requests or multi-turn interactions, whereas this benchmark is explicitly designed to evaluate whether harmful objectives can be achieved through sequences of individually benign actions. While the underlying threat model is not new, the paper provides a concrete and systematic way to study it. This contribution is consequential because it expands the scope of safety evaluation beyond settings that may overestimate agent robustness.

Strength 2: The benchmark appears sufficiently realistic and comprehensive to provide actionable insights about current systems.

The benchmark includes multi-step workflows spanning multiple services and attack categories, moving beyond simplified single-task evaluations. The experimental results suggest that agent safety performance can differ substantially between direct and decomposed task formulations. Although the realism of the benchmark could be further validated, the evaluation nevertheless provides useful evidence that decomposition may represent a meaningful blind spot in current safety assessments. The practical relevance of this finding increases the potential value of the benchmark as an evaluation tool for future agent-safety research.

Critical weaknesses

Weakness 1: Limited validation that DECOMPBENCH reflects realistic decomposition attacks.

A central motivation of the paper is that existing benchmarks fail to capture how real adversaries may decompose harmful objectives into seemingly benign subtasks. However, while the benchmark is carefully designed, the paper provides limited evidence that its manually constructed decompositions resemble those used in practice. In particular, the benchmark's attack workflows are derived from hand-authored seed tasks and DAG templates, rather than empirical attack traces or studies of attacker behavior. As a result, it remains unclear whether the benchmark captures a broadly relevant threat model or a particular interpretation chosen by the authors. This concern is consequential because the paper's claims about the limitations of existing evaluations depend on the realism and representativeness of the benchmark.

Weakness 2: The empirical findings may be difficult to disentangle from the benchmark construction process.

The paper's main experimental result is that agents exhibit weaker safety performance when harmful objectives are decomposed into individually benign subtasks. While this observation is interesting, the benchmark is explicitly designed to produce tasks with this property. Consequently, it is difficult to determine whether the results reveal a previously underappreciated safety vulnerability or largely validate the benchmark's construction methodology. Without stronger evidence that the benchmarked scenarios correspond to realistic deployments, the broader significance and generalizability of the reported safety degradation remain somewhat uncertain. This limits my confidence in the extent to which the empirical conclusions extend beyond the benchmark itself.

Questions for authors
  1. Representativeness of the benchmark

The paper's conclusions depend heavily on the assumption that the decompositions encoded in DECOMPBENCH are representative of realistic adversarial behavior. Could the authors provide additional evidence or justification that the hand-authored attack graphs and decomposition strategies reflect how real attackers would decompose harmful objectives in practice, rather than a benchmark-specific construction?

  1. Robustness to benchmark design choices

DECOMPBENCH is built from manually designed seed attacks, attack categories, and DAG templates. How sensitive are the benchmark's conclusions to these design decisions? For example, is there evidence that alternative task constructions or decompositions would lead to similar safety degradation trends?

  1. Interpretation of the empirical findings

The benchmark is explicitly designed so that harmful objectives are decomposed into individually benign-looking subtasks. Given this design, how should the observed safety degradation be interpreted as evidence of a broader safety vulnerability rather than as a consequence of the benchmark construction itself? Additional discussion or analysis clarifying this distinction would strengthen the paper.

  1. Support for claims regarding existing benchmarks

A central motivation of the paper is that existing agent-safety benchmarks may overestimate safety by failing to capture decomposition attacks. Could the authors provide additional evidence that the failure modes exposed by DECOMPBENCH are not already covered by prior benchmarks, and that performance on existing benchmarks is insufficient to characterize the risks evaluated here?

Reviewer_WWBVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DECOMPBENCH evaluates tool-using agents against harmful objectives split into apparently benign subtasks. It contains 253 tasks across eight harm categories, constructed from 101 manually authored seed graphs and 335 capabilities. Seven agents are evaluated on monolithic requests and decomposed requests with conversation history cleared between subtasks. Attack success increases substantially, including from 0% to 68.25% for GPT-5-mini. Validation includes safety-guard assessments of subtasks, human agreement checks, and evaluation of resulting environment changes.

Critical strengths
  1. Evaluation measures completed harmful actions. Environment checkpoints go beyond judging whether an agent produces an unsafe response. This makes the observed increases in attack success consequential, especially because 74.9% of checkpoints are solely rule-based.
  2. The benchmark supports executable workflows across services. Manually authored seed graphs and concrete tool environments provide a useful resource for studying cumulative misuse. Consistent vulnerability across seven agents strengthens the empirical finding.
Critical weaknesses
  1. The central benign-subtask criterion isn't fully established. Section 4 explicitly introduces artifact wrapping and file indirection to hide harmful operations or values. A harmless-looking instruction can still request a harmful action through its referenced artifact. High benign-label rates on isolated subtask prompts therefore don't establish benignness of the executable subtasks. This distinction matters because benign composition is the benchmark’s main claimed differentiator.
  2. The comparison combines several changes without separating their effects. Decomposition introduces procedural guidance, masking, and history resets; execution-budget matching is not reported. No ablation isolates these factors. The weak abliterated 8B baseline doesn't resolve this because it isn't capability-matched to the evaluated agents. The results demonstrate an effective combined attack, but provide less evidence about which mechanism drives its success.
Questions for authors
  1. Did guards and human annotators inspect the referenced files and scripts when judging subtask benignness? What fraction remains benign when assessed with the local information available to the executor?
  2. How do results change when history is retained and when masking is removed? Please also specify and match total execution budgets across conditions. These controls would clarify what the benchmark measures.
4 comments
AuthorsOpenReview ↗

Response to Reviewer WWBV (1/2): Subtask Benignness

Part 1 of response

Thank you for the thoughtful review and for recognizing the value of evaluating agent safety through what the agents actually accomplish. We appreciate your assessment that the consistency of the findings across models makes DeCompBench a useful resource for studying agent misuse.

Next, we address your questions below.

Q1: Subtask benignness

Thanks for the question. When decomposing the tasks and running them on an agent, we agree that benign-subtask files can still request a harmful action, but when running the decomposed subtasks, the attacker only cares about the benignness of subtasks as long as they are not refused by the agent. The low rates of refusal for our subtasks is evidence in itself that the subtasks were mostly considered benign. 196/253 (77%) of the tasks involve creating files, while the refusal rate is still very low, limited to a maximum of 16.60%.

Also, the dataset artifact released by us is the harmful monolithic tasks, and the users are free to use any decomposer of their choice to decompose the tasks: our decomposer (which uses mapping and wrapping operators) is just one instantiation, and a stronger attacker can use different operations that make the subtask files look even more benign.

Additionally, most of the files and scripts are created during the run of the previous subtasks, so it is not possible for the authors to make them benign by hand while the run is still going on, and hence the best we could do is use an LLM to automate making them benign-looking with our operators. The guards and human annotator evaluated the subtask instructions independently of the execution runs, without inspecting the referenced files or scripts.

We would like to clarify that the benign subtask isolation property is part of the harmful task creation methodology and it applies to the process before the harmful task construction. We will note in the paper that this is limited to the prompts.

...continued below

AuthorsOpenReview ↗

Response to Reviewer WWBV (2/2): Conversation History, Masking, and Execution Budgets

Part 2 of response

Q2: Conversation history, masking, and execution budgets

Thanks a lot for the suggestion. We did conduct your suggested experiment and found it to be enlightening, but we would like to clarify that such an experiment is useful to design the attacker but is not necessarily a comment on the benchmark : our final released benchmark is the monolithic harmful task benchmark and users are free to use any attack strategies on top, our paper just instantiates one such decomposer/attacker. From the experiment results, our attacker (masking and no history) is still the strongest of all and has the lowest safety refusals with the highest attack success rate.

These results also point to better defense design strategies: monitors considering trajectories across subtasks (rather than in isolation) can be much better at catching decomposition attacks.

For the experiment, we compared four GPT-5-mini runs on the same 253 tasks. We used run 1 from the paper as the baseline, then reran the original decompositions with history retained. We also regenerated the decompositions with masking disabled and evaluated them twice, once with history cleared and once with history retained. All four runs used OpenHands’ LLMSummarizingCondenser, so retained history was subject to summarization. All four runs were run under matched conditions, they all used the same limit of 150 steps per subtask.

RunSafety Refusal Rate (%)Attack Success Rate (ASR) amongst non-refused tasks (%)
Baseline (original run 1) Masking, no history6.3270.89%
Masking, history retained32.02%44.77%
No Masking, history retained36.76%36.87%
No Masking, no history16.60%54.98%

We see that between history and masking, history has a larger effect on the safety refusal in these runs. From a baseline of 6%, the safety refusal rate increased to 32% with history retained (+26 percentage points) and to 17% with masking removed (+10 percentage points): this means that history has more clues of harmful behavior than the files/scripts created in process (which our decomposer masked with operators). Additionally, changing the masking while keeping history retained only changed the refusal rate by ~4%, further emphasizing the point.

On the attack success front, we see that retaining the history lowers the ASR more than switching off masking : compared with the baseline of 71%, the attack success fell to 45% with history retained (-26 percentage points) and to 55% with masking removed (-16 percentage points): this points towards the summarization for history being lossy and not being good enough. Removing masking still lowers the ASR from 70% in the baseline to 54%, pointing towards suboptimal decomposition strategies that the model may have used in the absence of masking. (All trends are the same when considering ASR across all tasks instead of non-refused)

Both these results mean that attackers using decomposition attacks (in real life or in academic papers) are strongest when the history across subtasks is not retained, and the files/scripts are masked to hide harmful behavior. These also mean that to defend against decomposition attacks, monitors should look across subtasks.

Please let us know if there are any additional concerns or follow-up questions. Looking forward to hearing from you!

Thanks,

Authors

Dear Reviewer,

As the deadline for the discussion approaches, we wanted to follow up to ensure that our responses have adequately addressed your concerns. If there are any additional questions or points for discussion, we would be happy to address them promptly.

We look forward to your reply and sincerely appreciate your time and effort in reviewing our work.

Thanks,

Authors

Dear Reviewer,

Since today is the last day of the rebuttal discussion, we wanted to check if our responses have addressed your concerns and if there are any additional concerns we could answer. If not, we would be really grateful if you reconsider your score.

Thanks,

Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up to Reviewer VMX5: Comparison with MT-AgentRisk

We would like to make the following points re failure modes w.r.t. MT-AgentRisk.

(a) MT-AgentRisk checks for subtasks that run within a single session. An attacker could also just distribute their subtasks across sessions: for this setting, we show that when distributed across sessions, the subtasks lead to very low refusal rates and high attack success rates.

(b) There is a difference in the distribution of tasks, which makes our tasks more complex.

    (i) Our tasks can be divided into many more subtasks: on average, our tasks can be divided into 6 subtasks, while theirs can be divided into 3 subtasks. MT-AgentRisk does not specify the exact harm categories, so we could not compare.

    (ii) Our tasks involved on average 2.6 services, while theirs involved ~1.

    (iii) The task construction methodology is very different: we use a DAG-based construction to create the harmful tasks. We both use LLMs for decomposition, though.

(c) As mentioned previously, results also show that their monolithic harmful tasks are refused much less than ours, pointing towards our tasks being more harmful. On the other hand, agents refuse our subtasks much less than theirs, suggesting they more often treated our subtasks as benign.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.