acceptodds
Under review as a conference paper at ICLR 2027

Rolesplit: Heterogeneous Role Allocation In Agent Configuration Poisoning

Abstract

LLM agents rely on skills and memory for task guidance and on tool descriptions to interpret available operations. These distinct functions raise a security question: can distributing complementary attack instructions across components make an unauthorized action more likely, even when the legitimate task is completed correctly? We introduce RoleSplit, a configuration-poisoning approach that assigns attack roles according to component function. A poisoned skill supplies a procedural reason for an additional operation, while a poisoned memory annotation makes the requirement depend on normal task records for its applicability and arguments. In both constructions, poisoned tool metadata misrepresents the operation’s public effect as private archival. Across model services from four families, we compare these allocations with placing both roles in one component using the same attack-role text. Allocation gains vary across services. On Qwen 3.7, skill–tool placement achieves disclosure alongside correct private delivery in 30/56 cases, compared with 0/56 and 2/56 when both roles are placed in the skill or tool description, respectively. Changing the injected policy reference reduces memory–tool disclosure, linking the attack to task applicability. Attack effectiveness thus depends on how instructions are distributed across component functions as well as what they say. Code and data are available at https://anonymous.4open.science/r/rolesplit-049C/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, Agent Security, Configuration Poisoning, Heterogeneous Role Cooperation, Tool Poisoning, Memory Poisoning, Skill Poisoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_b6FaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RoleSplit, a configuration-poisoning attack that distributes complementary malicious roles across heterogeneous components of an LLM agent. The attack separates an additional-operation requirement RR from a false consequence description DD. In the skill–tool construction, a poisoned workflow makes an additional operation appear procedurally required, while poisoned tool metadata falsely describes its public effect as private. In the memory–tool construction, a policy-bearing memory annotation makes the additional operation task-dependent and binds its arguments to task-relevant records, while tool metadata again misrepresents the operation's effects.

Critical strengths

The controlled placement comparison is well designed. The main experiments preserve identical instantiated attack-role text across the joint and complete conditions, making it possible to isolate placement more cleanly than a comparison between separately engineered attack prompts.

I also appreciate that the authors anticipate a natural concern about the complete baselines being poorly structured. The supplementary experiment introduces an explicit coherent single-component procedure that specifies the full private-then-audit workflow; joint placement remains substantially more effective on Qwen 3.7. This strengthens the central empirical claim.

The use of environment-state scoring is another strength. Attack success requires an actual public disclosure while maintaining correct private task completion, rather than merely detecting malicious language in the model's output.

The paired statistical analysis is appropriate. Comparisons use matched content instances with exact McNemar tests and Holm correction, and the paper reports joint-only versus comparator-only outcomes. This provides substantially more information than aggregate ASR alone.

The task-applicability intervention in M+T is particularly convincing. Changing only the malicious annotation's applicable-policy reference sharply reduces attack success while preserving the relevant task facts and correct private answer. This supports the interpretation that the attack exploits the relationship between poisoned guidance and task-relevant state rather than simply the presence of malicious text.

Finally, the trusted-information study provides useful defensive insight: trusted task boundaries and tool-effect information can substantially reduce attack success while preserving performance on the evaluated legitimate tasks.

Critical weaknesses

My first concern is novelty. Distributed or conjunctive malicious content across multiple sources has already been explored in prior work, including attacks that distribute sub-payloads across skills or tool metadata. The novel distinction here appears to be the allocation of semantically complementary roles across heterogeneous components according to component function. I find this distinction meaningful, but currently the paper does not fully isolate whether the observed gains come specifically from role-to-function alignment versus more generic effects of source separation, prompt position, or context structure.

A direct role-swapping control would strengthen this claim substantially. For example, one could retain two-component heterogeneous placement but place DD in the workflow/memory and RR in tool metadata. If the proposed function-aligned placement substantially outperforms this misaligned distributed condition, the claimed mechanism would be much clearer.

Second, the evaluation is quite narrow despite the large number of trajectories. The two main studies are synthetic variants of private-notification tasks with unauthorized public disclosure as the adversarial objective. This controlled setting is appropriate for a causal study, but it is unclear whether the phenomenon generalizes to qualitatively different agent actions such as file modification, code execution, external API calls, purchases, or credential access.

Third, the threat model assumes that the attacker can modify multiple heterogeneous configuration components. In many realistic systems, skills/memory and tool metadata may belong to different trust domains or providers. The paper would benefit from a clearer discussion of practical deployment scenarios in which one adversary controls both sources, and how the increased attack effectiveness should be weighed against the stronger compromise assumption.

Fourth, each content-condition pair is evaluated with a single rollout. The paired analysis controls content variation well, but it does not quantify within-instance stochastic generation variance. Repeated rollouts for at least a subset of models and conditions would increase confidence that the observed placement effects are stable.

Finally, the two constructions should not be compared by absolute ASR: S+T and M+T use different tasks and different requested thinking modes. The paper largely treats them separately, but I think this limitation should be made more explicit.

Questions for authors
  1. Can the authors evaluate a heterogeneous but role-misaligned placement, e.g. DD in skill/memory and RR in tool metadata, to distinguish functional alignment from generic payload distribution?

  2. Does RoleSplit improve attack success for qualitatively different unauthorized operations beyond private-to-public information disclosure?

  3. What realistic system or supply-chain scenarios allow a single attacker to modify both the workflow/memory and tool metadata? Are these assumed to originate from the same third-party provider?

  4. Can the authors repeat a subset of the main experiments with multiple rollouts per content to estimate stochastic variability?

  5. Are the S+T and M+T success rates intended only for within-construction comparison? If any cross-construction conclusion is intended, how do the authors account for the different task structures and thinking modes?

3 comments

Thank you for the detailed review and for recognizing the matched placement design. We evaluated role reversal, a second unauthorized action, and repeated calls on Qwen 3.7.

Reversed placement. In the same matched Qwen 3.7 follow-up, Joint and Reversed achieve 34/56 and 28/56 for S+T disclosure, and 47/56 and 27/56 for M+T disclosure. The paired outcomes include 21 Joint-only and 15 Reversed-only successes in S+T, and 24 and 4, respectively, in M+T. Reversal holds two-component delivery fixed while changing which component carries each role and the context in which it is presented. Together with the shared-record modification results below, these findings indicate an allocation effect that depends on the construction and requested operation (Section 5.3, Table 5, and Appendix F.2).

Other unauthorized actions. We evaluated an unauthorized shared-record status change while keeping the legitimate private notification. For S+T, Joint achieves 35/56, compared with 0/56 for Reversed, 0/56 for Complete S, and 1/56 for Complete T. For M+T, Joint, Reversed, and Complete T all reach 56/56, and Complete M reaches 47/56. The S+T benefit therefore extends to this shared-state action, while the M+T result shows a ceiling among three placements (Table 5 and Appendix F.1).

Dual-component access. Our threat model assumes that the attacker controls a Skill or Memory entry and has prepared a tool with poisoned registration metadata. When the compromised Skill or Memory entry is invoked, it explicitly instructs the agent to install or enable this specific attacker-prepared tool. Once the tool is registered, its poisoned description becomes part of the agent's configuration. In the main Joint allocation, R resides in the Skill or Memory entry, while D resides in the registered tool metadata. This establishes the two-component configuration studied in RoleSplit without requiring the attacker to directly modify the description of an already-registered tool.

Repeatability. We added two repeat executions for each of the three main-study placements on the same 56 Qwen 3.7 contents per construction. Including the original run, Joint succeeds on 30/56, 30/56, and 30/56 S+T contents and 49/56, 49/56, and 48/56 M+T contents, exceeding both Complete placements in every round. The runs retain each construction's generation settings and state-based endpoints (Appendix F.3, Table 14).

Within-construction comparisons. Yes. We compare placements on the same contents, with the same outcome definition and generation settings within each construction. S+T and M+T use different tasks, success criteria, and requested thinking modes. Their absolute success rates describe their respective settings (Section 4.1, Table 1, and Figure 2).

Thank you for the additional experiments and clarifications. The role-reversal control, second unauthorized action, and repeated runs address several of my concerns.

One question remains about the role-to-function alignment claim: for S+T disclosure, the Joint-only and Reversed-only outcomes are relatively close (21 vs. 15). Could you clarify how strongly this result supports functional alignment, as opposed to a more general benefit from distributing the payload across components?

Thank you for this important distinction. We agree that S+T disclosure alone provides limited evidence for a specific role-to-function alignment mechanism. Joint succeeds on 34/56 contents versus 28/56 for Reversed, but the paired discordant counts (21 Joint-only versus 15 Reversed-only) do not establish a strong difference. Both distributed placements substantially exceed the complete single-component placements (Complete S: 0/56; Complete T: 6/56) in this follow-up, so a more general benefit from distributing the roles is consistent with the S+T disclosure result.

Our evidence for allocation sensitivity beyond distribution alone comes from the different outcomes under the same two-component delivery structure when the roles are exchanged:

Matched Qwen 3.7 settingJointReversedJoint-only / Reversed-only
S+T, public disclosure34/5628/5621 / 15
M+T, public disclosure47/5627/5624 / 4
S+T, shared-record modification35/560/5635 / 0
M+T, shared-record modification56/5656/560 / 0

These comparisons (Section 5.3; Appendix F.2, Table 13) show that which component carries which role can matter, but the effect is not uniform: it is pronounced in M+T disclosure and S+T record modification, modest in S+T disclosure, and unidentifiable at the 56/56 ceiling for M+T record modification. They do not by themselves separate semantic component function completely from presentation context or source authority. Accordingly, our conclusion is an allocation effect conditional on the construction and requested operation, not universal superiority of the function-aligned placement. This is the intended interpretation in Section 6.

Reviewer_L3JjOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies distributed poisoning and how heterogeneous components change attack effectiveness and execution.

Critical strengths

Sorry I couldn't find any strengths.

Critical weaknesses

(1) I don't think the idea is novel enough to make the cut for ICLR. The literature has shown many times that such attacks work in different settings and therefore I don't believe there is much contribution in showing one more variation also works.

(2) The experiments are very weak. They test on a limited synthetically generated set of their own, for which the details are not clear. Also there are no comparisons to show that the attack is working better than the case for single component case.

Questions for authors

see above

3 comments

Thank you for raising the concerns about the baselines, novelty, and evaluation details.

Single-component comparisons. The submitted manuscript compares Joint with Complete S/M and Complete T in Table 2 (Table 1 in the revision). Each complete baseline places both roles in one component and uses the same instantiated role texts and task content as Joint. On Qwen 3.7 S+T, Joint achieves 30/56 versus 0/56 for Complete S and 2/56 for Complete T. The revision also reports the ordered complete tool-side comparator under the main S+T generation settings. Joint achieves 76/96 versus 33/96 for the comparator, and both deliver the correct private notification on 96/96 contents (revised Table 2 and Appendix D). Several model settings also show competitive complete placements, reported in the same main table. We also repeated the three main-study placements twice on the same 56 Qwen 3.7 instances per construction. Joint remained ahead of both Complete placements in every round (Appendix F.3, Table 14).

Contribution. Prior work establishes that distributed malicious content can succeed. RoleSplit asks whether assigning the same complementary roles to components with different functions changes the outcome. The matched placement comparisons isolate allocation from changes in the instantiated role texts. The added reversal retains two-component delivery and tests an alternative assignment, with effects that vary by construction and requested operation (Sections 2–3 and 5.3).

Evaluation details and scope. The equity-notification and reimbursement-notification tasks each instantiate 56 synthetic contents. The instances vary task facts and record presentation, and every placement for a given content shares its underlying task (Section 4.1 and Appendices B–C). We score the executed state for both correct private delivery and qualifying disclosure. M+T additionally checks that the disclosed fields match the target record (Section 4.2). The revision also evaluates an unauthorized shared-record status change (Table 5). These controlled tasks establish where allocation changes outcomes within the evaluated workflows and operations.

Thanks to the authors for the rebuttal and sorry for the late response. Unfortunately, I believe my assessment to be majorly correct despite your rebuttal. In addition, for future improvement, I would like to suggest that you rewrite the paper in a more academic style (the writing is very suboptimal) and also show how you evaluated that the synthetic tasks are sensible -- this can be done with human review of the tasks, making sure they are diverse etc. -- incase these measures haven't been taken already.

Thank you for your suggestions regarding the writing and synthetic task validation. We manually inspected the task templates and a subset of generated instances. As noted in the abstract, our anonymous code repository provides the complete task templates and all evaluated task instances, allowing reviewers to directly examine their design, plausibility, and diversity.

Reviewer_433WOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether distributing complementary attack instructions across agent configuration components makes them more effective. RoleSplit places a requirement for an additional operation in a skill or memory record, and a misleading description of that operation’s consequences in tool metadata. The goal is to disclose sensitive information publicly while still completing the requested private notification correctly.

The experiments compare distributed placement with role deletions and complete payloads placed in a single component. They cover two synthetic task templates and model services from four families. Distributed placement produces large gains in several settings: on Qwen 3.7, skill–tool success rises to 30/56, versus 0/56 and 2/56 for the complete single-component placements. The paper also tests policy applicability, stronger single-component procedures, and trusted-information interventions.

Critical strengths

The placement comparisons are well controlled and show meaningful effects. Keeping the instantiated attack-role text fixed makes the comparison more informative than comparing unrelated payloads. The authors report paired outcomes, uncertainty intervals, and corrected statistical tests. They also acknowledge competitive single-component attacks on some models. The additional procedure comparison in Appendix D addresses an important alternative explanation: that the single-component payload simply lacked an explicit follow-up plan.

The evaluation measures harmful execution alongside correct task completion. Success requires actual public disclosure and exactly one correct private message, rather than a suspicious response or proposed tool call. This captures a security failure that ordinary task-success evaluation would miss. Changing only the memory annotation’s policy reference also provides useful evidence that the attack can depend on task applicability and record selection.

Critical weaknesses

The task diversity is limited despite the large number of trajectories. The main studies use two notification templates, with instances mostly varying names, identifiers, amounts, and record order. Both attacks ultimately add a public message described as private archival. This establishes the phenomenon in a controlled setting, but leaves uncertainty about other workflows and malicious effects. One rollout per condition also leaves repeatability across service calls unmeasured; the reported intervals describe variation across contents rather than repeated execution of the same inputs.

The experiments establish a placement effect more clearly than a functional role-matching mechanism. The main distributed condition always puts the requirement in the skill or memory and the false consequence description in tool metadata. There is no reversed allocation or comparable control that distributes the text without matching these proposed functions. Differences in presentation, context position, and perceived source authority could therefore contribute to the gains. This matters because heterogeneous role allocation is the paper’s main distinction from existing distributed poisoning attacks.

Questions for authors

Assessment-changing control: What happens when the two roles are reversed—placing the requirement in tool metadata and the false consequence description in the skill or memory? This would help distinguish functional matching from a more general benefit of distributing the payload.

Clarification: How were the final requirement and consequence templates selected? Were they revised using results from the evaluated endpoints or task instances? Please distinguish exploratory development from held-out evaluation.

Assessment-changing validation: Can you repeat a representative subset across several calls and independently written payload paraphrases? This would show whether the placement advantage is stable beyond the particular wording and single recorded rollout.

Clarification: What realistic compromise path gives one attacker control over both components, especially the registered tool description? The evaluation assumes this access; a concrete deployment example would help establish the practical scope of the threat.

1 comment

Thank you for recognizing the matched placement controls and state-based evaluation. We added a reversed placement, a shared-record task, repeated calls, and a wording variant.

Role reversal. On a separate matched Qwen 3.7 follow-up, Joint and Reversed achieve 34/56 and 28/56 for S+T disclosure, and 47/56 and 27/56 for M+T disclosure. The paired outcomes include 21 Joint-only and 15 Reversed-only successes in S+T, and 24 and 4, respectively, in M+T. For shared-record modification, Joint and Reversed yield 35/56 and 0/56 in S+T, and both reach 56/56 in M+T. Reversal holds two-component delivery fixed while changing which component carries each role and the context in which it is presented. The results show a placement effect that depends on the construction and operation (Section 5.3, Table 5, and Appendix F.2).

Template development. Exploratory development used the Qwen 3.7 model service also used in the final experiments. The 56 final evaluation instances in each construction were separate from the instances used in exploration. For the reported matched comparisons, we used the same fixed instantiated role texts across placements for each content and report all placement conditions in the final design (Sections 3.2–3.3 and Appendix B).

Repeated calls and wording. On Qwen 3.7, we repeated each of the three main-study placements twice using the same original 56 contents per construction and the original generation settings. Together with the original main run, Joint succeeds on 30/56, 30/56, and 30/56 S+T contents and 49/56, 49/56, and 48/56 M+T contents. Joint exceeds both Complete placements in every round (Table 14 and Appendix F.3).

We drafted another wording variant and used the same 56 main-study contents in each construction. Joint, Complete S/M, and Complete T yield 28/56, 0/56, and 16/56 for S+T and 47/56, 27/56, and 20/56 for M+T. Although the S+T Joint advantage over Complete T was smaller with this wording, Joint still retained an advantage (Table 3 and Appendix F.5).

Dual-component access. Our threat model assumes that the attacker controls a Skill or Memory entry and has prepared a tool with poisoned registration metadata. When the compromised Skill or Memory entry is invoked, it explicitly instructs the agent to install or enable this specific attacker-prepared tool. Once the tool is registered, its poisoned description becomes part of the agent's configuration. In the main Joint allocation, R resides in the Skill or Memory entry, while D resides in the registered tool metadata. This establishes the two-component configuration studied in RoleSplit without requiring the attacker to directly modify the description of an already-registered tool.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.