acceptodds
Under review as a conference paper at ICLR 2027

Plant, Persist, Trigger: Sleeper Attack on Large Language Model Agents

Abstract

Large Language Model (LLM) agents remain vulnerable to safety threats from the external environment, where attackers inject adversarial content into external observations such as tool-returned data, webpages, or MCP context, causing harmful agentic behaviors such as unsafe actions or incorrect outputs. Existing studies typically focus on single-interaction attacks, where the agent observes adversarial content and immediately exhibits harmful behavior within one user request. However, we show that adversarial content can also persist across interactions served by the same agent, making such threats harder to detect and mitigate. Specifically, adversarial content may persist in the agent state, remain dormant across interactions, and later be activated by a benign user query. We formalize this type of safety threat as Sleeper Attack. To evaluate it, we construct a benchmark with 1,896 instances covering six real-world harmful outcomes, three attack strategies, and three agent state targets: session context, memory, and reusable skills. Experiments on seven strong open-source and closed-source LLMs show that state-of-the-art LLM agents remain vulnerable to Sleeper Attack, even when they achieve low attack success rates under a single-interaction baseline. Our code and data are available at https://anonymous.4open.science/r/skdvnfu23ihr9wdscnksf1asdffsaef.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
agent safety, planning in agents, environment interaction, agent evaluation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_AxKmOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies sleeper attacks on LLM agents, where malicious content is saved during one interaction and causes harmful behavior during a later, benign request. The authors compare three attack strategies across session context, memory, and reusable skills, using a 1,896-instance benchmark and seven models. They also examine longer-term persistence, test defenses, and reproduce the attacks in OpenCode. The results show that resistance to immediate attacks does not necessarily carry over to later interactions, and that an initially unsuccessful attack can still leave harmful content in the agent’s state.

Critical strengths
  1. The framework is simple and coherent. The planting–triggering formulation brings memory and skill attacks into a common setting. The distinction between malicious instructions and corrupted facts is useful, since an agent can follow the user’s request yet act on poisoned information.

  2. The experiments provide useful evidence of persistent risk. Fresh-session replay shows that an initially unsuccessful attack can still cause harm later. The OpenCode evaluation also supports the findings beyond simulated tool outputs.

Critical weaknesses
  1. The cross-state results are difficult to interpret. Memory agents retrieve all saved notes, while skill agents must read the relevant skills before acting. The attack templates also differ across settings. These differences make it unclear how much of the ASR gap comes from the state type itself.

  2. The evaluation rules need further validation. In Table 17(b), both annotators marked 9 of 100 evaluation configurations as incorrect. The paper does not explain whether these errors were corrected or whether correcting them changes the main results. The reported 97% annotation agreement does not address this issue.

  3. The defense evaluation does not report its impact on normal tasks. Table 4 reports ASR but not benign-task success under each defense. Since Runtime Defense blocks memory and skill updates, it may also interfere with legitimate tasks. Without these results, it is difficult to assess whether the reduction in ASR comes at an acceptable cost.

Questions for authors

Q1. Both annotators marked nine configurations as incorrect in Table 17(b). Were these corrected before computing the reported results? What impact do these errors have on the ASR?

Q2. An update call alone does not confirm that the malicious content was saved. Please explain how retention is checked and how attacks that already succeed during planting are excluded.

Q3. Runtime Defense substantially reduces ASR. Please also report benign-task success under each defense, including tasks that require legitimate memory or skill updates.

4 comments
AuthorsOpenReview ↗

Response to Reviewer AxKm — Part 1

R1. Controls and implementation differences in cross-state comparisons (W1).

  • We thank the reviewer for pointing out the implementation differences that need further clarification in the cross-state comparison.

  • Our comparison uses a paired design: for the same base case, the user task, attack objective, evaluated model, and evaluation criteria remain consistent across state settings. Memory and Skill reads and writes are also implemented through a unified tool-calling framework.

  • These controls allow us to compare the risks of the same attack task across different state settings.

  • Memory provides saved information through retrieval, while Skill provides reusable procedures. These reading, writing, and usage patterns reflect how the different states operate. Attack templates are adapted to the different state interfaces and differ only in the instructions that induce Memory and Skill tool calls. Within each strategy × state setting, all models use a fixed template. We will clarify this in the paper.

R2. Corrections to the nine evaluation configurations and their impact (W2 and Q1).

  • We thank the reviewer for raising this issue.

  • Given the scale of the benchmark, we use an LLM to generate evaluation configurations and assess their quality through independent human auditing of a sample.

  • Manuscript Table 17(b) presents the independent audit results before correction. We corrected the nine configurations that both annotators judged incorrect before computing the final results. The ASR reported in the paper uses the corrected configurations.

  • We further analyzed the impact of retaining these problematic configurations. Without correcting the nine configurations and using their original judgments, the overall ASR is 31.864%, only 0.045 percentage points higher than the corrected result of 31.819%. The largest change in any model’s ASR is 0.106 percentage points, and the model ranking remains unchanged.

R3. Checking retention and excluding attacks completed during planting (Q2).

  • We thank the reviewer for raising this issue. We determine whether malicious content has been successfully saved using the execution status of the update operation and the write acknowledgment returned by the tool, and verify content retention through code-based checks.

  • We also check whether the attack objective is achieved at each interaction stage. If the final attack objective defined for a case has already been achieved during the planting stage, that trajectory is not counted as a successful Sleeper Attack.

R4. Benign task success under different defenses (W3 and Q3).

  • We thank the reviewer for the suggestion. We additionally report benign task success under each defense in Table 6.

Table 6. BSR-under-attack (%) under different defenses.

ModelNo DefenseRule DefenseGuard DefenseRuntime Defense
Qwen3.5-Plus57.659.161.663.1
Qwen3.5-Flash61.161.364.263.1
DeepSeek-R154.429.330.936.0
Llama-3.3-70B46.758.058.947.1
Average54.951.953.952.3
  • Average task performance changes relatively little across defenses. The average BSR is 54.9% without defense and 51.9%, 53.9%, and 52.3% under the three defenses, respectively.

  • Defenses can both block normal operations and reduce attack interference. For both Qwen models and Llama, the scores remain stable or improve after adding defenses. Together with the reduction in ASR, this indicates that these models can maintain normal task performance while reducing attack risk. One possible explanation is that defenses reduce attack interference with the original task, partially offsetting the impact of additional checks.

  • DeepSeek-R1 shows a more pronounced decline. Its score decreases from 54.4% without defense to 29.3%–36.0%. The trajectories show additional confirmation requests, paused execution, and failures to continue and complete the task, suggesting that the defenses may lead it to adopt a more conservative execution strategy.

AuthorsOpenReview ↗

Looking forward to hearing from you

Dear Reviewer AxKm,

We sincerely appreciate your time and thoughtful feedback on our work.

We would greatly appreciate any further feedback on our responses and would be happy to address any remaining questions or concerns. If you feel that our responses have adequately addressed your concerns, we would sincerely appreciate it if you would consider revisiting your evaluation.

Thank you again for your time and consideration.

Best regards,

The Authors

I thank the authors for the clarifications and additional experiments. Most of my concerns have been addressed. Overall, I will raise my score from 2 to 3.

We thank the reviewer again for their valuable suggestions and positive assessment of our work.

Reviewer_2MAXOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the attacks on LLM agents in which harmful/adversarial content is introduced to the agent via external sources and are stored in the persistent state. This content later triggers an attack through a future benign user request in a separate interaction. The authors define three attack strategies: delayed instruction execution, eliciting missing details from the user, and corrupting stored facts. They build a 1,896-instance simulated benchmark across six harm categories, evaluated with rule-based success checks. Across seven LLMs, they report that models with low single-interaction attack success can be much more vulnerable to Sleeper Attacks, with skills the most vulnerable state type and fact corruption the most effective strategy. Additional experiments cover defenses, trigger distance, model scaling, and a replication in the OpenCode agent framework.

Critical strengths
  • I appreciate the paper for introducing a timely problem with a useful unifying formulation. Prior work studies these attacks in isolation, for specific components such as memory or skill files. This paper unifies them under a single plant-then-trigger formulation while introducing multiple attack strategies. This framing offers a shared vocabulary for the area and enables comparisons that prior work does not support.

  • Each attack strategy is well motivated, and I appreciate that the authors supported and illustrated them with concrete examples/cases.

  • I also appreciate the broad evaluation that the authors have done. The benchmark is quite substantial in itself, with multiple instances of varying attack strategies, harm categories, and diverse application domains. The evaluation covers not only open but closed source models as well (Gemini, GPT, Qwen, DeepSeek, Llama) and a size sweep within one family (Qwen3 4B–32B). This makes the evaluation more practical from an average user’s perspective. The breadth of the study suggests that the observed vulnerabilities are not specific to a single model or scenario, and makes the benchmark a useful resource for future work on agent safety.

Critical weaknesses
  • I believe the overall design of the experiments carried out did not isolate the effect of the persistent state from its original injection in context. In the benchmarks, the planting and triggering occur in consecutive interactions of the same session ( as shown in section 4.1) and the policy conditions on all prior interactions ( as shown in Eq. 1, line 150). Hence, the adversarial output in the memory and skills conditions remains visible when the trigger arrives. Hence, the success of an attack in these conditions, I believe, cannot be linked to a persistent state alone. This would further blur the difference between “session” and other state types since the session context would always be present. This effectively in some ways weakens the claims the authors make in cross-state comparisons. That being said, the authors do present a good control experiment to address this design weakness by the fresh session replay experiment as presented in Table 6. However, in Table 6, they only cover 2 out of the 7 initially tested models. The results in Table 6 show that the visible context difference substantially changed the Attack Success Rate(ASR). I encourage the authors to report memory and skill results for all models with the trigger issued in a fresh context.

  • The central framing of the paper is cut by the fact that the planting step is itself a successful single-interaction injection. The paper’s main claim is that resistance to single-interaction attacks “does not reliably carry over” across interactions, and the abstract states “substantial cross-interaction vulnerability even when single-interaction attack success is low”. However, Table 5 itself shows that during the planting interaction itself, agents write attacker-supplied content into the persistent state in 65.6 to 84.1% of runs. Hence, the models evaluated are simply not resisting/evading the injection in the first interaction. They follow it while not doing any harmful action. The results would be better characterised as showing that decomposing a harmful action into a benign-looking write and a later execution bypasses refusal. That would be a meaningful finding, but a different one from the claim that robustness degrades across interactions. I suggest revising the paper accordingly and treating write compliance as a single-interaction failure in its own right.

  • A low ASR could also represent the model’s inability to use the persistent state instead of robustness. This makes the model comparisons seemingly unreliable. This is because memory and skill-based attacks would only succeed if the agent first writes and then uses the state. This effect is quite substantial if you compare tables 2 and 6. For example, Qwenn 3.5 Flash only has 11.3% PIC-skill ASR overall but 100.0% among runs with a successful write (as also shown in section 4.3). Hence, I urge the authors to consider adding per-model write rates and per-model ASR based on successful writes to truly support the model rankings and scaling conclusions.

  • The paper is quite dense and hard to follow at times, with some of the terminology being inconsistent. For example, some of the same concepts appear under different names like “persistent-state” and “agent-state”, “session” and “interaction”. The text also heavily relies on abbreviations like LIP, PIE, PIC, etc., which makes the rest of the analyses quite hard to infer without constantly looking back. I would highly appreciate it if the authors work towards making the paper more readable and easier to follow.

Questions for authors
  • Can you report memory and skill results for all seven models with the trigger in a fresh context, so that only the stored content is visible?
  • Please report, for each model, the write rate and the ASR conditional on a successful write. Does the model ranking in Tables 2 and 3 hold under this measure? Do larger Qwen3 models also write to and retrieve from persistent state more often?
  • How do you assess the 65.6–84.1% write rates in Table 5 with the claim that the evaluated models resist single-interaction attacks?
7 comments
AuthorsOpenReview ↗

Response to Reviewer 2MAX — Part 1

R1. Fresh-session evaluation across seven models (W1 and Q1).

  • We thank the reviewer for the suggestion. We extended the fresh-session evaluation from two models to all seven models to examine whether content retained in Memory and Skill can still trigger attacks after the original conversation is cleared.

  • Table 3 compares ASR in the original execution and a fresh session on the same cases.

Table 3. ASR (%) in the original execution and a fresh session.

ModelOriginal ASRFresh-Session ASR
Qwen3.5-Plus52.648.5
Qwen3.5-Flash19.920.4
Gemini-3-Flash77.278.5
Gemini-3.1-Pro71.675.7
GPT-5.454.959.6
DeepSeek-R158.766.3
Llama-3.3-70B32.228.3
  • Attacks still succeed against all seven models after clearing the original conversation, and their relative ranking remains unchanged. The results are broadly stable, indicating that the fresh-session setting does not substantially change the conclusions of the original model comparison. Content saved in Memory or Skill can continue to trigger attacks in subsequent interactions.

R2. Direct ASR and write rate (W2).

  • We thank the reviewer for pointing out this wording issue.

  • We will clarify the distinction between the two results: a low Direct ASR indicates that directly inducing the model to achieve the harmful objective is difficult, while a high write rate indicates that the model may still accept unauthorized malicious writes to persistent state. Such writes create the conditions for achieving the final attack objective in subsequent interactions.

  • This result shows that directly asking the model to execute harmful instructions immediately is unlikely to succeed. However, when the process is divided into a planting stage and a trigger stage, inducing the model to complete the planting stage is easier, creating more opportunities to achieve the final attack objective.

  • We will revise the Introduction and the corresponding result analysis accordingly.

AuthorsOpenReview ↗

Response to Reviewer 2MAX — Part 2

R3. Per-model write rates, conditional ASR, and Qwen3 scaling (W3 and Q2).

  • We thank the reviewer for the suggestion. We additionally report each model’s benign write rate, write rate under attack, and conditional ASR after writing in Table 4.

Table 4. Benign write rates, write rates under attack, original ASR, and conditional ASR (%) across the main models.

ModelMemory: Benign Write RateMemory: Write Rate under AttackMemory: ASRMemory: Conditional ASRSkill: Benign Write RateSkill: Write Rate under AttackSkill: ASRSkill: Conditional ASR
Qwen3.5-Plus100.086.936.942.5100.084.647.055.6
Qwen3.5-Flash100.043.78.018.4100.023.85.523.0
Gemini-3-Flash100.095.872.876.092.796.071.774.7
Gemini-3.1-Pro100.095.445.447.6100.099.888.688.8
GPT-5.4100.063.120.532.4100.059.339.967.3
DeepSeek-R1100.078.131.440.3100.081.047.358.3
Llama-3.3-70B96.067.727.841.188.071.517.123.9
  • Most models can correctly use both types of write tools.

  • Models with higher original ASR retain higher conditional ASR when write rates are taken into account. Our evaluation checks whether writing succeeds, and cases with unsuccessful writes are not counted as attack successes. Therefore, models with high original ASR must also have high write rates to support that ASR. For example, Gemini-3.1-Pro and Gemini-3-Flash rank first and second, respectively, in both original and conditional ASR on Skill.

  • The rankings of some models do change under conditional ASR. On Skill, GPT-5.4 has lower original ASR than DeepSeek-R1 (39.9% versus 47.3%) but higher conditional ASR (67.3% versus 58.3%). One possible explanation is that GPT places greater trust in the saved state after writing, while inducing GPT to write harmful information is more difficult than inducing DeepSeek to do so. Together, these factors reverse the ranking under the two measures.

  • We particularly thank the reviewer for suggesting this additional analytical perspective. We also provide results for the open-source Qwen3 models in Table 5.

AuthorsOpenReview ↗

Response to Reviewer 2MAX — Part 2 (Continued)

Table 5. Benign write rates, write rates under attack, original ASR, and conditional ASR (%) for Qwen3.

ModelMemory: Benign Write RateMemory: Write Rate under AttackMemory: ASRMemory: Conditional ASRSkill: Benign Write RateSkill: Write Rate under AttackSkill: ASRSkill: Conditional ASR
Qwen3-4B100.034.411.633.796.723.22.510.9
Qwen3-8B100.054.212.723.2100.032.16.119.1
Qwen3-14B100.052.712.022.7100.059.313.322.4
Qwen3-32B100.060.125.742.2100.078.523.630.1
  • Smaller models can perform explicitly authorized benign writes. Qwen3-4B achieves first-interaction write rates of 100.0% on Memory and 96.7% on Skill, while Qwen3-8B reaches 100.0% on both states.

  • Larger model size generally accompanies higher write rates under attack. From Qwen3-4B to Qwen3-32B, the Skill write rate increases consistently from 23.2% to 78.5%. The Memory write rate increases overall from 34.4% to 60.1%, with only a small decline from 8B to 14B (54.2% to 52.7%).

  • Qwen3-32B has the highest write rate under attack, ASR, and conditional ASR on both states. Its conditional ASR reaches 42.2% on Memory and 30.1% on Skill, indicating that, among these models, the larger model is more readily induced to perform malicious writes and also exhibits the greatest security risk after writing.

  • The lower ASR of smaller models may partly reflect their limited ability to coordinate normal tasks with additional malicious instructions. In LIP and PIE in particular, the attack often requires the model to accept an additional write request while handling a normal task, then continue the attack process in subsequent interactions. Although smaller models can perform explicitly requested benign writes, they may ignore additional instructions or fail to complete subsequent steps coherently in these more complex instruction settings, resulting in lower overall ASR.

Reviewer_VvMyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Sleeper Attack, an attack pattern against LLM agents that focuses on multi-turn settings. More specifically, Sleeper Agent first plants malicious content into an agent’s persistent state (i.e., memory, skills, etc.); then, at a later turn, a benign trigger carries out the attacker’s intended task. The authors build a benchmark of 1,896 such instances across different threat vectors and harmful payloads, and evaluate on a variety of language models. They find that for the most part, single-turn robustness does not generalize to the multi-turn setting, and many existing models are vulnerable to persistent-state attacks.

Critical strengths
  1. This is a clear and timely problem. As agentic harnesses continue to incorporate more types of persistent state in increasingly autonomous environments, there is a need for a systematic evaluation of the associated attack surface.
  2. The Sleeper Agent benchmark seems to be well-designed, and I appreciate the details provided in Section 3.2. The authors explain how they extend the existing ToolEmu environment to account for the persistent-state threat vector, along with the three different types of attacks that they implement. Combined with the variety of payloads and scenarios considered, the benchmark is quite diverse in coverage.
  3. The experiments in Section 4 are thorough and demonstrate some neat results. The Persistent Information Corruption (PIC) attack seems to be the strongest out of the three; I was surprised to see that models that are robust in other settings (i.e., Qwen3.5-Flash) are completely vulnerable to PIC attacks. Ablations on the trigger interaction turn demonstrate that the attack is not brittle and has some staying power.
Critical weaknesses
  1. It is not clear to me whether Sleeper Attack is meant to be a genuinely novel attack, or a generalization of existing attack strategies. The authors mention in Section 2 that there have been a variety of attacks proposed against both agentic memory and agentic skills. I would recommend that the authors explain how exactly Sleeper Attack is different from existing schemes. On the other hand, if Sleeper Attack is meant to be a generalization, then I think this should be made more explicit and the authors should demonstrate how existing attack methods fit within the formulation of Section 3. The authors should also pivot to explaining the core contributions and novelty of their proposed benchmark.
    • Along the same lines, I didn’t have any problem following the mathematical notation in Section 3.1, but it does feel a little too vague/abstract at the moment. I think the presentation of the technique would benefit from the use of some concrete examples. For instance, the authors already give an example for Latent Instruction Planting (LIP) in the first paragraph of the Sleeper Attack sub-section; at a minimum, including some examples for the other two attack strategies would be helpful.
  2. I think the authors should better motivate how some of these attacks could occur in the wild. Session context attacks make sense; these essentially correspond to stale tool calls that are kept in context. But attacks involving skills seem harder to mount; most of the time, skills are bootstrapped / created by a trusted user who is interested in improving their own agentic workflows. It seems less likely that an agent will arbitrarily create a skill for itself after browsing the web or accessing some other untrusted environment. Could the authors better specify their threat model and assumptions here, and ideally provide an example of an end-to-end attack? I think this is especially important given that the authors find that skill-based attacks are (on average) the most potent type of attack.
  3. Would it be possible for the authors to explain how they picked the defenses to evaluate for Table 4? I appreciate the fact that the authors considered this in the first place, but the curated selection seems to be a bit ad-hoc; LlamaGuard for instance is several years old, and I imagine no longer close to state-of-the-art. I would encourage the authors to test their attacks against a couple more contemporary schemes, although I don’t believe an exhaustive evaluation is necessary.
  4. A smaller nitpick, but would the authors be able to evaluate their attacks against some newer models? I think it would be interesting to see how Sleeper Attack fares against models like GPT-5.6 Sol or Qwen3.8. Of course, I understand that there might be time and/or cost limitations. I do appreciate that the authors attempted to evaluate a wide variety of models in Table 2.
Questions for authors

Thank you for submitting your work to ICLR 2027. I found this paper to be interesting, and I think it addresses a timely problem. The benchmark proposed in this work seems to be well-designed and I like the diversity of attack techniques and threat vectors considered. The results in Section 4 are also thorough and include a variety of ablation studies. I am currently leaning towards acceptance, although I would appreciate it if the authors address some of my concerns above. I also have the following questions about the work in its current state:

  • The authors implement their benchmark within an LLM-emulated sandbox. Have the authors considered the security of the sandbox itself, and taken steps to ensure that an agent cannot escape the sandbox and potentially cause real harm? It might not be a realistic concern in this scenario, but given some of the recent sandbox escapes I think it might be worthwhile to be extra prudent.
  • I am interested in how the proposed benchmark compares to other existing benchmarks. For instance, CIMemories (https://arxiv.org/abs/2511.14937) considers information leakage within the context of memory-level attacks. Are there any threat vectors and/or attack strategies present within these existing benchmarks that might be useful to include in Sleeper Attack?
  • Small question: in Table 3, the authors consider the benign task success rate under attack. Is there any reason why the authors do not consider the utility of their agents when not under attack? I think it would be good for reference to know how an undefended agent does in the typical scenario.
6 comments
AuthorsOpenReview ↗

Response to Reviewer VvMy — Part 1

R1. Contribution, relationship to prior work, and PIE/PIC examples (W1).

  • We sincerely thank the reviewer for the valuable suggestions.

  • Sleeper Attack is intended as a generalized formulation of cross-interaction attacks.

  • Our core contributions and novelty lie in

    • We establish a unified formulation of cross-interaction attacks by explicitly separating attack planting from attack execution, bringing previously studied attack patterns into a common conceptual framework.

    • We design a paired evaluation benchmark in which each base case is evaluated across a direct single-interaction baseline and all three persistent-state settings, enabling controlled comparisons of vulnerabilities across agent states.

    • We further enrich the benchmark along two complementary dimensions: three representative strategies synthesized and extended from existing work, and six categories of real-world harm. These dimensions enable a more comprehensive assessment of the vulnerabilities across different persistent-state types.

  • Relationship to and differences from prior work. We will add a method-by-method mapping to this framework in Section 2, covering the injection entry point, the contaminated persistent state, and the subsequent trigger. For example, AgentPoison poisons memory or knowledge bases to influence subsequent retrieval and behavior; MINJA uses interactions with the agent to insert malicious records into memory and influence execution during later user queries; and SkillJect uses poisoned skill content to influence subsequent tool use. These methods share the mechanism of persistent content influencing later behavior, while differing in attacker privileges and injection entry points. Our unified evaluation framework further supports systematic comparison of risks across attack mechanisms and persistent-state types using common tasks and evaluation criteria.

  • Connecting the formalization to concrete scenarios. We will add the following examples after the strategy definitions in Section 3.1, following the presentation of the existing Latent Instruction Planting example.

    • Proactive Information Elicitation: An instruction planted in the first interaction directs the agent to request sensitive information it has not yet obtained, under the pretext of a verification procedure in a later task. When the user subsequently makes a normal request, the agent actively asks for this information and uses it for the attack.

    • Persistent Information Corruption: In the first interaction, the saved financial-report recipient address is replaced with the attacker’s address. When the user later asks the agent to send the report to the finance team, the agent uses the corrupted address and sends the report to an unauthorized recipient.

R2. Practical conditions for skill attacks (W2).

  • We thank the reviewer for suggesting that we clarify the practical conditions under which skill attacks occur.

  • We consider agents that support skill maintenance: skills are created and added by the user before the attack, and the agent can read and update the skills to record task experience, revise existing procedures, and reuse them in subsequent tasks. This setting has practical support. For example, Hermes Agent supports modifying existing skills through skill_manage, while EvoSkill automatically proposes and validates revisions to existing skills based on execution feedback.

  • In our threat model, the attacker controls only the tool-return content. During a normal task, the agent reads this content, mistakenly adopts its malicious instructions as a skill update, and uses its own permissions to write them into an existing skill. The attack is triggered when a later related request invokes that skill again. Thus, even if a skill was initially created by a trusted user, subsequent updates can introduce untrusted content.

    • For example, the agent already has a skill for preparing and sending reports. During a normal task, a manipulated tool response induces it to add an extra recipient to that skill. When the user later makes a normal request to send a report, the agent reuses the updated skill and sends the report to the unauthorized recipient.
AuthorsOpenReview ↗

Response to Reviewer VvMy — Part 2

R3. Choice of defenses (W3).

  • We thank the reviewer for suggesting that we clarify the choice of defenses. In fact, our evaluation already includes recent state-of-the-art defense models, each with a different focus:

  • We use Llama Guard 4-12B, a general-purpose LLM guard released in 2025.

  • We uses AgentDoG1.5-Qwen3.5-4B, an agent-focused guard released in 2026, for runtime checks of tool returns and state-write checks.

  • We will explicitly identify the model versions, check locations, and blocking mechanisms in the text and tables so that readers can better understand our defense settings.

R4. Supplementary results on newer models (W4).

  • We thank the reviewer for the suggestion. We additionally evaluated Qwen3.8-Flash and GPT-5.6-Sol, with the results reported in Table 1.

Table 1. ASR (%) for the additional models. LIP, PIE, and PIC aggregate over Session, Memory, and Skill.

ModelLIPPIEPICSleeper AverageDirect Average
Qwen3.8-Flash12.010.038.020.011.3
GPT-5.6-Sol0.70.016.75.80.0
  • The two models achieve overall Sleeper ASRs of 20.0% and 5.8%, compared with Direct ASRs of 11.3% and 0.0%, respectively.

  • PIC still exposes substantial residual risk. Qwen3.8-Flash and GPT-5.6-Sol achieve PIC ASRs of 38.0% and 16.7%, respectively, indicating that PIC remains a significant security threat even for newer models.

R5. Experimental safety and sandbox escape (Q1 and Ethics Concerns).

  • We thank the reviewer for their attention to experimental safety. We clarify the safety boundaries of our main experiments in terms of the task environment and the tool-calling workflow.

    • First, our main experiments use simulated tools and data and do not access real accounts, perform real transfers, or disclose real users’ private data. We follow ToolEmu’s tool-simulation design: the accounts, transactions, and user privacy data involved in the tasks are simulated, and no real financial or user databases are connected. For example, when the simulator returns “transfer successful,” this only means that the LLM simulator has generated a simulated result that allows the agent’s execution trajectory to proceed. It involves no real accounts or transfers and does not connect to real MCP services or financial APIs.

    • Second, all tool calls in our main experiments, including terminal commands, receive responses generated by an LLM simulator. Model-generated commands are not executed as command-line programs on the host, which also removes the conditions required for sandbox escape. Specifically, the agent generates tool names and arguments according to predefined tool-interface specifications. The backend sends them to the LLM simulator as text input; after the simulator generates a simulated result, the backend parses it and returns it to the agent. For example, when the agent issues a terminal-tool call containing an rm -rf deletion command, the command and its arguments are sent to the LLM simulator only as text input. The simulator generates a simulated execution result, which the backend then returns to the agent. The backend does not pass the command to the host’s terminal for execution, so no files or directories on the host are actually deleted. Memory and Skill updates are handled by backend code that parses requests through predefined interfaces, extracts the text, and updates state storage for later interactions to read. The updated content is not executed as program code.

  • Sandbox escape requires crossing an actual execution or access boundary, for example by exploiting a reachable service or misconfigured permissions. The escape described in OpenAI’s incident report involved models exploiting a vulnerability in an accessible package-proxy service to obtain Internet access. In our main experiments, the models cannot access a real package-proxy service.

  • We will make these safety boundaries clearer in the experimental setup and ethics discussion.

AuthorsOpenReview ↗

Response to Reviewer VvMy — Part 3

R6. Connection to CIMemories (Q2).

  • We thank the reviewer for recommending CIMemories and suggesting that we clarify the connection to related work.

  • Our benchmark design is rooted in a broad and comprehensive range of real-world harm outcomes. The privacy risks examined by CIMemories fall within the information-leakage categories already covered by our benchmark. Its evaluation focuses on whether individual information attributes in memory should be disclosed given the task and recipient. CIMemories could serve as a useful reference for further refining our evaluation with more fine-grained disclosure conditions and criteria. We will further clarify this point in the manuscript, and add a comparison with CIMemories in the related work section.

R7. Benign task success without attacks (Q3).

  • We thank the reviewer for raising this point. We additionally evaluated benign task success without attacks and compared it with BSR-under-attack, as shown in Table 2.

Table 2. Benign task success rate (%). All columns aggregate over Session, Memory, and Skill; Direct is excluded.

ModelNo-Attack BSRLIP BSR-under-attackPIE BSR-under-attackPIC BSR-under-attackOverall BSR-under-attack
Qwen3.5-Plus92.277.478.012.257.0
Qwen3.5-Flash80.975.374.122.458.2
Gemini-3-Flash85.686.885.61.159.3
Gemini-3.1-Pro91.879.476.33.654.4
GPT-5.490.971.860.932.055.5
DeepSeek-R191.172.267.716.953.2
Llama-3.3-70B67.362.146.315.842.1
  • Most models achieve high task-completion rates without attacks. Six of the seven models have an average benign success rate above 80%, and four exceed 90%.

  • Within the same model family, the stronger model generally performs better without attacks. For example, Qwen3.5-Plus outperforms Qwen3.5-Flash (92.2% versus 80.9%), and Gemini-3.1-Pro outperforms Gemini-3-Flash (91.8% versus 85.6%).

  • Mechanistically, PIC corrupts state information needed for the normal task, causing the agent to use incorrect arguments in subsequent operations; attack success therefore often coincides with benign-task failure. In contrast, the additional harmful behaviors induced by LIP and PIE can occur alongside normal task completion, allowing relatively high BSR to persist under attack. Overall, task-completion rates are lower under attack, while a substantial level of benign-task completion remains. The gaps range from 22.7 to 37.9 percentage points, with the largest gaps for DeepSeek-R1 and Gemini-3.1-Pro at 37.9 and 37.4 points, respectively.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.