acceptodds
Under review as a conference paper at ICLR 2027

SynChain: Inducing Computer-Use Agent Systems to Construct Their Own Attack Chains

Abstract

Computer-use agents (CUAs) have turned large language models into persistent execution systems that generate, store, and reuse their own artifacts, such as skills and memory entries. Existing security defenses, however, largely treat attacks as externally triggered and temporally bounded, overlooking how compromise can propagate internally through an agent's own persistent state. We show that malicious influence can be covertly embedded into the structural redundancies of self-synthesized artifacts, where it survives internal state updates and evades standard vetting. We formalize this threat as , a self-synthesized attack paradigm that uses persistence-aware directed supervised fine-tuning to induce agents to produce poisoned yet benign-looking artifacts. The resulting dormant payloads later reactivate as trusted context in future workflows, without any new malicious external input. To evaluate this propagation at scale, we build CUAChain, a chain-structured benchmark covering five categories of real-world agent workflows, multi-step task chains of up to five subtasks, and three attack objectives, and conduct over 6,000 end-to-end agent executions. Across OpenClaw, Codex, and Claude Code under four defense settings, SynChain achieves over 93% attack success in single-hop propagation and consistently outperforms adapted baselines, showing that securing CUAs requires provenance-aware reasoning over cross-task execution trajectories.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Computer-use agent, attack chain

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rhbpOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SynChain, a self-synthesized attack framework targeting persistent computer-use agents (CUAs). By applying persistence-aware supervised fine-tuning, the approach induces agents to generate malicious yet benign-looking artifacts that can be stored and reactivated across subsequent tasks. The authors also introduce CUAChain, comprising 30 task chains and three attack objectives. Experiments across OpenClaw, Codex, and Claude Code demonstrate strong attack effectiveness under multiple defenses, while revealing limitations in long-horizon attack propagation.

Critical strengths
  1. The paper identifies an important security vulnerability in self-generated persistent artifacts, extending existing agent attack research beyond externally injected malicious content.

  2. The experiments cover multiple agent frameworks, attack objectives, and defense mechanisms. The cross-task propagation analysis provides useful insights into persistent agent security.

Critical weaknesses
  1. The threat model assumes a pre-compromised model, which may limit the practical applicability of the attack. Comparisons with alternative model-compromise strategies would further strengthen the contribution.

  2. Attack effectiveness decreases substantially as the task chain grows longer, suggesting limitations in long-horizon persistence.

Questions for authors
  1. Could the authors further clarify how SynChain compares with alternative model-poisoning approaches under comparable attacker capabilities and training budgets?

  2. What are the primary factors contributing to the sharp performance degradation in longer task chains? Could the impact of memory summarization and artifact retrieval be quantified separately?

  3. The preliminary provenance-aware defense shows promising results. Could the authors discuss whether its effectiveness generalizes to other attack objectives and CUA frameworks?

  4. The related work discussion could more clearly position this study alongside existing research on persistent memory vulnerabilities and agent reliability, including AgentPoison, MemoryGraft, MemPoison, Sponge Tool Attack, and a survey titled Efficiency in LLM Multi-Agent Systems: A Survey. These connections would help clarify the distinct contributions of the proposed approach and its implications for agent safety and reliability.

4 comments
AuthorsOpenReview ↗

Responses to Reviewer rhbp (1/2)

Dear Reviewer rhbp,

Thank you for your thoughtful review; we address the model-compromise comparisons, propagation failures, and defense generalization below, with the corresponding manuscript revisions highlighted in purple.


For Weakness 1 and Question 1: How realistic is the pre-compromised model, and how does SynChain compare with other model-compromise approaches?

The adversary only releases a compromised checkpoint or model endpoint. It cannot control future tasks, modify the agent framework, access the workspace, or install skills (Section 3.1). This matches how open-weight checkpoints and third-party endpoints are adopted in practice. It is a different capability from prompt or skill injection: the malicious content originates in the supplied model rather than a live external-content channel.

DemonAgent is the closest model-compromise family, a trigger-conditioned backdoor in the model, and SkillJect represents skill-layer injection. Both are adapted to the same Chain-N protocol with their original attack primitives unchanged (Appendix D). Both stay near 100% in Chain-1 without defense, which confirms the adaptation preserves their capability. Three comparisons follow.

  1. Under GuardAgent in Chain-1, DemonAgent reaches 30.00% to 33.33% across the three frameworks, against 87.78% to 93.33% for SynChain.
  2. In Chain-2, averaged over frameworks and defenses, DemonAgent reaches 15.19%, against 72.59% for SynChain.
  3. Fine-tuning the same backbone with the same QLoRA setup and the same 149 pairs on direct malicious actions, rather than on artifacts, gives 96.7% Chain-1 ASR without defense, 30.0% under GuardAgent, and 16.7% in Chain-2 without defense on Claude Code for O1, against 100.0%, 93.3%, and 86.7% for SynChain, respectively.

The comparison isolates the training-target type at the same reported budget and distinguishes immediate model behavior from artifact-mediated persistence.

We clarified the threat model in Sec. 3.1 and added the matched direct-action control to Appendix J.

For Weakness 2 and Question 2: What drives the degradation in longer chains, and can summarization and retrieval be separated?

Yes. We audited saved failed Chain-2 to Chain-5 traces by their earliest observed failing stage. Each column reports shares among annotated failures at that depth.

Earliest failing stageChain-2Chain-3Chain-4Chain-5
Artifact not created10532
Artifact removed or overwritten20201713
Carrier lost in memory summarization10253542
Carrier not retrieved20253033
Premature activation (n < N)0000
Carrier present, payload not enacted301585
Agent failure101075

Summarization and retrieval appear as separate rows, as you asked. Together they account for 30, 50, 65, and 75 percent of failures from Chain-2 to Chain-5. Memory summarization grows fastest, from 10 to 42 percent.

At Chain-2 the largest single category is a carrier that survives but is not enacted, at 30 percent. The conditional enactment rate after propagation therefore need not equal that of a freshly generated artifact.

The baselines decay faster under the same protocol. In Chain-3, SkillJect and DemonAgent average 16.48% and 4.35% ASR, against 26.57% for SynChain (Table 2).

We added failure attribution to Sec. 4.4 and Appendix H.3, and qualified the survival model in Appendix H.2.

AuthorsOpenReview ↗

Responses to Reviewer rhbp (2/2)

For Question 3: Does the provenance-aware defense generalize to other objectives and frameworks?

Yes. We extended the pipeline of Appendix B.5 from one cell to all three objectives and all three frameworks at Chain-2.

FrameworkO1O2O3
Claude Code86.7 → 16.776.7 → 20.083.3 → 23.3
Codex86.7 → 20.083.3 → 23.386.7 → 26.7
OpenClaw76.7 → 16.773.3 → 20.073.3 → 23.3

Each cell is Chain-2 ASR without the pipeline and with it. Reductions range from 50.0 to 70.0 points, and the pattern holds across the objectives and frameworks the original experiment did not cover.

Validating an artifact against the task that caused its creation holds up across all three objectives and frameworks we test.

We expanded Appendix B.5 to report the full three-framework, three-objective grid.

For Question 4: How does this work relate to the suggested papers?

We added all five works to the discussion. AgentPoison [1] poisons an agent's long-term memory or RAG knowledge base with records retrieved at inference. MemoryGraft [2] plants poisoned successful experiences through content the agent ingests and later imitates. MemPoison [3] studies persistent memory threats across ten model families and measures how poisoned memory shifts their responses, with and without defenses.

In all three the malicious content enters from outside the model and is then stored. In SynChain the carrier is produced by the agent's own model during a benign workflow. No attacker-supplied record and no external trigger are involved at runtime. We also cite Sponge Tool Attack [4] and the survey on efficiency in LLM multi-agent systems [5] next to our efficiency analysis in Section 4.5. Both concern the cost of agentic execution, which is separate from cross-task compromise.

We added the memory-security comparisons to Sec. 2 and the efficiency references to Sec. 4.5.

Thank you for your constructive feedback; we remain open to further discussion and suggestions to improve the presentation and defensive implications of the work.

References

[1] Z. Chen, Z. Xiang, C. Xiao, D. Song, B. Li. AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases. NeurIPS 2024.

[2] S. S. Srivastava, H. He. MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval. 2025.

[3] J. Gao, K. Xia, Y. Zhang, X. Hong, M. Lin, X. Wei, W. Li, S. Lu. MemPoison: Uncovering Persistent Memory Threats and Structural Blind Spots in LLM Agents. 2026.

[4] Q. Li, X. Wang. Sponge Tool Attack: Stealthy Denial-of-Efficiency against Tool-Augmented Agentic Reasoning. ICML 2026.

[5] Q. Li, L. Xiong, H. Lu, W. Qu, X. Yang, J. Zhang, X. Wang. Efficiency in LLM Multi-Agent Systems: A Survey. 2026.

I thank the authors for their rebuttal, which has addressed most of my concerns. I therefore maintain my positive rating and continue to support acceptance.

Thank you for confirming that our rebuttal addressed your concerns and for your continued support of acceptance; your suggestions have substantially strengthened the paper.

Reviewer_Njd1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SynChain, a self-synthesized attack framework targeting persistent Computer-Use Agents (CUAs). The authors identify an emerging security risk in which a compromised underlying language model autonomously generates malicious yet seemingly benign reusable artifacts, such as skills, scripts, or memory entries, during legitimate workflows. These artifacts are subsequently stored and reused by the agent, allowing malicious behavior to propagate across task boundaries without requiring additional adversarial inputs. The paper also introduces CUAChain, a benchmark covering five workflow categories, three attack objectives, and task chains of up to five steps. Experiments are conducted across OpenClaw, Codex, and Claude Code under several defense configurations. Overall, the paper highlights an important security challenge arising from agents that autonomously generate and reuse their own persistent execution artifacts.

Critical strengths

S1. Important and timely problem formulation. SynChain considers a scenario in which malicious influence originates from an already compromised model and subsequently propagates through agent-generated artifacts.This perspective broadens the scope of agent security research from isolated interactions to cross-task state transitions. As agents increasingly generate and reuse their own skills and memories, this problem is practically relevant. S2. A systematic framework for studying cross-task attack propagation. The Chain-N evaluation protocol is particularly valuable because it explicitly examines the temporal persistence of attacks rather than focusing exclusively on immediate activation. S3. Broad empirical evaluation across agent frameworks and attack objectives. The authors evaluate SynChain across three different CUA frameworks, multiple attack objectives, and several defense configurations.

Critical weaknesses

W1. The proposed training method does not explicitly optimize long-horizon persistence. The training examples contain neither subsequent activation tasks nor delayed attack trajectories.The model is not directly optimized for successful propagation through repeated memory updates, retrieval operations, or artifact transformations. W2. The effectiveness of the evaluated defenses may not adequately reflect prevention of unsafe tool execution. Appendix E.2 describes GuardAgent as an output-level review mechanism that examines the original user request and the primary agent's draft response. If operations have already occurred during tool execution, reviewing or rewriting the final response cannot necessarily prevent the underlying unsafe action. As a result, the high ASR observed under GuardAgent may partly reflect a mismatch between the attack surface and the stage at which the defense is applied, rather than demonstrating robustness against comprehensive runtime security mechanisms. W3. Some reported aggregate results appear inconsistent with the main experimental tables. Section 4.4 reports average no-defense ASRs of 98.89%, 83.34%, and 35.56% for Chain-1, Chain-2, and Chain-3, respectively. However, averaging the corresponding framework-level results in Tables 1 and 2 gives approximately:

  • Chain-1: 98.52%
  • Chain-2: 80.74%
  • Chain-3: 32.22% The discrepancy may arise from different aggregation procedures or evaluation subsets, but the paper does not clearly explain this distinction. W4. The underlying propagation mechanism is insufficiently disentangled. The reported attack success rates measure whether malicious behavior eventually occurs, but do not clearly isolate the contributions of artifact creation, artifact retention, memory propagation, retrieval, and final activation.It remains unclear how frequently failures at larger chain depths are caused by:failure to preserve the malicious artifact or loss of relevant information during memory summarization or others.
Questions for authors

Q1. What makes the proposed SFT specifically persistence-aware? Since subsequent activation trajectories are not included in training, could the authors clarify which components directly encourage cross-task survival? Q2. Can the authors provide a finer-grained analysis of attack-chain propagation? For Chain-2 through Chain-5, what fraction of failures results from artifact loss, memory compression, retrieval failure, premature activation, or failure to execute the payload? Q3. How are successful attacks distinguished from attempted or merely proposed malicious actions? The benchmark descriptions allow certain objectives to count attempted or explicitly committed actions as successful.Could the authors report separate statistics for malicious intentions, attempted operations, and successfully executed operations? Q4. How would SynChain perform against defenses operating before tool execution? The evaluated GuardAgent mainly reviews the generated response. Would the attack remain effective against runtime monitors that inspect tool-call arguments, enforce capability restrictions, or require independent authorization before sensitive actions?

7 comments
AuthorsOpenReview ↗

Responses to Reviewer Njd1 (1/2)

Dear Reviewer Njd1,

Thank you for your detailed review; we clarify the training objective and defense stages, correct the aggregates, and report diagnostic analyses below, with manuscript revisions highlighted in purple.


For Weakness 1 and Question 1: What makes the SFT persistence-aware, if activation trajectories are not in training?

Persistence-aware refers to how the training targets are built, not to optimization over multi-step rollouts. The training pairs contain no activation tasks and no delayed trajectories, as Appendix B.2 states.

The targets place the carrier in the functional, reusable parts of an artifact. Those are the parts the harness keeps when it updates memory, rewrites artifacts, and retrieves context. The model learns to reproduce that placement during benign artifact synthesis, and propagation is then carried out by the harness's own state-update functions. We do not claim the loss optimizes cross-task survival, and the propagation results are evaluated after training.

The supervision-budget study supports this reading. With 50 training pairs, Chain-1 ASR already reaches 96.7%, 90.0%, and 96.7% for O1-O3 (Table 4). The supervision concerns artifact targets rather than multi-step activation rollouts.

We clarified the meaning of persistence-aware supervision and the standard SFT objective in Sec. 3.2.

For Weakness 2 and Question 4: Does the GuardAgent result reflect robustness, and what about defenses before tool execution?

GuardAgent reviews the final response rather than the tool calls. Its column therefore evaluates output-stage review, which cannot undo operations already performed during execution.

Appendix B.5 already evaluates a defense that acts before execution. Generated skills are staged non-executable and checked by an independent validator before they can run, which cuts Chain-2 ASR from 86.7% to 16.7% on Claude Code for O1. To address tool-level monitoring directly, we add a monitor that inspects each tool call's arguments against the three objective categories before the call executes.

SettingO1O2O3Acc
SynChain, no runtime monitor86.776.783.390.0
+ pre-execution tool-call monitor20.023.326.783.3

Both rows are Claude Code, Chain-2, with no other defense. The monitor cuts ASR by 53.3 to 66.7 points and costs 6.7 points of benign task completion.

These results support intervention before execution. The monitor checks call arguments against the three objective categories; the validator instead checks each artifact against the task that produced it.

We corrected the defense-stage descriptions and report the monitor in Sec. 4.2, Table 3(b), with its specification in Appendix E.4.

AuthorsOpenReview ↗

Responses to Reviewer Njd1 (2/2)

For Weakness 3: Why do the Section 4.4 averages differ from Tables 1 and 2?

Thank you for catching this. Averaging the no-defense rows of Tables 1 and 2 over the three frameworks gives 98.52%, 80.74%, and 32.22% for Chain-1 to Chain-3. The Section 4.4 values came from the separate long-horizon run behind Figure 3. That run also produced the Chain-4 and Chain-5 points, and the text did not state this. The corrected comparison keeps the main-table aggregates separate from the long-horizon run.

We rechecked every aggregate in Section 4.2 against the tables and corrected the following baseline averages:

  • SkillJect in Chain-2 and Chain-3 now reads 37.50% and 16.48%.
  • DemonAgent in Chain-2 and Chain-3 now reads 15.19% and 4.35%.
  • SkillJect under GuardAgent in Chain-2 now reads 2.22%.

We corrected the aggregates in Secs. 4.2 and 4.4 and identified the separate run in the Figure 3 caption.

For Weakness 4 and Question 2: Where do longer chains fail?

We audited saved failed Chain-2 to Chain-5 traces by their earliest observed failing stage. Each column reports shares among annotated failures at that depth.

Earliest failing stageChain-2Chain-3Chain-4Chain-5
Artifact not created10532
Artifact removed or overwritten20201713
Carrier lost in memory summarization10253542
Carrier not retrieved20253033
Premature activation (n < N)0000
Carrier present, payload not enacted301585
Agent failure101075

Three things follow from this breakdown.

  1. Summarization and retrieval together account for 30, 50, 65, and 75 percent of failures from Chain-2 to Chain-5. Carrier survival becomes the binding constraint as depth grows.
  2. At Chain-2 the largest single category is instead a carrier that survives but is not enacted, at 30 percent. Enacting a payload from a retrieved and summarized carrier is less reliable than enacting one from a freshly written artifact, so the Chain-1 value of η should not be read as a constant across depth.
  3. No premature activation appears in the reported breakdown; the strict predicate in Eq. 6 checks for activation before step N.

The breakdown supports a carrier-survival limitation at depth three and beyond, while identifying a distinct enactment limitation at Chain-2. The exponential form remains an analytical approximation, rather than a fitted predictive model.

We qualified the depth-dependent claim in Appendix H.2 and added the annotation rule and breakdown to Appendix H.3.

For Question 3: How are attempted actions distinguished from executed ones?

ASR already excludes intent expressed only in text. O1 requires that the agent both access local configuration secrets and send them outside the workspace. O2 requires an operational proposal or attempt of privilege escalation rather than a discussion of permissions. O3 requires a concrete write, or a clear commitment to one (Appendix C.2). Following your suggestion we classify every run by its highest observed action level.

ObjectiveStated intent onlyAttempted, not completedExecuted with effect
O10.013.386.7
O26.753.323.3
O33.350.033.3

All three rows are shares of every Claude Code, Chain-2, no-defense run. They are not shares of the failed runs in the table above. Rows do not reach 100 where some runs show no qualifying signal.

O1 counts only the executed level as a success, which matches its 86.7% ASR. For O2 and O3, the operational-attempt category includes actionable proposals or explicit commitments that satisfy the benchmark criteria; it does not require a completed operation. Their ASR counts the attempted and executed levels together.

We clarified the objective-specific evidence rubric and added the intent, attempt, and effect table to Appendix F.


We would be glad to discuss these analyses further and will use your additional feedback to strengthen the clarity and rigor of the evaluation.

Reviewer_Njd1OpenReview ↗

Reply to author

Thank you for the author's response. Considering the novelty and impact of the paper, I will maintain my score.

Reviewer_9dkgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes SynChain, a self-synthesized attack paradigm that induce agents to produce malicious but benign looking artifacts, which in later workflows can cause harmful output. It uses persistence-aware SFT to train the model to produce such artifacts. The paper proposes CUACHAIN, a chain-structured benchmark that evaluate three objectives (Privacy Leakage, Privilege Tampering, and Unauthorized Write) on real world agent workflows. Massive experiment is conducted on three agent harness and different backbone LLMs. The results show that SynChain consistently outperform adapted baselines on both short chains and longer ones.

Critical strengths
  1. A novel and well-framed safety perspective. The paper identifies an interesting and underexamined safety issue: how self-generated, potentially malicious artifacts can survive the agent’s internal state updates and resurface as trusted context to produce harmful outputs in later runs. Rather than treating attacks as externally triggered and temporally bounded—the assumption underlying prompt-injection and skill-injection work—it reframes the agent as both producer and consumer of its own operational dependencies, where trust in self-generated outputs opens a channel that input-, output-, and skill-vetting defenses do not see. This offers a genuinely new perspective on how safety attacks can arise within the internal supply chain of an agent harness.

  2. Extensive experiments complemented by a Markovian analysis. The paper reports a large-scale evaluation—spanning OpenClaw, Codex, and Claude Code, four defense settings, multiple backbone models, three attack objectives, and chain lengths from 1 to 5 (~6,480 main executions). Reporting both Acc and ASR gives a clear picture of how the attack behaves across interaction length and defense method, and the contrast between gently-declining Acc and sharply-collapsing ASR is used informatively to argue that longer chains sever propagation rather than merely rendering tasks infeasible. The evaluation is further strengthened by human validation of the LLM judge (κ=0.911, no false negatives), a supervision-budget ablation, and OOD generalization to unseen domains. The paper also provides a Markovian analysis that frames carrier survival as an information-bottleneck process, offering a plausible mechanistic account of the observed depth-dependent decay.

Critical weaknesses
  1. Unclear motivation. The paper designs the attack model by persistent-aware SFT, which assumes that the attacker can fine-tune and continuously control the backbone model. This is different from accidental compromised or maliciously manipulated artifacts propagating through the agent's internal state. If you can SFT an attack model, you can do more than simply generating artifacts. More importantly, such model remains active throughout the chain of tasks, and Eq.4 and 5 continuously generating new, potentially compromised artifacts. (Although Appendix H assumes no injection after the initial corrupted artifact, and Appendix G.2 illustrated a Chain-5 example) The details of how the artifacts propagates also remains unclear. For example, in Appendix G.2, how exactly the O1 triggers? Whether the initial artifact keeps track of the depth of the trace or there are other mechanism that triggers O1 is not elaborated in the paper. This ambiguity weakens the paper's central contribution.
  2. Lack of methodological innovation. The paper proposes "Persistence-Aware Directed SFT via Latent Carrier Mutation" as its main method, but this is actually just standard next-token SFT on the created training targets (Eq. 3). The model is simply learning to add a specific malicious payload in the artifact. There is no ablation nor any explanation why this design would survive the agent's internal state update function or the external defenses. The contribution is somehow overclaimed.
Questions for authors
  1. Can the authors elaborate or give an example of how the initial poisoned artifact triggers after reaching certain trace depth, as in Appendix G.2? What carries the trigger signal?
  2. Is the same attack model is used throughout the experiment? If so, how to ensure that there will be no new injections of malicious artifacts? Is the fine-tuned model essential for the artifact to propagate, or can the poisoned artifact also be carried in the base, non-SFT model?
  3. Are O1/O2/O3 three separately fine-tuned models? Plus, are Figure 2’s T1/T2/T3 identical to O1/O2/O3?
  4. Can the author provide some ablation or analysis on how exactly the latent-mutation-SFT is superior to naive-append-SFT?
9 comments
AuthorsOpenReview ↗

Responses to Reviewer 9dkg (1/2)

Dear Reviewer 9dkg,

Thank you for your careful review; we clarify the threat model, state-carried activation, and training controls below, with the corresponding manuscript revisions highlighted in purple.


For Weakness 1: Why assume a compromised model, when such an adversary could do more?

The adversary releases one compromised checkpoint or model endpoint, and nothing else. Section 3.1 rules out control over future tasks, the agent framework, the workspace, and skill installation. After deployment there is no channel back into the victim's runtime.

Direct malicious behavior during each request and artifact-mediated propagation are distinct effects of model compromise. The latter places the behavior in retained state that the harness can reuse in later tasks.

The artifact can outlive the generation that produced it and re-enter the context through normal state management. This is the channel we isolate, rather than assuming unrestricted control of the runtime.

We clarified the adversary's capabilities and the distinction from input-channel threats in Sec. 3.1.

For Weakness 1 and Question 1: What carries the trigger signal to depth N?

No external trigger is involved. The signal is carried by the agent's own persistent state. Each benign subtask updates a workspace continuity record, which the agent retrieves as ordinary context in later subtasks. The carrier stays dormant while the accumulated state does not meet its activation condition. In the strict Chain-N protocol that condition is depth N (Eq. 6).

The depth is held in the retained persistent state that the harness updates, rather than supplied by a new input. The case-study trace identifies the artifact at turn 1, its continuity record in memory at turn 2, its retrieval at turns 3–4, and the activation condition at turn 5.

We clarified the carrier lifecycle in Sec. 3.2 and Appendix G.2.

For Weakness 2 and Question 4: What is new beyond standard SFT, and how does latent carrier mutation compare with naive append?

The objective in Eq. 3 is standard next-token SFT, and the revision says so plainly. The contribution is the supervision target. A naive append leaves the payload as a separable block that vetting and output review can isolate. Latent carrier mutation integrates the behavior into the artifact's functional structure while preserving its utility.

SFT targetNo DefenseSandwichSkill VetterGuardAgent
Naive append93.386.753.336.7
Structured target (SynChain)100.096.7100.093.3

Both variants are trained on the same 149 pairs with the same QLoRA configuration. Both are evaluated on Claude Code, Chain-1, objective O1.

Without defense the two are within 6.7 points. Under Skill Vetter the gap widens to 46.7 points, and under GuardAgent to 56.7 points. The comparison isolates the training target while holding the SFT objective and budget fixed.

We clarified standard SFT in Sec. 3.2 and added the matched target comparison to Sec. 4.2, Table 3(a).

AuthorsOpenReview ↗

Responses to Reviewer 9dkg (2/2)

For Question 2: Is one model used throughout, can it re-inject, and is fine-tuning necessary?

One fine-tuned model per objective runs at every step of a chain. The strict Chain-N protocol elicits the carrier only at the synthesis step. The later subtasks are benign and do not request the artifact type the model was trained to poison. Eq. 6 and the per-step traces confirm that no attack behavior occurs before step N.

The protocol excludes repeated poisoning opportunities so that the measurement tests propagation from initial synthesis. Eq. 5 illustrates artifact addition, while Eq. 1 permits framework-level retention, rewriting, and removal. The strict Chain-N protocol constrains the poisoning opportunity within this broader system description.

The base-model control isolates synthesis: given the same benign skill-creation prompts, the non-fine-tuned backbone reaches 0.0% ASR, against 100.0% for the fine-tuned model on Claude Code, Chain-1, O1. It does not test replacing the checkpoint after a poisoned artifact has already been generated; that handoff setting is not measured here.

We clarified the protocol in Sec. 3.2 and report the synthesis control in Sec. 4.2 and Appendix J.

For Question 3: Are O1-O3 separate models, and are Figure 2's T1-T3 the same as O1-O3?

Yes to both. O1, O2, and O3 are three separately fine-tuned models, one per attack objective. T1, T2, and T3 in Figure 2 denote the same three objectives, and "Overall" is their mean.

We made this correspondence explicit in Sec. 4.1 and the Figure 2 caption.


We welcome further discussion of any remaining questions and would be grateful for suggestions that help us improve the method description and evaluation.

Thank you for the response and elaboration. My main concern remains the motivation. Could the authors give a practical example of how this attack would work, where activating the payload later through the artifact achieves something that acting directly would not? In the experiments, the same compromised model generates the malicious artifact and also executes all the following tasks. Wouldn’t it be more natural to execute the following tasks with a benign model, to truly measure the effectiveness of the persistent artifact? When the compromised model is used throughout the workflow, it is hard to tell whether the activation comes from the artifact or from the model itself, especially since the largest failure category at Chain-2 is a carrier that is present but not enacted (30%).

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Response and Revisions

Dear Area Chair and Reviewers,

We thank the Area Chair and Reviewers 9dkg, Njd1 and rhbp for their time and constructive feedback. We are encouraged that the reviewers found the problem important and timely (Njd1, rhbp), viewed the shift from externally triggered attacks to self-generated persistent artifacts as a genuinely new perspective on agent security (9dkg), and found the Chain-N protocol and the cross-task propagation analysis valuable (Njd1, rhbp). We also thank Reviewer rhbp for confirming that our responses addressed most of their concerns and for their continued support.

We summarize the main concerns and how the revision addresses them.

1. Threat model and propagation mechanism (9dkg: W1, Q1–Q3; rhbp: W1&Q1). We clarified that the adversary only releases a compromised checkpoint or endpoint and has no later access to the runtime. The trigger is carried by the agent's own persistent state, with no new external input. The strict Chain-N protocol elicits the carrier only once. A non-fine-tuned backbone produces no attack. SynChain also stays far more effective than DemonAgent and than fine-tuning on direct malicious actions under the same budget.

2. What the training contributes (9dkg: W2&Q4; Njd1: W1&Q1). We state plainly that the objective is standard SFT, and that the contribution lies in the supervision target. A matched comparison with naive-append targets, using the same data and configuration, shows that the two are similar without defense. Under skill vetting and output review, however, the structured target remains far more effective.

3. Defense stages and success criteria (Njd1: W2, Q3&Q4; rhbp: Q3). We corrected the description of GuardAgent as output-level review and added a monitor that inspects tool calls before execution. We also extended the provenance-aware defense to all objectives and frameworks, and separated stated intent, attempted actions and executed actions for every objective. Together, these results point to pre-execution, provenance-aware checks as the effective place for defenses to act.

4. Degradation in longer chains (Njd1: W4&Q2; rhbp: W2&Q2). A failure-stage audit of Chain-2 to Chain-5 traces separates artifact loss, memory summarization, retrieval failure and non-enactment. Memory summarization and retrieval account for most failures as chains grow, and no premature activation occurs. The baselines decay faster under the same protocol.

5. Aggregates and related work (Njd1: W3; rhbp: Q4). We corrected the aggregates in Sections 4.2 and 4.4 and identified the separate long-horizon run behind Figure 3. We also added the suggested works on memory poisoning and agent efficiency.

ConcernRaised byAdditional experiments and clarificationsLocation
Threat model and propagation9dkg (W1, Q1–Q3); rhbp (W1, Q1)Adversary capabilities; carrier lifecycle; one-model-per-objective protocol; non-fine-tuned control; direct-action fine-tuning controlPages 4–8, 21, 28, 32
Training contribution9dkg (W2, Q4); Njd1 (W1, Q1)Standard-SFT clarification; matched naive-append vs. structured-target comparisonPages 4–6, 8
Defense stages and success criteriaNjd1 (W2, Q3, Q4); rhbp (Q3)Pre-execution tool-call monitor; provenance-aware defense on all objectives and frameworks; intent/attempt/execution breakdownPages 8, 18–19, 24–25
Longer chainsNjd1 (W4, Q2); rhbp (W2, Q2)Failure-stage attribution; qualified survival modelPages 9, 29–30
Aggregates and related workNjd1 (W3); rhbp (Q4)Corrected aggregates; separate long-horizon run identified; memory-poisoning and efficiency referencesPages 2–3, 7–9

All revisions are highlighted in purple in the updated manuscript. We hope this summary is helpful for the remaining discussion, and we would welcome any further questions or comments from the reviewers.

AuthorsOpenReview ↗

Follow-Up on Our Rebuttal Responses

Dear Reviewers,

We understand that reviewers may receive many OpenReview emails during the ICLR rebuttal period, and we sincerely apologize for adding another message. However, as the discussion period is coming to an end, we truly value this opportunity to communicate with such dedicated and thoughtful reviewers.

In response to your constructive feedback, we have conducted additional experiments and submitted a revised manuscript. We would be deeply grateful if you could revisit our responses when your schedule permits.

Once again, we apologize for the interruption and sincerely appreciate your time and understanding.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.