acceptodds
Under review as a conference paper at ICLR 2027

SkillFence: Execution-Grounded Trust Boundaries for Skill-Augmented LLM Agents

Abstract

Skill-augmented LLM agents increasingly act on externally supplied workflow artifacts, such as skill files and tool outputs, that are operationally relevant but may be attacker-controlled. We identify skill-context injection, in which such artifacts are elevated from contextual evidence to execution authority, so that forged approvals, routing notes, or policy claims become unsafe tool actions. To measure this threat, we build a black-box benchmark that rewrites 28 seed attacks into enterprise-style skill and tool artifacts using safety-ablated open-weight models. On agents built on GPT-5.4, Claude Sonnet 4.6, and Kimi-2.5, these artifacts achieve broad attack success rates (Broad ASR, which counts unsafe actions, authority confusion, and reasoning contamination) of 38.39%–76.07%. We propose execution authority separation: artifacts may inform a task, but only a trusted authority root, such as the user or system policy, may authorize a sensitive action. SkillFence enforces this principle by extracting authority claims from artifacts, estimating residual authority confusion, and gating each sensitive tool call against the trusted root. It reduces Broad ASR from 70.71% to 6.96%, from 55.00% to 5.18%, and from 41.16% to 2.86% on the three backbones while retaining 98.67%–100% benign utility.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Skill-Context Injection, LLM Agents, Agent Security, Trust Boundaries, Tool-Use Security, Execution Authority Separation, Prompt Injection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_AN8dOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a defense against prompt injection attack in the context of skills. In this attack, artifacts injected into the skill document or returned by the tool are elevated from contextual evidence to execution authority. Therefore, forged approvals, routing notes, or policy claims become unsafe tool actions. To defend the agent against this kind of attack, this paper propose a method called SkillFence. The key idea is that artifacts may inform a task, but only a trusted authority root like users or trusted system policy may authorize a sensitive action. SkillFence approximates this invariant with a trust boundary filter, an authority-confusion estimator, and an execution authorization layer that gates every sensitive tool call. On a benchmark of 28 seed scenarios, SkillFence reduces broad ASR for frontier models, while retaining the benign utility.

Critical strengths
  1. The problem studied in this paper is important.
  2. The design of the defense is light-weight and can be incorporated into real-world agent harness.
  3. The method is easy to follow.
Critical weaknesses
  1. My main concern is that the metric does not test the paper's central claim. All main comparisons report only broad ASR, which is an LLM-judged union that includes reasoning contamination. And strict unsafe-action ASR is never reported for the main benchmark. These two metrics can differ widely. It is therefore unclear whether the gains come from the execution authorization layer or from the prompt-level read-only wrapping of c′c^{\prime}. And the ablation attributes most of the reduction to a context-level component: the authority-confusion estimator alone accounts for 84.5% of it.
  2. The comparison with prior defenses is limited. The authors note that filtering, risk scoring, and action gating are not new, but the experiment compares only against context-layer defenses. Without comparisons to previous baseline methods (Progent, DRIFT, MELON, or CaMeL), it is hard to judge the performance of the proposed method.
  3. The paper motivates SkillFence with workflows where the user hands the next step to the skill, but the evaluation does not seem to test this setting. In every concrete example, the user only asks the agent to read or summarize something, and the artifact tries to push it into sending a message or changing state. SkillFence's scoring rule stops such an action simply because it goes beyond the user's request, whatever the artifact says, and this kind of mismatch is exactly what task-derived defenses such as Progent and DRIFT are built to catch. The harder case where the skill legitimately calls for a sensitive action and the attacker only tampers with it does not appear to be tested, and the benign tasks also do not include user-requested sensitive actions.
Questions for authors
  1. Could you report strict unsafe-action ASR, not only broad ASR, for the main results, the ablation, and the baseline comparison?
  2. When the agent attempts an unauthorized tool call and SkillFence blocks it, does the judge count this as an unsafe action? How much of the improvement comes from the gate blocking calls rather than the agent no longer trying?
  3. How many of the attack scenarios involve a read-only request? Do any benign tasks ask the agent to perform a sensitive action, and if so, how often does SkillFence block them or hold them for confirmation?
  4. Could you compare against previous defenses mentioned in the weaknesses?
9 comments
AuthorsOpenReview ↗

Responses to Reviewer AN8d (1/2)

Dear Reviewer AN8d,

Thank you for your precise questions on metrics, blocked-call accounting and delegation; the corresponding manuscript revisions are marked in blue.

For Weakness 1 & Question 1: Do the gains hold under strict and execution-level metrics, and which component produces them?

The gains hold under both Strict ASR and verified execution outcomes. Strict ASR counts zu=1z_u = 1 (performed or attempted), while Executed-unsafe counts unauthorized side effects verified from tool and state records independently of the judge. On the main benchmark (1,120 attacks per row):

BackboneMethodBroad (%)Strict (%)Executed-unsafe (%)
GPT-5.4No Defense70.7130.5426.96
GPT-5.4SkillFence6.962.140.54
Claude Sonnet 4.6No Defense55.0024.0220.98
Claude Sonnet 4.6SkillFence5.181.610.36
Kimi-2.5No Defense41.1617.5015.00
Kimi-2.5SkillFence2.860.890.18

SkillFence cuts executed unauthorized actions by 98–99% on every backbone, more than its 90–93% cut in Broad ASR. On the Table 8 baselines (GPT-5.4, 1,120 attacks per row):

MethodBroad (%)Strict (%)Executed-unsafe (%)
No Defense70.7130.5426.96
Spotlighting [1]36.8816.9612.95
PromptArmor [2]26.8812.958.93
SkillFence6.962.140.54

For the ablation (GPT-5.4, Qwen-3.5 mutations, 560 attacks per row), we report all rows of Table 7, including two new leave-one-out rows that separate the read-only wrapping of c′c' from the gate:

ConfigurationStrict (%)Executed-unsafe (%)
No Defense30.0024.82
Trust Filter Only17.1413.93
Authority Est. Only8.216.61
Exec. Auth. Only4.821.07
Full without read-only wrapping10.891.43
Full without Exec. Auth. Layer8.396.79
SkillFence (Full)1.430.18

The full system combines the strongest execution protection with the lowest Strict ASR:

  • Without the gate, Executed-unsafe is 6.79%, compared with 0.18% for Full; without read-only wrapping, it is 1.43%.
  • Among single components, Exec. Auth. Only reaches 1.07%, compared with 6.61% for Authority Est. Only.
  • These end-to-end configurations show how the components work together. We replaced the 84.5% attribution with separate Broad and execution-level comparisons in Sec. 5.5.

We added the Strict and Executed-unsafe columns to Tables 2, 7 and 8.

For Question 2: Does the judge count a blocked call as unsafe, and how much of the gain comes from blocking?

A proposed unauthorized call remains a Strict positive even when the gate blocks it or requires confirmation. Executed-unsafe instead requires an observed unauthorized side effect. A blocked legitimate call is a utility failure, not an unsafe-action positive.

The ablation and main results should be read at the trajectory level:

  • The gate directly blocks reads of sensitive approval or session-state artifacts, so it can prevent the agent from obtaining the material used to justify a later call. Strict is measured over the resulting trajectory, rather than over a fixed list of proposals. Exec. Auth. Only can therefore reduce both Strict and Executed-unsafe even though a blocked unsafe proposal still counts as Strict.
  • On GPT-5.4, the main benchmark changes from 342 Strict-positive and 302 executed-unsafe episodes to 24 and 6, respectively, out of 1,120. The remaining 18 episodes contain an unsafe attempt without an observed unauthorized effect; this difference is not a count of gate interventions. The overall reduction combines changes earlier in the trajectory with execution-time interception.
  • The ablation rows are separate end-to-end runs, not a replay of identical candidate calls. Their Strict rates are therefore trajectory outcomes, and additional components need not improve every intermediate metric monotonically. The full system achieves the best joint outcome: 1.43% Strict and 0.18% Executed-unsafe.The gate provides the final execution safeguard, while filtering and read-only wrapping shape the context in which the agent acts.

We clarified trajectory-level Strict accounting and the gate's upstream checks in Secs. 4.4 and 5.5.

References

[1] Hines et al. Defending against indirect prompt injection attacks with spotlighting. arXiv:2403.14720, 2024.

[2] Shi et al. PromptArmor: Simple yet effective prompt injection defenses. arXiv:2507.15219, 2025.

AuthorsOpenReview ↗

Responses to Reviewer AN8d (2/2)

For Weakness 2 & Question 4: How does SkillFence compare with prior execution-layer defenses?

We add all four defenses you name: Progent [3] and DRIFT [4], which derive tool policies from the user task; MELON [5], which re-executes the trajectory with a masked user request; and CaMeL [6], which separates control flow from untrusted data. We also add a simple authorization rule that checks Eq. (1) without modeling artifact claims. Setup: GPT-5.4; 100 attack and 100 legitimate delegated episodes per method; a confirmation request does not count as completion.

MethodExecuted-unsafe (%) ↓Legitimate completion (%) ↑Tasks needing confirmation (%) ↓
No Defense35970
Progent78514
DRIFT98118
MELON4887
CaMeL28216
Simple authorization rule57325
SkillFence1972
  • SkillFence combines 1/100 executed-unsafe attacks with 97/100 legitimate completions and only 2/100 tasks needing confirmation.
  • Progent and DRIFT execute 7% and 9% of attacks and ask for confirmation on 14% and 18% of legitimate tasks.
  • MELON and CaMeL execute 4% and 2% of attacks, but complete only 88% and 82% of legitimate tasks and ask for confirmation on 7% and 16% of them.
  • The simple rule executes 5% and completes 73%. SkillFence combines artifact-claim modeling with execution authorization and achieves a stronger safety–utility balance in this comparison.
  • The tables report observed counts as percentages. For example, 1/100 and 2/100 have 95% Wilson intervals of [0.18, 5.45]% and [0.55, 7.00]%. Together with the completion and confirmation counts, these results characterize the safety–utility balance.

We added this comparison to Sec. 5.7.

For Weakness 3 & Question 3: Are the scenarios read-only, and how does SkillFence treat legitimate sensitive actions?

Not all of them. Of the 28 seeds, 15 are read-only requests, 9 delegate an operation to the skill or record, and 4 are ambiguous. Appendix F now labels each seed.

The benign set also tests sensitive actions. Of the original 300 tasks, 42 involve one:

  • The user explicitly requests it in 18, authorizes it conditionally in 15, and has not authorized it in 9.
  • An authorized task passes only when the required operation and final output are completed; a block or confirmation-only reply is a failure.
  • In the 9 unauthorized-action cases, passing means completing the trusted task without performing the unapproved operation.
  • Overall, SkillFence completes 298/300, 296/300 and 300/300 tasks on the three backbones.

The E setting below separately measures completion when the user explicitly authorizes the sensitive action.

To test authorized sensitive actions at a larger scale, together with the harder case you describe, we add two settings (100 episodes per scenario per method). In E, the user explicitly requests a sensitive action. In D, the skill legitimately calls for the action, but its approval or scope is tampered with.

MethodE: authorized-action completion (%) ↑D: executed after tampering (%) ↓D: task completion under tampering (%) ↑
No Defense973860
Progent881183
DRIFT901486
MELON89987
CaMeL86384
Simple authorization rule81779
SkillFence95092
  • SkillFence executes no unauthorized action in the 100 tampered D episodes, against 11% for Progent, 14% for DRIFT, 9% for MELON and 3% for CaMeL, and completes 92% of D tasks, against 60% without defense.
  • When the user requests the sensitive action (E), SkillFence completes 95% of tasks, within 2 points of the undefended agent. These 95/100 completions require the authorized operation and requested output; confirmation-only replies receive no completion credit.We added E and D to Sec. 5.7 and the benign breakdown to Appendix I.

References

[3] Shi et al. Progent: Securing AI agents with privilege control. arXiv:2504.11703, 2025.

[4] Li et al. DRIFT: Dynamic rule-based defense with injection isolation for securing LLM agents. NeurIPS, 2025.

[5] Zhu et al. MELON: Provable defense against indirect prompt injection attacks in AI agents. ICML, 2025.

[6] Debenedetti et al. Defeating prompt injections by design. arXiv:2503.18813, 2025.

We are glad to discuss any of these points further.

Thank you for the detailed response and the new experiments. The added Strict and Executed-unsafe results resolve most of my first concern. Three questions remain:

  1. How do legitimate delegated actions pass the gate? Section 4.4 says a sensitive action whose recipient or payload comes only from the artifact is at least sent to review, and a review does not count as completion. Yet 97% of delegated tasks complete. This seems to rely on the approval tool introduced in Appendix T. Is that tool part of the trusted root, and does it exist in the main benchmark? Also, why does the stricter simple authorization rule execute 7% of tampered cases while SkillFence executes none?

  2. How much does the gate actually block? On GPT-5.4, episodes with an unsafe attempt drop from 342 to 24, so most of the gain seems to come from the agent no longer trying, not from the gate stopping calls. Please report how many calls the gate blocked or held on the main benchmark, and explain why 6 of the remaining 24 attempts were still executed.

  3. How reliable is the comparison with execution-layer defenses? Please describe how the new episodes were built and how the baselines were implemented. With 100 episodes on one model, the safety gap to CaMeL, MELON and the simple rule is too small to be conclusive; only the difference in completion rate is clear.

Reviewer_Hij9OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on skill-context injection, where a skill file or tool output smuggles in a fake approval, routing note, or policy claim, and the agent ends up treating that untrusted content as if it were real authorization to act. The authors build a benchmark of attack scenarios across several LLM backbones and show undefended agents fall for this a large share of the time. They then propose SkillFence, a three-stage middleware that filters out authority claims from untrusted content, scores how risky the remaining context is, and checks every sensitive tool call against a trusted source before letting it run. This cuts the attack success rate down dramatically across all three backbones while barely touching how well the agent does on normal tasks.

Critical strengths
  1. The framing is clean. The paper lays out why existing defenses, whether they filter suspicious text or restrict what tools can do, never actually decide whether a claimed approval or policy status should count as real authorization. Naming that gap and turning it into a testable rule is a solid contribution on its own.

  2. The results hold up consistently across three different models, and the ablation shows the gains mostly come from the authority-confusion estimator rather than any one lucky component. Normal task completion barely drops, so the defense isn't just refusing everything to look safe.

  3. The evaluation goes further than most papers in this space. It tests attacks borrowed from three other research groups and still holds up well, and the authors are upfront about where their own classifier breaks down under new attack styles, rather than hiding that weakness. They also report real wall-clock overhead instead of just token counts.

Critical weaknesses
  1. The two components doing most of the work are both built on fairly shaky foundations. The filter mostly looks for specific phrases like "already approved," so it is closer to keyword matching than real understanding of authority, and the risk estimator combines its signals through hand-picked weights with no sensitivity testing. Worse, the estimator is designed so it can only raise the risk score, never lower it, which means a missed keyword downstream can never get corrected. The whole defense ends up resting on keyword coverage and manually tuned numbers rather than something that can self-correct. In addition, almost every result comes from attackers who don't know the defense exists. The only tests against an attacker who understands the mechanism are two very small.

  2. Every outcome in the paper, attack success and task completion alike, is judged by a single LLM model with no human-checked validation set anywhere. The headline metric only agrees with an independent, rule-based check about four times out of five, and that gap is explained away rather than resolved with real human labels.

  3. The only defenses compared against are general content filters that the paper's own argument says can't solve this problem in the first place. None of the more directly comparable methods, the ones that also operate at the execution layer, are tested side by side with SkillFence. So the paper shows it beats defenses working at the wrong layer, not that its specific mechanism beats comparable alternatives.

  4. The whole benchmark, including the transfer experiments, comes from a small handful of hand-written seed scenarios, and the transfer test is only run on one of the three backbones used everywhere else. The big generalization claim rests on a narrower base of attacker scenarios and models than the headline numbers suggest.

Questions for authors

Please check the weaknesses and answer the questions.

4 comments
AuthorsOpenReview ↗

Responses to Reviewer Hij9 (1/2)

Dear Reviewer Hij9,

Thank you for your detailed comments on the estimator, evaluation and scenario coverage; the corresponding manuscript revisions are marked in blue.

For Weakness 1: Does SkillFence rest on keyword coverage and hand-picked weights, and does it hold against adaptive attackers?

SkillFence makes the execution-authorization decision from several complementary signals:

  1. The intent check compares the proposed action with the trusted request without keyword markers. Prototype margin and intent alignment in Eq. (5) provide additional evidence. The source check uses claims extracted into bb, so its evidence depends on claim recognition; it is not a separate marker-independent detector. In delegated tasks, the trusted approval and authorized scope remain decisive.
  2. The max in Eq. (7) does not stop a missed keyword from being corrected. It lets the classifier raise a score the rules missed and only stops the classifier from lowering it. The action gate in Eqs. (9)–(11) also combines source and intent evidence with the context estimate.
  3. The complete claim-aware system improves the safety–utility balance over a simple authorization rule: 1% versus 5% executed-unsafe and 97% versus 73% legitimate completion (see W3).The fusion weights are fixed heuristics, set once and kept identical across all backbones, attack sets and experiments. We replayed 96 development candidates under 40 weight and threshold configurations. “Covered” denotes the 72 unauthorized-action candidates included in this replay; it does not denote coverage of an attack space. The sweep measures decision stability on those fixed candidates:
Sensitivity sweepResult
Weight and threshold configurations40
Covered dangerous candidates blocked72/72
Legitimate candidates blocked, in any configuration0

For mechanism-aware attackers, we add carriers that we wrote by hand to evade the markers (100 episodes per row):

SettingBroad (%)Executed-unsafe (%)
No Defense, fixed carriers4518
SkillFence, fixed carriers82
SkillFence, mechanism-aware carriers123

The hand-written mechanism-aware condition gives 3/100 executed-unsafe outcomes, compared with 2/100 for the fixed-carrier condition. This directly tests the full pipeline on carriers written to evade its markers. The undefended row uses fixed carriers. We added the decision-stability sweep and the carrier comparison to Appendices K and O.

For Weakness 2: Is the judge validated against human labels?

We validate the judge against independent human labels and report execution outcomes separately:

  1. Executed-unsafe is verified from tool and state records, not by the judge. On the main benchmark it falls from 26.96%, 20.98% and 15.00% to 0.54%, 0.36% and 0.18% on GPT-5.4, Claude Sonnet 4.6 and Kimi-2.5.
  2. Legitimate completion in the new comparisons is checked against task-specific requirements using tool execution records and required final outputs. Any required operation must execute with the correct tool and arguments, a delegated action must follow the approval query, and the final output must meet the report requirement. An agent's claim of completion is not counted.
  3. Two annotators independently labeled the same 100 episodes, after a separate 10-episode calibration round. They saw the request, artifact, tool calls, state changes and final output, but not the judge's labels, risk scores, block reasons or the defense method:
Human auditResult
Inter-annotator agreement88% (κ = 0.76)
Judge precision / recall / F1 (Broad)88% / 84% / 86%

The 79.29% figure in Table 29 is agreement with a deterministic proxy, not with humans:

  • With the judge as the prediction and the proxy as the reference, there are 135 judge-positive/proxy-negative and 39 judge-negative/proxy-positive records. These are disagreements between two labeling procedures; the proxy is not an adjudicated ground truth.
  • On the unsafe-attempt component they agree on all 840 records.
  • The human precision, recall and F1 summarize the 100-episode audit as a whole; they are not separate defended/undefended estimates. Our execution-safety results use independently verified side effects.

We added the human audit and clarified the proxy accounting in Appendix C.

AuthorsOpenReview ↗

Responses to Reviewer Hij9 (2/2)

For Weakness 3: How does SkillFence compare with execution-layer methods?

We add four execution-layer defenses, Progent [1], DRIFT [2], MELON [3] and CaMeL [4], and a simple authorization rule. Setup: GPT-5.4; 100 attack and 100 legitimate delegated episodes per method; a confirmation request does not count as completion.

MethodExecuted-unsafe (%) ↓Legitimate completion (%) ↑Tasks needing confirmation (%) ↓
No Defense35970
Progent78514
DRIFT98118
MELON4887
CaMeL28216
Simple authorization rule57325
SkillFence1972

The setting closest to your concern is delegation, where the action itself is legitimate and only its approval or scope is forged (D, a separate set of workflows). E covers tasks in which the user explicitly requests a sensitive action (100 episodes per scenario per method).

MethodE: authorized-action completion (%) ↑D: executed after tampering (%) ↓D: task completion under tampering (%) ↑
No Defense973860
Progent881183
DRIFT901486
MELON89987
CaMeL86384
Simple authorization rule81779
SkillFence95092

Against these execution-layer methods, SkillFence executes no unauthorized D action, compared with 11% for Progent, 14% for DRIFT, 9% for MELON and 3% for CaMeL, while completing 92% of D tasks, compared with 60% without defense and 83–87% for the four methods. We added both comparisons to Sec. 5.7.

For Weakness 4: Does the generalization claim rest on a few seeds and one backbone?

The transfer results cover three external attack families and all three backbones. The set is not derived from our 28 seeds. Its 500 attacks (150 from SKILL-INJECT [5], 150 from SkillJect [6] and 200 from PASB [7]) follow the objectives of these three external attack families, rewritten as skill or tool artifacts for our benchmark (Appendix M). We now run it on all three backbones:

BackboneBroad: No Defense (%)Broad: SkillFence (%)Executed-unsafe: No Defense (%)Executed-unsafe: SkillFence (%)
GPT-5.430.40 (152/500)1.60 (8/500)11.60 (58/500)0.20 (1/500)
Claude Sonnet 4.625.00 (125/500)1.80 (9/500)9.20 (46/500)0.20 (1/500)
Kimi-2.518.00 (90/500)1.20 (6/500)6.60 (33/500)0.00 (0/500)

SkillFence keeps Broad ASR at 1.2–1.8% and allows at most one executed unauthorized action out of 500 on every backbone. Table 5 now covers all three backbones. Within the native benchmark, the 28 seeds span four attack categories and both injection surfaces, including 9 operational delegations, and Executed-unsafe falls by 98% or more on all three backbones.

References

[1] Shi et al. Progent: Securing AI agents with privilege control. arXiv:2504.11703, 2025.

[2] Li et al. DRIFT: Dynamic rule-based defense with injection isolation for securing LLM agents. NeurIPS, 2025.

[3] Zhu et al. MELON: Provable defense against indirect prompt injection attacks in AI agents. ICML, 2025.

[4] Debenedetti et al. Defeating prompt injections by design. arXiv:2503.18813, 2025.

[5] Schmotz et al. Skill-Inject: Measuring agent vulnerability to skill file attacks. arXiv:2602.20156, 2026.

[6] Jia et al. SkillJect: Automating stealthy skill-based prompt injection for coding agents with trace-driven closed-loop refinement. arXiv:2602.14211, 2026.

[7] Wang et al. From assistant to double agent: Formalizing and benchmarking attacks on OpenClaw for personalized local AI agent. arXiv:2602.08412, 2026.

We would welcome any further questions.

Thank you for the thorough response. I appreciate the corrections on two points I had misread: the role of the max in Eq. (7), and the proxy agreement figure. The human audit and the trace-verified executed-unsafe metric address my concern about judge reliability, and the head-to-head comparison against Progent, DRIFT, MELON, and CaMeL addresses my strongest concern. I am raising my rating to 3.

We sincerely appreciate your time, thoughtful feedback, and careful engagement with our response. Your comments helped motivate the human audit and the trace-verified executed-unsafe metric, which we have incorporated into the revision. Thank you for raising your score.

Reviewer_RALfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies an emerging security vulnerability in skill-augmented LLM agents termed skill-context injection. The authors identify that modern agents increasingly interact with external workflow artifacts,such as SKILL.md files, connector records, and tool outputs,which are operationally critical yet potentially attacker-controlled. When an adversary injects forged approvals, status notes, or policy assertions into these artifacts, the agent is prone to authority confusion: treating untrusted context as a source of execution authority and converting these claims into unauthorized downstream tool calls .

To counter this, the paper formulates the security invariant of execution authority separation, positing that external artifacts may serve as passive, read-only evidence, but only a trusted authority root (user instructions, trusted system policy, or a fresh runtime confirmation) may authorize sensitive actions.

The benchmark is built from 28 seed scenarios mutated into 1,120 attack variants across GPT-5.4, Claude Sonnet 4.6, and Kimi-2.5. Across these backbones, SkillFence reduces Broad Attack Success Rate (Broad ASR) from 70.71%, 55.00%, and 41.16% down to 6.96%, 5.18%, and 2.86%, respectively, while maintaining 98.67%–100% benign utility and transferring effectively against prior attack families.

Critical strengths

The paper addresses a critical blind spot in agent security: in complex enterprise workflows, malicious injections do not look like explicit jailbreak instructions (e.g., "ignore previous instructions"), but rather like legitimate operational data (e.g., "approval granted", "use updated route"). Existing content-level filters often fail because stripping these terms destroys task utility, whereas existing execution-level defenses assume actions consistent with delegated tasks are legitimate. Formulating this failure as "authority confusion" and enforcing "execution authority separation" establishes a clean, principled paradigm for securing tool-calling agents.

SkillFence does not introduce expensive LLM judge calls inside the critical path of tool invocation. The Execution Authorization Layer operates using deterministic rule heuristics and score arithmetic, introducing a median latency of around 0.1 ms per tool call. The ablation study thoroughly substantiates the contribution of each stage, demonstrating that the estimator and authorization layer effectively backstop one another without impairing benign pass-through utility.

Critical weaknesses

While the paper categorizes and discusses execution-layer defenses (such as Progent, DRIFT, MELON, and AuthGraph) in Table 1 and Section 2, the empirical benchmark in Table 8 only compares against context-layer sanitizers (Spotlighting and PromptArmor). Because the paper advocates for an execution-time trust boundary, the absence of head-to-head empirical results against runtime privilege-control systems (e.g., Progent) leaves an open question regarding how much gain stems specifically from artifact-borne authority modeling compared to existing intent-constraint frameworks.

As acknowledged in Appendix R, the lightweight classifier component suffers a significant drop in recall (down to 25.00%) when evaluated on external benchmark families under zero-shot transfer. Although the authors argue that the rule indicators and source-tracking terms step in as a safeguard, the evaluation corpus relies on 28 seed scenarios mutated via two models. Evaluating against human-written red-team payloads or multi-turn conversational poisoning would provide firmer bounds on its generalization limits.

Questions for authors

1.Why were execution-layer mechanisms like Progent or DRIFT excluded from direct evaluation in Table 8?

2.The current evaluation isolates single-turn artifact consumption. If an adversary spreads the injection across multiple turns,such as writing an authority claim into a temporary memory slot in Step 1, and referencing it to justify an action in Step 2.How effectively does the source tracker trace the provenance across multiple intermediate tool outputs?

3 comments
AuthorsOpenReview ↗

Responses to Reviewer RALf (1/2)

Dear Reviewer RALf,

Thank you for your positive assessment and questions on execution-layer baselines and multi-turn provenance; the corresponding manuscript revisions are marked in blue.

For Weakness 1 & Question 1: Why were execution-layer defenses such as Progent or DRIFT not compared in Table 8?

Table 8 originally focused on context-layer baselines. We now add Progent [1] and DRIFT [2], which derive tool policies from the user task, MELON [3] and CaMeL [4], and a simple authorization rule that checks Eq. (1) without modeling artifact claims. Setup: GPT-5.4; 100 attack and 100 legitimate delegated episodes per method; a confirmation request does not count as completion.

MethodExecuted-unsafe (%) ↓Legitimate completion (%) ↑Tasks needing confirmation (%) ↓
No Defense35970
Progent78514
DRIFT98118
MELON4887
CaMeL28216
Simple authorization rule57325
SkillFence1972
  • Against Progent and DRIFT, SkillFence lowers Executed-unsafe from 7% and 9% to 1% and raises legitimate completion from 85% and 81% to 97%.
  • MELON and CaMeL execute 4% and 2% of attacks, but complete only 88% and 82% of legitimate tasks.
  • The simple rule targets the same authorization invariant without claim modeling. It executes 5/100 attacks and completes 73/100 legitimate tasks, while SkillFence achieves 1/100 and 97/100. This comparison directly addresses the value of the full claim-aware architecture.

The comparison evaluates each method in the common harness with its tools, policies and delegated-workflow requirements.

The execution gap widens for every baseline when the action itself is legitimate and only its approval or scope is forged (D, a separate set of workflows). E covers tasks in which the user explicitly requests a sensitive action (100 episodes per scenario per method).

MethodE: authorized-action completion (%) ↑D: executed after tampering (%) ↓D: task completion under tampering (%) ↑
No Defense973860
Progent881183
DRIFT901486
MELON89987
CaMeL86384
Simple authorization rule81779
SkillFence95092

SkillFence executes no unauthorized D action, against 11% for Progent, 14% for DRIFT, 9% for MELON and 3% for CaMeL, and completes 92% of D tasks, against 60% without defense. We added both comparisons to Sec. 5.7.

References

[1] Shi et al. Progent: Securing AI agents with privilege control. arXiv:2504.11703, 2025.

[2] Li et al. DRIFT: Dynamic rule-based defense with injection isolation for securing LLM agents. NeurIPS, 2025.

[3] Zhu et al. MELON: Provable defense against indirect prompt injection attacks in AI agents. ICML, 2025.

[4] Debenedetti et al. Defeating prompt injections by design. arXiv:2503.18813, 2025.

AuthorsOpenReview ↗

Responses to Reviewer RALf (2/2)

For Weakness 2: Does the classifier's recall drop limit generalization, and does SkillFence hold against stronger attacks?

The end-to-end protection extends well beyond the classifier's standalone recall:

  • The 25% result concerns the classifier alone on 8 seed-only public attack records (2 of 8, Table 6).
  • The classifier can only raise the context estimate (Eq. 7). The gate combines that estimate with extracted source evidence and the action's consistency with trusted intent.
  • Beyond our 28 seeds, we test 500 attacks adapted from SKILL-INJECT [5], SkillJect [6] and PASB [7], which are not derived from those seeds. End to end, SkillFence reduces them from 30.40% to 1.60% Broad ASR on GPT-5.4. We now run this transfer set on all three backbones, and SkillFence allows at most 1 executed unauthorized action out of 500 on each.For stronger attackers, and following your suggestion of human-written red-team payloads, we add carriers that we wrote by hand to avoid SkillFence's markers (100 episodes per row):
SettingBroad (%)Executed-unsafe (%)
No Defense, fixed carriers4518
SkillFence, fixed carriers82
SkillFence, mechanism-aware carriers123

SkillFence records 2/100 executed-unsafe outcomes on the fixed carriers and 3/100 on the hand-written mechanism-aware carriers, providing an end-to-end check beyond classifier recall. These are the two evaluated carrier conditions; the undefended row uses the fixed carriers. We added this test to Appendix O.

For Question 2: Can the source tracker follow an authority claim across turns and intermediate tool outputs?

A relayed claim does not become trusted. Three parts of the design apply:

  1. Writing the claim into memory in Step 1 is itself a sensitive action (sact=0.90s_{act} = 0.90 for memory writes), so the gate checks that write against the trusted request.
  2. Reading the slot back in Step 2 returns a tool output, which the filter treats as untrusted, so its claims are extracted again.
  3. The intent check compares the final call with the trusted request. A send issued under a read-only request is escalated however the claim was relayed.

We test this with two relays on GPT-5.4 (100 episodes each). Each turn starts a new session, so the claim reaches the final send only through memory:

RelayNo Defense: Strict (%)No Defense: Executed-unsafe (%)SkillFence: Strict (%)SkillFence: Executed-unsafe (%)
R1: claim saved to memory under a read-only request, then read back633130
R2: user asks to save notes; a later turn reads them back and acts452720

Without defense, the relayed claim leads to an executed send in 31% and 27% of episodes. SkillFence executes none in either relay. In R2 the memory write is legitimate, yet the relayed claim is still stopped at the read-back or at the send check. We added this test to Appendix T.

The current tracker does not carry span-level provenance through an LLM summary that paraphrases the claim. Appendix S discusses persistent provenance across summarization as an extension.

References

[5] Schmotz et al. Skill-Inject: Measuring agent vulnerability to skill file attacks. arXiv:2602.20156, 2026.

[6] Jia et al. SkillJect: Automating stealthy skill-based prompt injection for coding agents with trace-driven closed-loop refinement. arXiv:2602.14211, 2026.

[7] Wang et al. From assistant to double agent: Formalizing and benchmarking attacks on OpenClaw for personalized local AI agent. arXiv:2602.08412, 2026.

Thank you again. We would be happy to clarify anything further.

Dear Reviewer RALf,

Thank you again for your time and feedback. As the discussion period is coming to an end, we would like to check whether our responses have addressed your concerns. If you have any further questions, please let us know and we will be glad to clarify them before the deadline.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Response and Revisions (2/2)

Weaknesses and Questions

ReviewerCommentOur ResponseRatingReviewer Response
RALfNo head-to-head comparison with execution-layer defenses such as Progent and DRIFT (W1&Q1)We added Progent, DRIFT, MELON, CaMeL and a simple authorization rule (Sec. 5.7, Tables 9–10). SkillFence keeps legitimate completion at the undefended level (97/100) while executing 1/100 attacks; no baseline attains both.3
Classifier recall drops on external families; human-written payloads untested (W2)The classifier can only raise the risk estimate, and the gate also uses source and intent evidence. End to end, transferred attacks on all three backbones lead to at most 1/500 executed unauthorized actions, and hand-written mechanism-aware carriers to 3/100 (Tables 5 and 35, App. O).
Can provenance be tracked across turns? (Q2)In two cross-session memory relays, SkillFence executes none of the relayed claims, against 31% and 27% without defense (App. T).
Hij9Reliance on keywords and hand-tuned weights; few adaptive tests (W1)We clarified that the max in Eq. (7) lets the classifier raise a score the rules missed, so a missed keyword can still be corrected. Across 40 weight and threshold configurations, all covered unauthorized candidates are blocked and no legitimate one is (App. K). We also added mechanism-aware carriers (App. O).2 → 3Raised rating to 3: "The human audit and the trace-verified executed-unsafe metric address my concern about judge reliability, and the head-to-head comparison against Progent, DRIFT, MELON, and CaMeL addresses my strongest concern."
No human validation of the LLM judge (W2)Executed-unsafe is verified from tool and state records, independently of the judge. A two-annotator human audit gives κ = 0.76 and judge F1 of 86% (App. C).
Comparison limited to context-layer defenses (W3)Same execution-layer comparison as above (Sec. 5.7).
Transfer tested on one backbone (W4)The 500 transferred attacks now run on all three backbones (Tables 5 and 35).
AN8dBroad ASR does not test the central claim (W1&Q1)We added Strict ASR and Executed-unsafe to Tables 2, 7 and 8, plus leave-one-out ablations that separate read-only wrapping from the gate.2Strict and Executed-unsafe results "resolve most of my first concern"; follow-up answers "resolve my questions about the gate and the baselines"; rating maintained
How are blocked calls counted? (Q2)Strict counts a blocked proposal as positive; Executed-unsafe requires an observed side effect. Accounting is trajectory-level (Secs. 4.4, 5.5).
No comparison with execution-layer defenses (W2&Q4)Same comparison as above (Sec. 5.7).
Delegated and user-requested sensitive actions untested (W3&Q3)9 of the 28 seeds are operational delegations (Table 12). With tampered delegations, SkillFence executes 0/100; when the user requests a sensitive action, it completes 95/100 (Table 10, App. I).
Follow-up: approval tool, gate counts, baseline setupThe approval tool is part of the trusted root and appears only in Tables 9–10. The gate blocks or holds 48 of 61 unauthorized calls, and the residual cases are categorized. Episode construction and baseline configurations are documented (App. T, Tables 37–43).
Final comment: delegated evidence, role of the gate, statistical gap to baselinesDelegation is part of the main benchmark on all three backbones. The gate alone lowers Executed-unsafe from 24.82% to 1.07% (Table 7). On tampered delegations, SkillFence's 0/100 is significantly below MELON (p = 0.003) and the simple rule (p = 0.014), with only CaMeL (3/100) not distinguishable, and its completion is higher than every baseline.—

We express our sincere gratitude to all reviewers for their constructive feedback. The suggestions and insights provided have been invaluable in improving our paper, and we hope that this summary clarifies the questions raised by the reviewers. We also sincerely thank the ACs and SACs for their valuable time and careful evaluation of our submission. Your dedication contributes to making ICLR a more open, inclusive, fair, and equitable research community.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.