acceptodds
Under review as a conference paper at ICLR 2027

Hijacking the Instruction Hierarchy in LLM Agents: From Semantic Injection to Persistent Compromise

Abstract

Major LLM providers have introduced instruction hierarchy training to strengthen model security overall against prompt injection. However, with the rise of LLM agents and skill-sharing ecosystems, third-party skill text has become a new attack surface: its benign metadata enters high-authority positions through the agent's own placement after installation, and the model's hierarchical instruction following transfers to the skill body along the metadata-body mapping, which attackers can exploit in agent settings to compete against the model's own safety alignment. We also find that mainstream LLMs readily detect explicitly malicious code but poorly recognize natural-language text carrying equivalent malicious intent. These two factors lead the agent to generate and execute malicious scripts aligned with attacker intent, and even to write malicious textual instructions into local memory files whose effects persist after the skill is removed. This elevates indirect prompt injection from transient behavior disruption to substantive and persistent system compromise. We propose HierJack and present a comprehensive study of how LLM agents expose the instruction hierarchy to hijacking. Systematic evaluation across 16 LLM-agent combinations yields an average one-shot attack success rate (ASR) of 88.46% and an average persistent ASR of 69.57%. Porting the payloads of InjecAgent and ChatInject to skill-based delivery vehicles raises their ASRs by 46.4 and 36.9 percentage points over the original external-data delivery. These results show that the LLM agent shifts the security boundary from model-side to agent-side privilege granting. Once an agent grants third-party content high authority, model compliance becomes an attack amplifier when it outcompetes the model's safety alignment on specific content. Safety alignment must therefore extend to both the optimization of privilege tiering and the recognition of semantically encoded malicious intent.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Instruction hierarchy training, Indirect prompt injection, LLM agent security, Memory poisoning, Safety alignment

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_vGwYOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces HierJack, an attack exploiting metadata‑body trust propagation in LLM agents with third‑party skill ecosystems. Under on‑demand skill loading, high‑authority skill metadata transfers trust to low‑priority skill body content without forging message roles. Attackers deploy semantic payloads with no executable malicious code and achieve persistent compromise via agent memory poisoning. Evaluations span four agents and four LLMs, including ablation, baseline comparison against prior IPI work, stealth audits and preliminary defense tests. The work demonstrates that instruction‑hierarchy training, a model‑side security defense, can become an attack amplifier given flawed agent context assembly.

Critical strengths
  1. Novel attack mechanism distinct from prior prompt‑injection work. Unlike role‑forging attacks e.g. ChatInject, HierJack leverages native agent registry mapping instead of tampering message roles. It uncovers a non‑trivial risk: model‑side instruction‑hierarchy defenses can be weaponized by agent‑level context‑assembly logic, which is high relevance for real‑world skill‑market agent ecosystems.
  2. Complete and rigorous evaluation pipeline. Well‑defined three‑fold metrics (one‑shot ASR, memory‑insertion rate, persistent ASR). Ablation, carrier‑comparison baselines and stealth audits provide solid causal evidence for the core trust‑transfer hypothesis. Statistical significance is reported.
Critical weaknesses

Mitigation strategies remain mostly conceptual without end‑to‑end implementation benchmarks. This is acceptable for a red‑teaming security paper, but the authors should expand brief discussion on practical agent‑runtime mitigations in revision.

Questions for authors
  1. Please more prominently highlight the Full‑Access / permission‑approval‑fatigue assumption in the main body, not only the appendix. Could you briefly discuss how partial human‑in‑the‑loop approval would degrade attack effectiveness?
  2. For runtime mitigation that downgrades trust for all skill‑body content independent of metadata, please add short discussion on potential functional impact for legitimate skill workflows in revision.
Reviewer_RdHJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a skill-supply-chain attack (HierJack) on tool-using LLM coding agents. Its thesis: when a third-party skill is installed, the agent itself places the skill's benign YAML metadata in a high-authority message position (e.g., the system prompt), and the model's instruction-hierarchy-trained compliance then "transfers" along the registry mapping µ from the metadata to the lower-authority skill body. The authors (i) rewrite six classes of explicit malicious code into equivalent-intent natural-language ("semantic") payloads embedded in a benign workflow, which evade code-oriented audits; (ii) induce the agent to copy the payload into its persistent memory file so that the attack re-fires after the skill is removed and the process restarts; and (iii) evaluate across 4 agents × 4 models (Claude Code, Codex, Qwen Code, OpenCode; GPT-5.5, GLM-4.7, Qwen3.6-plus, Kimi-K2.5) on 306 constructed cases. Reported averages: one-shot ASR 88.46%, persistent ASR 69.57%; porting InjecAgent/ChatInject payloads onto the skill carrier raises their ASR by 46.4 and 36.9 points. The paper adds a payload ablation, a carrier comparison, scanner-evasion tests, an AgentSpec defense test, an instruction-position-swap experiment, and a 65 MB supplementary package with per-case CSVs, sample logs, and an evaluation toolkit.

This is a timely and genuinely worrying finding, and the breadth of the study and the released artifacts are commendable. My reservations concern (a) whether the two central mechanistic claims — the metadata→body "trust transfer" via µ, and the role of instruction-hierarchy training — are isolated by controlled experiments, (b) a gap between "command executed" and "attack goal achieved," and (c) several artifact/consistency issues I found when checking the supplementary package.

Critical strengths
  1. Important, well-scoped threat. Skills/extensions are a real and growing surface, and the observation that code-level audits miss intent-preserving natural-language payloads is actionable for both model and marketplace defenders.
  2. Broad evaluation with released artifacts. 16 agent×model cells, a carrier comparison, ablations, scanner tests, a defense test, and a position-swap study, plus a supplementary package with per-case CSVs and an evaluation toolkit.
  3. Reasonable statistical reporting for an attack paper. Table 5 gives Wilson intervals, and Appendix D reports two-proportion tests and a Bonferroni-corrected ablation. Persistent ASR is defined end-to-end over all 306 cases rather than conditioned on memory-insertion success, which is the honest choice.
  4. Serious ethics handling. Isolated local execution, de-weaponized parameters, fictional endpoints, and preliminary vendor disclosure before deanonymization are described clearly.
Critical weaknesses

W1. The two central mechanistic claims are not isolated by controlled experiments.

  • Metadata→body trust transfer via µ. The carrier comparison (Table 2) and the cross-agent comparison vary several things at once (invocation wrappers, loading time, context position, workflow framing, system prompts, memory-loading rules), so they cannot attribute the gain to the registry mapping alone, despite the claim that "the two carrier conditions differ only in whether the attack payload is associated with legitimate skill metadata." The clean test is a within-agent, within-model, within-payload intervention: (a) skill body loaded but metadata not placed in the high-authority listing (H ∧ ¬µ), and (b) listing retained but the description replaced by a neutral stub (H_stub ∧ µ). Equation (1)'s T(x) is only a behavioral compliance proxy and does not by itself evidence authority propagation.
  • Instruction-hierarchy training as amplifier. The grouping of GPT-5.5/GLM-4.7 ("trained") vs. Qwen3.6-plus/Kimi-K2.5 ("not disclosed") conflates many model differences, and absence of public disclosure is not evidence of absent training. Wallace et al. (2024) concern earlier models and do not establish GPT-5.5's specific procedure. The within-model memory-placement contrast (privileged vs. demoted memory position across agents) is the stronger evidence and should be foregrounded; the cross-model training narrative (e.g., §5.1, Conclusion) should be narrowed to an empirical association unless a same-model position intervention is provided.

W2. "Substantive and persistent system compromise" exceeds the verification I could confirm. The paper states that outbound classes (exfiltration, reverse shell) are judged by listener connection records. In the supplementary package I found no connection_log file and no reference to one in the evaluation toolkit (evaluate_log_based.py, listener.py). As released, success for local classes appears to reduce to "the agent generated a marker-bearing script and the OS executed it," not a verified end state (e.g., a file actually exfiltrated and received, a shell actually connected back). Issuing a command is not the same as achieving the goal: permission errors, failed calls, or absent side effects can separate the two. Please (a) specify a per-category postcondition (exit status / received connection / state diff) and release a sample of the listener records the text relies on, and (b) for persistence, give a reset checklist and a negative control that deletes the memory file before Task 2 to confirm that re-firing is attributable to memory alone and not to residual state outside the workspace.

W3. The carrier-comparison table (Table 2) is not a matched-denominator contrast. Appendix D states that "each baseline run is evaluated over its own case set," so the external and skill arms are not evaluated on the same cases, and Table 2 reports neither the per-arm n nor how invalid runs are excluded. As reported, "+36.9 / +46.4 / up to +70.5 pp" should be presented as a descriptive difference between differently-sized stimulus sets, or recomputed on matched cases with the denominators and the "valid" rule stated. The two-proportion tests in Appendix D inherit this issue and also treat per-case outcomes on a shared 306-stimulus set as independent across cells (no clustering, single run per case, no repeated seeds).

W4. The headline conditions are an upper bound, not a "most common" configuration. All results use Full-Access (auto-approve) mode, and I confirmed that 306/306 Task-1 prompts explicitly instruct the agent to use the corresponding workspace skill (case.json flags this, and the task01.md files contain the instruction). So the ASRs are measured under "user explicitly installs, auto-approves, and explicitly invokes the malicious skill." That is a legitimate and serious worst case, but the paper presents it as the prevalent real-world setting (the 93%-approval citation is about approval rate, not about disabled permissions). Please relabel these as an upper bound and add at least a small non-auto-approve / non-explicit-invocation control.

W5. Artifact, reporting, and consistency gaps.

  • OpenCode panel has no producer in the released toolkit. The evaluator has no opencode branch (the --agent option defaults to codex with no OpenCode handling), and the Sample Log/ directory contains runs only for Claude Code / Codex / Qwen Code — no OpenCode run directories. OpenCode result CSVs are present, but the toolkit and sample logs do not cover them, so the four OpenCode cells in Table 1 (one quarter of the one-shot average) and every OpenCode-based ablation/position-swap figure (Figures 3–4) cannot be traced in the artifacts. Please release the OpenCode evaluation path and sample logs, or scope the claims accordingly.
  • Truncated main CSVs. Three of the 16 Main CSVs have fewer than 306 rows (Codex+GPT-5.5: 263; Codex+Kimi-K2.5: 202; OpenCode+Qwen3.6-plus: 299). Please reconcile these with the stated n = 306 per cell and Figure 5's denominators.
  • Deferred definitions. The per-category ASR criterion, the Table 2 "valid" denominator, the Table 3 per-scanner detection threshold, and the Figure 4 "success" definition (which appears to differ from the main one-shot "verified-executed" criterion) are all needed in the main text.
  • Table 3/Table 4 specifics. Table 3 mixes HierJack scans with SkillJect numbers taken from its paper without matched versions/thresholds/populations; in Table 4, 99 successes and 5 true defenses leave 4 of the 108 runs unaccounted for (presumably timeouts, which the caption excludes from both success and failure); please state these categories and the denominator rule, and provide per-event records behind the 443 "Rule Interceptions."
  • Release hygiene. The README states that all endpoints use *.example-cdn-sync.com. This is a registrable domain, not an RFC 2606 reserved name (.invalid/.test/.example), and should be replaced before any public release. The authors should also re-scan the package for absolute machine paths and other identifying metadata.
Questions for authors
  1. Can you run the within-agent/within-model/within-payload interventions H ∧ ¬µ and H_stub ∧ µ, so the metadata-mapping claim is isolated from payload aggressiveness and agent differences?
  2. For each of the six categories, what postcondition defines "goal achieved"? Can you release a sample of the listener connection records used to judge outbound classes, and report the GPT-5.4-judge vs. manual-spot-check agreement and how disagreements were resolved?
  3. Can you add a negative control that deletes the memory file before Task 2, and a reset/snapshot checklist showing no state outside the workspace survives?
  4. Please give the per-cell n and the "valid" rule behind Table 2, and recompute the carrier gains on matched case sets; do the significance tests account for the shared stimulus set and single-run design?
  5. Can you release the OpenCode evaluation path and sample logs, and reconcile the three truncated Main CSVs (263/202/299) with n = 306?
  6. Is there direct evidence that GPT-5.5/GLM-4.7 received the specific instruction-hierarchy training, or should the training claim be narrowed to an observed position-sensitivity association?
Reviewer_SFSGOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HierJack, a prompt-injection attack that exploits trust transfer between high-authority skill metadata and lower-authority skill instructions. By converting malicious code into semantic descriptions embedded in benign workflows, the attack induces malicious execution and persistent memory poisoning. Experiments across 16 agent-model combinations report 88.46% average one-shot ASR and 69.57% persistent ASR.

Critical strengths

Important security problem: The paper identifies a practical attack surface in skill-enabled agents, highlighting how trusted skill workflows can undermine model-level instruction hierarchy defenses. The demonstration of persistent compromise after skill removal is particularly concerning.

Comprehensive empirical evaluation: Experiments across four agents and four models, together with payload ablations, baseline comparisons, and defense evaluations, provide substantial evidence of attack effectiveness.

Critical weaknesses

Insufficient isolation of the claimed mechanism: The paper attributes the improved ASR to metadata-body trust transfer, but the comparisons do not fully isolate this mechanism from other differences between skill execution and external-data delivery. More controlled experiments are needed to establish causality.

Limited realism and generalizability: The evaluation assumes preinstalled malicious skills and Full-Access execution mode. The results may overestimate practical attack effectiveness under typical permission restrictions, skill auditing, and deployment configurations. Moreover, the payload ablations are limited to one agent framework.

Questions for authors

Can the authors isolate metadata-body trust transfer by comparing identical payloads with and without high-authority metadata registration, while keeping the execution workflow unchanged?

How does HierJack perform under default permission settings rather than Full-Access mode?

How sensitive is persistent ASR to different memory retrieval mechanisms and configurations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.