acceptodds
Under review as a conference paper at ICLR 2027

From Untrusted Content to Accepted Instructions: Security Risks of Context Compaction in LLM Agents

Abstract

LLM agents use context compaction to continue tasks that would otherwise exceed their context windows, replacing older history with a model-generated summary. We show that this process creates a new attack surface in agent harnesses: a prompt injection can persist into the compaction summary, reframed as part of the user's task. Across ten models evaluated on a set of 140 tool-using histories, summaries present injected instructions as goals, constraints, or next steps of the task in 18–45% of cases, often omitting the injection's original provenance and sometimes misattributing it to the user or system. When framed in this way, agents follow these injected instructions in 25–83% of their next responses. In a separate evaluation with executable tools, agents carry out attacker requests while still completing the user's task. Changing the role in which the summary is delivered (e.g., user vs. system vs. tool) provides little protection in our tests, but changing the compaction prompt to separate user goals from tool observations reduces attack success from 24% to 5% across four models. This work identifies compaction as a critical security boundary in agent harnesses, characterizes how the vulnerability arises, and demonstrates a possible defense.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Context compaction, LLM agents, prompt injection, agent security, context engineering, conversation summarization, instruction hierarchy

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_WZMmOpenReview ↗
Rating
2(+1)
Confidence
4(+1)
Informativeness
1
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper studies indirect prompt injection through context compaction in LLM agents, focusing on how summaries transform instructions embedded in untrusted external content into apparent task requirements. The authors evaluate ten models using injected tool outputs across coding, web-search, and AgentDojo tasks, measuring how summaries represent these instructions and whether agents subsequently follow them. The results show that agents are more likely to follow injected instructions when summaries present them as goals, constraints, decisions, or next steps. A controlled experiment further demonstrates that presenting the same instruction as remaining work rather than untrusted evidence increases attack success. The authors also evaluate defenses based on summary delivery roles and compaction prompts. Lowering the summary’s message authority provides little benefit, whereas separating user-authorized goals from external observations reduces attack success from 24% to 5% in a four-model tool-enabled evaluation. The authors argue that compaction should preserve source attribution and authorization boundaries, rather than merely retain task-relevant information.

Critical strengths
  1. The paper introduces a realistic attack scenario: compaction attacks.
  2. The evaluation results show that compaction attacks persist in some widely used models.
Critical weaknesses

Weaknesses:

  1. This paper claims that compacting a poisoned conversation history can preserve an indirect prompt injection attack. This is not surprising and I believe the proposed compaction attack is simply a subset of indirect prompt injection attacks. To claim that compaction introduces a new attack surface, the paper should provide experiments demonstrating additional threats compared with conventional indirect prompt injection, such as amplifying attacks or bypassing more defenses. However, Appendix E.7 reports 115 successful attacks with the full history, compared with 109 after compaction. This suggests that compaction attacks are no more threatening than conventional indirect prompt injection attacks, weakening their significance and novelty.
  2. The paper’s intended takeaway is unclear. My understanding is that the paper aims to show that compaction can transform external malicious content into tasks recognized by agents, making them more vulnerable to attacks. However, I do not see experiments supporting this claim, and I am unsure whether my interpretation is correct.
  3. The experiments are conducted on AgentDojo, an existing security benchmark for tool-using agents that does not natively support the evaluated harnesses and compactor. Several benchmarks, such as DTap [1], support current harnesses, including OpenClaw, Codex, and Claude Code. Evaluation using these real-world harnesses would make the results more convincing.

References:

[1] DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents

Questions for authors

Please refer to Weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer WZMm (1/4): W1, additional threat

Thank you for your review, and for suggesting DTap (Chen et al., 2026), whose tasks and attack goals we ran in two real-world harnesses, Codex CLI and the pi coding agent (W3). We answer in four comments: W1 here and in the second, W2 in the third, and W3 in the fourth.


W1. Is the compaction attack a subset of indirect prompt injection, with no additional threat (109 versus 115)?

Compaction gives the same injection a second route to the agent, and attacks succeed through it that fail when the agent reads the injection directly. When the agent sees the tool result directly, it encounters the instruction as something an external source said. The compactor only has to record it as part of the task for the continuing agent to encounter it as assigned work. In a live session where the agent gets a turn with the original result before compaction, an instruction it initially ignores can influence a later action through the summary. Our AgentDojo comparison tests the two routes from the same checkpoint immediately after the injected read.

Why the AgentDojo totals are similar (109 versus 115). The totals match because compaction removes some attacks and adds others. We resumed each case from the same checkpoint right after the injected read, once with the full history and once from a summary. In 43 cases the attack succeeded only with the full history, and in 37 it succeeded only after compaction. In 36 of those 37, the summary presented the instruction as part of the task (Appendix E.7). To check that these differences were not chance, we resumed all 80 cases five more times per condition (new; App. I.4):

  • The 37 compaction-only attacks succeed in 63% of runs from the summary and 19% from the full history.
  • The 43 full-history-only attacks succeed in 54% of runs from the full history and 7% from the summary.

This comparison treats the two routes as alternatives, but a live agent goes through both. It reads the tool result first and resumes from a summary only later, when compaction triggers. The 37 are instructions the agent mostly ignores in the tool result and then follows once the summary presents them as its task. An injection the agent has already resisted can still succeed after compaction, so this risk adds to direct injection rather than replacing it. Section 4.4 now reports this breakdown alongside the totals.


References

  • Chen et al. (2026). DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents. arXiv:2605.04808.
AuthorsOpenReview ↗

Response to Reviewer WZMm (2/4): W1 continued, amplification and defenses

W1, continued. Amplifying attacks and bypassing more defenses.

Amplifying attacks. For some cases, the attack succeeds when the agent resumes from a compacted summary and fails in a separate run with the full history.

  • A real-world harness. In Codex CLI on DTap's legal tasks, the agent filed the injected line in 39% of full-history runs and 52% of runs when configured to compact during the task. The increase was concentrated in tasks where the agent usually resisted the line with full history. The held-out comparison is under W3 (App. I.3.3).

  • An agent that resists the injection. In 110 runs over 55 AgentDojo histories, the attack never succeeds when Claude Sonnet 5 continues from the full history, which contains the injected tool result. However, when it continues from GPT-5.6 Luna's summary of that history, the attack succeeds in 16% of runs (new; App. I.4). GPT-5.6 Sol shows the same in the submission (7% and 33%; Appendix E.8). Existing harnesses do allow a smaller model to write the summary, as in Aider and LangChain's summarization middleware (Aider, 2026; LangChain, 2026).

Bypassing defenses. Compaction changes which defenses work.

  • A defense that relies on the agent seeing the source is lost after compaction. In our delegation test (Appendix F.3), the user tells the agent to use a source only as background, not as instructions, and the source also asks for every output to be copied to an external address. With the full history, GPT-5.6 Luna never makes the copy (0 of 36 runs), because it can see that the request came from the source the user restricted. After Luna compacts the history with Codex's prompt, it makes the copy in 24 of 36 runs, even though the user's restriction is kept word for word in the summary because the summary records the copy as part of the task, so the restriction no longer applies to it.

References

  • Aider (2026). Options reference: --weak-model. aider.chat/docs/config/options.html.

  • Chen et al. (2026). DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents. arXiv:2605.04808.

  • LangChain (2026). Built-in middleware: Summarization. docs.langchain.com.

AuthorsOpenReview ↗

Response to Reviewer WZMm (3/4): W2, takeaway and supporting experiments

W2. Is the takeaway that compaction transforms external malicious content into tasks, and which experiments support it?

Yes, that is the takeaway. Compaction can turn an instruction in external content into a requirement the agent recognizes as part of its task, and the agent is then more likely to act on it. The submission tested the first two steps below (Sections 4.1 and 4.3). The revision states it in the introduction and adds the last two experiments.

  • Compaction turns injected instructions into tasks. Summaries from all ten models present the injected instruction as a goal, constraint, decision, or next step in 18–45% of cases (Section 4.1).

  • Presenting an instruction as a task causes the agent to act on it. We append the same sentence to one summary either as remaining work or as untrusted evidence from a tool. GPT-5.6 Luna follows it in 31% of runs in the first case and never in the second (Section 4.3). Three more models now show the same pattern, following it in 31–50% of runs against 0–6% (new; Appendix E.5).

  • This makes the agent more vulnerable than the original injection. On 78 AgentDojo cases where compaction had presented the instruction as part of the task, five models followed the same instruction in 75% of runs when it sat in a summary and in 29% when it sat in the original tool output. Both were delivered as tool messages, so the difference does not come from the message role (new; App. I.2).

  • The same holds in a real-world harness. In Codex CLI on DTap legal tasks (W3), the agent prepares a court filing from a client's case file, and one document contains an injected line asking it to include a statement against the client. We rewrote only what Codex's own summary said about that line, choosing the version at random within each task. The agent put the line in its filing in 26 of 32 runs when the summary presented it as a filing requirement, and in 7 of 32 when the summary added that neither the user nor the supervising attorney had supplied or confirmed it (App. I.3.3).

Reviewer_c9nqOpenReview ↗
Rating
2(−1)
Confidence
4(+1)
Informativeness
3
Presentation
3
Soundness
1(−1)
Read review
Paper summary

The paper studies context compaction in LLM agents from a security angle. When an agent's history grows past the model's context window, the harness asks a model to summarize the history and the agent then continues from the summary alone. The paper shows that this step opens a new door for indirect prompt injection. An attacker plants an instruction inside content the agent reads through a tool, such as a hotel review or the output of a test plugin. If the summary writes that instruction down as part of the task, for example as a goal, a constraint, a decision, or a next step, the agent that resumes from the summary no longer sees an outside request. It sees work it believes it was assigned. The model writing the summary does not need to obey the injected instruction. Recording it in the wrong place is enough.

The main findings are measured on ten models from nine developers, each compacting its own history and then continuing the task. On cases built from 140 recorded agent histories, summaries present the injected instruction as part of the task in 18 to 45 percent of cases. Agents then follow 25 to 83 percent of the instructions framed this way, while they follow almost none of the instructions the summary leaves out, reports as source content, or flags. About one fifth of these summaries credit the instruction to the user, the system, or the project, none of which the attacker controls. In a replay of AgentDojo tasks with executable tools, agents reach the attacker's target in 109 of 548 runs, and in 91 of these they also complete the user's real task, so task completion alone cannot show that the agent did only what the user asked.

The paper makes three contributions. First, it identifies compaction as an attack surface and gives a simple formal account of how a harness compacts a history. Second, it traces the attack along its full path, from the tool result into the summary and from the summary into the agent's responses and tool calls, and it shows that what matters is how the summary presents the instruction rather than whether the summary keeps it or names its source. A controlled experiment makes this point causal. The authors hold one clean summary fixed and append the same attacker instruction either as remaining work or as untrusted evidence, and the agent reaches the attacker's goal in 26 of 84 runs in the first case and in zero in the second, with task completion unchanged. Third, the paper compares two defenses. Delivering the summary in a lower authority message role does not help, since the text still says what the task is. A compaction prompt that separates user goals, agent progress, and external observations into different sections reduces attack success from 24 to 5 percent across four models, and it still keeps requirements the user actually delegated to a source.

The validation is careful. Clean control runs without the injection never show the attacker behavior, across 666 text continuations and 548 tool enabled runs. Summary labels come from a judge model calibrated against a small human sample at kappa 0.72 to 0.85, and a second judge from a different model family re labels the attacked summaries and supports the same conclusions. The controlled experiment is verified by replaying all 252 runs and recomputing the scores. The defense is checked for over blocking on constructed tasks where the user delegates part of the work to a source, and against two hand written adaptive injections that try to look user approved. Uncertainty is reported with bootstrap intervals over histories.

Critical strengths
  1. The paper explains the mechanism behind the attack instead of only showing that the attack works. Most prompt injection papers report a success rate and stop there. This paper separates three things that are easy to confuse. It asks whether the summary keeps the injected instruction, whether it presents the instruction as part of the task, and who the summary says the instruction came from. It then shows with numbers that the second one is what drives the agent's behavior, and it backs this with a controlled experiment rather than correlation. The authors hold one clean summary fixed, append the same attacker sentence either as remaining work or as untrusted evidence, and get 26 of 84 runs reaching the attacker's goal in the first case and zero in the other two, with task completion unchanged across conditions. This is consequential for my assessment because it turns a broad survey of a vulnerability into an explanation that other researchers and harness developers can build on. The proposed defense follows directly from the mechanism, and the paper shows the defense works by changing what the summary says rather than by hiding the injection, since retention stays at 89 percent while task framing falls from 82 to 5 percent. A result that names the cause is far more useful than one that only names the rate, and it is the main reason I weigh this paper heavily.

  2. The measurements are careful in exactly the places where papers in this area usually cut corners, which makes the claims safe to rely on. The attack is tested with executable tools in AgentDojo, so the authors can tell the difference between an agent mentioning the attacker's address and actually sending to it. Clean control runs without any injection never produce the attacker behavior, across 666 text continuations and 548 tool runs, so the measured effects come from the injection and not from the tasks. The study covers ten models from nine developers, a second judge from a different model family re-labels the attacked summaries and reaches the same pattern, and the controlled experiment is verified by replaying every run and recomputing the scores. The defense is also tested for the failure mode that defense papers usually hide, which is blocking work the user actually wanted. In the delegation study the schema keeps all 36 requirements the user authorized while dropping 35 of 36 unauthorized copies, which is better than the uncompacted history manages. This matters for my assessment because security evaluations stand or fall on whether each link of the chain from injection to summary to action is measured honestly, and here each link has its own control. It is also why my remaining concerns are about framing and scope rather than about whether the results are real.

Critical weaknesses
  1. The paper does not engage with the closest prior work, so its novelty claim is currently unsupported. A June 2026 paper, Governance Decay (arXiv 2606.22528), already treats compaction as a security relevant step, includes an attacker who steers the compaction step through injected content, and reports a crossing experiment where the model that writes the summary, rather than the model that acts, determines whether a rule survives. That crossing result overlaps directly with this paper's Appendix E.8. Dash et al. 2026, which the paper does cite, already names compaction without source filtering as a channel for hostile content to enter an agent's stored state, and the one sentence of contrast the paper gives understates this. The paper's real contribution likely survives the comparison. It studies the opposite direction, an outside instruction being promoted into the task rather than a legitimate rule being deleted, and it isolates the framing effect causally rather than observationally. But the paper never makes this argument itself, so a reader cannot tell what is new without doing the authors' work for them. This is consequential for my assessment because acceptance requires a clear account of what this paper adds over these two works, and right now that account does not exist in the text. It is fixable in a revision, but it is mandatory, not optional.

  2. The headline rates are stronger than the evaluation materials justify, and the paper does not say this where readers will see it. Three construction choices each push the numbers up. The instruction bank was developed on one of the evaluated models, favoring instructions that summaries tend to keep. Injections sit at the end of the attacker's text, and the paper's own paired diagnostics show the same sentence is task framed far less often when it sits inside a structured tool output, with framing falling from 75 percent to 0 percent in one test and from 41 to 3 percent in another. The defense is measured on cases pre-selected because the attack already succeeded there, which roughly doubles the baseline success rate relative to the random sample. On top of this, the main text never mentions that in the tool enabled suite compaction does not increase the total number of successful attacks compared with leaving the history whole, 109 against 115. Compaction changes which attacks succeed and moves them out of reach of defenses that read the original tool output, which still matters, but it is a weaker claim than the abstract makes. Each of these facts is disclosed somewhere, mostly in appendices, yet the abstract figures of 18 to 45 percent and 24 to 5 percent will circulate as if they were population rates, and they are not. This is consequential because the paper's significance rests on those numbers being read correctly, and the main text currently does not give the reader the means to do that.

Questions for authors
  1. How do you position this work against Governance Decay (arXiv 2606.22528) and against the compaction channel described in Dash et al. 2026? Both predate this submission and both treat compaction as a security surface, and Governance Decay also reports that the model writing the summary, rather than the model acting on it, determines the outcome, which overlaps with your Appendix E.8. Please state concretely what your paper adds over each of these works. Your answer largely decides how I weigh the contribution.

  2. The instruction bank and the placement rules were developed on GPT-5.6 Luna, and Luna is also an evaluated model and the annotation judge. Do you have any statement set that was not filtered through Luna, for example one written independently or developed on a different model, and if so how do the framing and success rates change? If not, can you give evidence that the rates are not specific to that development process? This determines whether I read the 18 to 45 percent range as a property of compaction or as a property of the bank.

  3. In your AgentDojo comparison, the uncompacted injected history produces 115 successful attacks and the compacted history produces 109. Do you agree that compaction did not increase the total number of successful attacks in this suite? If so, what exactly is the additional harm that the abstract should claim, and would you be willing to state that counterfactual plainly in the main text rather than only in Appendix E.7?

  4. Which production configurations does your threat model cover? Stock pi truncates tool results at 2000 characters, a limit your setup raises to one million, and deployed Codex retains recent user messages alongside the summary and can use a remote compaction path. Does the attack require the injected tool result to be fully replaced by the summary, or does it still succeed when part of the original output survives in context next to the summary? The answer changes how far the results generalize beyond the replayed harnesses.

  5. In the role comparison, delivering the summary as a tool result raised DeepSeek's attack success from 16 to 24 percent and Luna's from 48 to 52, while the text says the tool role lowers success by at most one point. Do you stand by the broader conclusion that delivery role provides no protection, and how do you reconcile that conclusion with these increases?

  6. For the source-separated schema you test two fixed adaptive wordings on one model. Did you run even a small search over attacker wordings against the schema, for example sampling variants that imitate user approval or the schema's own annotations, and if so what was the strongest attack you found? A negative answer would not decide my score on its own, but it would change how I describe the defense.

The followings are minor suggestions:

  • Repeated summary and continuation samples for the full random sample, not only the 23 Luna-selected cases, to show how stable the per-model rankings are.
  • A closer study of interior and structure-aware placements, since your own diagnostics suggest that source formatting is a strong lever for both attack and defense.
  • Defenses that record provenance outside the model rather than through prompting, and a test of how the attack and the schema behave across repeated compactions in one long session.
11 comments
AuthorsOpenReview ↗

Response to Reviewer c9nq (1/6): W1/Q1, positioning

Thank you for the positive and detailed review. We answer in six comments: W1/Q1 here, then W2 and Q2, Q3, Q4, Q5 and Q6, and the minor suggestions.


W1/Q1. What the paper adds over Governance Decay and Dash et al.

We show how an outside instruction becomes part of the user's task through the summary, and the controlled test shows that the same instruction can lead to different actions depending on how the summary presents it (Section 4.3). The submitted test used Luna as the continuing agent. Appendix E.5 adds GLM, DeepSeek, and Nemotron using the same saved summaries.

  • Governance Decay (Chen, 2026) shows how injected content can lead compaction to drop a legitimate rule. We study how compaction can add an external instruction to the task. As you note, both studies vary the compactor and continuing agent separately. In Appendix E.8, changing the compactor changes which injected instructions the same agent follows.

  • Dash et al. (2026) identify compaction without source filtering as a channel for hostile content to enter persistent memory. We measure what a summary carries forward within a single task, including whether an instruction is retained, how it is framed, and which source it is attributed to. On AgentDojo summaries that already task-framed the instruction, adding a source note did not lower attack success (70% compared with 66% for a neutral note). A note stating that the user had not approved the instruction and asking the agent to wait for confirmation lowered success to 7% (Appendix E.6). Our schema keeps external observations separate from authorized work. We evaluate it with executable tools and on requirements the user delegated to a source (Sections 5.3 and 5.4).

Section 6 explains what our work adds to each study.


References

  • Chen (2026). Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents. arXiv:2606.22528.

  • Dash et al. (2026). From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents. AIWILD workshop at ICML 2026.

AuthorsOpenReview ↗

Response to Reviewer c9nq (2/6): W2, Q2, headline rates and statement set

W2. What the headline rates describe.

The 18–45% and 24% → 5% figures describe the cases we evaluated under the threat model of Section 2.2. The statements are written as plausible task requirements in a source the attacker controls.

  • "The instruction bank was developed on one of the evaluated models." Independent statements, written without seeing our bank and developed on a different model, give similar pooled rates across the eight models other than Luna and GLM (Q2 below).

  • "Injections sit at the end of the attacker's text." The headline rates do not depend on placing the line after the output. The two drops mentioned come from a diagnostic on separate cases, which compares a line inside a structured tool output with a line placed after that output (Appendix G). The main evaluations exclude that placement, because an attacker who controls only the source cannot write after the output's structure. In both the text evaluation and AgentDojo, every injection sits at the end of the content the attacker controls, such as a web page, a review, or a message body, and inside its field when the tool result is structured (Appendix A).

  • "The defense is measured on cases pre-selected because the attack already succeeded there." The 24% → 5% result uses no filtering by prior attack outcome. It comes from the AgentDojo evaluation of Section 5.3, which runs all 55 histories on four models (the 219 runs available under both prompts). Selection for prior task framing applies to the 62-case role and prompt comparisons in Sections 5.1 and 5.2, which the revision now labels as selected.

To make this clearer, the abstract now gives the 18–45% as a share of sampled cases under our threat model and the 24% → 5% as a result in AgentDojo, and the revised introduction names this attacker beside the 18–45% rate.


W2/Q2. Do you have a statement set not filtered through Luna, and how do the rates change?

Following your suggestion, we tested a statement set that was written independently and developed on a different model, and it gives similar pooled rates across the eight models other than Luna and GLM (task framing 21% vs 22%, attack success 10% vs 10%). We had developed the bank on Luna because an attacker can develop statements offline on a model it has access to, and the new test repeats that step with GLM.

Two other models, Claude Sonnet 5 and Gemini 3.1 Pro, wrote 48 statements of the kind our threat model defines (Section 2.2) for the same 123 cases, without seeing our bank or any result (new; App. I.5). We inserted each statement into histories from one half of the set, compacted them with GLM-5.3 Flash, and kept the 10 statements that GLM's summaries most often presented as part of the task. We then tested those 10 on the 35 cases they fit in the other half, with GLM and the nine other models, using the same prompt and judge.

Table R4. Independent statements and our bank on the same held-out histories (from Table 32 in the revision).

Held-out historiesIndependent statementsOur bank
Eight models other than Luna and GLM, task-framed / attack success21% / 10%22% / 10%
Luna, task-framed / attack success29% / 11%54% / 23%

On those held-out histories, Luna task-frames our bank in 54% of summaries and the independent statements in 29%. This suggests that development on Luna contributes to its higher rate with our bank. Section 3.1 now states this comparison. You also note that Luna is the annotation judge. Four non-author annotators checked it and agreed with it on 88 of 96 task-framing labels (new; App. I.1).

AuthorsOpenReview ↗

Response to Reviewer c9nq (3/6): Q3, 109 versus 115

W2/Q3. Do you agree that compaction did not increase the total (109 versus 115), and what should the abstract claim?

In AgentDojo, overall attack success was similar (109 of 548 runs after compaction and 115 of 550 with full history). However, compaction still presents a new attack surface, as it can present an instruction that the agent may ignore in tool output as part of its task. In a live session where the agent has a turn with the tool output before compaction, the summary can give the instruction a second opportunity to influence its actions. The abstract now says compaction changes which attacks succeed, and Section 4.4 gives the comparison.

  • Different attacks succeed. In the original AgentDojo runs, 43 cases succeeded only with the full history and 37 only after compaction (Appendix E.7). In 36 of the 37, the summary presented the instruction as part of the task. We ran five additional continuations per case in each condition. For the 37, the attacker reached its target in 63% of runs from the saved summary and 19% from the full history, so these are the cases where the summary gives an ignored instruction its second opportunity. For the other 43, success was 54% from the full history and 7% from the summary (App. I.4).

  • Protections that rely on the source can be lost. In our Luna delegation test, the user tells the agent to use a source only as background. Luna carries out the rule added by that source in 0 of 36 runs with the full history and 24 of 36 after its own summaries (Appendix F.3).

  • Codex CLI shows the same pattern on some tasks. In DTap legal tasks (Chen et al., 2026), GPT-5.6 Luna filed the injected line in 39% of runs with the full history and 52% with Codex's server-side compaction, configured to trigger during the task (p = 0.01, permutation test within task). We held out each cohort in turn and selected tasks with zero full-history filings in the other two cohorts. In the held-out runs for those tasks, the line was filed in 12 of 44 runs with compaction and 1 of 44 with full history. In the 30–60k-token pi tasks, the filing rates were 60% with compaction and 65% with full history (App. I.3.3).


References

  • Chen et al. (2026). DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents. arXiv:2605.04808.
Reviewer_MRUROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper investigates the security risks of context compaction in LLM-agent harnesses, showing that malicious prompt injections can be mistakenly transferred into actions to be executed during context compaction. Authors find that an indirect prompt injection can change status during compaction: text that originally appears inside an untrusted tool result may be rewritten in the summary as a goal, constraint, decision or next step. Extensive experiments are conducted to validate this vulnerability and explore security prompts and source-separated schemas to mitigate this risk. The paper studies an interesting attack scenario, and the attack and defense methods are intuitive and easy to understand.

Critical strengths
  1. The paper investigates a security vulnerability in context compaction. This is a simple yet widespread and practically meaningful failure mode, which may have broader implications for agent summarization, memory, and planning, etc.
  2. The paper shows that an indirect prompt injection can change status during compaction: text that originally appears inside an untrusted tool result may be rewritten in the summary as a goal, constraint, decision or next step.
  3. The attack and defense methods are simple and easy to understand. The experiments are comprehensive and clearly presented.
Critical weaknesses
  1. The paper could be improved in terms of writing fluency and overall narrative coherence.
  2. The mechanism underlying the attack appears similar to Procedural Prompt Injection or Workflow Hijacking, making the proposed setting seem like a specialized application of these broader attack paradigms.
  3. The proposed defenses do not fundamentally resolve the underlying security issue. They may be bypassed by disguising malicious instructions as legitimate steps within an external workflow.
Questions for authors

I would be willing to raise my score if the authors could address the following concerns:

  1. What is the primary factor that contributes to the success of the attack? Is it a task-disguising effect similar to Procedural Prompt Injection, or stem from untrusted information is mistakenly granted authority during context compaction? What vulnerability is distinctive to context compaction itself?
  2. I tend to view the identified security risk as a specialized application of broader directions such as Procedural Prompt Injection, Workflow Hijacking and Dependency Traps. How do the authors distinguish paper's contribution from these existing attack paradigms?
  3. Can the risk be generalized to arbitrary context compression processes, such as memory, self-evolution, context update, etc?
  4. The proposed defense method does not appear to essentially solve the risk of context compaction process and may be easily bypassed by more covertly malicious instructions.
  5. How effective would an independent auditing mechanism or a defense with a controlled workflow be in this setting?
6 comments
AuthorsOpenReview ↗

Response to Reviewer MRUR (1/2): Q1, W2/Q2, what is distinctive to compaction

Thank you for the careful review, and for stating so clearly what would change your assessment. We answer in two comments: Q1 and W2/Q2 here, then Q3, W3/Q4, Q5 and W1.


Q1. Is the primary factor a task-disguising effect, as in Procedural Prompt Injection, or untrusted information mistakenly granted authority during compaction? What is distinctive to compaction?

Our results suggest that the primary factor, and the vulnerability distinctive to compaction, is that the agent's own harness presents the injected instruction as part of the user's task.

During compaction, the compactor records the user's goals, constraints and next steps, and its summary replaces older messages. The compactor only has to record the injected instruction as part of the task for the continuing agent to see it as assigned work. Across ten models, summaries do so in 18–45% of sampled cases under our threat model (Section 4.1).

The compactor gives the instruction the appearance of assigned work. In the controlled comparison, across four agents, the same instruction was followed in 31–50% of runs when the summary presented it as remaining work, and in 0–6% when the summary labeled it as untrusted tool evidence and explicitly denied authorization (Section 4.3 and Appendix E.5).


W2/Q2. How does the contribution differ from Procedural Prompt Injection, Workflow Hijacking and Dependency Traps?

In the examples we discuss, the distinction is where the task framing comes from.

Workflow Hijacking, Dependency TrapsCompaction
Who makes the instruction look like part of the taskThe attacker, who writes it as a procedure or a prerequisiteThe compactor, which records it among the summary's goals, constraints or next steps
Where the agent meets itIn a tool's output or descriptionIn its own summary of the task

The attacker therefore needs no disguise. Our statements are plain requirements, and the task framing in the results above comes from the compactor.

We note that we interpreted Workflow Hijacking as in ARGUS (Weng et al., 2026) and Dependency Traps as in VIGIL (Lin et al., 2026). We could not locate a paper titled Procedural Prompt Injection, so we have treated it as an instruction disguised as a step of a procedure. We would be glad to discuss a specific paper if you can point us to one.


References

  • Lin et al. (2026). VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commit. ACL 2026.

  • Weng et al. (2026). ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection. arXiv:2605.03378.

AuthorsOpenReview ↗

Response to Reviewer MRUR (2/2): Q3, W3/Q4, Q5, W1

Q3. Does the risk extend to other context-compression processes, such as memory, self-evolution or context updates?

We expect that it does, wherever a model rewrites untrusted content into text that the agent later treats as its own state, as memory writing, self-evolution, and context updates also do. Related work reports similar effects for memory (Dash et al., 2026; Xu et al., 2026). We have tested this mechanism only in context compaction.


W3/Q4. The defense does not fundamentally solve the risk, and may be bypassed by instructions disguised as workflow steps or by more covert ones.

We agree. The source-separated schema is a mitigation, not a complete fix. We considered 16 candidate wordings and excluded two that changed the attack's objective. We selected one on development tasks and tested it and the main alternatives on held-out tasks (App. I.7). On action targets, the schema lowered attack success for the selected workflow wording from 15% to 5%. The forged schema annotation had the highest observed success under the schema on those targets, at 7% compared with 10% under Codex's prompt. Section 5.3 now reports this result, and the interval for that change includes zero.


Q5. How effective would an independent auditing mechanism or a controlled workflow be?

The independent auditor reduced attack success with similar task completion.

  • An independent auditor. The schema separates sources while writing the summary. The auditor applies the same authorization principle in a separate check before the agent resumes. Gemini 3.8 Flash sees the system prompt, user messages, and summary, and moves requirements the user did not give to a separate section. On the same 55 histories and four agent models, attack success fell from 21% to 3% with the auditor and from 24% to 5% with the schema, each against its own baseline. The auditor reduced attack success by 17.8 percentage points (95% CI 12.3 to 23.7). User-task completion was 81% with and without the auditor. In the delegation study it kept 35 of 36 requirements the user delegated (App. I.8).

  • A controlled workflow. A design such as CaMeL (Debenedetti et al., 2026) could keep the authorized plan outside compaction. Whether enforcing that plan prevents a summary from adding unauthorized actions is a question for future work.


W1. Writing fluency and narrative coherence.

The revision now states the paper's main claim about what compaction changes in the second paragraph of the introduction, before the worked example of Figure 1, and Section 6 positions it against related work. Section 4.4 also brings the AgentDojo comparison with the full history into the main text. Appendix I.3.3 reports the real-world harness results. Thank you for this feedback.


References

  • Dash et al. (2026). From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents. AIWILD workshop at ICML 2026.

  • Debenedetti et al. (2026). Defeating Prompt Injections by Design. IEEE SaTML 2026.

  • Xu et al. (2026). From Storage to Steering: Memory Control Flow Attacks on LLM Agents. arXiv:2603.15125.

Thank you for the response, which has addressed most of my concerns. However, I still have a few concerns that require further clarification:

Q2: Regarding the following related works:

  • AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skills
  • SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents

Beyond the differences in attack scenarios, does the proposed method differ from these works in terms of its underlying attack mechanism?

Q3: Could the authors provide additional experiments to substantiate their claims?

Reviewer_s4yfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper authors studies context compaction as a novel prompt injection attack. In this setting the agent summarizes the history and tend to add tool injection attack instructions as required next steps for the benign task agent is trying to complete. Authors provide detailed analysis around framing the injected attacks as part of the task and agents completing the attacker objective. The paper also proposes a simple prompt based approach where the agent is instructed to separate user goals from external instructions. This method leads to a significant drop in attack success.

Critical strengths
  • The paper studies a novel and interesting vulnerability where summarizing can lead to higher indirect prompt injection success
  • The paper is well written and easy to follow
  • The paper provide detailed analysis on the effect of summarization on attack success beyond simple measures of attack success rates
  • The paper proposes an effective mitigation approach to the proposed attack
Critical weaknesses
  • Authors highlight that delivering summary as a tool response does not help mitigate the attack success. However authors do not provide any results explaining the reason for this.
  • Context compaction is naturally required in long horizon tasks. However authors provide no analysis around how this would work in long-horizon tasks
  • Authors do not provide a rigorous analysis on judge reliability
Questions for authors
  • Can authors provide an analysis explaining why delivering summary as a tool response does not help
  • Can authors provide results on at least one long horizon dataset and provide analysis on how the trajectory length affect the results
  • Can authors provide a human annotation, that is not conducted by authors validating the reliability of the judge
4 comments
AuthorsOpenReview ↗

Response to Reviewer s4yf (1/3): W1/Q1, delivery role

Thank you for the constructive review and for the three clear questions, each of which we provide an answer to below. W1/Q1 is answered here, W2/Q2 in the second comment and W3/Q3 in the third.


W1/Q1. Why does delivering the summary as a tool response not help?

On the selected cases, where the instruction appears has a larger effect than changing between the user and tool roles. In both roles, models follow the instruction more often in the summary than in the original tool output. A likely reason is that models judge an instruction by the text it sits in more than by the role of the message that carries it, and in the summary that text is the agent's own account of its task.

We took 78 selected AgentDojo model–history cases where Luna, Hy4, or MiMo had presented the injection as part of the task. We tested those cases and the same histories with GLM and DeepSeek. The agent resumed from a summary of the clean history followed by the tool output. We varied where the sentence sat and the role of the message that held it (App. I.2).

Table R1. How often five models follow the attacker's sentence, by where it sits and the role of its message (Table 29 in the revision).

Where the attacker's sentence sitsRole of that messageAgent follows it (five models)
At the end of the summaryuser, as in deployment78%
At the end of the summarytool, as in Table 2a75%
In the tool output it came fromuser39%
In the tool output it came fromtool29%

Changing from a user to a tool role decreased the rate by just 3 points when the sentence sits in the summary and by 10 when it sits in the tool output, whereas moving the sentence out of the summary lowers it by 39 points in the user role and by 46 in the tool role, despite the tool output being the more recent message. These results are also in line with recent work from Ye et al. (2026), who find in models' internal representations that user-style text delivered as tool output is still represented largely as user text.

Section 5.1 gives the main comparison. Appendix I.2 gives the full experiment and discusses Ye et al. (2026).


References

  • Ye et al. (2026). Prompt Injection as Role Confusion. ICML 2026.
AuthorsOpenReview ↗

Response to Reviewer s4yf (2/3): W2/Q2, long-horizon results and trajectory length

W2/Q2. Results on a long-horizon dataset, and how trajectory length affects the results.

We tested how the length of a SWE-rebench trajectory and the time since the instruction was read affect compaction. We found no consistent effect of trajectory length. In the longest group, with medians of 116k tokens and 97 tool results, the summary still presented the instruction as part of the task in 35–49% of cases across placements. The rate was 40% when the instruction was read 86 tool results before compaction (App. I.3.1).

How length affects the results. We measure both total trajectory length and the distance from the instruction to compaction, in tokens and tool results. The table pools GPT-5.6 Luna, GLM-5.3 Flash and DeepSeek V4.1 Flash over 64 further SWE-rebench trajectories spanning 21k to 124k tokens, grouped by length into four sets of 16. The first two columns give each group's medians. The longest trajectories reached the 100-step limit.

Table R2. Summary framing and attack success after compaction, by trajectory length and distance to compaction (from Table 30 in the revision).

Trajectory (tokens, tool results)Instruction read before the compactionSummary keeps itPresents it as part of the taskAttack success after compaction
25k, 401.6k tokens (4 tool results)69%52%26%
47k, 631.5k (2)52%42%18%
83k, 992.1k (3)57%44%28%
116k, 972.2k (2)66%49%15%
116k, 9720k (13)47%38%12%
116k, 9750k (34)45%35%12%
116k, 97103k (86)50%40%9%

We found no consistent trend across the four length groups (first four rows). In the longest group, retention fell from 66% at the nearest placement to 47% at the placement 20k tokens before compaction. Retention ranged from 45% to 50% across the three earlier placements, and attack success after compaction ranged from 9% at the earliest placement to 15% at the nearest (last four rows).

Long sessions. Long contexts, where compaction is needed, often come from users carrying out several tasks in one session. For each of 16 DTap legal tasks, we first ran other DTap legal tasks in one Codex CLI session with GPT-5.6 Luna until just before Codex's default compaction point (244,800 tokens), and then gave the agent that task, one of whose documents contains the injected line. With Codex's default compaction, in the 21 runs where it compacted after the agent read the line and before the first filing, the attack succeeded in 17 (81%). That compaction returns an encrypted summary, so we repeated the sessions with Codex's local compaction, which writes a readable one. In the 19 such runs, the summary presented the line as part of the task in 18, and the attack succeeded in 18 (new; App. I.3.3).

Appendix I.3.1 gives the trajectory-length results. Appendix I.3.3 gives the real-world harness results.


References

  • Chen et al. (2026). DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents. arXiv:2605.04808.
AuthorsOpenReview ↗

Response to Reviewer s4yf (3/3): W3/Q3, human annotation

W3/Q3. Human annotation by non-authors to validate the judge.

Four non-author annotators validated the judge, and they agree with it on 88 of 96 task-framing labels (κ = 0.83; new; App. I.1).

The annotators are four researchers in NLP and AI safety, three in industry and one doctoral student. None is an author or took part in the project, and none was told the paper's hypotheses. Each annotator works alone from the paper's rubric (Appendix D), after six practice examples with feedback, and without seeing model names or the judge's labels. Two annotators label every example, and half of the 48 summaries are ones our judge labeled task-framed.

Table R3. Agreement of the non-author annotators with the judge and with each other (from Table 27 in the revision).

LabelAgreement with the judgeAgreement between the two annotators
Task framing: is the instruction presented as part of the task?92% (88 of 96 labels)92% (44 of 48 summaries)
Retention: how much of the instruction does the summary keep?91% (87 of 96 labels)94% (45 of 48 summaries)
Response: does the agent's response follow the instruction?96% (23 of 24 labels)92% (11 of 12 responses)
Stated source: where does the summary say the instruction came from?58% (14 of 24 labels)42% (5 of 12 summaries)

Agreement is lowest on stated source. The annotators agree with each other on 5 of 12 summaries and with the judge on 14 of 24 labels. The judge estimates that 39% of task-framed summaries name the tool output as the source (Section 4.1). Reweighting with the annotators' labels gives 32%, with a 95% interval of 15% to 51%.

Additionally, several key outcomes, such as AgentDojo's action targets and DTap's attack success, do not require an LLM judge.

AuthorsOpenReview ↗

Follow-up to Reviewer s4yf

We sincerely thank the reviewer again for their time and thoughtfulness in their review, which helped us clarify and improve our work. We have revised the manuscript in response to each point and hope our replies have answered their questions.

We would be grateful for any further feedback on the revision, and we are happy to address any remaining questions or continue the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.