acceptodds
Under review as a conference paper at ICLR 2027

Twin Agent: Context Residual Compression for Privilege Separated Agents

Abstract

Large language model (LLM) agents are vulnerable to security risks, such as prompt injection attacks from untrusted context that manipulate downstream reasoning and tool use. Existing secure-by-design approaches mitigate this risk by separating untrusted observations from privileged execution and careful control of information flow, but often degrade utility and require extensive task-specific engineering. We thus propose Twin Agent, a general privilege separation design pattern inspired by residual coding in the agent context. Twin Agent consists of two nearly symmetric agents: an Explore Agent that inspects untrusted information and a Safe Agent that executes privileged actions. The Explore Agent is conditioned on the Safe Agent's current context and communicates only compact hints to the Safe Agent about the next action to take. This design reduces the information needed to preserve task utility and thus achieves a better security–utility tradeoff, which we empirically verify by measuring how utility and attack success change as the length of hints varies. We evaluate Twin Agent on long-horizon software engineering tasks with SWE-bench Lite and on heterogeneous multi-tool interaction tasks with AgentDojo and DecodingTrust-Agent. Across both benchmarks, Twin Agent preserves high task utility while preventing prompt injection attacks, outperforming both undefended agents and privilege separation baselines.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, Prompt Injection, AI Agent Security, Indirect Prompt Injection, Agentic AI Safety, Privilege Separation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XoiAOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Twin Agent, a framework consisting of two symmetric agents with similar architectures, namely the Explore Agent and the Safe Agent, to defend large language model (LLM) agents against prompt injection attacks. The authors evaluate Twin Agent on SWE-bench, AgentDojo, and DecodingTrust-Agent. The experimental results demonstrate that Twin Agent achieves low attack success rates while largely preserving task utility.

Critical strengths

(1) The problem is relevant to the security of tool-using agents. The architecture is simple and does not require major changes to existing agent workflows. (2) The evaluation covers three benchmarks and includes adaptive attacks. The results show that restricting information flow can substantially reduce attack success rates.

Critical weaknesses

(1) The security argument in Section 4.3 is questionable. The authors claim that the hint channel is the only pathway through which untrusted information can influence the Safe Agent. However, in the SWE-bench implementation, the Safe Agent can read repository files and receive shell command outputs, both of which may also contain untrusted content.

(2) The adaptive attack results are less convincing than the paper suggests. The attack succeeds in only 11 out of 232 instances. The authors rerun these instances five times and report no additional successes, but this does not explain why the initial attacks failed.

(3) The baseline comparisons are not entirely fair. The dual-agent baseline prohibits communication between agents, resulting in zero task utility on SWE-bench-injected. In the comparison with CaMeL, the authors reduce its retry budget from 10 to 3 and acknowledge that this reduction negatively affects its utility. Nevertheless, the resulting numbers are used to support the claimed advantage of Twin Agent in the security–utility trade-off, which raises concerns about the fairness of the comparison.

(4) The presentation quality of the figures and tables is rather unprofessional (Figures 1, 2, and A2, and Table 3), making them difficult to read and interpret.

Questions for authors

(1) The main contribution appears to be Residual Compression, but its implementation seems to rely on context-dependent hint generation with a length constraint. The paper lacks comparisons with existing methods under identical communication budgets and defense settings. Without such comparisons, it is difficult to attribute the observed improvements to the proposed mechanism.

(2) Section 4.3 treats the hint channel as the only attack surface, whereas Section 4.1 allows the Safe Agent to inspect repository files and shell outputs. This creates some confusion regarding the threat model and its underlying assumptions.

(3) The results for CaMeL should be reported using its original retry budget, or at least compared under matched computational constraints.

2 comments

We thank the reviewer for the constructive feedback and thoughtful suggestions.

1. SWE-bench trust boundary (W1/Q2)

Please see Section 4.1 and Appendix A for the threat model in SWE-bench-injected. Repository files and shell command outputs are trusted sources because we protect the repository, while the attacker injects through the GitHub issue description.

2. Adaptive attack interpretation (W2)

LLMs sample next tokens with probabilities. Attacks can succeed with some probability rather than always winning or always losing. We rerun the attack five times and find no additional success, showing the attack patterns aren't strong enough to commit a stable attack.

3. Baseline fairness (W3/Q3)

The Dual Agent baseline is a typical design pattern that prohibits communication between agents, so that it can ensure security by design. However, this design pattern may fail on complex tasks, such as SWE-bench. Our experiment provides practical evidence for this weakness.

For CaMeL, we explicitly disclosed the reduced retry budget (max_attempts=3 instead of 10) in Appendix A. CaMeL is computationally expensive, and full evaluation with 10 retries exceeded our available budget. We also conducted preliminary comparisons on a smaller subset of tasks in our environment and did not observe a meaningful utility difference between these settings. Furthermore, even using the approximately 35% utility reported in the original CaMeL paper instead of our measured 30.9%, Twin Agent still achieves substantially higher utility (62.9%).

4. Contribution of residual compression (Q1)

Residual compression is not merely enforcing a short hint. The Explore Agent conditions its message on the Safe Agent's current history and communicates only the missing information needed for the next decision, rather than summarizing the full untrusted input. This distinction is formalized in Eqs. (11)–(12) and reflected in the prompts in Appendix E, including the option to output HINT: null when no additional information is needed.

Few methods limit hint budgets. Existing dual-agent design patterns allow no communication between the two agents, which we have already implemented and compared. We also compared with CaMeL on AgentDojo, which allows careful information control. Adapting CaMeL to SWE-bench is nontrivial, as it requires manually designed security policies and tool-specific integration to support open-ended, iterative debugging tasks.

5. Presentation (W4)

We appreciate the feedback and will improve the readability and labeling of Figures 1, 2, and 3, and Table 3. We would appreciate it if the reviewer can elaborate on this.

We thank the reviewer again for the constructive feedback. As the discussion period is coming to an end, we would greatly appreciate it if you could take a look at our rebuttal and let us know whether it addresses your concerns. We would be happy to provide any further clarification.

Reviewer_T5tfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes Twin Agent, a privilege-separated LLM-agent architecture in which an Explore Agent observes untrusted content but cannot execute privileged actions, and a Safe Agent executes privileged actions but receives only bounded, residual hints from the Explore Agent. The central idea—conditioning the Explore Agent on the Safe Agent’s current trajectory and communicating only task-relevant residual information under a length budget—is intuitive and relevant to secure agent design. The paper evaluates on SWE-bench-injected, AgentDojo, and DecodingTrust-Agent.

Critical strengths
  1. The architecture is clearly motivated and easy to understand. The residual-hint idea is a reasonable relaxation of strict privilege separation and preserves the ReAct-style interaction pattern.
Critical weaknesses
  1. The base argument of the proposed method is "attacking the latest, strongest models requires long injections, so the attacker needs to send a lot of information to succeed". This is interesting, but no evidence is provided in the paper.
  2. The security claims are stronger than the evidence supports. Twin Agent does not eliminate prompt injection by design. It reduces the attack surface and then relies on a length limit plus an LLM-based prompt-injection detector. The Explore Agent is fully compromised in the copy-and-paste phase, and malicious hints pass the detector in 57/232 instances.
  3. Baseline coverage and fairness are incomplete. In Table 1, merely CaMeL is a SOTA method and it's only evaluated on AgentDojo. More baselines should be compared, such as [1,2,3].
  4. Attack coverage is limited. AgentDojo uses only the default “important_instructions” attack and no adaptive attack. DecodingTrust-Agent uses a restricted subset and only one attack style. Given that the paper argues the defense is general across heterogeneous agent systems, broader attack coverage and cross-model evaluation would be important.

References:

[1] Chen, S., Piet, J., Sitawarin, C., & Wagner, D. (2025). {StruQ}: Defending against prompt injection with structured queries. In 34th USENIX Security Symposium (USENIX Security 25) (pp. 2383-2400).

[2] Wu, T., Zhang, S., Song, K., Xu, S., Zhao, S., Agrawal, R., ... & Zhou, W. (2025, May). Instructional segment embedding: Improving llm safety with instruction hierarchy. In International Conference on Learning Representations (Vol. 2025, pp. 93728-93758).

[3] Chen, Y., Li, H., Zheng, Z., Wu, D., Song, Y., & Hooi, B. (2025, July). Defense against prompt injection attack by leveraging attack techniques. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 18331-18347).

Questions for authors

Please reply to the Weaknesses.

4 comments

We thank the reviewer for the constructive feedback and thoughtful suggestions.

1. Evidence for the role of hint length (W1)

The statement that no evidence is provided is inaccurate. Section 5.3 and Figure 3a explicitly evaluate utility and adaptive attack success across hint budgets from 0 to 200 characters (and an unlimited-budget setting). The attacks use PAIR with 100 iterations and 10 repetitions. Appendix B (Fig. A1) further ablates the defense components across budgets.

We do not claim that every attack against a frontier model requires a universally long injection. Rather, our empirical hypothesis is that restricting the communication budget makes successful attacks harder while residual compression preserves task-relevant information.

2. Security guarantees and the detector (W2)

We agree that Twin Agent does not eliminate prompt injection by design or offer a formal security guarantee. We did not make such a claim. Thank you for pointing this out. We will clarify that the architecture reduces the attack surface by design in the revised paper.

However, the defense does not reduce to detector accuracy. The attacker must both evade the detector and induce the Safe Agent to perform an unauthorized action within the same short hint. A benign hint can easily pass the detector but does not achieve the attack goal; a more explicit attack may be effective but easier to detect. This joint constraint is central to our security argument in Section 4.3.

In fact, we already consider a threat in the main text where the Explore Agent is fully compromised, and our experiments directly support this distinction. In Table 3, the attack against the Explore Agent has 100% copy-and-paste success, meaning that the Explore Agent is fully compromised. Malicious hints pass the detector at least once in 57/232 instances, while only 11/232 instances achieve the malicious objective (4.7% ASR).

3. Additional baselines (W3)

Our main comparisons isolate architectural approaches to privilege separation (Dual Agent and CaMeL), because typical model-level defenses are available only on small-to-mid-sized models, limiting their utility on complex tasks such as SWE-bench. We agree that model-level defenses are also useful points of reference. We conducted additional experiments with Meta-SecAlign as a SOTA baseline for model-level defenses.

On SWE-bench, Meta-SecAlign-8B and Meta-SecAlign-70B showed 0.0% and 5.5% utility on SWE-bench due to their limited capabilities, compared to 62.5% utility for Twin agent.

On AgentDojo, here are the results (Meta-SecAlign-70B is finetuned based on Llama-3.3-70B-Instruct):

Model / MethodBenign utilityAttack success rate
Gemini-2.5-Flash + plain agent61.9%41.5%
Gemini-2.5-Flash + Twin Agent62.9%0.11%
Llama-3.3-70B-Instruct + plain agent52.6%26.8%
Meta-SecAlign-70B + plain agent84.5%1.9%
Llama-3.3-70B-Instruct + Twin Agent46.4%0.53%
Twin Agent achieves the lowest ASR in both model groups while largely preserving the utility of the corresponding base models. In particular, it reduces ASR below that of Meta-SecAlign-70B without requiring any model fine-tuning.

Adapting CaMeL to SWE-bench is nontrivial, as it requires manually designed security policies and tool-specific integration to support open-ended, iterative debugging tasks. Moreover, the 0.0% utility of the strictly separated Dual Agent baseline in Table 1 highlights the difficulty of preserving task utility without flexible information flow.

4. Attack coverage (W4)

We agree that broader evaluation would strengthen our results. We evaluated standard prompt-injection attacks on AgentDojo and DecodingTrust-Agent, and adaptive attacks on SWE-bench, across three models. Broader attack and model evaluations were limited by our computational budget.

We thank the reviewer again for the constructive feedback. As the discussion period is coming to an end, we would greatly appreciate it if you could take a look at our rebuttal and let us know whether it addresses your concerns. We would be happy to provide any further clarification.

Thanks for the reply. However, it doesn't address the proposed weaknesses.

W1: I don't think my words are hard to understand, but the authors twisted my opinion to refute me. I don't think GPT-5.2 can represent "the latest, strongest models" at the deadline of ICLR, when even GPT-6 Astra is released. GPT-5.2 is five generations behind GPT-6.

W2: The authors acknowledge the over-claimed statement.

W3: The authors merely add one result of a new baseline. The new baseline significantly outperforms the authors' method in terms of benign utility, while the attack success rate is likely low.

W4: This weakness is not tried to address.

Thank you for your feedback. We would like to clarify a few points.

W1. We apologize for the imprecise wording and will remove the claim about "the latest, strongest models." Our empirical findings are limited to the models evaluated in our paper.

W3. Meta-SecAlign-70B is obtained by additionally fine-tuning Llama-3.3-70B-Instruct, with a training procedure designed to preserve benign task utility. In contrast, Twin Agent is a training-free architectural defense that aims to preserve, rather than improve, the underlying model's utility. Therefore, we believe the appropriate utility comparison is against the corresponding base model: Twin Agent achieves 46.4% utility versus 52.6% for Llama-3.3-70B-Instruct, while reducing ASR from 26.8% to 0.53%. Its ASR is also lower than Meta-SecAlign's 1.9%.

W4. As explained in our previous response, we follow standard benchmark evaluation protocols. AgentDojo's "important_instructions" attack subset is a typical setting used in related prior studies. DecodingTrust-Agent includes many tasks that do not involve prompt injection, so we focus on the CRM and Finance suites, which exhibit the highest prompt-injection success rates under environment injection, as explained in Appendix A.

Reviewer_iK8iOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an architecture called Twin Agent based on the principle of privilege separation, designed to defend LLM agents against prompt injection attacks in open environments. The approach splits the agent into an Explore Agent, tasked with observing the untrusted environment, and a Safe Agent responsible for executing privileged operations. It further incorporates a residual compression mechanism, through which the Explore Agent transmits only compact, context-aware hints to the Safe Agent.

Critical strengths
  1. This paper models the design of secure agents as a compressed information flow problem and introduces the concept of residual communication, offering a fresh perspective on privilege-separation architectures.
  2. As a general design pattern, the Twin Agent approach can be seamlessly integrated into existing, mainstream agent workflows—such as ReAct—without the need for cumbersome task-specific protocol engineering or hard-coded communication interfaces, making it highly practical for real-world applications.
  3. The authors conducted a comprehensive evaluation using SWE-bench, AgentDojo, and DecodingTrust-Agent. Furthermore, they designed adaptive attacks based on the PAIR framework to test robustness under worst-case scenarios, thereby demonstrating the effectiveness of their proposed method.
Critical weaknesses
  1. The paper sets empirical limits on hint lengths for different tasks; however, it lacks theoretical guidance or an adaptive mechanism for automatically determining or dynamically adjusting the optimal information budget.
  2. When comparing against the CaMeL baseline, the authors reduced the max_attempts parameter to improve computational efficiency, which led to a significant drop in the baseline's utility. This artificial weakening of the baseline may result in an overestimation of the relative advantages of the Twin Agent framework.
  3. A core security assumption of the Twin Agent framework is that short injections are unlikely to bypass the detector and compromise the frontier model. However, if the Explore Agent is fully compromised (a 100% success rate for an attacker), the system's security effectively degrades to the binary classification capability of the prompt-injection detector used by the Safe Agent. Once this detector is bypassed, the entire line of defense collapses.
  4. The concept of residual compression relies on the assumption that the Safe Agent’s current contextual trajectory is correct. If the Safe Agent reaches a dead end early on due to a lack of information, the residual hints generated by the Explore Agent, based on that erroneous trajectory,may fail to provide the necessary corrective guidance.
  5. Although the paper notes that API costs roughly double, it does not sufficiently address the impact of the dual-agent architecture on practical deployment and user experience, specifically regarding the increased inference latency and the higher number of interaction turns, which are particularly relevant for long-horizon tasks like those in SWE-bench.
Questions for authors
  1. How can the optimal hint character budget for various task scenarios be determined theoretically or automatically, rather than relying on empirical, manual grid search?
  2. Does reducing the number of retries for the CaMeL baseline lead to an underestimation of its utility? Is it possible to provide a fair, head-to-head comparison under an equivalent computational budget?
  3. When the Safe Agent generates an erroneous reasoning trajectory due to a lack of initial information, how does the Explore Agent’s residual mechanism prevent the accumulation of errors and facilitate effective course correction?
  4. For open-source models with limited instruction-following capabilities, do the strict format constraints required by the Twin Agent framework remain stable and effective?
2 comments

We thank the reviewer for the constructive feedback and thoughtful suggestions.

1. Hint budget selection (W1/Q1)

Thank you for suggesting we develop a theory and introduce an adaptive mechanism to automatically adjust the hint budget. Both are hard and interesting improvements.

We don’t have a theoretical basis or an automatic mechanism for determining the hint budget.

2. CaMeL comparison (W2/Q2)

We explicitly disclosed the reduced retry budget (max_attempts=3 instead of 10) in Appendix A. CaMeL is computationally expensive, and full evaluation with 10 retries exceeded our available budget.

We also conducted preliminary comparisons on a smaller subset of tasks in our environment and did not observe a meaningful utility difference between these settings.

Furthermore, even using the approximately 35% utility reported in the original CaMeL paper instead of our measured 30.9%, Twin Agent still achieves substantially higher utility (62.9%).

3. Security beyond the detector (W3)

“Once this detector is bypassed, the entire line of defense collapses” is not true. An attacker must simultaneously evade the detector and induce malicious behavior in the Safe Agent under a restricted communication budget. Bypassing the detector alone is trivial with a benign hint, but such a hint would not achieve the malicious objective. Conversely, more explicit malicious instructions may be more effective but also more detectable.

In fact, we have already considered such threats in the main text, and our experiments directly support this distinction. In Table 3, the attack against the Explore Agent has 100% copy-and-paste success, meaning that the Explore Agent is fully compromised. Adaptively optimized malicious hints pass the detector at least once in 57/232 instances, while only 11/232 instances achieve the malicious objective (4.7% ASR).

4. Incorrect trajectories (W4/Q3)

“The concept of residual compression relies on the assumption that the Safe Agent’s current contextual trajectory is correct.” is incorrect; it does not make that assumption. The Explore Agent observes the current trajectory to determine what information is missing, including information needed to correct an erroneous hypothesis. It cannot provide instructions directly; otherwise, the detector or the safe agent will block the hint.

Whether correction succeeds is an empirical question. Our results show that Twin Agent preserves SWE-bench utility (62.5% success rate versus 61.2% for the original agent), while Appendix C (Fig. A2) provides examples of reduced trajectory deviation with residual hints.

5. Computational overhead (W5)

As reported in Figure 3b on API costs, this is approximately 2× the original SWE-Agent cost at 50–100-character budgets. We acknowledge that additional cost and latency overhead may have an impact on practical deployment and user experience.

6. Open-source models (Q4)

Hint length is enforced programmatically, independently of model instruction-following. As a result, the strict format constraints required by the Twin Agent framework can remain stable and effective.

We thank the reviewer again for the constructive feedback. As the discussion period is coming to an end, we would greatly appreciate it if you could take a look at our rebuttal and let us know whether it addresses your concerns. We would be happy to provide any further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.