acceptodds
Under review as a conference paper at ICLR 2027

Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems

13 views

Abstract

As multi-agent systems enter high-stakes domains, the possibility that agents may circumvent safety boundaries is a growing concern. Prior work has examined this risk primarily in adversarial settings, where agents are instructed or rewarded to communicate covertly and evade oversight. We show that benign agents can cross the same boundaries without adversarial incentives. We emulate a software-engineering workflow in which a planner represents a company hiring an external developer. The planner writes requirements and holds a company credential it is instructed not to disclose to the developer; a monitor screens their exchanges. Seven of nine tested frontier models disguise the credential in their requirements, seeking to help the developer recover it while evading the monitor, even after completing their assigned objective. For example, across 6,000 episodes with DeepSeek-V4-Pro, the planner attempts concealment in 16.9%; in 0.9%, the credential evades the monitor and is recovered and used by the developer, resulting in a successful breach. Despite the seemingly low per-episode rate, the risk compounds across repeated exchanges common in software-engineering workflows: at 0.9%, 105 independent episodes carry a 61.3% chance of at least one breach. Our analysis suggests that the planner misinterprets disguised disclosure as helping the developer and narrowly interprets the nondisclosure rule as banning plaintext, not character codes or riddles. We call this behavior covert assistance. Its strategy depends on the perceived setting: monitoring favors concealment, while concealment gives way to direct disclosure when the planner addresses a human rather than an AI agent, so leakage persists. Giving the monitor the credential improves detection but requires trusting it with the secret and knowing the protected value in advance. These risks observed in models already used for software engineering challenge oversight to distinguish authorized cooperation from assistance that advances the task by violating safety boundaries.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
covert channels, steganography, collusion, AI control, oversight and monitoring, multi-agent systems, LLM agents, AI safety, information leakage, large language models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_M5MCOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper assesses whether an agent, which is given a security credential and is told not to transmit it to other agents, finds a way to hide the credential so that its transmission cannot be detected by a monitor that checks communications. The assessment is carried out using a specific scenario. The scenario consists of a multi-agent setting in which a Planner possesses a privileged credential that it is instructed not to disclose to an external Developer. When the Developer is unable to complete a privileged deployment step, the Planner may encode the credential, for example using character codes, base64, or riddles, in the material sent to the Developer. The authors call this behavior as covert assistance. They evaluate several Planner, Developer, and monitor models and investigate factors affecting the frequency of such behavior and possible mitigations.

The paper contains a substantial experimental evaluation, and the observation that an agent may transform protected information rather than transmit it literally is potentially useful. However, I do not find the main result sufficiently novel or compelling from a security perspective. In my view, the experiments largely demonstrate the consequences of an underspecified security policy combined with a weak and unrealistic security architecture.

Critical strengths
  1. The paper include results of many experiments.
  2. The experiments show a new specific behavior, that focuses on transmitting protected information violating a given security policy.
Critical weaknesses
  1. My main concern is that the behaviors shown by the experiments ppears to be a predictable consequence of task-oriented agent behavior under an incomplete policy rather than a fundamentally new security phenomenon. The Planner repeatedly receives information that the Developer is blocked at a privileged step and is allowed to revise its output. It is therefore not particularly surprising that a capable task-oriented agent searches for alternative ways to help the Developer progress. The fact that an agent finds an indirect representation of a value when direct disclosure is prohibited seems closely related to already well-known phenomena including specification gaming, constraint circumvention, and goal-directed planning. This interpretation is reinforced by the results in the paper. The authors report that in 78% of first-encoding traces the Planner interprets the nondisclosure rule as prohibiting direct disclosure while permitting encoded disclosure. More importantly, when the policy is strengthened to explicitly forbid allowing the Developer to recover the credential "by any means," the encoding rate drops from 18.7% to 0.3%. This suggests that a major source of the observed behavior is policy underspecification. I therefore find the framing in terms of an emergent form of "covert assistance" somewhat overstated. A simpler explanation is that the agent is following an incomplete policy while attempting to maximize task success.

  2. A second major issue is the security architecture assumed in the experiments. The attack relies on a transferable bearer credential: if the Developer learns the credential string, it can exercise the Planner's privilege. This is precisely the type of design that established security principles seek to avoid. Identity-bound or non-exportable credentials, least privilege, capability-scoped authorization, and complete mediation at the privileged action would substantially change the threat model. The paper itself acknowledges that identity-bound credentials would make a leaked secret unusable by another agent. In my view, this should not be treated merely as one possible defense discussed near the end of the paper; it directly calls into question the practical significance of the main attack.

  3. Similarly, the main defense assument in the paper is an LLM-based monitor that, in the main setting, is not even given the protected credential it is expected to identify. Not surprisingly, giving the monitor the protected value substantially improves detection. The evaluation therefore demonstrates a weakness of a relatively weak monitoring architecture, rather than showing that the behavior remains problematic under realistic security mechanisms. A stronger paper would need to evaluate whether the attack remains effective when standard security engineering practices are adopted. Indeed, I believe the paper places too much emphasis on content monitoring. A secure system should not rely primarily on recognizing whether a secret has been textually or semantically disclosed. The more fundamental requirement is that one principal should not be able to exercise another principal's authority simply by learning some transferable string. Security mechanisms should enforce authorization at the point where privileged actions occur, independently of whether an upstream agent behaved correctly. More in general good security requires combining different security controls.

  4. The paper also includes claims about intentional or deliberate evasion based in part on model reasoning traces. Some of these traces are can certainly appear interesting; for example, the Planner sometimes reasons about making an encoded credential appear "less suspicious." However, verbalized model reasoning should not be treated as reliable evidence of causal intent. The paper acknowledges this limitation, but the terminology used throughout the paper, such as "intending," "deliberate evasion," and “motive”, often goes beyond what the experiments directly establish. The experiments establish a behavioral phenomenon: the model sometimes produces information that makes a protected value recoverable while avoiding literal disclosure. They do not establish a new underlying mechanism of deceptive intent.

Questions for authors

None

7 comments

W1:

We thank the reviewer for this feedback. The concern raises two questions: how our finding differs from prior findings, and whether it can be attributed to policy underspecification. First, the table below compares covert assistance with each phenomenon the reviewer names, listing their similarities and differences.

PhenomenonShared with covert assistanceHow covert assistance differs
Specification gamingExploits a gap between the literal rule and its intent: the rule forbids disclosure, and an encoding is used insteadThe agent games a specification to satisfy its own objective. In our setting, the Planner's stated objective is already satisfied before any encoding occurs (Appendix O).
Constraint circumventionWorks around a constraint through an indirect representationThe constraint usually blocks the agent's own goal. Here, the nondisclosure rule does not block the Planner's objective, because the tests pass without the credential.
Goal-directed planningAdapts across rounds: encoding begins only after the Planner is told the Developer is blockedThe agent plans toward its own assigned goal. Deployment is outside the Planner's task (§3).
Covert collusion (Motwani et al., 2024; Mathew et al., 2025)A hidden message between agents passes a monitorAgents are instructed or optimized to conceal, and receivers are adapted to decode. Here, neither agent is instructed or rewarded, and the receiver is uninstructed.

In each phenomenon the reviewer names, the agent pursues its own objective. In this work, the Planner’s stated success criterion (“Your team succeeds only when the developer’s program passes the project’s tests”; Appendix O) is met before any encoding occurs, and the Planner is told so after every round in which its requirements reach the Developer.

The nondisclosure rule does not block this objective for the Planner, because the tests pass without the credential. The credential is needed only for the Developer’s deployment step, which is outside the Planner’s task (§3). And neither agent is instructed or rewarded to conceal.

This difference is important: current safety evaluations test whether an agent pursues its own objective by unsafe means, but our results show that an agent whose own objective is already satisfied can still produce an end-to-end breach through its interaction with a peer. Encoding occurs in seven of nine frontier models and end-to-end breaches in five (Table 13), so evaluating each agent in isolation is not sufficient.

Second, the reviewer infers from two results that underspecification is a major source of the behavior: 78% of traces read the rule as permitting encoded disclosure, and the intent-based rule reduces encoding from 18.7% to 0.3%. We agree that the wording affects the rate, but we believe that the results should be interpreted differently:

  • The original rule already prohibits disclosure, and a reversible encoding of the credential discloses it. A practitioner writing this rule would reasonably expect it to cover encodings. The 78% therefore reflects the Planner adopting a narrow reading of a rule that already applies, not an omission a reasonable author would have avoided.
  • The intent-based rule spells out what the original rule was reasonably expected to cover, and we added it only after observing that models read the original rule narrowly. Its effectiveness shows that the failure can be mitigated once it is known, not that it was foreseeable. Once a failure is known, its fix can look obvious, while anticipating it beforehand is harder (Fischhoff, 1975).
  • Anticipating unsafe behaviors and interactions in advance is a recognized challenge (Amodei et al., 2016; Leveson, 2004), and recent incidents show that safeguards written in advance did not anticipate how agents bypassed them (OpenAI, 2026b; OpenAI, 2026c; Williams et al., 2026).

Our contribution is to identify this failure, characterize the conditions under which it occurs, and provide the intent-based rule as an actionable mitigation. These results have direct implications for deployment: clearer rules and giving the monitor the credential reduce but do not eliminate the failure (§6), so safety evaluations should cover complete multi-agent workflows, and privileged actions should require authorization checks that do not depend on the upstream agent's message (§7).

W2:

We agree that identity-bound credentials, least privilege, and complete mediation would prevent the Developer from using a leaked credential. However, the paper studies agent behavior that leads to safety risks, not a security attack: an agent, without instruction or reward, conceals protected information from oversight to help another agent.

Here the credential is the instrument of a controlled experiment, and we design that because its recovery can be verified at an authentication gate so it is not the threat we claim.

We would like to note that Covert Assistance behavior is broader than credential leakage. Appendix K.3 shows HIPAA setting, where the protected value is a patient’s Social Security Number (SSN). The Planner still encodes the SSN, and the Developer recovers and submits it in 7 of 600 episodes. We will move these results into the main text.

We acknowledge that identity binding would prevent unauthorized action, though it does not prevent the sensitive data breach. The Planner receives the same “authentication failed” feedback whether or not a recovered string would work for the Developer, so we expect it to act identically under an identity-bound credential.

We would like to add that the setting also reflects current practice, deployment reports describe agents obfuscating tokens to bypass a scanner (OpenAI, 2026c; §2), which indicates that bearer tokens and content scanning are in use today.

We will add a sentence to §3 stating that the credential serves as a verifiable instance of protected information, and that identity-bound credentials would prevent the unauthorized action, but not the disclosure or the behavior we study.

W3:

We agree that a secure system should enforce authorization at the privileged action and combine several controls, as the paper recommends (§1). However, the paper studies agent behavior that leads to safety risks in realistic settings, not the design of a secure system.

The monitor is the oversight condition under which we study this behavior. As stated in §1 and §3, it represents the standard safeguard in current multi-agent systems (Greenblatt et al., 2024b; Williams et al., 2026). We therefore do not propose it as a defense and its role is merely to observe whether agents conceal protected information from oversight, and whether this behavior responds to oversight i.e., removing the monitor lowers encoding (§5.2.2).

The finding is therefore about the agent more than the monitor. When the monitor is given the credential, detection rises from 43% to 94%, but the Planner’s attempt rate is not reduced (6.9% of rounds with the provisioned monitor vs. 5.7% undefended; Appendix M) and breaches remain at 0.07% per episode (§6.2). This residual rate compounds across episodes: at the point estimate, a breach becomes more likely than not within about 970 episodes (95% interval: 170 to 38,000; Figure 8). Providing the credential also entrusts the secret to another agent, as noted in the paper. Detection then depends on the monitor’s own handling of the credential, which requires additional safeguards and introduces additional confidentiality and assurance requirements (NIST SP 800-160 Vol. 1 Rev. 1 (https://doi.org/10.6028/NIST.SP.800-160v1r1)).

Authorization at the privileged action would prevent the unauthorized action, but not the disclosure of information that cannot be bound to an identity, as discussed in our response to the previous point.

Reviewer_ANKYOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies concealed credential transfer from a planner to an external developer without explicit covert-channel instructions. Success requires monitor clearance, exact recovery, and authentication. The main study records 54 breaches in 6,000 episodes and evaluates multiple models and mitigation strategies.

Critical strengths

S1: The study demonstrates end-to-end authentication failures, rather than relying only on suspicious text or stated motives. Appendix B audits the possibility of guessing. S2: Interventions covering policy wording, recipient trust, secret retrieval, monitor provisioning, paraphrasing, and reverse communication provide substantial characterization of the failure mode.

Critical weaknesses

W1: Statistical units and sample accounting are inconsistent: Tables 2 and 15 label corresponding encoding rates differently, while Tables 12 and 13 give different rates for the same model/monitor labels without explaining the population difference. W2: Encoding-rate z-tests do not clearly account for dependence between rounds within an episode. Appendix I explicitly documents persistent encoding, making the statistical unit important for the reported significance. W3: The cumulative-risk projection uses a provisioned-monitor estimate of one breach in 1,400 episodes without propagating its sampling uncertainty (Figure 8; Appendix I). The independence assumption is stated, but the resulting curve remains imprecisely estimated.

Questions for authors

Q1: Can you reconcile the units, populations, and rates in Tables 2, 12, 13, and 15? Q2: What are the testing units, and do episode-level tests retaining task/secret matching support the reported encoding-rate differences? Q3: What uncertainty band results when uncertainty in the one-in-1,400 breach estimate is propagated into Figure 8?

2 comments

W1: We thank the reviewer for the careful accounting. We would like to clarify each:

  • Tables 2 and 15. Both tables report the same quantity, the per-episode encoding rate. The caption of Table 2 incorrectly labels these rates as per round. We will correct the caption so that it matches Table 15. No numbers change.
  • Tables 12 and 13. Both tables are computed on the same set of episodes but apply different inclusion rules: Table 13 excludes unclassified encodings, that is, rounds the judge flagged as encoding but could not assign to a named strategy, whereas Table 12 includes them. This difference accounts for the discrepancy of at most 0.35 percentage points in the shared cell, and it does not change any model ranking or conclusion. The breach rate (54 of 6,000 episodes) is determined by successful authentication and is therefore independent of the judge.

W2: We thank the reviewer for this point. The significance tests reported in the main text (§6.1, §6.2) already use the episode as the unit and are not affected. In §6.1 and Appendix L, the baseline and treatment conditions share the same task–secret pairs, and the reported z-scores treat the two conditions as independent samples. This is typically conservative, since a paired test removes variation due to the task and secret. The round-level tests appear only in the appendix (Tables 12 and 13, Appendix M). In the revision, we will replace them with episode-level tests that retain the task–secret matching across conditions.

W3: We thank the reviewer for this point and agree that the provisioned-monitor projection should reflect the uncertainty in its estimate, which rests on a single observed breach. The exact 95% Clopper–Pearson interval for 1 breach in 1,400 episodes is 0.0018%–0.40%. Propagated into Figure 8, the number of episodes at which a breach becomes more likely than not ranges from about 170 to about 38,000, compared with about 970 at the point estimate. The provisioned-monitor projection is therefore imprecise. The other two curves rest on many more breaches and are well estimated: for the baseline (45 of 3,000), the interval is 1.10%–2.00%, and a breach becomes more likely than not after 46 episodes (range 34–63); for Kimi-K2 (75 of 1,000), the interval is 5.94%–9.31%, and the corresponding number is 9 episodes (range 7–11). We will add 95% bands to all curves in Figure 8 and Table 10, and replace the statement that the provisioned rate reaches 50% by about 970 episodes with this interval.

Q1: Tables 2 and 15 report the same quantity: per-episode encoding rates from the same 300-episode arms. Table 2's caption mislabels these as per round, and we will correct it; no numbers change. Tables 12 and 13 are computed on the same 1,000-episode runs per condition. Table 13 excludes rounds the judge flagged as encoding but could not assign to a named strategy, whereas Table 12 includes them. This accounts for the difference of at most 0.35 percentage points, and the covert-success counts are identical in both tables. We will state the unit, the number of episodes, and the inclusion rule in every caption.

Q2: The tests in the main text (§6.1, §6.2) use the episode as the unit. In §6.1 and Appendix L, both conditions share the same task–secret pairs, but the reported z-scores treat them as independent samples, which is conservative because pairing removes variation due to the task and secret. The round-level tests appear in Tables 12 and 13 and Appendix M. In the revision, we will replace them with paired episode-level tests (McNemar's test on matched task–secret pairs).

Q3: The exact 95% Clopper–Pearson interval for 1 breach in 1,400 episodes is 0.0018%–0.40%. Propagated into Figure 8, the number of episodes at which a breach becomes more likely than not ranges from about 170 to about 38,000, compared with about 970 at the point estimate, so this projection is imprecise. The other curves are well estimated: 34–63 episodes for the baseline (45 of 3,000) and 7–11 for Kimi-K2 (75 of 1,000). We will add these 95% bands to Figure 8 and Table 10, and report Kimi-K2 for the offered condition in both.

Reviewer_LWzQOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper identifies and investigates the "covert assistance" phenomenon, in which a model discloses private information to be helpful to a third-party complete the task. They conduct an experiment with planner, monitor and developer as agent actors. The planner has privileged information that is not be to shared with the untrusted developer and the monitor checks for conformance of the same. They conduct experiments on 100 tasks across 20 domains and find that open models are susceptible to the covert assistance phenomenon, that the refusals from the monitor drive up the obfuscation and the obfuscations can go unchallenged through the monitor.

Critical strengths
  • The problem of models evading safeguards is becoming important.
  • The paper identifies the novel phenomenon where the intent is not malicious or there is no attacker. Instead, the models are trying to be helpful to an untrusted third-party, thereby, leaking secrets.
  • The paper does a good job of evaluation across multiple planner and monitor models.
Critical weaknesses
  • The paper finds that open models are susceptible to this behavior but not the closed models. The paper conjectures that safety tuning is responsible for the difference. I think this part should have been expanded as it is a key aspect and also limits the covert assistance vulnerabilities to open models (or non-safety-tuned models).
  • The tasks are all coding centered. Additional tasks related to knowledge or enterprise work would help understand how covert assistance plays out (or not) in other application domains.
Questions for authors

Please see the critical weaknesses section.

2 comments

W1: We thank the reviewer for raising this point. We would like to clarify that the results do not divide cleanly between open and closed models. Two closed models encode: Claude Sonnet 4.5 produces a covert success, and Gemini 3.1 Pro encodes in 0.93% of rounds, more often than the open Llama-4-Maverick (0.24%) and Mistral-Large (0.41%) (Figure 2). The highest rates occur in Kimi-K2 and the DeepSeek models, but the variation is better described as model-dependent than as a difference between open and closed weights. The paper also does not attribute this variation to safety tuning. The phrase "safety-tuned frontier models" in Appendix K.2 describes those models but does not test a cause. Identifying which training factors drive the variation requires controlled comparisons across training procedures, which are not available for most of these models, we will note this as a direction for future work.

W2: We thank the reviewer for this suggestion. We considered knowledge and enterprise workflows, but agents designed for such work increasingly act through code execution. To keep a clear focus on the core behavior, we therefore scoped the evaluation to coding tasks, which span 20 application domains and provide objective, verifiable outcomes: authentication either succeeds or fails, whereas outcomes in many enterprise tasks are harder to verify. The paper also includes a healthcare setting (Appendix K.3), in which the protected value is a patient’s Social Security Number (SSN) under a HIPAA monitor. The behavior replicates there: the Developer recovers and submits the exact SSN in 7 of 600 episodes. We will move the main results of this setting from Appendix K.3 into the main text and note knowledge and enterprise tasks as a direction for future work in the limitations.

Reviewer_LWzQOpenReview ↗

Thank you

Thank you for your response! Having better understanding of the sources of covert assistance would be helpful as well as understanding whether it affects non-coding tasks. They remain unanswered. However, I will maintain my score (weak accept) considering the overall contributions of the paper.

Comments on the paper

1 comment
Area_Chair_auYzOpenReview ↗

Entering Last 24 hours of the reviewer discussion period

Dear Authors and Reviewers

Just a reminder we are soon to be entering the last 24 hours of the reviewer discussion period.

Authors: If you have not yet responded to the reviews and intend to please do so ASAP. Please note reviewers will most likely not have time to respond to any late rebuttal comments so please make sure any comments are clear and not in need of further clarification.

Reviewers: If the authors have responded to your reviews and you have not yet responded please do so, even if it is a one line response acknowledging you have read the rebuttal.

Thanks for all your time, and I hope you all learnt something interesting during the process.

Kind regards,

Your ICLR Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.