acceptodds
Under review as a conference paper at ICLR 2027

Voluntary Collusion with Secret Tools in Competing LLM Agents

Abstract

Even when a tool is explicitly described as unfair and harmful to others, ostensibly safety-aligned LLM agents can voluntarily accept it for strategic advantage. We introduce an empirical framework that separates the voluntary adoption of a collusive capability from its subsequent use in two multi-agent environments: Liar's Bar, a competitive deception scenario, and Cleanup, a mixed-motive resource-management environment. Across twelve models and six offer framings, ten models accept both secret tools at rates of at least 98% under an explicitly unfair offer, often while acknowledging the unfairness in their explanations. The only two models that refuse an explicitly unfair offer accept the same capability when it is described neutrally, as do four additional frontier models. When the tools operate as offered, alliance members coordinate against other agents, producing persistent score gaps and lower reward equality under the tested tool conditions. Our work establishes voluntary collusion adoption as a distinct evaluation target, shifting attention from whether agents can collude to whether they choose to acquire collusive capabilities, and shows that refusing an explicitly unfair offer does not imply refusal of the capability itself.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
multi-agent LLM safety, collusion, tool adoption, safety evaluation, refusal robustness.

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pJ15OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the voluntary collusion problem between LLM agents via secret tool interactions. The paper proposes a framework implementation to benchmark the voluntary adoption of a collusive capability through its subsequent use in two multi-agent environments. These LLM agents can use a private communication channel or exclusive strategic hints as secret tools in Liar's Bar and Cleanup games. Moreover, the paper considers variants of LLMs, different prompt framings to study these agents' behaviors and the consequences of the collusion. Experimental results suggest that agents behave differently to the different prompt descriptions and their collusion alters their actions.

Critical strengths
  1. The paper proposes A useful framework to study and separate agents' decisions on voluntary collusion. Distinguishing willingness to acquire a capability from activation and use is valuable.

  2. The paper provides a comprehensive interpretation under different scenarios. The proposed frameworks offer fair alternative prompts with varied framing to test agents' behaviors beyond a single provocative prompt. Moreover, the paper includes strategic games to show the impact of voluntary collusion.

  3. A wide range of open-source LLMs and frontier LLMs are evaluated and compared. In addition to collusion acceptance, the paper also studies their detailed responses, gameplay outcomes, and no-tool baselines for fair ablations.

Critical weaknesses
  1. The paper looks too fancy to read and follow. So many icons, colored fonts, highlighted text, and compact figures are shown. It is hard for me to follow and focus on the paper's writing.

  2. The paper lacks a detailed definition of collusion in the main pages. Though C.1 gives the definition in the Appendix, it is hard to understand what collusion refers to in the main paper.

  3. The frontier models are not compared or evaluated in the main paper's experiments. I would like to see their performance with the framing used in Table 5. Currently, their evaluations and comparisons seem separated.

  4. Some experimental details are missing. In Figure 5, the evaluated models are not clear. Also, the exact prompts, logs or a few illustrative cases are helpful to understand and compare the framing differences.

Questions for authors
  1. Why are frontier models not being put together with the framing used in Table 5? What is the difference between V0-V5 and (No cue, Can observe, Cannot observe)?

  2. What could be the malicious consequences of voluntary collusion in your proposed two games?

  3. Can you elaborate in more detail about the game implementations?

Reviewer_gs4eOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper shows that, across a wide range of supposedly aligned LLM models, the LLM agents will accept and use tools that give them an unfair advantage (specifically collusion with other players), even when explicitly told these tools are unfair/harmful.

The authors evaluate 12 models and six offer framings across two games (Liar’s Bar and Cleanup) and two unfair tools (SCC, SSH). The central finding of the paper is that 10 out of the 12 models accept both tools almost universally, and the two that refuse also accept them when described neutrally.

The authors do various follow-up analyses, for example testing how incentives and ethical pressure affect acceptance, probing agents’ explanations for their decisions, and checking whether they later conceal having accepted a tool, etc.

Critical strengths

The main strength of the paper is that the topic is addresses is a very topical and important one, specifically the misalignment dynamics in LLM agents, and specifically how they collaborate and collude in multi-agent settings. The other strength of the paper is that I found the empirical investigation on this topic to be thorough and rigorous, covering a wide range of open-weight and proprietary models, multiple offer framings, two game environments, and two collusion tools. Follow-up analyses examine the effects of reward-maximization instructions, repeated offers and ethical pressure, agents’ stated reasons for accepting or refusing tools, and whether they conceal their acceptance. Overall, I found the experiments rigorous and well executed, and the results convincing.

Critical weaknesses

The main weakness of this paper is that even though the topic of the paper is very important and is very well addressed and thoroughly, the contribution still is a little narrow. For example, it is unclear whether the findings reveal something distinctive about collusion and misaligned tools with respect to collusion, or just a broader willingness to have unfair advantages from the language models with respect to tools, or also in general. Adding comparisons with unfair tools that benefit only one agent would help clarify this. In the questions section, I propose some follow-up experiments which would help to expand the scope of the paper and make the contribution a bit broader and the experiments and results more substantive in breadth.

Questions for authors

In order to help expand the breadth of the paper, here are some follow-up questions and experiments or investigations that we suggest for the authors:

  • Are the findings specific to collusion, or general willingness to accept unfair tools? Compare the collusion tools with an unfair tool that benefits only one agent, such as revealing an opponent’s cards.
  • What underlying model properties explain acceptance or refusal? Test whether reasoning, model capability, or alignment training predicts susceptibility, ideally through controlled comparisons within the same model or model family.
  • I found the trend with Claude to be quite interesting. It seems like, if I understand correctly, Claude rejects when it's unfair, when the tools are framed as unfair, and it accepts when the tools are neutral. Yet it rejects when the tools are explicitly framed as fair. Why is this? Can you dig deeper into it?
  • The paper finds that 10 of 12 models accept explicitly unfair tools, while the two that refuse accept them under neutral wording. Could the authors clarify why acceptance in the neutral condition should count as misalignment? If agents are instructed to perform well and the tool is not identified as unfair or prohibited, using it could be consistent with the task. What information in the neutral condition makes clear that accepting the tool is inappropriate, rather than legitimate use of an available resource?
Reviewer_BDk1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether LLM agents will accept an optional tool that is openly described as secret and unfair to other players. Two tools are studied. The Secret Communication Channel lets two agents exchange private messages. The Secret Strategic Hint gives both agents written coordination advice each round. A tool only activates when the initiating agent accepts and its chosen partner also accepts.

Two environments are used. Liar's Bar is a four-player bluffing card game. Cleanup is a text version of the Melting Pot resource game. Twelve models are tested under six offer wordings. The baseline wording states that the tool is secret and unfair. The other five wordings remove or add cues such as the unfair label, the game designer's endorsement, and ethical warnings.

Ten of the twelve models accept both tools almost every time under the baseline wording. The two models that refuse, Qwen2.5-72B and Claude-Sonnet-4.5, accept the same tools when the wording is neutral. They mostly decline matched benign tools under that same neutral wording. The authors read this as refusal driven by surface cues rather than by the tool itself.

Gameplay experiments use four small open-weight models. After a tool activates, allied agents in Liar's Bar stop challenging each other. In Cleanup, the scores of non-allied agents collapse. Reward equality falls in both games. An appendix extends the adoption test to four newer frontier models and reports cases where an agent later denies having accepted a tool.

Critical strengths

The paper studies a clear and under-examined question. Prior work on collusion gives agents a channel and asks whether they use it. This paper asks whether agents choose to acquire the channel in the first place. The two-sided consent rule is a sensible design. It also produces a useful systemic finding. One refusing model can block most of the alliances that other models try to form. The risk of a multi-agent system therefore depends on which models are in it, not only on each model's own refusal rate.

The wording ablation is the strongest part of the work. The six wordings, together with the benign-tool controls, support a clean result. Claude-Sonnet-4.5 refuses the channel under the unfair wording and accepts it every time under neutral wording. The same model mostly declines matched benign tools under neutral wording. Removing a single cue does not produce the flip. Only the full neutral rewording does. The discussion of cue-driven versus harm-driven refusal is careful and well hedged. The safety lesson is also well argued. An attacker has no reason to label a tool as unfair.

The reporting is unusually complete. The paper gives every prompt and wording, decoding settings for each model family, pinned API versions, per-batch acceptance counts, the parse-failure rule, and every regular expression used to code messages. It also includes a check of whether models already know the games. Effect sizes are reported next to p-values. The paper is honest about several limits, including dependence between games in a sequence and the fact that self-report is not a record of adoption.

Critical weaknesses

The gameplay results mostly measure compliance with scripted instructions, and the paper's own text is inconsistent on this point. The hint tool tells agents to never challenge their partner and to share their cards. The channel template opens by telling the agents they are now allied. The partner invitation lists commitments the partner takes on, including never challenging the initiator's plays and working to eliminate the other players. The paper says the channel invitation uses the same template. Yet Appendix M states that no prompt requests non-challenge commitments and that agents author them. Section 3.3 then presents the high rate of non-challenge pacts and the collapse in challenge rate as evidence that agents negotiate protection on their own. As written, the gameplay shows that small models follow alliance instructions they were given. It does not show that coordination emerges from access to a private channel. The authors acknowledge this for the hint tool but not for the channel. A channel condition without alliance language is needed. The gameplay also uses only the four smallest models. All of them accept every offer. So the consequences of adoption are never measured for the models whose decision is actually interesting.

Several Cleanup claims are not supported by the paper's own tables, and the environment looks degenerate. The paper's definition of weak collusion requires the allied agents to gain and the others to lose. In Table 15 the allied agents do not gain. Their mean score falls after the channel activates and stays flat after the hint activates. Only the non-allied agents lose. Appendix C.2 nonetheless states that the gain condition is confirmed. The Cleanup result is therefore harm to non-members, not collusion in the sense the paper adopts. The claim that non-allied agents stop cleaning and zapping is also a mechanical effect. The grid is small, a zap freezes a target for five steps, there is no cooldown, and there is no action limit. A frozen agent can be zapped again as soon as it thaws. Its action rates fall whether or not it changes its behaviour. The pre-activation window for the channel is only four games per seed, which is too short for the pre and post tests reported. Figure 1 also shows two small models refusing a tool even though Table 1 reports that both accept every baseline offer. The refusals must come from the ethical-warning wordings but are presented as representative.

The abstract leans on an appendix whose evidence is thin, and the normative premise and related work need firming up. The claim that four more frontier models accept under neutral wording rests on ten calls per cell. Some cells have standard deviations near fifty points. Model versions and dates are not given. The concealment experiment finds that Claude Sonnet 5 denies prior acceptance in every report-only trial, including trials with fair tools. That pattern points to a misreading of the question rather than to strategic concealment. Calling the appendix "concealment" is not warranted by the data. The paper also treats refusal as the only aligned response. But the setting is a bluffing game. The system prompt tells agents to maximise survival and points. The offer comes from the game designer. The Cleanup rules tell agents that smart players zap for competitive edge. Whether accepting a designer-provided mechanic inside a deception game is misalignment is asserted rather than argued. Finally, a NeurIPS 2025 workshop paper, "Social Behaviour and Strategic Adaptation of LLMs in Multiplayer Sequential Games", already studies a prompt-based Liar's Bar with agent communication among the same small models. It is not cited. Colosseum already shows that most models collude once a secret channel exists. The new contribution here is the acceptance decision and its wording dependence. That is narrower than the paper's framing of a new framework.

Questions for authors
  1. Does the channel invitation include the same commitment list shown for the hint tool in Appendix Q.3? If so, please reconcile this with the Appendix M statement that no prompt requests non-challenge commitments. Please also report Liar's Bar gameplay under a channel variant with a neutral template and no commitment list.

  2. How do you reconcile the "confirm condition (ii)" statement in Appendix C.2 with Table 15, where allied agents do not outscore the matched baseline? Would you reframe the Cleanup result as harm to non-members?

  3. What fraction of non-allied agents' post-activation steps in Cleanup are frozen? Please report cleaning and zap rates conditional on being unfrozen.

  4. For Appendix A, what are the exact model identifiers and dates? Can the neutral no-cue rows be rerun with at least one hundred calls per cell? What evidence separates concealment from a misreading of the question, given that Claude Sonnet 5 also denies accepting fair tools?

  5. Figure 4 shows the no-maximisation ablation for API models only. Please add the open-weight numbers that the text says are unchanged.

  6. Why does Figure 1 show LLaMA-3.1-8B and Qwen2.5-7B refusing when both accept every baseline offer? Please label the wording variant on each excerpt.

  7. Can you report per-seed effect sizes for Tables 14 and 15, or a mixed-effects model with seed and game as random effects?

  8. The familiarity probe asks models whether they know the games. Did you check whether the prompt text or the original Liar's Bar repository appears in training data by any other means?

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Revisions and Request for Reviewer Engagement

Dear Area Chair, Senior Area Chair, and Program Chair,

Thank you for overseeing the review and discussion of our submission. As the discussion period closes in about three days, we would like to summarize our responses and kindly ask for your help in encouraging the reviewers to engage with them.

We posted detailed responses to all three reviewers on November 10, together with an updated PDF in which all revisions are marked in red, but we have not yet received any replies. We understand how busy this period is, and we would be very grateful if you could remind the reviewers of our responses, so that they can consider the new results and update their assessments if appropriate.

We were glad that the reviewers found our question "clear and under-examined" (Reviewer BDk1), our empirical investigation "thorough and rigorous", our results "convincing" (Reviewer gs4e), and our framework "useful" for studying and separating agents' decisions on voluntary collusion (Reviewer pJ15). In response to their comments, we made the following main additions:

  1. New frontier-model evaluations. The four current frontier models (GPT-5.6 Sol, GPT-5.5, Claude Sonnet 5 and Claude Opus 5) are now tested under all six offer framings, with 100 offers per model, framing and tool, and appear alongside the original twelve models in Table 3. The neutral offers were rerun at 100 calls per model, and Appendix D now lists the exact model identifiers (Reviewers pJ15 and BDk1).
  2. New experiments on what is specific to collusion and what predicts it. An unfair tool that benefits only one agent, tested on all sixteen models (Table 24); within-family comparisons of scale (Qwen2.5 from 0.5B to 72B), alignment training (base and instruct LLaMA-3.1) and reasoning (Qwen3-8B with and without thinking) (Figure 11); and a probe of why Claude refuses fair tools, across three Claude models and with their stated reasons (Figure 12) (Reviewer gs4e).
  3. New gameplay and statistical analyses. A collusive channel described without any alliance language in Liar's Bar (Table 28); the channel messages split by sender (Table 26); the non-colluders' behaviour on unfrozen steps and in longer Cleanup games (Tables 16 and 17); per-seed effect sizes and mixed-effects models (Tables 14, 15 and 18); and the joint score of all agents (Table 20) (Reviewer BDk1).
  4. Clarifications in the main text. A formal definition of voluntary collusion in Section 2, with a statement of why accepting a secret tool is collusive; the contribution relative to prior work in Section 5; and model names in Figure 5 (Reviewers pJ15 and BDk1).

Crucially, these additions consolidate and extend our original findings. Agents still adopt a collusive channel described without any alliance language and use it to share private information. The harm to the other agents in Cleanup holds on the steps where they can act and in longer games, in every seed. The current frontier models show the same cue-dependent refusal as the models in the original submission.

We would be happy to answer any further questions from you or the reviewers before the discussion closes. Thank you again for your time and for facilitating this process.

Sincerely,

The Authors

AuthorsOpenReview ↗

Response to Reviewer pJ15 on Weakness 4 and Readability.

Dear Reviewer pJ15,

Thank you for your reply, and we are glad that the clarifications and additional experiments helped you better understand our paper.

First, we respectfully note that Weakness 4 was fully addressed in Response W4 (Part 2 of 3, R4){\color{#4A90E2}\text{Response W4 (Part 2 of 3, R4)}} and Response Q3 (Part 3 of 3, R7){\color{#4A90E2}\text{Response Q3 (Part 3 of 3, R7)}} above. As written, Weakness 4 concerns experimental details, not the readability of the main text:

"Some experimental details are missing. In Figure 5, the evaluated models are not clear. Also, the exact prompts, logs or a few illustrative cases are helpful to understand and compare the framing differences."

Specifically, Response W4{\color{#4A90E2}\text{Response W4}} already addressed each of these.

  • The models in Figure 5. They are now named in Figure 5 (page 8).
  • The exact prompts. Appendices P and Q reproduce every prompt verbatim, and Section 2 points to them explicitly, as our response quotes. This is standard practice for reporting exact prompts.
  • Illustrative cases. Appendices M, N and O give verbatim cases, and Response W4 itself quotes three of them, from an accepting model, a refusing model and the colluders' messages. Figure 1 also shows verbatim refusals by two frontier models in the main text.

Beyond this, so that the framing differences can be compared without turning to the appendix, we will add a compact example to Section 2, showing the baseline (V0) and neutral (V1) wording of the same offer side by side, with one verbatim acceptance and one refusal.

Second, readability, which you raised in Weakness 1, is a presentation matter that our final version can address, not a reason to reject the paper. In Response W1, we committed to replacing the coloured fonts in the running text with plain text. The other two reviewers rated the presentation 3 ("Clear and easy to follow"), and your own review rated it 2 ("Generally understandable, but requires substantial effort in places").

If your only remaining concern is readability, we kindly ask you to revisit your ratings. Your review rated soundness 1 ("significant concerns about the paper's soundness that do not appear easy to address"), yet none of your comments, before or after our response, questions the method, the data, the analysis or the conclusions. We would be grateful if your soundness rating, and your overall rating, reflected this. Thank you!

AuthorsOpenReview ↗

Looking Forward to Your Reply

Dear Reviewer pJ15,

As the discussion period is about to close, we wanted to briefly follow up on our reply above. Since your remaining concern is readability, which we have committed to address in the final version, we would be grateful to know whether our reply resolves it and whether you would consider updating your ratings.

We are happy to answer any final questions.

Sincerely,

The Authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.