acceptodds
Under review as a conference paper at ICLR 2027

Free-Form Communication Is Why Language Models Outperform Bayesian Agents in Negotiation

Abstract

In a solver-verified five-party negotiation scenario where language models and exactly specified Bayesian negotiators occupy interchangeable seats, tables of language models reach agreement far more often, in 0.958 of games for five Claude Opus 5 seats against 0.233 for five Bayesian negotiators. Agreement requires unanimity, so closing a deal is a coordination problem and the ability of LLMs to engage in free-form talk independently of proposing or accepting deals is what allows them to perform better. Three tests support this account: making the five Opus seats mute, so they still move but send no messages, costs 0.200 in agreement rate, as much as seating one mute Bayesian agent. The second test asks what the talk channel must carry; when each seat truthfully reveals its preferences once, every all-Bayesian table reaches agreement, whereas signals that stop short of revealing preferences plateau at an agreement rate of 0.400. The same declaration protocol adds less as language-model seats replace Bayesian ones, because free-form talk already conveys preferences. Attaching a language-model interface that speaks and listens for the Bayesian seat lets its table reach agreement as often as an all-Opus table, with the Bayesian policy still conditioning every proposal. A mute Bayesian seat lowers agreement at every capability tier, and an omniscient Bayesian seat helps Haiku tables but hurts Opus tables, so an algorithmic seat helps only when it knows something the table lacks and can communicate it. Designers should give every seat a free-form communication channel, wrap any mute policy such as an algorithmic participant in a language-model interface, and not seat a policy that knows nothing the table lacks, since the defaults otherwise cost a fifth of agreements. Experimental data and the interaction framework will be released on publication.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent Communication, Negotiation, Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_a4mDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why LLM agents outperform Bayesian negotiators in a five-party negotiation benchmark. Through communication ablations, preference-declaration experiments, and LLM interfaces for algorithmic agents, the authors argue that free-form communication improves coordination by conveying preference information and helping agents converge on feasible agreements.

Critical strengths
  1. The paper asks an interesting question rather than only reporting that stronger LLMs perform better.

  2. The experiment testbed is well controlled. The exact solver guarantees that every instance has a feasible agreement and allows every outcome to be evaluated compared to the full feasible set.

  3. The hybrid agent result is interesting.

  4. The writing is easy to follow.

Critical weaknesses
  1. The core causal claim is stronger than what the experiments establish. The evidence doesn't sufficiently support free-form communication is why LLMs outperform Bayesian agents in negotiation, as muted Opus agents still outperform the Bayesian agents.

  2. The paper argues that the relevant payload of communication is preference information. However, the experiments does not establish that preference revelation is the causal reason why natural-language communication improves LLM performance.

  3. Only one particular Bayesian reference policy is evaluated. The claim about Bayesian agents is too broad.

Questions for authors
  1. How do the authors justify the main causal claim with the all-muted experimental result?
  2. Can agents retain the same communication budget but be prohibited from revealing preferences?
  3. How robust is the main effect to other frontier models?
Reviewer_DBorOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper looks at agents, either LLM-based or "Bayesian" that negotiate. The paper shows that 'tables' with LLM agents reach agreement more often than Bayesian agents. This result is attributed to the fact that LLMs are able to reveal preferences to each other through a 'free-form' message channel or 'cheap talk'. This result is obtained through ablation.

Critical strengths

The paper studies a potentially interesting problem, is well written (I did not spot language or methodological issues) and the results seem thorough.

Critical weaknesses

The paper is hard to read (this already starts with the first sentence of the introduction - is this an observation, a conclusion?) There's a lot of terminology that may be clear to people familiar with the specific topic but not to a more general AI/ML audience. There is not a clear problem statement or research question formulated.

Moreover, the results concern specific (mostly commercial) LLM models, and a fairly small set of instances. What do these results tell us beyond these models? Furthermore, the baseline seams to be rather weak (making the results not very surprising). It's no surprise that being able to communicate your preferences, leads to better coordination.

I think that the bar for this type of research (experiments involving LLMs, which pretty much anyone can do nowadays) should be much higher in terms of clarity, significance and impact.

All in all I'm not particularly impressed by this work or the results.

Give the generality of the ICLR conference I think there are probably more fitting venues for this kind of work.

Questions for authors

none

Reviewer_ocLAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors compare LLMs with fully specified Bayesian negotiators in solver-verified, five-party negotiations and find that LLM-only groups reach agreement much more often. Through controlled tests of communication and agent capabilities, they show that free-form preference sharing drives this advantage.

Critical strengths
  • The paper presents an analytical study and reports an interesting finding: free-form communication may help explain why LLMs outperform Bayesian models in negotiation.
Critical weaknesses
  • The proposed method is evaluated in a relatively narrow setting: solver-verified, five-party negotiations. It is unclear whether the approach generalizes to negotiations with different numbers of participants, other task structures, or settings without solver-based verification. The authors should clarify the intended scope of the method and discuss how its assumptions may limit its applicability to broader negotiation scenarios.

  • The paper’s main claim that “free-form communication is why language models outperform Bayesian agents in negotiation” appears to rely primarily on empirical results based on a limited set of LLMs. These results may show an association, but they do not by themselves establish that free-form communication is the reason for the performance difference. The claim would be more convincing with a more rigorous analysis and provides stronger evidence for the proposed explanation; otherwise, the authors should qualify the claim.

Questions for authors
  • Have authors think of any theroical proof for the hypothesis/claim?
  • What are the key tehcnical contribution to the commuity? Who are the stackholders of this applicaiton?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.