- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper introduces a Kaiji-inspired card game to study negotiation, cooperation, and betrayal among LLM agents across different models and conditions. For the main model, proposals to tie are 93.5% of matches, yet agents break agreements 10.2% of the time, 70% of which are to win at the other models' expense through deception. The paper also does various extension analyses, such as removing memory, testing the game in which opponents are assigned, which creates a new failure mode (53% of the time) in which both agents sometimes accommodate the other’s conflicting proposal, preventing the intended tie. Scarcity worsens survival without a clear increase in betrayal rates, and reasoning makes the main model betray more often and can help it profit at non-reasoning agents’ expense, but its effects vary across models. Overall, the paper contributions a new environment and an initial investigation of how model choice, memory, and interaction protocols shape cooperation and betrayal.
Critical strengths
The main strength of the paper is that it tackles a very topical and important subject: the collaboration, deception, and interaction dynamics across LLM agents of different types in multi-agent settings. As LLM agents become more proliferated in the real world, this becomes a very important topic. The paper contributes a creative new environment which tests this and does extensive and thorough analyses across various ablations and experiments. The thoroughness of the experiments, in terms of the various ablations and extension analyses, is the main strength of the paper.
Critical weaknesses
The main weakness of the paper is that the evaluation is limited to one game, which makes it unclear which findings generalize beyond Kaiji’s particular rules and incentives vs. LLM agents in general. Also several findings deserve deeper investgation:
- why do some agents choose the losing alternative when breaking a pact?
- why does adaptation under scarcity fails to improve outcomes?
- why agents strongly favor ties despite the financial rewards for additional stars?
In particular, it remains unclear whether agents explicitly calculate interest and expected returns or respond only loosely to these incentives. Qualitative analysis of these decisions, targeted incentive experiments, and replication in another environment would strengthen the contribution. Some passages also feel AI-generated and are difficult to parse, especially the definitions and financial mechanics.
Questions for authors
- Do the same findings hold in other games, or do they depend on Kaiji’s set-up specifically?
- Why do some agents break a pact by playing the losing card? Are they making mistakes, forgetting the agreement, or intentionally helping their opponent?
- Why does scarcity increase eliminations without making betrayal more strategically effective?
- Why do agents favor ties despite the ¥4 million reward for each extra star? How do they weigh safe escape against potential profit?
- Do agents explicitly calculate compound interest and expected returns, or only consider money qualitatively? How does this differ between reasoning and non-reasoning models?