- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper presents an empirical study of whether off-the-shelf LLM coding agents can generate executable policies for zero-shot coordination with independently developed partners. Given task descriptions, policy interfaces, and rollout tools, each coding agent iteratively writes, evaluates, and revises policy code in an isolated session with a budget of up to two hours, without access to the actual cross-play partners. The LLM weights remain fixed, and the resulting decentralized policies act on local observations and private memory without deployment-time LLM calls. Evaluation covers two-player Overcooked V2, Yokai, and Hanabi through self-play, inter-seed cross-play, and cross-model pairings, with additional tests on two new Overcooked layouts and external Hanabi bots. The reported results show substantial inter-seed cross-play gains over the selected RL baselines on Overcooked V2 and Yokai. In Hanabi, supplying H-group conventions improves compatibility with a convention-following bot, while self-play and inter-seed cross-play remain below strong RL baselines. The paper also compares policy-generation objectives and model capabilities to examine how these factors affect the performance and compatibility of generated policies.
Critical strengths
-
The generated policies are executable and inspectable, with no deployment-time LLM calls. The reported inter-seed cross-play gains on Overcooked V2 and Yokai are substantial relative to the evaluated baselines.
-
Independent runs, cross-model pairings, and new layouts provide evidence beyond self-play. The Hanabi tests with human-proxy and convention-following bots also extend evaluation beyond the generated-policy population.
Critical weaknesses
-
The paper offers limited methodological novelty or new insight into coordination. It uses existing coding agents with rollout-guided code refinement, and Co-pi-tree already develops executable cooperative policies through interaction with a source partner [1]. The independent-generation setting is a relevant distinction, but the analysis provides little explanation of why the resulting policies are compatible or under what partner conditions that compatibility breaks down.
-
The comparisons rely largely on early ZSC baselines and do not adequately cover subsequent population-based methods. For example, Overcooked V2 includes Self-Play, State-Augmented, Other-Play, and FCP, but omits MEP [2], HSP [3], COLE [4], and GAMMA [5]. Without comparisons to these methods in the same environment and under a common evaluation protocol, the reported gains do not establish an advantage over a sufficiently representative set of ZSC approaches.
-
Partner generalization is insufficiently evaluated. The main Overcooked V2 and Yokai results use partners produced through the coding-agent generation setup, leaving robustness to independently designed conventions largely untested. Evaluation against a common, fixed, diverse, held-out partner pool, as in ZSC-Eval [6] and ICRL4AHT [7], is missing. Inter-seed and cross-model compatibility provide limited evidence for generalization across a broader partner distribution. The Hanabi human-proxy and convention-following bot tests provide useful external-partner evidence, but the claims about human compatibility lack direct human-AI evaluation.
-
The sources of coordination gains are not isolated. There is no comparable-budget, same-model control separating rollout feedback from additional generation and selection, and the SP-versus-XP experiment changes both the objective and selection rule. The mutual co-adaptation interpretation is also stronger than the code evidence: in Appendix G.4, the cook infers signalling conventions while the signaller follows a fixed mapping. Without targeted ablations, shared priors, fixed conventions, fallback behavior, and online adaptation remain competing explanations.
[1] Zhang, B., et al. Distilling LLM Reasoning into an Interpretable Policy Tree for Human-AI Collaboration. arXiv:2606.08596, 2026. Paper.
[2] Zhao, R., et al. Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordination. AAAI, 2023. Paper.
[3] Yu, C., et al. Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biased. ICLR, 2023. Paper.
[4] Li, Y., et al. Cooperative Open-ended Learning Framework for Zero-Shot Coordination. ICML, 2023. Paper.
[5] Liang, Y., et al. Learning to Cooperate with Humans using Generative Agents. NeurIPS, 2024. Paper.
[6] Wang, X., et al. ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordination. NeurIPS Datasets and Benchmarks, 2024. Paper.
[7] Jing, Y., et al. Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork. ICML, 2026. Paper.
Questions for authors
-
How do MEP, HSP, COLE, and GAMMA perform under the same environment and evaluation protocol?
-
How do the methods compare on a common, diverse, held-out partner pool, and in direct human-AI experiments?
-
How much do rollout feedback and online convention inference contribute when generation budgets, selection rules, and ordinary task memory are controlled?
2 comments
I’m remaining my rating. My main concern remains that the Overcooked V2 and Yokai results don’t show whether the cross-play gains hold when all methods face the same diverse set of unseen partners.
We thank the reviewer for their positive comments on our results. We appreciate their feedback and address their concerns below.
We kindly refer the reviewer to our response to reviewer fPME regarding the main contributions of the paper. In light of these contributions, we strongly disagree that the paper provides no new insight.
We differ from Co-π-tree in several substantive ways:
Independent policy generation. Co-π-tree constructs and refines policies through interaction with an externally supplied source partner. This is essentially best-response optimisation against a fixed partner, which can be formulated as a single-agent problem by treating the partner as part of the environment. This development setting is fundamentally different from the ZSC setting considered in our work, where policies must become mutually compatible despite being developed independently using self-play feedback.
General-purpose coding agents. Co-π-tree uses a specialised scaffold, validated solely on Overcooked. We demonstrate that off-the-shelf coding agents can generate effective executable policies across three established coordination benchmarks.
More demanding coordination challenges. Existing methods already achieve strong IXP in the original Overcooked environment, which poses limited coordination challenges (as highlighted in the Overcooked V2 paper). We evaluate on Overcooked V2 and Yōkai, which introduce challenges including partial observability, asymmetric information, and test-time convention formation. Importantly, these benchmarks remain unaddressed by the existing ZSC methods.
Complementary evidence from Hanabi. We demonstrate competitive cross-play with human-proxy bots and strong compatibility with Wilflame when supplied with convention descriptions (Section 5.2). Our hybrid method further achieves the highest mean cross-play returns with both partner types while retaining competitive SP and IXP (Section 5.5, Table 3).
Hence, our findings extend substantially beyond Co-π-tree.
We agree that an in-depth analysis of agent behaviour is important. We kindly refer the reviewer to our response to reviewer fPME on our significant additions to our qualitative analysis. We also refer the reviewer to our response to reviewer tPkP under “On characterising failure conditions.”
We kindly refer the reviewer to our response to reviewer wVgq on our experiments with GAMMA and MEP. We find that both these methods perform very poorly on Overcooked V2.
Appendix E compares single-shot generation, the coding agent’s first attempt, and iterative refinement with rollout feedback. Iterative refinement improves mean IXP over the first attempt in all 30 model–environment comparisons and over single-shot generation in 25 of 30. Access to the coding harness alone does not consistently improve coordination, highlighting the contribution of iterative feedback.
Appendix I provides learning curves showing improvements in SP and IXP over successive attempts. Section 5.4 further compares models under SP and XP objectives: stronger models achieve higher SP under the SP objective, alongside a larger SP–IXP gap, and higher IXP under the XP objective.
Generation budgets are already controlled through a common two-hour wall-clock limit per run. We are unclear what is meant by controlling “ordinary task memory.”
We kindly refer the reviewer to our response to reviewer tPkP on mutual co-adaptation, where we provide additional examples illustrating richer forms of test-time adaptation beyond the belief-mapping example initially presented.
We kindly refer the reviewer to our response to reviewer tPkP on the pitfalls of evaluating against a common diverse held-out partner pool and why we believe it is not indicative of ZSC performance in the real world.
Such studies are costly, which motivates AH2AC2 (Dizdarević et al., 2025): a benchmark specifically designed to provide inexpensive, reproducible human-proxy evaluation partners trained on human gameplay. We use these proxies alongside the convention-following Wilflame bot. As reported in Section 5.2, generated code policies achieve XP returns comparable to established baselines with the human proxies and substantially higher returns with Wilflame. Our hybrid policies further improve these results, achieving the highest mean XP returns with both partner types (Section 5.5, Table 3).
We agree that direct human evaluation is important, but it lies beyond this paper’s scope. Our results establish compatibility with these bots and provide encouraging evidence for human–AI coordination. We will make the scope of this evidence clearer in the paper.