- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes PopGen, a framework for generating semantically diverse teammate populations for zero-shot coordination. It first generates diverse behavior descriptions using LLM-based oversampling, then translates them into executable policy code through iterative refinement with VLM feedback. The generated teammates are used to train a cooperator via population play. Experiments across four environments demonstrate improved coordination with unseen teammates, while ablations investigate semantic diversity, behavior adherence, and policy generation methods.
Critical strengths
-
Practical and efficient teammate generation. Directly generating executable policies avoids the expensive RL training required by reward-function-based approaches such as SemDiv. The experiments demonstrate improvements in generation efficiency and behavioral adherence, making this a practically valuable alternative.
-
Promising empirical results. PopGen achieves strong zero-shot coordination performance across four environments. The analyses of population diversity, robustness to unseen teammates, and the diversity-quality trade-off provide useful empirical insights.
Critical weaknesses
-
Limited methodological novelty over SemDiv and insufficient positioning. The central idea of using LLMs to generate semantically diverse teammate behaviors and grounding them into executable agents was already explored by SemDiv (Li et al., 2025). PopGen's main differences appear to be replacing sequential behavior generation with oversampling and replacing reward-function-based policy training with direct policy code generation. While useful, these modifications seem incremental, and the paper should more clearly distinguish its contributions from SemDiv. The related work should also discuss Macop (Learning to Coordinate with Anyone, Yuan et al., 2023), which explores adaptive generation of diverse teammates based on coordination difficulty. This is particularly relevant to the paper's proposed future direction of online adaptive behavior generation.
-
Insufficient evaluation of the claimed improvements over SemDiv and semantic diversity. The comparisons with SemDiv are fragmented: Figure 5 compares text generation strategies, representing SemDiv only through multi-turn prompting, while Figure 6 compares policy code against reward-function-based training. Neither evaluates the complete SemDiv teammate generation pipeline under a unified protocol. Although the original methods have different test-time assumptions, a controlled end-to-end comparison of their generated populations would strengthen the claims. Furthermore, the semantic diversity metric relies on distances between text embeddings, which may not reflect meaningful differences between the resulting policies. The authors should validate this relationship and clarify the embedding inputs and normalization procedure (the embedding model itself is specified in Section 5.1).
Questions for authors
See Critical Weaknesses
2 comments
Thank you for the detailed response. The new analysis of semantic diversity and the clarification of the embedding procedure address part of my concern. The efficiency and adherence gains are also valuable. The comparison with SemDiv remains at the component level, and I still view the methodological contribution as incremental. I therefore maintain my rating of 3.
Reply to reviewer ogVZ
We thank the reviewer for the thoughtful review and for recognizing PopGen as a practical and efficient alternative to RL-based teammate generation, with strong empirical results across four environments. We address each concern below.
Novelty and positioning relative to SemDiv (W1). We agree that SemDiv first proposed generating teammates from behavior descriptions, and PopGen builds on this idea. Our contribution is to identify and address the two bottlenecks that limit it in practice.
Together, these changes make it practical to generate large, complex populations, which a reward-function pipeline cannot reach at a similar budget. We will state these differences from SemDiv explicitly in the related work.
We thank the reviewer for pointing us to Macop, which generates teammates that are hard for the current cooperator to coordinate with and continually trains the cooperator against them. This is closely related to the online adaptive generation we propose as future work, where PopGen could generate behaviors targeting the cooperator's weaknesses. We will discuss Macop in the related work and conclusion.
End-to-end comparison with SemDiv (W2). We separated the two stages on purpose, so each comparison changes one component while the other is held fixed. Figure 5 compares behavior generation methods on the same environments, and Figure 6 compares policy code and reward functions from the same set of behaviors. We did not run the full SemDiv pipeline for two reasons. First, SemDiv assumes a different test-time setting: at execution, an LLM uses a natural-language description of the teammate's behavior to select the cooperator's best policy head, whereas we assume the cooperator has no information about its partner at test time. Second, SemDiv filters trained policies with manually set, per-environment thresholds (e.g., on task reward and on reward with existing partners), which are hard to reproduce faithfully in new environments. We therefore compared the two components that transfer to our setting, behavior generation and agent grounding. PopGen generates more diverse behaviors than SemDiv, and the policy-code population also yields a stronger cooperator (Figure 6). We will clarify this reasoning in the paper.
Validity of the semantic diversity metric (W3). We agree that distances between text embeddings need not reflect differences between the resulting policies. To test this, for every pair of PopGen policies in each environment we computed both the semantic diversity between their behaviors (Equation 2) and the Jensen–Shannon divergence (JSD) between their trajectory distributions, introduced in TrajeDi. As most PopGen policies are deterministic, we add Gaussian noise (σ=0.25) to continuous actions. The table reports the Spearman correlation ρ between the two over all pairs, with Mantel permutation p-values. A discount factor γ sets the comparison horizon, from one-step actions (γ=0) to whole trajectories (γ=1).
The correlation is positive in all 12 settings and significant (p<0.05) in 10, so policies whose descriptions are semantically farther apart also act more differently in the environment. We will add this analysis to the appendix.
Thank you for suggesting we clarify the embedding inputs and normalization. The embedding model takes as input the textual description of each behavior, generated by the LLM in Stage 1. We normalize embeddings to unit length, and Equation 2 takes the Euclidean distance between unit vectors, 2(1−ek⊤eℓ), ranging from 0 (identical) to 2. We will clarify this in Section 4.1.