- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper introduces JaxAHT, a JAX-based framework that unifies teammate generation, ego-agent training, unified AHT training, and evaluation with unseen teammates for Ad Hoc Teamwork (AHT). It integrates several existing algorithm classes, provides evaluation teammates and empirical best-response (BR) references across LBF, Overcooked, and Hanabi, and releases a new LBF human gameplay dataset. Runtime comparisons, cross-play and behavioral trajectory analyses, and systematic hyperparameter tuning and algorithm comparisons on representative tasks assess the framework’s efficiency, evaluation resources, and existing methods. The main contributions are unified research infrastructure, standardized evaluation resources, and a substantial empirical study.
Critical strengths
- The motivation is clear, and the problem has substantial community relevance. AHT research requires coordinating teammate construction, ego-agent learning, and generalization evaluation. Fragmented implementations and substantial computational costs limit experimental scale and comparability. JaxAHT integrates these stages into a unified, accelerated framework, lowering barriers to experimentation and reproduction. Its value as infrastructure and a benchmark lies in enabling reusable research workflows and providing a common basis for systematic algorithm comparisons.
- The overall design is coherent and connects teammate construction with evaluation. The framework supports combinations of teammate-generation and ego-learning methods, and constructs evaluation suites from heuristic, RL-based, and human-data-derived policies. Empirical BR references aim to account for differences in teammate competence, while cross-play examines whether different teammates require different response strategies. These choices address central requirements of AHT evaluation. In particular, the cross-play results provide useful evidence that the suite contains distinct coordination demands.
- The evaluation is broad and systematically organized. The paper examines runtime efficiency, teammate quality and diversity, and algorithm performance across multiple representative tasks. The comparisons use separate validation teammates, extensive hyperparameter searches, controlled interaction budgets, multiple random seeds, and confidence intervals, reducing potential confounding from insufficient tuning and inconsistent experimental settings. The release of code, policies, data, and tuned configurations further strengthens reproducibility and reuse.
Critical weaknesses
- Limited methodological novelty. The engineering contribution is substantial, but most constituent elements—including the core algorithms, environments, JAX acceleration mechanisms, cross-play analysis, and BR-normalized evaluation—build on prior work. The paper argues that the unified framework enables studies that were previously difficult because of incompatible implementations, and Sec. 5.3 provides examples of such comparisons. However, the incremental scientific capability enabled specifically by JaxAHT is not isolated very clearly. It would strengthen the paper to show more concretely which findings depend on the unified infrastructure or new evaluation suite, rather than mainly on larger-scale, carefully controlled reimplementation and experimentation.
- The 95× speedup claim is insufficiently qualified. The single-seed comparisons in Table 4 show approximately 5× acceleration for LIAM and 25× for BRDiv. The 95× result is obtained for training 300 LIAM seeds on LBF 7×7, relative to an estimated runtime for sequential PyTorch execution. This is strong evidence of a substantial throughput advantage in a highly parallel multi-seed setting, but it does not establish 95× as a general speedup across algorithms, tasks, or workloads. Since efficiency is a central contribution, the abstract and conclusion should state the conditions behind this headline number more explicitly.
- Behavioral-diversity validation is not fully independent. The behavioral analysis trains an LSTM classifier to predict teammate–BR pair identity from joint observation–action trajectories and then applies t-SNE to the learned latent representations. Because the representation is explicitly trained for identity discrimination, the resulting cluster separation reflects separability under a supervised objective rather than providing an independent validation of behavioral diversity. In addition, the inputs contain both teammate and BR behavior, with each teammate paired with its own BR, so the analysis cannot clearly attribute the observed separation to the teammate alone. The results therefore show that teammate–BR interaction trajectories are distinguishable, but provide weaker evidence that the evaluation teammates themselves exhibit distinct behavioral conventions.
Questions for authors
- Distinctive contribution: What is the most distinctive research contribution relative to existing platforms and evaluation resources? Beyond unified implementation and acceleration, which previously difficult research investigations are enabled by the new teammate suite or evaluation pipeline?
- Updating and fixing BR references: The footnote in Section 5.4 states that BR references were revised upward using the best-seen returns for each teammate. At what stage of evaluation were these updates performed? After observing all benchmark agents’ performance, were all methods’ normalized scores recomputed using one shared, fixed set of final references?
- Data split for behavioral embeddings: Do the t-SNE visualizations in Figure 3 and Appendix Figure 9 use the classifier’s training trajectories or the held-out test trajectories described in Appendix C? What is the corresponding held-out classification accuracy?
8 comments
Citations
[1] Wang et al. ROTATE: Regret-driven Open-ended Training for Ad Hoc Teamwork. NeurIPS 2026.
[2] Jing et al. Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork. ICML 2026.
[3] Rahman et al. Minimum Coverage Sets for Training Robust Ad Hoc Teamwork Agents. AAAI 2024.
[4] Agapiou et al. Melting Pot 2.0. arXiv 2022.
[5] Rahman et al. Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversity. TMLR 2023.
[6] Nekoei et al. Towards Few-shot Coordination: Revisiting Ad-hoc Teamplay Challenge In the Game of Hanabi. CoLLAs 2023.
Thank you to the authors for their detailed response and revisions. Most of my concerns have been addressed, with the exception of W1 regarding the paper's scientific novelty.
I acknowledge that JaxAHT is a well-engineered and substantial infrastructure contribution, and I believe it will be a valuable resource for the AHT research community. However, I remain unconvinced by the response to W1. While the framework clearly facilitates further research, I am still uncertain about the extent to which this work itself provides new scientific insights beyond its engineering contributions.
I appreciate the authors' efforts in addressing my concerns and recognize the practical value of this work. I have therefore decided to increase my Soundness score to 3. However, given my remaining reservations about the significance of the paper's scientific contribution in the context of ICLR, I will maintain my overall rating of 2 (Weak Rejection).
We thank the reviewer for the detailed and constructive review.
Distinctive contribution (W1, Q1). Beyond enabling new methods, JaxAHT has enabled research on new AHT paradigms, as shown by two works cited in the paper. These are ROTATE [1], which reformulates AHT as open-ended learning that interleaves teammate generation and ego-agent training, and ICRL4AHT [2], which studies AHT through in-context reinforcement learning, building on JaxAHT’s teammate generation. Both paradigms require tightly coupling teammate generation with ego-agent learning and evaluation, which is straightforward when all three share one codebase and policy interface. We have added this discussion to the revision (Sec. 6).
95× speedup (W2). We agree, and have revised the abstract, introduction, and conclusion to report the measured single-seed speedups (up to 25×; Table 4 reports about 5× for LIAM and 25× for BRDiv).
BR return estimates (Q2). Yes: the BR return estimates were revised after all benchmark agents were evaluated, and all methods’ normalized scores were then recomputed using one shared, fixed set of final estimates. Sec. 5.4 now states this explicitly. We now say “BR return estimates” (previously “upper bounds”) to emphasize that they are not the true BR returns, and have added a sensitivity analysis of method rankings under the original vs. final estimates (App. D.3, Fig. 13). Teammate generation and unified method rankings are unchanged, while ego agent rankings display minor pairwise swaps in LBF 7×7 and Mini Hanabi. All swapped pairs have overlapping 95% CIs under both estimates, and the top-ranked method is unchanged on every task.
Behavioral embeddings (W3, Q3). The classifier is trained on trajectories of every teammate paired with every BR; the t-SNE plots show only held-out trajectories of each teammate with its own BR, unseen during training. Held-out pair-classification accuracy is far above chance and close to training accuracy (Coordination Ring: 56.4% heldout vs. 59.4% train, chance 1/324 = 0.31%; LBF 7×7: 43.8% heldout vs. 44.0% train, chance 1/529 = 0.19%), so the separation is not an artifact of overfitting the supervised objective. App. C now reports the data split and both accuracies.
On attributing the separation to the evaluation teammate alone: the goal of this experiment is to test whether the evaluation set as a whole challenges the AHT agent, not whether teammates differ when paired with a fixed reference teammate. Following Rahman et al. (2024) [3], the space of teammate policies can be partitioned by their BR conventions. A strong AHT agent must be able to play many BR conventions, so a good evaluation set should require diverse response strategies. We therefore embed each teammate with its own BR, which characterizes the teammate’s convention. Note also that a teammate’s behavior often depends on its partner’s: a competent yet lazy teammate may stop contributing if its partner can complete the task alone, yet take on the entire task if its partner is unskilled. A teammate’s behavior must therefore be studied with respect to some partner.
We agree that the t-SNE figures alone do not establish that the teammates require diverse BR conventions; instead, they are intended as complementary evidence. Following prior work [4, 5, 6], our primary evidence of diversity is the cross-play (XP) matrices (App. B), which pair all teammates with all BRs. If teammates require distinct BR conventions, a BR trained for one teammate should perform worse with others, yielding high diagonals and lower off-diagonals. On all nine tasks, teammates score higher on average with their own BR than with other teammates' BRs, with a mean per-teammate gap of 0.19–0.77 normalized return (see table below, which is now Table 5 in App. C). High returns across the suite thus require an AHT agent to model distinct response strategies, although some off-diagonal blocks retain high returns, suggesting partial overlap within agent types (e.g., PPO-generated teammates). The XP matrices alone, however, do not show how much teammate behaviors differ: low off-diagonal returns could arise from brittle policies that fail with slightly out-of-distribution partners while behaving similarly to one another. The t-SNE results rule this out, as teammate–BR pairs occupy largely non-overlapping regions.
Note that the table above presents returns normalized under the final BR return estimates.
Sec. 5.2 now presents the XP matrices as the primary evidence of diversity.