- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Z2SP addresses the coordination gap that arises when policies that perform well in self-play interact with unfamiliar partners. Its central idea is to adapt the ego policy toward the current partner's policy, seeking to recover the coordination behavior underlying that partner's self-play performance. After each joint transition, the method uses the partner's observation and executed action to update the ego actor through supervised imitation, while the ego continues to act on its own observations. Persistent adaptation retains the actor, optimizer state, and replay buffer across episodes; delayed history replay reuses past samples, and offline meta-initialization aims to reduce early adaptation costs. Experiments use five fully observable OvercookedV2 layouts and a common set of 20 unseen, self-play-trained partners from four algorithm families excluded from meta-training. Across 100 adaptation episodes with alternating roles, performance progressively approaches partner self-play. Additional experiments examine adaptation after partner switches, the importance of role coverage, auxiliary components, and online computational cost.
Critical strengths
-
Direct partner imitation is a simple way to obtain supervision without estimating reward-based policy gradients during deployment. Pairing partner actions with partner-view observations makes the update coherent under compatible observation encodings. The experiments demonstrate that this approach can substantially improve coordination with fixed, self-play-trained partners.
-
The common held-out partner set is a useful control. Comparing against the same partners and separating meta-training and test algorithm families provides more meaningful evidence than testing compatibility only among policies produced by the same training procedure. Multiple adaptation seeds and partner-level results help assess consistency.
-
The role ablation exposes an important boundary of the approach. The appendix also reports transient regressions and heterogeneous adaptation rates, making the evidence more informative than an endpoint average alone.
Critical weaknesses
-
The central persistent-adaptation result lacks a matched adaptive comparison. The baseline comparison covers the short-term setting, while the main long-horizon experiment measures Z2SP's improvement over its own initialization. TALENTS' online partner inference [1] and CooT's cross-episode context [2] provide relevant alternatives despite frozen network parameters. Comparisons under the same partners, information access, interaction budget, and role schedule are needed to establish Z2SP's relative value and interaction efficiency. Checkpoint returns also leave the cumulative reward earned during adaptation unresolved.
-
The methodological increment is modest, and the auxiliary gains are not adequately attributed. Conditional imitation learning already adapts a shared policy's strategy component through likelihood updates on observed partner actions [3]. Z2SP's full-actor, self-play target is distinct, but the supervised update offers limited novelty. The core-only ablation supports the basic mechanism; however, DHR adds optimizer updates, and meta-initialization adds offline supervision. Controls against ordinary replay with matched update/sample budgets and supervised initialization using the same offline examples are needed to establish the value of these designs. This attribution is particularly important given the modest methodological advance.
-
Near-self-play recovery depends strongly on role coverage. Training on partner-view observations does not necessarily teach the behavior required from the ego's role. The fixed-role results on More Fun and Fun Symmetries remain far below alternating-role performance, including under same-role evaluation (Table S3). Although explicitly acknowledged, this is a substantive restriction: the headline result depends on observing the partner in both responsibilities on these layouts. Full observability and compatible encodings alone do not supply that coverage.
-
Partner diversity is insufficiently characterized. The common held-out pool is a strength, but its selection procedure is under-specified, and different training algorithms do not necessarily produce different conventions or response requirements. Behavioral and response-diversity analysis, as motivated by ZSC-Eval [4], would clarify what the evaluation actually covers. Since all partners are self-play-trained, the results establish self-play recovery on this pool but provide limited evidence about when imitation remains effective across heterogeneous partner behaviors.
[1] Li et al. Adaptively Coordinating with Novel Partners via Learned Latent Strategies. NeurIPS, 2025. Paper.
[2] Wang et al. CooT: Learning to Coordinate In-Context with Coordination Transformers. ICML, 2026. Paper.
[3] Shih, Ermon, and Sadigh. Conditional Imitation Learning for Multi-Agent Games. HRI, 2022. Paper.
[4] Wang et al. ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordination. NeurIPS Datasets and Benchmarks, 2024. Paper.
Questions for authors
-
On two representative layouts, how does Z2SP compare with a persistent partner-inference or context-adaptation baseline under the same partners, observations, 100-episode budget, and role schedule? Please report cumulative interaction reward alongside frozen-checkpoint returns.
-
Do DHR and meta-initialization retain their gains against ordinary replay with matched update/sample budgets and supervised initialization using the same offline examples? Please report the additional training budget and complete meta-training settings.
-
Can imitation errors measured separately on ego-view and partner-view observations explain the fixed-role deficits and identify when partner-action supervision transfers successfully to the ego's role?
-
How were the five test policies per family generated and selected, and which distinct conventions do they represent? Does the result extend to a small independently specified partner family with compatible observations, actions, and role coverage?