acceptodds
Under review as a conference paper at ICLR 2027

Learning Cross-Context Teammate Signatures for Zero-Shot Coordination

Abstract

Zero shot coordination studies how agents cooperate with previously unseen teammates. A key challenge is to determine what information in interaction history actually matters for coordination, since teammate behavior is often entangled with the situations encountered. We argue that teammate representations should preserve coordination relevant conditional behavior rather than identity differences alone. We propose FSIG, which learns functional signatures by comparing teammate action predictions under shared queries. Behavioral supervision grounds these predictions in observed actions, while a consistency objective encourages agreement across matched histories. The learned signature then conditions the ego policy for adaptive coordination with unseen teammates. Across nine tasks spanning classic Overcooked, OvercookedV2, and Level Based Foraging, FSIG achieves competitive coordination performance. Further analyses show that interaction history provides additional predictive information, the functional consistency objective improves agreement across histories, and learned signatures affect controller behavior during coordination.

Primary area:
reinforcement learning
Keywords:
Zero-Shot Coordination, Ad Hoc Teamwork, Multi-Agent Reinforcement Learning, Teammate Modeling, Partner Generalization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_N32YOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work propose FSIG, which learns functional signatures by comparing teammate action predictions under shared queries. Behavioral supervision grounds these predictions in observed actions, while a consistency objective encourages agreement across matched histories.

Critical strengths

The paper offers a conceptually clean and well-motivated reframing of teammate representation. The distinction between “understanding the partner” and “responding to the partner” is articulated clearly and leads to a non-trivial design choice: defining the meaning of a history through its behavioral readout under shared queries rather than through latent-space alignment.

Critical weaknesses
  1. The signature’s marginal effect on OvercookedV2 is statistically inconclusive, but the paper’s claims are not always calibrated to this.
  2. The conceptual relationship to conditional neural processes and E3T-style history-conditioned prediction needs sharper formalization. FSIG’s shared-query behavioral readout is closely related to conditional neural processes (Garnelo et al., 2018) and to history-conditioned partner prediction in E3T (Yan et al., 2023). The paper mentions these works but does not provide a precise comparison of what is shared and what differs.
Questions for authors
  1. Tables 1–3 report mean ± sample SD over five training repetitions but do not include paired statistical tests for the main cross-play comparisons. Given that Table 4 already uses paired 95% CIs, please add paired bootstrap CIs or permutation tests for the main classic Overcooked, OvercookedV2, and LBF comparisons.
  2. The shared-query mechanism in Eq. (2)–(4) is conceptually close to CNPs’ target-point prediction and to E3T’s history-conditioned teammate modeling. What is the diffference betwen them?
Reviewer_GGCfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FSIG learns a teammate signature from interaction history for zero-shot coordination. It trains a history encoder and shared decoder to predict a teammate's action at a common query observation, while encouraging predictions from two histories of the same teammate to agree. The encoder is then frozen and used to condition a recurrent PPO controller. Across four Classic Overcooked layouts, three OvercookedV2 tasks, and two Level-Based Foraging (LBF) grids, FSIG has the highest mean return on six of nine tasks. The paper also reports a matched No-Signature control and several representation probes.

Critical strengths

S1. The shared-query construction is clear and useful. Comparing predictions at the same observation avoids treating two different histories as if they contained directly comparable situations. The training and zero-shot evaluation protocols are described in enough detail to follow the method.

S2. The paper includes useful controls and qualifications. FSIG is compared with a controller that receives no signature under a matched training setup. Its LBF gains over this control are small but consistent: +0.00994 and +0.01115, with all five paired repetitions positive on both grids. The paper distinguishes the highest mean from a significant gain.

Critical weaknesses

W1. The evidence for coordination-relevant signatures is indirect. The loss predicts teammate actions and makes predictions from two histories of the same checkpoint consistent; checkpoint identity defines the training groups (Sec. 4.1). That is a reasonable way to learn behavior, but query decisions are sampled uniformly. The objective does not distinguish actions that change the ego's best response from those that do not, or align different partners who call for the same response. No experiment directly tests that distinction. Since this is the paper's central representation claim, behavioral prediction and downstream return alone do not settle it.

W2. The matched control gives a mixed picture of the signature's contribution. On both LBF grids, FSIG improves on No-Signature with paired intervals above zero (Table 4). On all three OvercookedV2 tasks, the point estimates are positive but the intervals include zero. Classic Overcooked has no No-Signature row, although it contains some of the paper's most prominent cross-play comparisons. Adding that row would show whether the signature helps there. This matters because on LBF the No-Signature controller already exceeds every external baseline, so the external gap alone cannot measure the signature's contribution.

W3. The prediction results need a stronger reference point. FSIG barely improves on Context-only on GCS and Demo. On TTS the gain is larger, but its cross-entropy is worse than a uniform six-action predictor. An action-frequency baseline would make these numbers easier to interpret. Context-only also differs from FSIG in both its lack of history and its lack of the consistency loss. A history-conditioned predictor without that loss would help isolate what history adds.

W4. The consistency and intervention results are hard to connect to coordination. The consistency loss changes the signature predictions, but its measured return effects are inconclusive (App. B.5). In the TTS intervention, the main-text bootstrap interval excludes zero, while the paired-t interval on the same five repetition differences includes zero and the sign-flip test is not significant. Same and Different also have similar returns. These results suggest that the controller responds to the signature stream, but do not show that it uses teammate information rather than cues from the current trajectory.

W5. External comparisons are harder to read than the matched control. The baseline set changes across suites, and external methods use their own training populations and budgets. On Asymmetric Advantages, FSIG's difference from MEP is +121 against SP partners but −119 against E3T partners (Table 5); the overall mean hides that variation. I would put more weight on the matched No-Signature comparisons and show partner-source results more prominently. A budget-matched external comparison on a compatible task would help.

Questions for authors
  1. Can you test whether partners that call for the same ego response receive similar signatures, even when their overall behaviors differ?
  2. What is the FSIG–No-Signature difference on Classic Overcooked under the same controller training budget?
  3. How do the prediction results compare with an action-frequency baseline and with a history-conditioned predictor trained without the consistency loss?
  4. Does the consistency loss improve return across the nine tasks? How robust is the TTS intervention result across the reported analyses?
  5. Can you report the key Classic Overcooked contrasts by partner source and clarify the relative training budgets of FSIG and the external baselines?
Reviewer_rHbFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes FSIG, a teammate representation for zero-shot coordination with unseen partners. FSIG trains in two stages. First, a recurrent encoder maps an interaction history to a signature. A shared decoder uses this signature to predict the teammate's action at a query observation taken from a separate episode. The loss combines action prediction with a Jensen–Shannon consistency term between two histories of the same teammate. Second, the encoder is frozen, and a recurrent PPO controller learns to use the signature to maximize team reward. The authors evaluate on nine tasks from classic Overcooked, OvercookedV2, and Level-Based Foraging against held-out partners, comparing to population-based, teammate-modelling, and self-play baselines. FSIG reports the highest mean return on several tasks and competitive return on the rest. Further analyses show that history slightly improves teammate-action prediction, that the consistency loss improves agreement across histories, and that replacing the signature lowers return.

Critical strengths
  • The paper is easy to follow at a high level. The motivation, to represent how a teammate responds rather than who it is, is clear and makes sense for coordinated multi-agent control. Figure 1 and the shared-query idea make the method easy to understand.
  • The evaluation is broad, and include 9 tasks from 3 benchmark suites, including OvercookedV2 and LBF, test the method beyond the usual classic Overcooked layouts.
  • The statistical reporting is honest and rigorous. The paper reports paired confidence intervals, matched controls, and its own null results, which makes the evidence easy to assess.
  • The method is simple and cheap at deployment. It needs no test-time updates or query episodes, only a frozen encoder that updates within the episode.
Critical weaknesses
  • The central claim is not well supported in the results. In the TTS intervention a signature from the same teammate does no better than one from a different teammate. This suggests the controller relies on current-trajectory context rather than teammate-specific information.
  • The representation carries little behavioral information, as shown by the TTS predictor's cross-entropy being worse than a uniform guess, GCS and Demo being only slightly better, and history improving prediction by just 0.004–0.07 nats.
  • The results are mostly not significant, as shown in Table 4, where all three OvercookedV2 intervals against the No-Signature control include zero. Classic Overcooked lacks this control, and most intervals against FCP also include zero, leaving LBF as the only clear effect at about 0.01 return.
  • The comparison is hard to interpret because the baselines change between suites: LIAM, the closest teammate-modelling baseline, appears only in OvercookedV2, E3T and MEP only in classic Overcooked, and LBF has only two baselines.
Questions for authors
  • If same-teammate and different-teammate signatures give the same return on TTS, what teammate-specific information does the signature carry? Does the Same vs. Different gap appear on LBF, where the signature's effect is significant?
  • Why is the TTS predictor's cross-entropy worse than a uniform guess? Is the decoder undertrained, or can the ego's observations not predict the heuristic partners' actions?
  • Could you add the No-Signature control in classic Overcooked and run LIAM and E3T on all three suites, so that FSIG can be compared against teammate-modelling methods consistently?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.