acceptodds
Under review as a conference paper at ICLR 2027

Partially Observable Zero-shot coordination by Predicting Intention of Partner

Abstract

Zero-shot coordination in embodied settings requires acting while the partner is intermittently out of view, leaving existing methods with ambiguous partner representations and uncertainty over hidden partner states. We propose Predicting Intention of Partner (PIP) to jointly address these challenges. PIP uses a Joint-view VAE to distill richer training-time evidence from the union of both agents' local observations into a partner representation available from local observations alone. Partner-state Belief networks further infer the partner's hidden location and behavioral tendencies from the ego agent's interaction history. We evaluate PIP in Burrito-PO, Overcooked-PO, and a Melting Pot substrate, together with a human evaluation in Burrito-PO. PIP attains the highest mean performance among the compared methods across all three benchmarks. Human evaluation and diagnostic analyses further support coordination with unseen partners and the contributions of both components under partner occlusion.

Primary area:
reinforcement learning
Keywords:
Zero-Shot Coordination, Agent collaboration, Agent cooperation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Y3SSOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper considers the problem of zero-shot coordination in (spatially-induced) partially observable environments. Specifically, the authors focus on a class of methods that learn a VAE-based generative model that captures diverse play styles from gameplay data and then trains a cooperator policy against latent conditioned policies sampled from the generative model.

The authors introduce the Predicting Intention of Partner (PIP) framework that consists of two main components. They generative model is a joint-view VAE that conditions on the local observations of both players, to produce the latent. They also add explicit partner modelling objectives to boost policy learning in the (spatially-induced) partially observable setting.

Critical strengths
  1. The paper is easy to read and follow at a high level. The studied problem setting is quite clear and the proposed contributions are clearly stated.

  2. The presented experiment results are positive, with PIP outperforming the proposed baselines. The human subject experiments results are particularly encouraging.

Critical weaknesses

The paper has multiple critical weaknesses as described below, and hence I provide a Reject rating.

Problem Setting is not well motivated

  1. [Line 102] “To the best of our knowledge, our work is the first to address the failure modes that dynamic spatial partial observability induces in both partner generation and policy learning in ZSC”

  2. Why should we explicitly focus on (spatially induced) partial observability? What insights do we hope to gain from this perspective? More importantly, why are the general belief modelling approaches (LBS [4], OBL [5]) and partner modelling approaches ([1],[2],[3]) from prior work not applicable?

The proposed method introduces multiple novel design choices when simpler alternatives are available.

The paper also lacks the necessary ablations to justify these design choices. Additionally, some of these design choices appear very specific to grid-world-like environments, raising concerns about the scalability of the proposed approach.

  1. In Eq. (2), how is the union over observations computed? Does it scale to high-dimensional observations? Why not simply concatenate the local observations as is commonly done in the literature (e.g., MAPPO)?

  2. If the challenge is that the partner disappears from the field of view, how would distilling the posterior from the teacher help? The teacher, with the full POV, can trivially locate the partner, whereas the student’s local observation simply does not contain this information, making the task impossible.

  3. What is the motivation to maintain a student and a teacher VAE? Why can’t the teacher VAE be used to train the cooperator policy?

  4. Missing Ablations:

    1. [Line 239] Obs masking to remove “direction orientation shortcut” — what does this mean? What happens if you omit the mask?

    2. Action prediction target — can you ablate against the simple target of predicting the next action?

    3. Location belief

      1. Can you ablate against the simple case of predicting the next location?
      2. How is this information accessible during training? Are you assuming access to the underlying state s during training? If yes, could you pass s to the VAE as input?
    4. Why should the PPO update not affect the hidden belief state?

      1. What happens if it does?
      2. How important is u_t?
      3. L_aux is not defined

Experimental Details are not clear for reproducibility

  1. The paper introduces a large number of HPs, but says very little about how they are tuned. Which HPs are copied from the baselines, and which ones are tuned? Do you provide the baselines with the same HP tuning budget?

  2. It is unclear whether the agent is optimised using IPPO or MAPPO

  3. To my understanding, the Melting Pot env does not use a common reward, but the authors say they maintain the original rewards. This is confusing.

  4. It is unclear what the error bars in Tables 1-3 represent. Which part(s) of the entire pipeline are trained using independent random seeds?

  5. What do the Fixed sigma and Het sigma baselines in Figures 3–4 correspond to? Is one of them Gamma?

Baselines

  1. There is a large body of prior work on prediction of partners’ observations and actions ([1],[2],[3]). How does the proposed method compare to this?

References

  1. Georgios Papoudakis, Filippos Christianos, and Stefano V. Albrecht. Agent Modelling under Partial Observability for Deep Reinforcement Learning. NeurIPS, 2021.

  2. Arrasy Rahman, Ignacio Carlucho, Niklas Höpner, and Stefano V. Albrecht. A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learning. Journal of Machine Learning Research, 24(298):1–74, 2023.

  3. Dung Nguyen, Hung Le, Kien Do, Svetha Venkatesh, and Truyen Tran. Social Motivation for Modelling Other Agents under Partial Observability in Decentralised Training. IJCAI, pp. 4082–4090, 2023.

  4. Hengyuan Hu, Adam Lerer, Noam Brown, and Jakob Foerster. Learned Belief Search: Efficiently Improving Policies in Partially Observable Settings. arXiv:2106.09086, 2021.

  5. Hengyuan Hu, Adam Lerer, Brandon Cui, Luis Pineda, Noam Brown, and Jakob Foerster. Off-Belief Learning. ICML, PMLR 139:4369–4379, 2021.

Questions for authors

Please answer:

  1. Question 2 on problem setting
  2. Questions 1-4 on methods
  3. Question 1 on reproducibility
  4. Question 1 on baselines
19 comments

Table Reference Convention. Throughout our responses, numbered tables and figures (e.g., Tables 1–3, Fig. 2) refer to the main manuscript, letter-prefixed tables and figures (e.g., Table F.2) refer to the supplementary material, and Tables R9–R12 are provided directly in this rebuttal.


Clarification on Representation-Based Agent Modeling and Partner Generation

Before addressing the individual questions, we clarify the distinction between representation-based agent-modeling methods [1-3] and partner-generation methods such as GAMMA, GOAT, and PIP, particularly regarding their feature and optimization pathways.

In representation-based agent modeling, representations inferred from the controlled agent's interaction history directly condition its policy or action-value function. LIAM [1] employs an encoder-decoder architecture rather than a VAE, using reconstruction of other agents' observations and actions to learn a representation for policy conditioning, as illustrated by

zt=fϕ(htego),πC(atego∣htego,zt).z_t=f_{\phi}(h_t^{ego}),\qquad \pi_C(a_t^{ego}\mid h_t^{ego},z_t).

GPL [2] uses a variational belief posterior in its action-value computation, while Social Motivation [3] similarly uses learned agent representations for decision-making.

In contrast, GAMMA and GOAT use VAE latents to generate training partners rather than directly condition the Cooperator. GAMMA samples a latent from the prior and passes it to a pretrained VAE decoder to generate a partner policy. The Cooperator then learns through environment interactions and rewards obtained with these generated partners, following

z∼p(z),μz(atp∣htp)=pθ(atp∣z,htp).z\sim p(z),\qquad \mu_z(a_t^p\mid h_t^p)=p_{\theta}(a_t^p\mid z,h_t^p).

J(πC)=Ez∼p(z)[V(πC,μz)].J(\pi_C)=\mathbb{E}_{z\sim p(z)}[V(\pi_C,\mu_z)].

GOAT follows this pathway with adversarial latent selection. The pretrained partner generator remains fixed during Cooperator PPO training. Notably, GAMMA itself distinguishes direct latent conditioning as a separate, orthogonal approach in Appendix J.

PIP follows the same partner-generation pathway, using Joint-view VAE to distill a richer-view teacher posterior into a local-view student whose decoder generates training partners. Neither the VAE posteriors nor the sampled latent directly enters the Cooperator policy. Thus, PIP's distillation improves Partner Generation rather than directly inferring hidden partner states for Cooperator decision-making.

[PS1, PS2] Why viewpoint-induced partial observability, and how it relates to LBS, OBL and agent modelling

Why focus specifically on spatially induced PO in ZSC?

We acknowledge that spatial partial observability has been studied in MARL, including the works cited by the reviewer [1-3], and that PO has also been explored in ZSC, including Hanabi, OvercookedV2, and ERS. Unlike Hanabi, where information is hidden by game rules, spatial PO causes the partner to dynamically leave and re-enter the ego agent's field of view depending on their relative positions and movements. However, it is important to distinguish the coordination strategy compatibility problems primarily addressed in existing PO-ZSC research from the problems that dynamic spatial PO introduces into the ZSC learning process itself. In generative partner-based ZSC, diverse partner policies are generated to train a Cooperator capable of coordinating with unseen partners. Therefore, our motivation is not simply that such observation loss requires hidden-state inference, but that it can affect both Partner Generation and Policy Learning.

How does this differ from existing belief- and agent-modelling approaches?

LIAM [1], GPL [2], and Social Motivation [3] primarily model other agents under partial observability to improve decision-making, while LBS [4] and OBL [5] address belief-based decision-making and coordination. These approaches may be applicable or complementary to Policy Learning under PO, but do not directly investigate how limited observations affect the generation of diverse training partners in generative ZSC. Specifically, dynamic spatial PO introduces two distinct challenges:

(1) Partner Generation: Spatial PO can make distinct partner behaviors appear similar in local trajectories. When important behavioral segments occur outside the FOV, the VAE encoder may fail to distinguish behavioral styles, leading to overlapping latent representations and potentially limiting training partner diversity.

(2) Policy Learning: When the partner leaves the FOV, the cooperator loses direct access to its location and actions. In ZSC, the cooperator must coordinate with partners whose behaviors were not encountered during training, requiring it to infer hidden states and behavioral tendencies from its limited observation history.

Although these two problems are related, addressing one does not automatically resolve the other. PIP mitigates partner representation ambiguity through the Joint-view VAE and infers the location and actions of unobserved partners through Partner-state Belief.

What insights do we gain from studying Spatial PO?

For Partner Generation, Fig. 3 and Supplementary Fig. F.1 show improved partner-specific latent separation through Joint-view VAE distillation, despite the student using the same local observations as the baseline VAE. Fig. 4 further shows improvements in coverage and MMD, supporting better latent distribution quality.

For Policy Learning, Fig. 5 shows that removing the Belief message changes the policy's action distribution more strongly shortly after the partner leaves the FOV, with the effect decreasing as Time Since Last Seen (TSLS) increases. Fig. 6 further shows larger coordination reward reductions for partners with higher occlusion ratios. These findings indicate that the contribution of inferred partner information depends not only on partner visibility, but also on the timing and frequency of dynamic observation loss. Table 3 further shows that combining JV-VAE and Partner-state Belief achieves higher coordination performance than either component alone.

Therefore, our novelty claim in Lines 102–103 concerns not Spatial PO itself, but the joint analysis and mitigation of two distinct failure modes induced by dynamic spatial PO in ZSC, namely behavioral representation ambiguity during Partner Generation and hidden partner-state uncertainty during Policy Learning.

[M1] Forming the union view, its scalability, and why not concatenation

In Burrito-PO, each local observation is a map-sized grid with cells outside the agent's 7×77 \times 7 field of view masked, so both observations share the same coordinate frame. The union in Eq. (2) combines visible regions cell-wise, leaving cells unseen by both agents masked. Compared with direct concatenation, Concat(oti,otiˉ)\mathrm{Concat}(o_t^i,o_t^{\bar{i}}), our teacher input Concat(uti,oti)\mathrm{Concat}(u_t^i,o_t^i) has the same dimensions H×W×2CH \times W \times 2C and encoder memory requirements, adding only a parameter-free cell-wise operation. The union provides a unified spatial representation of both views, while otio_t^i identifies regions visible to the modeled agent. This additional operation is performed only during VAE training, since the teacher is not used during Cooperator PPO training or deployment. We will clarify these details in Appendix C.

[M2, M3] Motivation for teacher-student distillation and partner generation

We apologize that Sec. 4.1 did not sufficiently clarify the purpose of distillation. First, the teacher does not observe the full global state, but uses the union of both agents' local observations (Eq. (2)). More importantly, the student is not trained to reconstruct an unobserved partner's location. Since the purpose of the VAE is to generate diverse training partners, both encoders infer a posterior over the modeled agent's behavioral latent zz, rather than its hidden location. The teacher uses richer joint-view evidence to distinguish behavioral modes, while the student learns from local observations through posterior distillation. Although the student cannot recover unobservable information exactly, the forward KL in Eq. (4), considering only the distillation term and an unrestricted student distribution, has the optimum

qS∗(z∣τS)=EτT∣τS[qT(z∣τT)].q_S^*(z\mid\tau_S)=\mathbb{E}_{\tau_T\mid\tau_S}[q_T(z\mid\tau_T)].

Thus, the student learns the teacher posterior averaged over information unavailable in its local trajectory. Figures 3–4 and Supplementary Fig. F.1 show improved latent separation and distribution quality, while Table 3 shows a reward improvement from 163.3 to 181.6.

After distillation, the student decoder generates training partners using only local observations. The teacher decoder could also be used if it operates on local observations. However, we use the student decoder because it is trained on local-view trajectories with posterior distillation, whereas the teacher VAE learns its posterior from richer joint-view evidence. Since we have not directly compared the two decoders, we do not claim that the teacher decoder would necessarily perform worse.

[M4.1] Obs masking

The mask in Line 239 is applied to the agent-orientation channel only in the belief-module input. Since orientation is encoded at each agent's position, this channel directly provides a visible partner's facing direction, which could let the belief module shortcut Behavioral Belief prediction rather than rely on the observation history. The actor retains this channel for action selection, while cells outside the field of view are already masked across all channels at the environment level for both the actor and the belief module. Thus, omitting the mask would not expose any out-of-FOV information or change the spatial PO setting, but the mask is necessary to block this orientation shortcut.

[M4.2, M4.3.1] Ablation of simpler prediction targets

We conducted the requested ablations by replacing either the Behavioral Belief or Location Belief prediction target while keeping all other settings identical to PIP (Table R9). Replacing the reaction and tendency targets with next-step action prediction reduces reward from 249.0±12.2249.0 \pm 12.2 to 162.1±6.6162.1 \pm 6.6. For Location Belief, predicting the next location (Δ=1\Delta=1) with a one-hot target yields 84.0±5.284.0 \pm 5.2, while using the uncertainty-aware soft target(Eq. (6)) improves performance to 219.9±7.0219.9 \pm 7.0. These results support the benefits of multi-step Behavioral Belief targets and uncertainty-aware, multi-offset Location Belief prediction.

Table R9. Ablation of alternative Behavioral Belief and Location Belief prediction targets on Burrito-PO Open. All other settings are identical to PIP. Results are mean reward ±\pm SE.

Alternative prediction targetReward ↑
Next action162.1 ± 6.6
Next location (hard, Δ=1\Delta=1)84.0 ± 5.2
Next location (soft, Δ=1\Delta=1)219.9 ± 7.0
PIP (original targets)249.0 ± 12.2

[M4.3.2] How is this information accessible during training? Could you pass ss to the VAE?

Our setting is motivated by embodied agents, such as robots, that can access their own local observations and record their own positions and actions, but may not have access to the global state ss. Under this assumption, we consider each agent's logged positions and actions available as training-time supervision for the belief heads. Following the same motivation, we deliberately use the union of both agents' local observations rather than the global state ss for Joint-view VAE. We will clarify these information-access assumptions in Secs. 3 and 4.1.

Reviewer_B2XAOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors study zero-shot coordination, where an agent has to work with partners it never saw during training, in settings where the partner often moves out of the agent’s field of view. They argue that this kind of spatial partial observability causes two problems: the model of possible partners loses diversity, because different partner behaviors look the same from a limited view, and the agent loses track of where the partner is and what it is doing. To address the first problem, the authors train a partner model where a teacher sees both agents’ views during training and distills what it learns into a student that only sees the agent’s own view. They address the second problem by adding belief networks that predict the partner’s location and behavior from the agent’s own history, using logged partner information as supervision during training only. The authors evaluate their method on partially observable versions of different tasks and they run a human study in one of them.

Critical strengths
  1. The authors clearly motivate the problem and separate it from prior work. In Hanabi-style settings, the game rules hide information, while here hidden information comes from viewpoint and movement, which is closer to real embodied collaboration.
  2. The authors keep all the extra information on the training side. The teacher and the logged partner state only shape training, and the deployed agent acts only on its own observations.
  3. Their method (PIP) performs best across all three benchmarks, often by a large margin, and in Figure 2, 66 human participants got higher rewards with PIP and preferred it as a partner over the two strongest baselines.
  4. In Section 6.3, the authors show that removing the belief changes the agent’s actions most right after the partner leaves the field of view, and that the belief helps most when the partner is out of view more often. This supports the idea that the belief module does what it is supposed to do.
Critical weaknesses
  1. PIP uses extra information during training that the baselines may not get: the union of both agents’ views for the teacher, and logged partner state for the belief networks. It’s not clear whether the comparison is fair, or whether the baselines would also improve with the same training signals.
  2. The evaluation outside Burrito is narrow. The authors use a single Overcooked layout and a single Melting Pot task, and only compare against two baselines there. In Burrito, the authors pick the 12 test partners themselves and don’t say how many training seeds each method uses.
  3. The belief targets rely on many hand-designed choices and weights, like coarse grid cells, a reachability prior, and several discount and offset parameters. The authors don’t show how sensitive PIP is to these choices, and some of them seem tied to grid-world layouts.
  4. The name and framing talk about predicting the partner’s intention, but the belief networks predict location and action tendencies, not intentions. The main ideas, distilling from a teacher with extra information and adding belief prediction as an auxiliary task, are also known, and the related work doesn’t discuss work on learning with privileged information.
Questions for authors
  1. Do the baselines get the same privileged training signals as PIP? If you give them the same signals, how much of the gap remains?
  2. How sensitive is PIP to the design choices and weights of the belief targets?
  3. In Table 1, is the standard error over partners or over training seeds? How many seeds do you use, and how do you select the 12 test partners?
  4. Do the ablation results hold on other layouts, especially Ring, where PIP shows the largest relative gain?

Suggestion

  • In Figure 5, the authors match each line to its axis only by color, which is hard to follow in black and white. A legend would make the figure easier to read.
10 comments

Table Reference Convention. Throughout our responses, numbered tables and figures (e.g., Tables 1–3, Fig. 2) refer to the main manuscript, letter-prefixed tables and figures (e.g., Table F.2) refer to the supplementary material, and Tables R4–R8 are provided directly in this rebuttal.


[W1, Q1] Fairness of Privileged Training Signals and Controlled Ablation.

We acknowledge that the original baselines and PIP in Table 1 of main paper do not receive identical privileged training signals, so the main comparison is not strictly controlled for training-time information. However, we hope the reviewer will consider that determining what information to exploit during training and how to incorporate it into partner generation and decision-making under spatial PO is itself a central contribution of our work.

To provide a fairer, more controlled comparison, Table R4 extends the ablation reported in Table 3 of the main manuscript. Specifically, (a) and (b) correspond to the previously reported GAMMA and GAMMA + JV-VAE settings, respectively, with (b) exploiting union-view information during training. (c) further adds the same future-action and location supervision as PIP, using a shared RNN without a separate belief branch and with matched model capacity. Finally, (d) retains the same training signals but employs PIP's dedicated recurrent belief branch, gradient separation, and belief injection. Even with identical privileged training signals and comparable model capacity, a gap of +53 reward points remains between (c) and (d). This supports that PIP's gains cannot be explained solely by additional information, but also depend on how partner-state information is modeled and utilized for decision-making under spatial PO.

Table R4. Controlled comparison of privileged training signals and belief modeling on Burrito-PO (Open).

VariantMethodUnion ViewPartner StateBelief ModelingReward ↑
(a)GAMMA———163.3 ± 10.6
(b)+ JV-VAE✓\checkmark——181.6 ± 9.1
(c)Shared RNN + Aux.✓\checkmark✓\checkmarkShared RNN196.0 ± 16.3
(d)PIP (Ours)✓\checkmark✓\checkmarkDedicated Belief RNN249.0 ± 12.2

[W2a] Evaluation Diversity and Baseline Selection.

We acknowledge the reviewer's concern that the diversity of layouts and baselines evaluated beyond Burrito-PO remains limited. To address this, we are conducting additional experiments with GAMMA, GOAT, and PIP on another layout of Overcooked-PO and will share the results as soon as they become available. Regarding baseline selection, we chose GAMMA and GOAT because they are the most directly comparable generative-based ZSC approaches to PIP. While these additional experiments do not cover all possible task structures or ZSC methods, we hope they will help address the reviewer's concerns about evaluation breadth.

[W2b, Q3] Test Partner Selection and Training-Seed Robustness

As described in Appendix B.2 of our supplementary material, the 12 Burrito-PO test partners were selected following ZSC-Eval's BR-diversity procedure (Wang et al., 2024), which selects 12 of 24 behavior-preference candidates to maximize behavioral diversity based on best-response agents' event statistics, independently of PIP's performance. In the original submission, all methods used the same fixed training seed, and the reported SE reflected ten repeated evaluations aggregated over 12 held-out partners, not independent training seeds. This evaluation was intended to characterize coordination performance variability across diverse unseen partners and repeated episodes. To assess training-seed robustness, we retrained the target policies with three independent seeds per method across all benchmarks, keeping the preceding Partner Generation stages fixed. We report mean ±\pm SE across seed-level mean rewards in Table R5 and Table R6. PIP achieves the highest mean performance across all settings, supporting robustness to Cooperator training seeds. These results complement the original held-out-partner evaluation. We will clarify the partner selection and statistical aggregation protocols in the revised manuscript.

Table R5. Training-seed robustness on Burrito-PO. Results are mean ±\pm SE over three independent training seeds.

MethodOpenHallwayForced-CoordRing
FCP145.1 ± 2.667.9 ± 5.619.6 ± 1.247.0 ± 2.4
MEP124.9 ± 5.768.9 ± 5.618.3 ± 0.750.4 ± 1.6
E3T59.2 ± 3.343.2 ± 1.619.2 ± 1.029.28 ± 0.9
ERS95.6 ± 7.051.3 ± 7.923.3 ± 0.746.2 ± 3.1
GAMMA158.0 ± 8.577.14 ± 9.719.5 ± 3.454.1 ± 4.5
GOAT154.3 ± 5.771.9 ± 5.218.7 ± 0.470.6 ± 4.7
PIP (Ours)266.0 ± 8.594.5 ± 8.625.1 ± 2.1138.5 ± 9.3

Table R6. Training-seed robustness on Overcooked-PO and Melting Pot. Results are mean ±\pm SE over three independent training seeds.

MethodOvercooked-POMelting Pot
GAMMA312.4 ± 29.0177.2 ± 3.0
GOAT346.2 ± 10.1150.7 ± 3.9
PIP (Ours)399.0 ± 6.1223.5 ± 8.7

[W3, Q2] Sensitivity to belief-target hyperparameters

We conducted a one-at-a-time sensitivity analysis of seven key hyperparameters of the Partner-state Belief module, as shown in Table R7. Importantly, we use the same Partner-state Belief hyperparameters across all evaluated environments without environment-specific retuning.

Table R7. Sensitivity analysis of belief-target hyperparameters. Bold values indicate the default settings used in PIP. Results are reported as mean reward ±\pm standard error.

HyperparameterValueReward ↑
Coarse grid scale4 × 4209.0 ± 16.0
2 × 2249.0 ± 12.2
1 × 1203.7 ± 4.6
HmaxH_{\mathrm{max}}0203.8 ± 12.1
50249.0 ± 12.2
100191.6 ± 8.3
λbehavior\lambda_{\mathrm{behavior}}0.01164.4 ± 11.0
0.1249.0 ± 12.2
1.080.0 ± 5.9
λlocation\lambda_{\mathrm{location}}0.1171.7 ± 8.3
1.0249.0 ± 12.2
10.0147.8 ± 12.1
γrec\gamma_{\mathrm{rec}}0.8195.2 ± 6.7
0.9249.0 ± 12.2
0.97205.2 ± 11.4
γten\gamma_{\mathrm{ten}}0.8182.1 ± 8.9
0.9189.6 ± 10.0
0.97249.0 ± 12.2
KoffK_{\mathrm{off}}5203.3 ± 10.6
10249.0 ± 12.2
20218.1 ± 14.4

The default configuration achieves the highest mean reward among all tested settings. For Location Belief, changing the coarse-cell size or HmaxH_{\mathrm{max}}, which determines when the reachability prior is replaced by a uniform spatial prior, reduces coordination performance. For Behavioral Belief, varying the discount factors (γrec\gamma_{\mathrm{rec}}, γten\gamma_{\mathrm{ten}}) for the reaction and tendency targets, as well as KoffK_{\mathrm{off}}, which separates their prediction ranges, also affects performance. In particular, PIP is sensitive to the loss weights of Behavioral Belief (λbehavior\lambda_{\mathrm{behavior}}) and Location Belief (λlocation\lambda_{\mathrm{location}}), with larger weights substantially degrading performance. These results highlight the importance of balancing the two belief supervision objectives.

Regarding applicability beyond grid-world environments, Location Belief currently represents partner positions as distributions over coarse cells and uses a reachability prior based on the last observed partner location. These components could potentially be extended to continuous spaces through spatial discretization and distance-based reachability estimates. However, we have not yet evaluated PIP in continuous 3D environments. We will clarify this limitation and include the sensitivity results in the revised manuscript.

Reviewer_dzQMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies zero-shot coordination when partners are intermittently outside an agent’s local field of view. It proposes PIP, combining a Joint-view VAE for partner generation with explicit partner-state belief learning. A teacher using the union of both agents’ local observations guides a local-view student, whose decoder generates training partners. A separate recurrent belief module predicts partner locations and short- and longer-horizon action tendencies, and supplies its representation to a PPO-trained policy. Evaluation covers Burrito-PO, one Overcooked-PO layout, one Melting Pot substrate, and a 66-participant human study.

Critical strengths
  1. The problem is practically meaningful and the observation constraints are clearly specified. Intermittent partner visibility introduces challenges beyond unfamiliar partner conventions. Addressing both the training-partner distribution and hidden-state inference is a coherent response. The paper also distinguishes training-only information from deployment inputs: the deployed coordinator relies on local observations and its own inferred beliefs.
  2. The human evaluation provides meaningful evidence of practical effectiveness. The within-subject study uses randomized agent order and anonymized policies. Participants achieve higher rewards with PIP, and 48 of 66 rank it first. The inclusion of paired statistical tests and multiple-comparison correction strengthens this evidence. These results support improved cooperation with unfamiliar humans under the tested conditions.
  3. The experimental analysis goes beyond aggregate performance. Component ablations, belief prediction diagnostics, belief-message lesions, representation analysis, and capacity/supervision controls collectively provide useful evidence about the system. The substantial improvements on several layouts are encouraging, although not every reported mean advantage is conclusive.
Critical weaknesses
  1. The methodological novelty is limited and needs sharper justification. The framework combines established ingredients: latent-variable partner generation, richer-view teacher–student distillation, recurrent partner modeling, auxiliary prediction, and PPO. Its main contribution appears to be their integration for spatial partial observability, rather than a substantially new coordination mechanism. The paper needs to establish more clearly what this particular integration achieves beyond simpler combinations of existing approaches. In particular, the title’s “intention prediction” is operationalized through coarse location prediction and future-action distributions, rather than explicit inference of goals or subtasks. The empirical benefits are valuable, but do not by themselves establish a distinctive technical contribution.
  2. The controls do not fully isolate the source of the belief module’s advantage. Appendix F.4 acknowledges that the supervision control uses different action targets from PIP: current-action prediction versus discounted future-action distributions. Therefore, these comparisons do not cleanly separate the effects of predictive targets, a dedicated recurrent state, gradient separation, and the belief-injection interface. A stronger comparison would match the auxiliary targets and supervision while varying the architectural choices. Without it, the results establish that the complete system works, but provide weaker evidence that its specific belief architecture is necessary.
  3. Robustness across training runs and task structures remains insufficiently characterized. The manuscript does not clearly specify the number of independent target-policy training seeds or whether the reported standard errors capture training variability. Evaluating many episodes and held-out partners does not replace independent training repetitions. This is particularly relevant to weaker results such as Forced-Coord, where the mean advantage is small relative to the reported uncertainty. The supplementary environments also have narrow coverage: one Overcooked layout and one Melting Pot substrate. The rationale for selecting these tasks is reasonable, but does not establish robustness across different topologies and cooperation mechanisms. The Melting Pot small-to-large-map evaluation is useful evidence of spatial-scale transfer within the same task; it should remain distinct from cross-task generalization. The human study likewise supports effectiveness under its tested conditions, rather than broad adaptability across human coordination settings.
Questions for authors
  1. What is the strongest technical distinction between PIP and combining a generative-partner baseline with standard recurrent partner prediction? Which experiment most directly establishes this distinction?

  2. Does PIP retain its advantage when the comparison method uses identical future-action and location targets, matched capacity, and comparable optimization settings?

  3. How many independently trained target policies contribute to each result, and what is the aggregation unit for the standard errors? Do the gains persist across additional layouts or cooperation mechanisms?

6 comments

Table Reference Convention. Throughout our responses, numbered tables and figures (e.g., Tables 1–3, Fig. 2) refer to the main manuscript, letter-prefixed tables and figures (e.g., Table F.2) refer to the supplementary material, and Tables R1–R3 are provided directly in this rebuttal.


[W1, Q1] Novelty and Distinction from Existing Approaches.

We thank the reviewer for the opportunity to clarify our contribution. Rather than proposing a new coordination objective, we identify and jointly address two bottlenecks introduced by spatial partial observability (PO) in generative-based ZSC:

  • Limited partner-strategy coverage during generation
  • Uncertainty about hidden partners during decision-making

Our contribution lies not in simply combining existing components, but in identifying these challenges and designing complementary solutions tailored to spatial PO.

(1) Limited partner-strategy coverage during generation. Under spatial PO, distinct partner behaviors can appear as similar local trajectories. This can lead to overlapping latent representations and limit the diversity of generated training partners. To address this, JV-VAE distills richer joint-view information into a local-view student, improving the discriminability of behavioral representations.

(2) Uncertainty about hidden partners during decision-making. When a partner moves outside the FOV, the cooperator loses direct access to the partner's state. This is particularly challenging in ZSC, where the cooperator must coordinate with unfamiliar partners whose behaviors were not encountered during training. To address this, PIP maintains a separate recurrent belief state that predicts short- and long-term behavioral tendencies and current/future partner locations. Rather than relying on conventional next-action prediction alone, PIP learns discounted future-action distributions and uncertainty-aware location beliefs, which are incorporated into the actor's decisions under intermittent visibility.

Table F.2 provides the most direct comparison with a straightforward combination of generative partner modeling and recurrent prediction. Adding auxiliary partner prediction to a shared recurrent policy (88.7), even with matched capacity (96.0), does not reproduce PIP's performance (249.0). We acknowledge that these controls use different prediction targets and therefore do not isolate the contribution of each architectural choice. Nevertheless, this distinction is itself relevant to our motivation: simply adding conventional prediction supervision is insufficient in the evaluated setting, highlighting the importance of designing what to predict and how to use it under spatial PO. Table 3 further supports the complementary contributions of JV-VAE and Partner-state Belief.

Finally, we agree that “Intention” may overstate our model's capability. Our aim was to emphasize inferring hidden partner states and behaviors to guide coordination, rather than explicit goal inference. We will adopt the more precise term “Partner-state Belief” and clarify these distinctions in the revised manuscript. We appreciate the reviewer's constructive feedback.

[W2, Q2] Isolating the Contributions of Belief Architecture.

We agree that the original controls in Table F.2 use different prediction targets and therefore do not fully isolate the contributions of the individual design choices.

To address this concern, we designed additional target-matched controls using the same JV-VAE partner generator, identical discounted future-action and location prediction targets, auxiliary losses, approximately matched capacity, and PPO training settings. Specifically, we compare (a) a shared recurrent policy with matched auxiliary supervision, (b) a separate belief RNN allowing PPO gradients, (c) a separate belief RNN with stop-gradient and concatenation-based injection, and (d) our full PIP. In all variants, ground-truth partner states are used only for training-time auxiliary supervision and are never directly provided to the actor.

As shown in Table R1, all controls underperform PIP despite using identical prediction targets and comparable training settings.

Comparing (b) and (d) isolates gradient separation, and (c) vs. (d) isolates the injection interface. A dedicated belief RNN alone does not help when PPO gradients reach it ((b) < (a)); its benefit appears together with gradient separation. These results suggest that PIP's improvements cannot be explained solely by additional supervision or the choice of prediction targets, but also benefit from explicitly maintaining and effectively integrating learned partner-state beliefs into decision-making. We will incorporate these results and analyses into the revised manuscript.

Table R1. Target-matched belief architecture controls on Burrito-PO (Open). All variants use the same JV-VAE, prediction targets, auxiliary losses, and comparable model capacity and PPO settings.

MethodGrad PPO → BeliefInjectionReward ↑
(a) Shared RNN + Aux.N/AShared state196.0 ± 16.3
(b) Separate belief RNNYesAdditive185.1 ± 13.3
(c) Separate belief RNNNoConcat214.4 ± 9.3
(d) PIP (Ours)NoAdditive249.0 ± 12.2

[W3a, Q3a] Training-Seed Robustness and Statistical Reporting.

In the original submission, all methods were trained using the same fixed seed for a controlled comparison. The reported standard errors (SE) reflect variability across ten repeated evaluations, aggregated over 12 held-out partners, rather than variability across independent training seeds. This was intended to characterize coordination performance variability across diverse unseen partners and repeated episodes. However, we agree that training-seed robustness requires separate evaluation.

To address this, we retrained the target policies with three independent training seeds per method across all evaluation settings and report the mean ± standard error (SE) across seed-level mean rewards.

As shown in Tables R2–R3, PIP achieves the highest mean performance across all evaluated settings, with limited variation across training seeds. These results suggest that PIP's improvements are not specific to a particular training seed and remain consistent across independently trained policies. Nevertheless, we acknowledge that the performance improvement in Forced-Coord (FC) remains modest. Unlike other layouts, FC requires substantial cooperation from the partner to complete tasks, making it difficult for the target agent to obtain rewards through its own actions alone. Consequently, when partners do not cooperate effectively, reward signals can be sparse during training, making policy improvement particularly challenging. We believe this structural difficulty contributes to the relatively small performance differences among methods in FC. Overall, the additional results provide evidence of robustness across independent training runs, complementing the original evaluation across diverse partners and episodes. We will clarify the aggregation unit of the original SE and incorporate the additional training-seed robustness results into the revised manuscript.

Table R2. Training-seed robustness on Burrito-PO. Results are mean ± SE over three independent training seeds.

MethodOpenHallwayForced-CoordRing
FCP145.1 ± 2.667.9 ± 5.619.6 ± 1.247.0 ± 2.4
MEP124.9 ± 5.768.9 ± 5.618.3 ± 0.750.4 ± 1.6
E3T59.2 ± 3.343.2 ± 1.619.2 ± 1.029.3 ± 0.9
ERS95.6 ± 7.051.3 ± 7.923.3 ± 0.746.2 ± 3.1
GAMMA158.0 ± 8.577.1 ± 9.719.5 ± 3.454.1 ± 4.5
GOAT154.3 ± 5.771.9 ± 5.218.7 ± 0.470.6 ± 4.7
PIP (Ours)266.0 ± 8.594.5 ± 8.625.1 ± 2.1138.5 ± 9.3

Table R3. Training-seed robustness on Overcooked-PO and Melting Pot. Results are mean ± SE over three independent training seeds.

MethodOvercooked-POMelting Pot
GAMMA312.4 ± 29.0177.2 ± 3.0
GOAT346.2 ± 10.1150.7 ± 3.9
PIP (Ours)399.0 ± 6.1223.5 ± 8.7

[W3b, Q3b] Robustness Across Task Structures and Cooperation Mechanisms.

We acknowledge the reviewer's concern that our evaluation across different cooperation mechanisms remains limited, and agree that broader evaluation would be valuable. Nevertheless, we hope the reviewer will consider the consistent improvements already observed across distinct spatial constraints and coordination requirements.

Specifically, the four Burrito-PO layouts cover different spatial topologies, including open workspaces (Open), spatially separated roles (Forced-Coord), constrained circular paths (Ring), and distributed task stations (Hallway). PIP achieves the highest mean performance across all four layouts (Table 1). Furthermore, we observe consistent improvements across three distinct tasks and cooperation mechanisms: multi-stage cooking in Burrito-PO, ingredient- and recipe-based coordination in Overcooked-PO, and time-constrained synchronized mining in Melting Pot (Tables 1–2). Our human evaluation also shows favorable results in both coordination performance and participant preference (Fig. 2). To further assess robustness across different spatial configurations, we are currently conducting additional experiments on new Overcooked-PO layouts. We will share the results as soon as these experiments are completed.

While these results do not establish generalization to all possible cooperation mechanisms or human-AI coordination settings, we believe that the overall performance improvements across diverse spatial structures, three distinct coordination tasks, and human-partner interactions provide meaningful evidence of the effectiveness of our approach under spatial PO.

Comments on the paper

4 comments

We sincerely thank all reviewers for their valuable time, careful evaluation, and constructive feedback. We appreciate that the reviewers recognized several important strengths of our work.

In particular, B2XA explicitly highlighted that our spatial PO setting addresses a distinct challenge from conventional Hanabi-style PO in ZSC: while Hanabi hides information through game rules, our setting introduces dynamic information loss through agents' viewpoints and movements, which the reviewer recognized as more closely reflecting real embodied collaboration. Similarly, dzQM recognized the practical importance of intermittent partner visibility and the coherent motivation for jointly addressing partner generation and hidden-state inference under spatial PO. Y3SS also acknowledged the clarity of our problem setting and the presentation of our contributions. Moreover, both dzQM and B2XA recognized that privileged information is used only during training, while the deployed agent operates solely on local observations.

Regarding empirical validation, all three reviewers acknowledged PIP's performance improvements: B2XA highlighted its strong results across three benchmarks, dzQM noted substantial gains across several layouts, and Y3SS recognized its improvements over the compared baselines. We are particularly encouraged that all three reviewers positively evaluated our human study. Y3SS described its results as particularly encouraging, B2XA emphasized the improvements in human coordination and partner preference, and dzQM highlighted the rigorous within-subject design and statistical evaluation. Beyond aggregate performance, dzQM recognized the breadth of our component ablations and diagnostic analyses, while B2XA specifically noted that our belief-lesion analysis supports the intended role of partner-state belief under occlusion.

We also take the remaining concerns regarding technical novelty, controlled comparisons, and robustness seriously. In the following responses, we clarify our technical contributions and provide additional controlled ablations, independent training-seed evaluations, and analyses across spatial layouts. We hope these clarifications and additional results will help the reviewers assess our contributions more comprehensively.

Area_Chair_YHpyOpenReview ↗

Please review the authors’ rebuttal and update your assessment

Dear reviewers,

Thank you for your reviews, and thank you to the authors for their responses. Please read the authors’ rebuttal and discuss whether it addresses your main concerns. Please update your review and rating if warranted, and briefly explain your current assessment, including any concerns that remain unresolved. If your assessment is unchanged, please explain why.

Thank you for your continued engagement.

Dear Reviewers dzQM, B2XA, and Y3SS,

We sincerely thank all reviewers for their valuable and insightful reviews. Your comments and the subsequent discussions have helped us substantially improve the quality and completeness of our paper. Based on the discussion so far, we have revised the manuscript and uploaded the revised version. We will also incorporate the outcomes of the ongoing discussion into the final version.

Color coding in the revised manuscript

All changes made in response to the reviews are color-coded by reviewer as follows.

  • Red text — Reviewer dzQM
  • Blue text — Reviewer B2XA
  • Green highlight — Reviewer Y3SS
  • Purple text — Reviewers dzQM and B2XA
  • Blue text on a green highlight — Reviewers B2XA and Y3SS
  • Purple text on a green highlight — all three reviewers

Figure and table captions follow the same scheme. Most of the additional experiments and analyses from the rebuttal are collected in the new Appendix G (Additional Experiments), and the remaining changes are made in the main text and the relevant appendix sections.

Renaming the framework to PIPO

Reflecting the comments from Reviewers dzQM and B2XA, we have renamed our framework to Partner Inference under Partial Observability (PIPO). In our earlier responses, we stated that we would replace the "intention" framing with "Partner-state Belief." While revising the manuscript, however, we found that using this term for the whole framework would make the framework name identical to the name of one of its modules, Partner-state Belief. This could cause confusion, particularly in the module ablation results. We therefore clarified the framing as promised in our responses, using Partner-state Belief only for the belief module, and renamed the framework to PIPO instead. Every occurrence of PIPO is shown in purple. We kindly ask for the reviewers' understanding regarding this change.

Since the title cannot be changed during the discussion period, we have kept the current title for now. In the final version, we plan to retitle the paper "Zero-Shot Coordination under Spatial Partial Observability via Partner Inference," removing the term "Intention."

We thank the reviewers again for their time and constructive feedback.

Best regards, The Authors

Dear AC and Reviewers,

We have uploaded the revised manuscript and supplementary materials, with our responses to all reviewer comments integrated into both.

We sincerely thank the AC and the reviewers for their efforts during the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.