- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
For zero-shot coordination, this paper introduces belief-conditioned successor features (BFS), which summarize discounted future features conditioned on a belief over the partner's latent type. This paper approximates BSFs in MA-JEPA, an on-policy self-play (SP) algorithm that combines a learned history encoder with a temporal difference latent prediction objective. This objective provides reward-free supervision for long-term predictive representations. Experiments evaluate the proposed algorithm with promising results.
Critical strengths
The evaluation results seem promising.
Critical weaknesses
-
The theory assumes an exact belief, but MA-JEPA learns an approximate embedding z_t with no guarantee of posterior sufficiency. Proposition 5.4 only covers belief-independent policies, while the actual policy conditions on z_t . Thus the theoretical results do not directly justify the trained agent’s behavior.
-
Novelty seems limited. The distinction from explicit partner modelling (E3T, CBPR, etc) is clear, but the relationship to other latent-predictive / successor-style objectives (TD-JEPA, BYOL-γ, forward-backward representations, intention-conditioned value functions) is discussed only briefly. A more precise statement of what is new versus what is an adaptation of TD-JEPA to the multi-agent case would strengthen the novelty claim. TD-JEPA already combines a JEPA-style predictor with a temporal-difference target to recover successor features. As the coordination treated as a task in this paper, TD-JEPA should work in this coordination case.
-
Limited comparison to recent ZSC methods in the main text: TALENTS and PACE.
Questions for authors
How does MA-JEPA compare to other latent-predictive / successor-style objectives (TD-JEPA, BYOL-γ, forward-backward representations, intention-conditioned value functions)?
5 comments
Response to reviewer's concerns (2/2)
W3. Comparison with TALENTS and PACE
Thank you for this suggestion. We would like to explain why we chose these main baselines. SP, FCP, and E3T mainly differ in how the ego-agent meets diverse partners during training: self-play, a population of partners, or a mixture of the ego policy with a random policy. CBPR adds the inference of the partner's meta-task. Our research question is: beyond partner exposure, can the ego-agent coordinate better if it learns how the partner changes dynamics? For this reason, our main comparison focuses on these exposure-based and agent-modeling baselines.
We agree that TALENTS and PACE are relevant recent methods.
- TALENTS is particularly relevant because it combines a learned latent strategy space with strategy inference during interaction. Therefore, this gives a direct comparison between explicit strategy inference and our approach, which learns a history embedding through future-feature prediction and PPO. Its original setup uses a learned strategy library from offline trajectories, so we also report its adapted results separately.
- PACE addresses a related adaptation problem through context-aware exploration and peer identification. Its original setting uses context across multiple episodes. Our evaluation instead measures adaptation within a single episode, without prior interaction with the test partner. This protocol difference motivates reporting its adapted results separately. We nevertheless include both PACE variants in the empirical comparison.
The table reports the average over five layouts.
| Method | Shaped reward | Sparse reward |
|---|---|---|
| TALENTS | 160.3 | 72.8 |
| PACE without online adaptation | 123.9 | 68.1 |
| PACE with online adaptation | 153.8 | 80.8 |
| MA-JEPA | 161.9 | 136.7 |
Under this protocol, MA-JEPA and TALENTS obtain similar scores with shaped rewards. MA-JEPA has a larger advantage with sparse rewards. Table 3 in Appendix A.3 gives the per-layout means and 95% confidence intervals (Reviewer 9MgJ W4).
Revision. We added a footnote in Section 6 that points to this comparison. We also replaced the five-layout averages in Table 3 (Appendix A.3) with per-layout average performance and confidence intervals.
Closing. We thank the reviewer again for the insightful and constructive comments. We hope that these responses answer the questions of the reviewer, and we will be glad to answer more questions!
Dear Reviewer 1Rsn,
As the reviewer–author discussion phase is approaching its deadline (November 18, AoE), I would kindly ask you to review the authors' responses and provide an update on whether your concerns and questions have been adequately addressed.
As there are differing opinions among the reviewers on this paper, your updated assessment is particularly important in helping us reach a well-informed decision.
Thank you for your time and valuable contribution.
Response to reviewer's concerns (1/2)
We thank the reviewer for the careful and constructive review, and for finding our evaluation results promising.
Thank you for raising this important point. We acknowledge that the theory assumes an exact belief, and Proposition 5.4 covers belief-independent policies. We would like to clarify what each mathematical result establishes for the trained agent.
Proposition 5.3. As Section 5.2 states, Proposition 5.3 applies to any fixed policy π(ai∣s,b). Therefore, the Markov property, the Bellman recursion in Eq. (8), and its unique fixed point hold for belief-dependent policies. This Bellman recursion gives the TD target in Eq. (10). It does not give the same guarantee for the learned embedding or the jointly trained networks.
Proposition 5.4. Only the linearity in Proposition 5.4 needs a belief-independent policy. We do not apply this identity to the belief-dependent policy of MA-JEPA. As Appendix A.1 states, it motivates our design rather than guaranteeing the behavior of the trained agent. Specifically, it shows that the belief takes the role of the task vector w in standard SFs.
Learned representation. MA-JEPA does not aim to identify the partner's policy, actions, or memory. The goal is to learn how the partner changes the ego-agent's dynamics, so that the ego-agent can act well in the current situation. Two different partners can require the same ego action when they lead to the same future from that situation. Therefore, a representation can support this prediction without recovering the full posterior. However, the objective does not guarantee that zt keeps sufficient information. Our posterior probe in Section 6.5 shows that zt keeps partner information, but it does not show prediction sufficiency (
Reviewer mwRh Q2).Revision.We added the posterior probe to Section 6.5 and updated Appendix A.1 accordingly.Thank you for highlighting this connection. We agree that TD-JEPA is the closest algorithmic predecessor of MA-JEPA, and we have stated it in Appendix A.2. It studies latent successor prediction in a single-agent setting. MA-JEPA addresses a problem that does not occur there: the effective transition depends on an unobserved partner policy, so the process is not Markov in the state (Proposition 4.1). Therefore, we condition the successor recursion on a belief over the partner and analyze its fixed point on the belief-augmented process (Proposition 5.3). In practice, a history embedding approximates this belief. On-policy collection also keeps each history, action, and bootstrap target consistent with the rollout policy and the current partner. As a result, it avoids replay data that may mix behavior policies or partner dynamics.
We also agree that a comparison with representation learning baselines is important. We compare MA-JEPA with representative objectives, including Laplacian, contrastive, BYOL, and BYOL-γ, with the same encoder and RL networks (Figure 5). None of them approaches MA-JEPA beyond the Easy layout. In the original manuscript, this comparison was in Appendix F.1. We moved it to the main text.
Our ablations in Figure 7 also give two controls. Since TD-JEPA is an offline single-agent method, its closest on-policy adaptation is MA-JEPA w/o belief encoder. It keeps the latent TD objective but removes the partner-conditioned history. MA-JEPA w/o BSF predictor keeps the same history encoder, PPO objective, actor, and critic, but removes the BSF prediction. Therefore, it isolates the benefit of BSF beyond memory alone. Both controls stay far below MA-JEPA in average XPSP (
Reviewer 9MgJ W1).For FB [2] and ICVF [3], the main difference is what defines the task. Both methods condition the prediction on an ego-side variable, namely an ego objective or an ego intention, within fixed environment dynamics. In our setting, the reward stays fixed, and the hidden partner changes the effective dynamics Tθ of the ego-agent. Therefore, two histories can require different predictions even when they end at the same state and ego action (Proposition 4.1).
Revision.We moved the comparison with representation learning baselines from Appendix F.1 to Section 6.2.