- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes a new Like-Me inference architecture for multi-agent reinforcement learning. It uses separate embeddings for the ego agent and its partner and processes them through a shared representation mechanism to predict the agents’ actions. The architecture performs well at predicting other agents’ behaviour across several environments.
Critical strengths
-
The proposed architecture is intuitive and simple, and the experiments demonstrate its effectiveness.
-
The exposition, visualizations, and discussions in the paper are clear and easy to follow.
-
The fixed-transform variant is a nice demonstration of the representation-sharing idea.
Critical weaknesses
-
There is not enough evidence to show how the architecture performs in a broader range of zero-shot coordination settings, such as interactions with human players or artificial agents from substantially different policy families. The current partner distribution provides limited evidence of how the model would transfer to these more challenging settings.
-
It is clear that the motivation for predicting other agents’ behaviour is to improve coordination. However, the paper does not convincingly isolate how its particular partner representation improves the ego agent’s decisions.
Questions for authors
-
The paper discusses the challenges of finding good evaluation metrics and proposes a metric based on partner-blind control. I understand the motivation, but are there more objective and less artificial metrics that could also be used?
-
Could this architecture be applied to LLM settings? Could the neural networks be replaced with LLMs to build better AI agent systems following the paper’s idea?
-
The paper uses different embedding networks for the ego agent and its partner but shares the same RNN layer. How would the method perform with a shared embedding network and separate RNN layers? What if both the embedding networks and the RNN layers were separate?
2 comments
Thank you for the detailed response and additional experiments. The evaluations against different partner families and the encoder/RNN-sharing controls substantially address my questions about generalization and the architecture choices. The embedding interventions also provide useful evidence that the partner stream affects decisions and coordination, and I appreciate your acknowledgment that they do not fully isolate the contribution of partner-specific information. The distinction between predictive probes, behavioral interventions, and coordination returns is helpful. Overall, the response strengthens my positive assessment, and I will maintain my rating of clear acceptance.
We thank the reviewer for the constructive and thoughtful review. We respond to the reviewer's concerns below. We've updated the paper and marked changes in blue.
jbsN-W1: Generalisation to different partners
We agree that evaluation against a broader range of different partner families can give stronger evidence. We therefore evaluated the ego policies against held-out E3T, FCP and PBT partners, using 10 partners per family in each of the five Overcooked layouts. We have added these results to
Appendix E.Across the four partner families (including SP), fixed and learnt SP+LIMI each achieve higher mean returns than SP in 18 of 20 layout–family comparisons. Fixed and learnt E3T+LIMI exceed E3T in 11 and 13 comparisons, respectively. These results extend the evidence to ad-hoc coordination with different policy families, while showing that benefits depend on the partner family and layout.
We have not conducted human-player experiments. Testing coordination with human partners can be a potential extension.
jbsN-W2: How the partner representation affects decisions
We’ve added evaluation-time interventions that replace the partner embedding with zeros, its mean, or a mismatched embedding, with corresponding ego-stream controls. The protocol and results appear in
Appendix D. We measure immediate action-distribution changes and return drops. Partner mismatch lowers mean return in all configurations (layouts and methods). It supports behavioural use of the partner stream. However, it doesn’t isolate partner-specific information.jbsN-Q1: Alternative evaluation metrics
Held-out ad hoc return is a direct measure of coordination and does not require a partner-blind control. The behavioural measures can give an important complement to the partner-blind probes. We report ad-hoc returns against held-out partners and have added embedding interventions that measure changes in ego decisions and coordination return. Comparing normal and intervened returns tests whether the policy uses the partner stream, although it does not isolate partner-specific information.
These measures answer different but complementary questions.
jbsN-Q2: Applicability to LLM agents
Our goal is to study LIMI as a cognitive principle rather than develop an LLM-based system, but the idea could inform such agents. An LLM agent could reuse its own decision-making model to predict another agent’s behaviour from an estimated partner perspective.
Explicit perspective mapping may be particularly useful when agents have different visual observations, information access, or roles. For example, in multiplayer games. Applying our architecture would require suitable representations and training objectives rather than simply replacing each network with an LLM. We have not tested this extension and present it as a direction for future research.
jbsN-Q3: Encoder and RNN sharing
We added both suggested controls in
Appendix F: (i) a shared encoder with separate single-core RNNs, and (ii) separate encoders with separate single-core RNNs.All variants have the same total recurrent-state dimension and component widths, although parameter counts differ because of weight sharing. The original architecture achieves the highest mean return in all five layouts. Among the controls with separate GRUs, retaining separate encoders also gives higher mean returns in every layout. APG and NPG rankings vary across layouts and do not consistently follow return rankings. These results favour the original architecture in the tested settings.