acceptodds
Under review as a conference paper at ICLR 2027

Artificial Mirror Neurons: Modelling Others through Like-Me Inference in Multi-Agent Reinforcement Learning

Abstract

Humans can understand others partly by using their own experience as a reference, a principle known as Like-Me inference (LIMI). We investigate whether this principle provides a computational mechanism for partner modelling in multi-agent reinforcement learning. We propose a simple LIMI architecture that represents another agent in the ego agent's own embedding space by combining an estimate of the partner's perspective with the ego's own representation mechanism. We study two variants: a fixed perspective transform that requires no behavioural-prediction objective, and a learnt transform that learns the self–other mapping from partner behaviour. We further show theoretically that the benefit of adapting to a partner is bounded by how much partner information is encoded in the agent's hidden state. Across Overcooked, multi-robot coordination, and embodied locomotion tasks, LIMI improves ad hoc teamwork in several settings, particularly when coordinating with unseen partners is difficult. Probing frozen representations further shows that LIMI increases information about partners' actions and internal states. Representational similarity analysis also reveals partial alignment between how self and other encoders in learnt-perspective-transform agents organise the same events, consistent with mirror-like representational structure. Together, our results suggest that reusing one's own representations provides a simple mechanism for implicit partner modelling and behavioural generalisation.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Theory of Mind, Like-Me Inference, Ad Hoc Teamwork

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_jbsNOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a new Like-Me inference architecture for multi-agent reinforcement learning. It uses separate embeddings for the ego agent and its partner and processes them through a shared representation mechanism to predict the agents’ actions. The architecture performs well at predicting other agents’ behaviour across several environments.

Critical strengths
  1. The proposed architecture is intuitive and simple, and the experiments demonstrate its effectiveness.

  2. The exposition, visualizations, and discussions in the paper are clear and easy to follow.

  3. The fixed-transform variant is a nice demonstration of the representation-sharing idea.

Critical weaknesses
  1. There is not enough evidence to show how the architecture performs in a broader range of zero-shot coordination settings, such as interactions with human players or artificial agents from substantially different policy families. The current partner distribution provides limited evidence of how the model would transfer to these more challenging settings.

  2. It is clear that the motivation for predicting other agents’ behaviour is to improve coordination. However, the paper does not convincingly isolate how its particular partner representation improves the ego agent’s decisions.

Questions for authors
  1. The paper discusses the challenges of finding good evaluation metrics and proposes a metric based on partner-blind control. I understand the motivation, but are there more objective and less artificial metrics that could also be used?

  2. Could this architecture be applied to LLM settings? Could the neural networks be replaced with LLMs to build better AI agent systems following the paper’s idea?

  3. The paper uses different embedding networks for the ego agent and its partner but shares the same RNN layer. How would the method perform with a shared embedding network and separate RNN layers? What if both the embedding networks and the RNN layers were separate?

2 comments

We thank the reviewer for the constructive and thoughtful review. We respond to the reviewer's concerns below. We've updated the paper and marked changes in blue\textcolor{blue}{\text{blue}}.


jbsN-W1: Generalisation to different partners

We agree that evaluation against a broader range of different partner families can give stronger evidence. We therefore evaluated the ego policies against held-out E3T, FCP and PBT partners, using 10 partners per family in each of the five Overcooked layouts. We have added these results to Appendix E.

Across the four partner families (including SP), fixed and learnt SP+LIMI each achieve higher mean returns than SP in 18 of 20 layout–family comparisons. Fixed and learnt E3T+LIMI exceed E3T in 11 and 13 comparisons, respectively. These results extend the evidence to ad-hoc coordination with different policy families, while showing that benefits depend on the partner family and layout.

We have not conducted human-player experiments. Testing coordination with human partners can be a potential extension.


jbsN-W2: How the partner representation affects decisions

We’ve added evaluation-time interventions that replace the partner embedding with zeros, its mean, or a mismatched embedding, with corresponding ego-stream controls. The protocol and results appear in Appendix D. We measure immediate action-distribution changes and return drops. Partner mismatch lowers mean return in all configurations (layouts and methods). It supports behavioural use of the partner stream. However, it doesn’t isolate partner-specific information.


jbsN-Q1: Alternative evaluation metrics

Held-out ad hoc return is a direct measure of coordination and does not require a partner-blind control. The behavioural measures can give an important complement to the partner-blind probes. We report ad-hoc returns against held-out partners and have added embedding interventions that measure changes in ego decisions and coordination return. Comparing normal and intervened returns tests whether the policy uses the partner stream, although it does not isolate partner-specific information.

These measures answer different but complementary questions.

  • Hidden state probes evaluate predictive information.
  • Embedding interventions evaluate the behavioural use of it.
  • Returns evaluate coordination success.

jbsN-Q2: Applicability to LLM agents

Our goal is to study LIMI as a cognitive principle rather than develop an LLM-based system, but the idea could inform such agents. An LLM agent could reuse its own decision-making model to predict another agent’s behaviour from an estimated partner perspective.

Explicit perspective mapping may be particularly useful when agents have different visual observations, information access, or roles. For example, in multiplayer games. Applying our architecture would require suitable representations and training objectives rather than simply replacing each network with an LLM. We have not tested this extension and present it as a direction for future research.


jbsN-Q3: Encoder and RNN sharing

We added both suggested controls in Appendix F: (i) a shared encoder with separate single-core RNNs, and (ii) separate encoders with separate single-core RNNs.

All variants have the same total recurrent-state dimension and component widths, although parameter counts differ because of weight sharing. The original architecture achieves the highest mean return in all five layouts. Among the controls with separate GRUs, retaining separate encoders also gives higher mean returns in every layout. APG and NPG rankings vary across layouts and do not consistently follow return rankings. These results favour the original architecture in the tested settings.

Thank you for the detailed response and additional experiments. The evaluations against different partner families and the encoder/RNN-sharing controls substantially address my questions about generalization and the architecture choices. The embedding interventions also provide useful evidence that the partner stream affects decisions and coordination, and I appreciate your acknowledgment that they do not fully isolate the contribution of partner-specific information. The distinction between predictive probes, behavioral interventions, and coordination returns is helpful. Overall, the response strengthens my positive assessment, and I will maintain my rating of clear acceptance.

Reviewer_UwAWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose a Like-Me inference approach for Multi-Agent RL problems that learns a representation of a partner agent by representing through the self-agent's egocentric models. This amounts to a perspective transformion of the ego-obervation to the partner's perspective, which is then used as the input to a recurrent actor-critic algorithm. The authors test both a fixed and a learned transform and show that their approach improves partner modelling and inccrease return in simulated cooperative MARL tasks.

Critical strengths
  1. New methodology for Like-me inference, which represents other agents by re-using the agent’s own internal models. While the paper focuses on simple environments, the approach is well motivated, the claims are well supported and not exaggerated and the work is relevant for future developments in multi-agent RL.
  2. The authors demonstrate the ability of their approach to represent information about the other agent and to improve the return in MARL simulated tasks when compared to approaches that explicitly learn to predict the action of the other agents , even if the results are task-dependent.
  3. Very well written, structured and clear.
Critical weaknesses
  1. The tested environments are somewhat simple and the observations of each agent have a high degree of overlap with the partner agents’ own observation, meaning that the changes in perspective mostly boil down to some reordering of the features.
  2. Besides this assumption, it is also assumed that the agents share the same action space, due to Lmatch\mathcal{L}_{match}.

Minor:

  1. Improve readability of the figures and plots, fonts are too small (Figure 2, 3, 4 Table 1 and 2).
  2. Reordering bar pots by base algorithm would improve readability.
Questions for authors
  1. After you freeze the single core does the Fused core keep on being updated? If it does, then after a while the predictions of the Single Core wont match those of the Fused anymore.

  2. How do you decide when to freeze the Single Core? Did you try to keep updating it and use an EMA copy of the encoder for Lmatch\mathcal{L}_{match} instead of a completely frozen one (DDPG Critic style)?

  3. How could this algorithm be adapted to agents with different action spaces? In MA-Brax humanoid, for example, there is a 9-8 split of the joints between two agents. This would increase the impact of the approach to other areas interaction between robots with different embodiments.

  4. Could a similar method be used with higher-dimension modalities such as vision or in cases where each agent has only a partial observation of the environment and of the other agents?

Note: due to the high number of papers for review it was not possible for me to thoroughly read the Supplementary Materials. My appologies if some of these questions have an answer there.

3 comments
AuthorsOpenReview ↗

Response (1/2)

We thank the reviewer for the thoughtful and constructive review. We have responded to the reviewer's concerns below. We've updated the paper accordingly and marked the changes in blue\textcolor{blue}{\text{blue}}.


UwAW-W1 Highly Overlapping Observations

We agree that the original observations overlap and may allow simple feature reordering. However, reordering features only changes the viewpoint but it does not tell how an unfamiliar partner will act. Agents can observe the same partner state, but partners can follow different strategies, so predicting and coordinating with partners remains challenging.

To address this, we added partial-observation Overcooked experiments in Appendix C. Here, the fixed transform requires spatial shifting, cropping, and rotation based on the partner’s position and heading, beyond exchanging feature channels. It uses only ego-visible information. Unavailable regions are zero-padded, and the partner embedding is zeroed when the partner is out of view.

Fixed SP+LIMI and E3T+LIMI improve mean return over their baselines in three and four of five layouts, respectively. These results extend the results to partial ego observations. The complex sensory inputs remain a limitation. To extend this, we need a future benchmark that allows us to train multi-agent RL in 3D environments, and the benchmark should share the key characteristics of Overcooked, allowing diverse and incompatible strategies among agents to arise.


UwAW-W2 & Q3: Agents share the same action space

We can extend our method to agents with different action spaces. Learnt LIMI learn ego and partner action predictions through LmatchL_{match}, so these predictions need compatible output representations. However, this does not require identical available actions. For example, a shared network can be trained with multiple roles (embodiments) and conditioned on a role indicator. The network then produces predictions over a common output space, and a role-specific mask selects the relevant actions.

To address this, in Appendix G, we test this extension in MA-Brax Humanoid as suggested, where two roles control nine and eight actuators. The shared network predicts 17 action coordinates, and each role uses only its assigned coordinates. Because ego distillation (LcopyL_{copy}) includes both roles, the single core learns to predict both action sets and can then be reused for partner prediction. For Fixed LIMI, it doesn’t require a shared action space. Its main requirement is a known perspective transform that produces inputs so the ego encoder can process them. We have clarified these assumptions and added the Humanoid experiment in the appendix.

Fixed E3T+LIMI achieves the highest mean return. Learnt SP+LIMI also exceeds SP, suggesting potential benefits with unequal action spaces. However, improvements vary across variants. This inconsistency might be because of nonstationarity in the single core used for LmatchL_{match}. Given the limited discussion period, we haven’t implemented stabilisation methods. Exponential moving average (EMA) could be a potential candidate, as it’s proven to work in distillation. We plan to evaluate an EMA target for the single core and report the results in the final version.


UwAW-W3-4: Figure and table readability

We have enlarged Figures 2–4 and increased their font sizes. We have also increased the font sizes in Tables 1–2. The bars are now grouped by base algorithm to make comparisons easier.

AuthorsOpenReview ↗

Response (2/2)


UwAW-Q1–Q2: Single-core updates and EMA stabilisation

There is no two-stage training or point at which the single core is permanently frozen. Three parameters groups are trained for the learnt transform. Each iteration performs 3 updates in parallel.

  1. LppoL_{ppo} updates the ego encoder and fused core.
  2. LcopyL_{copy} updates the single core and its action/value heads to match the current fused policy and value estimate.
  3. LmatchL_{match} updates the partner encoder through the current single core, holding the single-core and head parameters fixed only for this loss.

Thus, continuous distillation lets the single core track the evolving fused policy, although it does not guarantee exact agreement. We have clarified this in the main text at the end of Approach 2. Appendix K provides detailed gradient routing.

Using EMA: We agree that these coupled updates (2 and 3 above) can add nonstationarity and that an EMA target may improve stability, so we have added an EMA comparison. In the table below, we run a preliminary experiment with Overcooked. EMA (with decay 0.99) does not yield a consistent improvement. Mean returns and APG are slightly lower for both methods, while NPG is lower for SP+LIMI and similar for E3T+LIMI. EMA provides no clear performance advantage in Overcooked. This could be because PPO can deal with nonstationarity to some extent. We will provide the full results of EMA on other environments in the final version.

Table: EMA vs Online Update in Overcooked. Numbers are averages across 5 Overcooked layouts.

Learnt methodTargetReturnAPGNPG
SP+LIMIOnline131.73 ± 3.37.174 ± .006.202 ± .005
SP+LIMIEMA128.59 ± 2.88.171 ± .003.195 ± .005
E3T+LIMIOnline149.59 ± 1.94.187 ± .005.216 ± .005
E3T+LIMIEMA146.55 ± 2.01.182 ± .004.217 ± .004

UwAW-Q4 Extension to Higher Dimensional Modalities

It has the potential to extend to 3D environments. There are existing studies on perspective transformation (exocentric to egocentric) in computer vision (Luo et al. 2024, Park et al. 2026). The direct way is to combine their model with our algorithm and train it on 3D MARL environments. However, we need to work on the 3D environments. Training MARL in complex 3D environments is another challenge.

Luo et al., Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videos, ECCV2024

Park et al., EgoWorld: Translating Exocentric View to Egocentric View using Rich Exocentric Observations, ICLR2026

W1: This paper’s main contribution is a representation learning method that improves downstream policy learning in predicting and coordinating with the partner. Naturally, the improvements in performance are impacted by how easy or difficult it is to learn the perspective taking, so I believe that not having a demonstration in more complex environments remains a weakness of this work. Nevertheless, I appreciate the new results under partial observability and believe that they add value to the paper, even though I don’t believe they’re an answer to W1.

W2: Again I appreciate the effort of the authors in providing new results that address W2 and strengthen the contributions.

I thank the authors for the remaining clarifications. Overall I maintain my assessment of this work and therefore, after consideration, will keep my score of 3.

Reviewer_4KfzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work contributes a instantiation of Like-Me Inference (LIMI) for partner modeling. The key idea is to use the ego agent's own representation space with a perpective transformation. Experiments show the approach tends to yield better performance compared to baselines (though not always).

Critical strengths
  • The LIMI setup is reasonable and intuitive.
  • The paper is generally well-written in most places and I could follow along
  • The analysis and experimental setups show the benefit of the inductive bias provided by the ego-representations.
Critical weaknesses
  • There have been previous work exploring such representations for policy learning. Chen et al. (IROS'23) appears to be closely-related work that also proposes using the ego-model with perspective taking. There are differences (from my understanding): their method is based on a state-space world model versus policy + MARL in this work. Nevertheless, there are overlaps in the key ideas and so the method should be contrasted against.
  • At this point it is unclear how the model deals with partial observability. What happens in the case that the partner agent is observing something not within the view of the ego agent.
  • Experiments are limited somewhat to simple domains so the generalization of such the setup to more complex domains (as mentioned in the limitations) remains an open question.
  • Could the authors elaborate more on why learnt transforms to do not always beat fixed transforms? Is this a problem that can be addressed with more data or are there limits to what can be learnt (especially given partial observability).

Refs: Chen et al. "Latent Emission-Augmented Perspective-Taking (LEAPT) for Human-Robot Interaction." IROS'23

Questions for authors

Please see weaknesses above.

2 comments

We thank the reviewer for the thoughtful feedback. We respond to the reviewer's concerns below. We've revised the paper and marked the changes in blue\textcolor{blue}{\text{blue}}.


4Kfz-W1 Discussing relevant work.

We thank the reviewer for suggesting LEAPT (Chen et al., 2023). Both our and their approaches reuse a self-model to understand the other agent through perspective taking. LEAPT uses a probabilistic world model to infer others’ observations and beliefs under partial observability. Our work instead constructs partner representations through the ego’s own representational machinery and integrates them directly into a recurrent MARL policy. The evaluation also differs. On one hand, LEAPT evaluates visual and belief inference, including in human-subject experiments. On the other hand, we study whether self-based partner representations improve ad hoc coordination with held-out partners, without partner-specific parameter updates at evaluation. While we read LEAPT, we also found MIRROR (Chen et al., 2022), which adapts self-models for human modelling and assistive communication. We have clarified these connections and distinctions in the revised manuscript line 150.

Reference

  • Chen et al. (IROS 2023) Latent Emission-Augmented Perspective-Taking (LEAPT) for Human-Robot Interaction

  • Chen et al. (RSS 2022) MIRROR: Differentiable Deep Social Projection for Assistive Human-Robot Communication


4Kfz-W2: Partial observability

We agree that the perspective transformation should be extended to partial observations where the partner is not always visible. We have added partial-observation implementation and experiments in Appendix C.

For fixed LIMI, when the partner is visible, fixed LIMI transforms only the features the ego can see into the partner’s frame. It zero-pads unseen regions and does not reconstruct what only the partner can see. When the partner is out of view, we zero the partner embedding so RNN receives a zero vector. In the discussion phase, we only evaluate this single design choice. The alternatives could be retaining the last embedding or maintaining a recurrent partner estimate. We will explore these design choices and include them in the final version.

Learnt LIMI receives partner-matching supervision at all timesteps, including when the partner is out of view. Fixed SP+LIMI improves mean return over SP in three layouts, increasing the layout average from 89.73 to 112.25. E3T+LIMI (fixed) improves over E3T from 126.41 to 133.25. Both SP+LIMI variants also improve mean APG and NPG over SP in every layout, while E3T comparisons are mixed. This is because the original E3T already has a partner prediction module, so it can have good partner modelling even without our LIMI.


4Kfz-W3 Generalisation to more complex domains

We agree with the reviewer. A more complex domain in ad hoc coordination could potentially be Multi-Agent Craftax (Omari et al. 2025), which requires the ability to perform open-ended learning. It makes the coordination even more challenging.

Omari et al. 2025, Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscale


4Kfz-W4 Learnt transforms do not always beat fixed transforms

The two variants have different strengths. The fixed LIMI is given a correct rule for changing perspectives that makes learning easier. However, changing perspective alone does not capture differences between how the ego and partner behave. The learnt LIMI can capture these differences because it is trained using the partner’s behaviour.

This flexibility may help it outperform the fixed transform. But the learnt transform could be harder to train under some conditions, so it does not always perform better. More data may improve the learnt transform, but it cannot fully resolve missing information under partial observability. Our current experiments do not isolate how much each factor contributes, so we present these as possible explanations.

Another issue in learnt LIMI is nonstationarity. The single core has to learn the mapping from the fused core (main policy). The fused core is being updated throughout training, causing nonstationarity. This could make the learnt transform harder to learn. We have added this to the limitations.

Reviewer_4KfzOpenReview ↗

Thanks for the response

Thank you for the response regarding related work and the discussion regarding partial observability and learnt transforms. I appreciate it and maintain my positive view and score of this work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.