acceptodds
Under review as a conference paper at ICLR 2027

Modeling and Recovering Social Reward Differences in Autism with Multi-Agent Reinforcement Learning

Abstract

The clinical assessment of Autism Spectrum Disorder (ASD) relies heavily on observing dyadic interaction, yet existing computational models are overwhelmingly single-agent frameworks that fail to account for reciprocal social dynamics. Here, we operationalize the Social Motivation Theory of ASD in a multi-agent reinforcement learning (MARL) environment, providing a testbed to evaluate how alterations in social reward shape movement patterns. By varying a scalar social reward parameter, , across eight conditions from baseline () to profound deficit (), joint policy optimization with Proximal Policy Optimization (PPO) generates behavioral phenotypes consistent with ASD without prescribing top-down behavioral rules. Mean peer distance provides a sensitive metric that scales inversely with (Spearman , ). Control experiments confirm that this effect is specific to social reward attenuation rather than a general reduction in reward magnitude, consistent with the atypical interpersonal distance reported in clinical cohorts. To determine whether these kinematic signatures are informative, we train a bidirectional LSTM on trajectory sequences. The network recovers the latent parameter at chance accuracy across nine classes. This recovery remains well above chance after removing explicit relational distance features and improves with extended observation windows. Additionally, mixed- pairings show directionally consistent coordination degradation, offering a preliminary reinforcement-learning account of the Double Empathy Problem. These findings demonstrate that the social reward differences hypothesized in ASD can be modeled in simulation and recovered from dyadic kinematic data.

Primary area:
applications to neuroscience & cognitive science
Keywords:
multi-agent reinforcement learning, computational psychiatry, autism spectrum disorder, social motivation theory, latent parameter recovery, behavioral phenotyping

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_uxsEOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper models the Social Motivation Theory of autism as a scalar weight ρ\rho on a peer-proximity reward in a dyadic version of the MPE simple_spread environment. Both agents share one PPO policy. Each agent is rewarded for proximity to the nearest landmark and, with weight ρ\rho, for proximity to its peer. ρ\rho is swept from full social reward down to near zero.

The paper reports that mean peer distance increases as ρ\rho falls which is expected and close to circular.

It also reports that a BiLSTM can classify ρ\rho from dyad trajectories, including after the distance features are removed, and that classification accuracy improves with longer observation windows before flattening. Finally, mixed-ρ\rho dyads coordinate slightly worse than matched ones, which the paper presents as an RL account of the Double Empathy Problem.

Critical strengths

Reasonable motivation. Existing RL models of autism are single-agent, and asking what changes when social reward is altered inside an interacting dyad is a fair question.

Critical weaknesses

The model has no meaningful connection to autism. The environment consists of a pair of point particles on a flat plane, with landmarks standing in for toys. Each particle is rewarded for being near the other, with that reward weighted by ρ\rho. Lowering ρ\rho makes the particles stay further apart.

This claimed behavioral result follows from the reward by construction. The agents are trained to reduce peer distance, so peer distance tracks ρ\rho. Calling this a "behavioral phenotype consistent with ASD" or a "computational biomarker" isn't supported.

Since the ML methods are off-the-shelf (PPO, MPE, BiLSTM), the paper's contribution depends entirely on this link to autism. Without that link, what remains is that agents rewarded for proximity stay close, which isn't an ICLR contribution.

The ρ\rho-recovery result likely measures policy identification rather than ρ\rho. The classifier section describes stratified cross-validation over trajectories and doesn't mention grouping by seed, so presumably every policy appears in both the training and test sets. If this is the case, the BiLSTM can learn each policy's idiosyncrasies and map them to a label without learning anything general about ρ\rho. Several results fit this explanation:

Without leave-one-seed-out evaluation, the reported accuracy can't be read as recovery of ρ\rho.

Questions for authors
  • Why is MPE simple_spread a suitable model environment for ASD? The agents are point particles rewarded for proximity to landmarks and to each other. Which aspects of autism does this capture, and what result could the model produce that isn't already determined by the reward function?

  • Were cross-validation folds split by trajectory or by seed? If by trajectory, what is the accuracy when entire policies are held out?

  • How were the mixed-ρ\rho dyads trained? If they used the shared policy, there is no agent ID or ρ\rho in the observation, so the network can't condition on ρ\rho. Both agents' updates would mix both reward signals, and the claim that each agent optimizes strictly under its own ρi\rho_i wouldn't hold.

Reviewer_phhkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an analysis of social behavior using a multi-agent reinforcement learning experiment. In particular, the paper introduces a component in the reward that is dubbed a "social reward" and observes - in a MARL setup - how such social reward influences the behavior of the agents. The paper also investigates the inverse relation, i.e., whether different observed behavior may correspond to differerent values for the social reward component, by trying to recover the latter from the former. The paper goes on to establish a relation between the observed patterns of behavior and ASD behavioral phenotypes.

Critical strengths

The overall motivation of the paper (using computational models to better understand or replicate ASD behavior patterns) is of general interest, given the prevalence and limited understanding of this spectrum of conditions in our current society. Additionally, I also found the combination of reinforcement learning and computational psychiatry an interesting aspect of the paper. Finally, in terms of experiments, the paper strikes me as thorough and rigorous.

Critical weaknesses

My main concern with the paper lies precisely on the relation established between the computationao setup and ASD. My concerns go in both directions:

  • On one hand, it seems to me that - computationally - the results observed are quite expectable: by changing the value assigned to social interactions, it is expectable that the behavior of the agents changes accordingly. Several works from the 2010s on optimal reward design for cooperative multiagent systems, emerging social awareness, etc. already reported similar results similar, even if in different contexts and for different purposes. As a result, it is not clear to me what is the novelty from a multiagent learning perspective.
  • On the other hand, I am not convinced that the observed behaviors correspond to ASD-specific patterns, and it would seem to me that they could plausibly arise in many other settings involving heterogeneous social preferences, trust, familiarity, group identity, antagonism, or social anxiety. Therefore, while the proposed model demonstrates that modifying a social reward signal affects behavior, I am not convinced that the conclusions provide meaningful evidence regarding ASD specifically.
Questions for authors
  1. What evidence suggests that the social reward parameter indeed models social motivation of people with ASD, instead of generic heterogeneous social preferences?
  2. Which results and conclusions can be understood as informative regarding ASD-specific mechanisms, and which arise from the properties of the model adopted?
Reviewer_gELKOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper turns the Social Motivation Theory of autism into a two-agent reinforcement learning environment. Each agent is rewarded for being near landmarks and, with weight ρ, for being near its partner. Across eight values of ρ, agents trained with PPO stay farther apart as ρ decreases. A bidirectional LSTM recovers ρ from movement trajectories at about 56% accuracy across nine classes (about 5× chance), and accuracy improves with longer observation. A preliminary experiment pairing agents with different ρ values shows a small drop in coordination, which the authors relate to the Double Empathy Problem. All results are in simulation.

Critical strengths
  • A clear, well-motivated idea: Moving computational models of autism from a single agent to an interacting pair is well-motivated. Expressing Social Motivation Theory as one interpretable reward weight makes the model easy to reason about and build on.
  • Good experimentation: The paper includes sensible robustness checks: it removes the distance features from the classifier, retrains for each observation length, and tries a different reward shape. It reports its training details fully and is upfront about its limitations. This makes the results trustworthy.
Critical weaknesses
  1. No connection to real human data: Every finding describes simulated agents. Without fitting or testing the model on any real interaction data, it is hard to judge claims such as peer distance being a “computational biomarker.” It is also hard to tell whether the model captures anything about autism beyond the assumption built into its reward. This is the main factor in my assessment.
  2. Few novel predictions. The central result is that agents rewarded less for being near their partner stay farther away, and that follows largely from the reward design. The paper doesn’t derive a prediction that is non-obvious from the setup and testable in people, which is how simple models usually prove their value.
Questions for authors
  1. Which of your findings do you see as not following directly from the reward function? Could you test any of them with existing human data?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.