acceptodds
Under review as a conference paper at ICLR 2027

When to Rely on a Partner? Reliability based Arbitration for Zero Shot Cooperation with Decision Process Impairment

Abstract

Training agents that cooperate with humans requires adapting to diverse partner behaviors, which cannot be fully anticipated before interaction. Zero shot coordination (ZSC) addresses this by exposing agents to a wide range of partner behaviors during training, covering differences in skill level, strategy, preference, and action stochasticity. Such coverage, however, presumes partners whose decision processes are intact. Agents deployed as assistive or collaborative partners must also work with people whose decision processes are impaired, as in cognitive and psychiatric conditions. Such impairment does not lower competence uniformly: it selectively affects behavior during the task, so the same partner may act competently in one situation and fail in another. A fixed cooperation policy is therefore not enough; the agent has to keep reassessing how much to rely on the partner as the interaction unfolds. Adapting behavior across different situations is a fundamental aspect of human cognitive control. Reliability based arbitration control has been identified as a mechanism that supports such adaptation by adjusting the influence of competing strategies according to their estimated reliability. Inspired by this principle, we propose Reliability based Dual Specialist Arbitration (RDSA). RDSA decomposes cooperation into two strategies, each based on a different assumption about the partner's reliability: a Partner-Reliant Specialist that relies on the partner to handle part of the task, and a Self-Reliant Specialist that completes the task on its own. During interaction, RDSA estimates each specialist's reliability and dynamically adjusts their relative influence on action selection. To evaluate RDSA, we constructed simulated partners with decision process impairments by applying targeted perturbations to perception, valuation, and action selection. Experiments showed that these partners exhibited behavioral variation distinct from skill level, with the differences depending on the situation. RDSA outperformed the best performing ZSC baseline and the best static mixture in most evaluated conditions, including conditions in which neither specialist outperformed the best performing ZSC baseline.

Primary area:
reinforcement learning
Keywords:
human-AI cooperation, reinforcement learning, zero-shot cooperation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2tMVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Summary: This paper addresses the challenge of zero-shot coordination (ZSC) when collaborating with human partners who exhibit decision process impairments (e.g., associated with cognitive or psychiatric conditions). Existing ZSC approaches typically assume partners have intact decision architectures and capture diversity primarily through skill levels, strategies, or action stochasticity. However, impairments in perception, valuation, or action selection manifest as situation-dependent behavioral variations rather than uniform declines in competence. To handle partners whose reliability shifts dynamically across states, the authors draw inspiration from neuroscience principles of cognitive control and propose Reliability-based Dual Specialist Arbitration (RDSA). RDSA frames cooperation through two complementary policies—a Partner-Reliant (PR) Specialist (assuming cooperative assistance) and a Self-Reliant (SR) Specialist (assuming non-cooperative, uniform partner actions)—and adaptively arbitrates their relative influence on action selection via Bayesian reliability estimation based on online observations of partner actions.

Contributions:

  • New Problem Formulation in ZSC: Identifies decision process impairments as a critical, under-explored source of partner diversity that induces state- and context-dependent behavioral variability, distinct from uniform skill-level differences.
  • RDSA Framework: Introduces an arbitration control mechanism tailored for multi-agent cooperation, decomposing the policy space into two opposing assumptions (PR vs. SR) and dynamically tuning their arbitration weight (𝑝𝑡𝑝_𝑡) via Dirichlet posterior-based reliability tracking.
  • Computational Impairment Modeling & Benchmark: Formulates controlled perturbations targeting perception (channel noise), valuation (weight pruning, MLP noise), and action selection (quantal response temperature λ\lambda), systematically evaluating human-like decision breakdowns in cooperative settings.

Validation:

  • Empirical Superiority: Evaluated across five Overcooked layouts (three standard and two risky layouts) under 15 layout-impairment conditions, RDSA achieves the highest overall average return (157.0) and significantly outperforms five prominent ZSC baselines (FCP, MEP, HSP, GAMMA, E3T), individual specialists, and fixed static mixtures in most conditions (𝑝Holm<0.01𝑝_\text{Holm}<0.01).
  • Construct & Behavioral Validity: Using Jensen-Shannon (JS) divergence against return-matched checkpoints across 308 probe states, the authors verify that the simulated impairments induce distinct, state-dependent behavioral divergence rather than simple skill-level degradation.
  • Robustness to Human Proxies: Validated against impaired Behavioral Cloning (BC) agents trained on real human gameplay data, where RDSA consistently maintains performance advantages across diverse perturbations.
  • Dynamics Analysis: Confirmed via tracking analysis and Linear Mixed Models (LMM) that the arbitration weight ptp_t accurately tracks real-time partner deviation and yields the strongest benefits when the advantage between specialists is pronounced.
Critical strengths
  • Novel Problem Formulation (Addressing Decision Process Impairments in ZSC): The paper successfully identifies and formalizes a highly practical yet underexplored source of partner diversity in zero-shot coordination—namely, decision process impairments (such as cognitive or psychiatric breakdowns affecting perception, valuation, or action selection). Shifting the focus from uniform skill-level declines to situation-dependent, localized behavioral failures is a consequential conceptual contribution because it bridges multi-agent reinforcement learning with human cognitive control principles, opening a vital pathway for deploying collaborative AI systems alongside impaired or vulnerable populations.
  • Principled Architecture and Robust Empirical Validation: The proposed Reliability-based Dual Specialist Arbitration (RDSA) framework provides a rigorous, neuroscience-inspired control mechanism that dynamically weighs complementary policies via online reliability estimation. This methodological design is strongly substantiated by extensive multi-layout evaluations across standard and risky Overcooked environments, where RDSA consistently outperforms prominent ZSC baselines and static mixtures (often succeeding even when individual specialists fall short), proving its practical effectiveness and robustness across diverse perturbation conditions.
Critical weaknesses
  • Reliance on Simulated Impairment Proxies: While the paper systematically perturbs computational model components (perception noise, MLP weight pruning, and quantal response temperature) to mimic human cognitive impairments, these models remain functional simulations rather than direct empirical validations with human clinical data. This abstraction leaves open questions regarding whether these specific perturbation topologies fully capture the nuance and behavioral distribution of actual human psychiatric or cognitive conditions in interactive environments.
  • Limited Layout and Task Generalization: The empirical evaluation is constrained entirely to variants of the Overcooked gridworld framework. Given that Overcooked features relatively discrete, short-horizon coordination dynamics with well-defined subtasks, the scalability and generalizability of RDSA's evidence accumulation and reliability tracking mechanisms to more complex, continuous, or long-horizon real-world human-AI collaboration tasks remain to be demonstrated.
Questions for authors
  • Clarification on Real-World Clinical Generalization: The current evaluation relies on simulated decision process impairments (noise on observation channels, MLP weight pruning, and quantal response temperature variations) to model psychiatric and cognitive breakdowns. Given that these are simplified proxies, how confident are you that the state-dependent reliability tracking and Dirichlet posterior updates will transfer effectively to noisy, non-stationary human-subject interactions in clinical settings? Could you discuss any pilot tests or qualitative expectations regarding how real human behavioral variability might violate the assumptions of the Partner-Reliant and Self-Reliant specialists?
  • Clarification on Arbitration Hyperparameter Sensitivity: The proposed framework relies on several fixed hyperparameters for reliability estimation and transition dynamics, such as the memory length (Tmem=40T_{mem} = 40), the smoothing coefficient (κ=0.1\kappa = 0.1), and the transition scales (Aα=12,Aβ=9A_\alpha = 12, A_\beta = 9). How sensitive is RDSA's performance to these hyperparameter choices across different task horizons or more complex, long-horizon collaborative environments outside of Overcooked?
Reviewer_qQFqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores zero-shot cooperation with partners with varying behaviors and introduce a RDSA framework that combines a partner reliant specialist that assume partner being cooperative while a self reliant specialist works under weaker assumption about a partner. For the evaluation, the paper presented the simulated partners with perturbation to perception, valuation, and action selection. The method achieves the highest average return and perform best in most layout-impairment combinations.

Critical strengths

The paper explores an interesting problem and is written well. Also, the paper includes broad experiment results with multiple impairment mechanisms.

Critical weaknesses

Although the paper describes to use cognitive and psychiatric impairment, the explored mechanisms are perturbation through engineering than empirically validated impaired human behavior. Thus, the paper does not show that this work reproduce meaningful human impairment.

In addition, RDSA involves online inference and adaptation. Thus, experimental results do not show whether the gains come from reliability-based approach or an online adaption compared to fixed zero-shot policies.

Questions for authors

Can you please provide results that the proposed perception, valuation, and action selection perturbation can generate behaviors of people with relevant decision process impairment?

It would be good to clarify and compare the RDSA with an online team modeling or any adaptation baselines

Reviewer_cePHOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents a model and experiments for studying collaboration between AI and another AI that exhibits decision-making impairment found in the cognitive science literature (or maybe I misunderstood this). Their model is a tabular, game-theoretic model of behavior (essentially use K-level iterated response is my impression) but either assuming a uniform partner (SR-specialist) or a greedily reacting to partner at each level (PR-specialist). They present results in a "risky" variant of overcooked which has an additional "subgoal" which they use to define controlled experiments studying collaboration with an agent that has impaired decision-making. They show that the jenson-shanon divergence between the impaired decision-maker's behavior and a "neurotypical" reference agent's behavior were high, which they argue showcases impaired decision-making.

Critical strengths

They choose an important problem within the multi-agent literature.

Critical weaknesses

This is a very hard paper to understand. I think the methods could be greatly improved.

From a "modelling" perspective or from a "human-inspired" AI perspective, I think the paper has a lot of room for growth in explaining/motivating/validating that human-like/human-inspired cognitive impairment that they study.

I don't think the paper is studying any kind of impairment, because there are many kinds of impairments that we may not care about. For example, do we care about agents that can only walk in straight lines? do we care about agents that have a million extra policy logits that they have to learn to ignore? Do we care about agents that can only move leftwards? Maybe we do, but this needs to be motivated.

Regardless of whether we're building better AI or we're doing a model of human cognition, the ontological grounding to phenomenon of interest must be clear. If we're building better AI, they we need to clearly describe the target "problem setting" of interest: the kind of cognitive impairment we expect agents to face and clearly describe how our grid-world experiments properly capture some aspect of the abstract structure. Ideally, we present a series of experiments increasingly the ecological validity of the experiments and show that our model continues to have an advantage whereas baselines can't (this is the strategy ML typically takes).

If we are doing modelling, there's a smaller emphasis on increasing the ecological validity of experiments but there is a much larger emphasis on creating experiments which abstractly capture a phenomenon of interest. This paper presents neither.

Given that, it just reads like a random model (that's tabular) with random results that are weakly tied to phenomenon/problems of interest.

Questions for authors

Why do tabular RL instead of Deep RL? I guess because iterated response? Can you explain why JS measures "human-like" impairment?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.