EMERGENT BEHAVIOURS IN POWER AND INFORMATION-ASYMMETRIC MULTI-AGENT REINFORCEMENT LEARNING

We study emergent social behaviors in asymmetric multi-agent reinforcement learning (MARL), where a strong agent possesses informational and intervention advantages over weaker agents. Such asymmetries can fundamentally alter how autonomous agents interact, potentially producing behaviors that affect autonomy, individual welfare, and collective outcomes. We introduce a formal framework that characterizes four behaviors—paternalism, manipulation, loss of agency, and social welfare loss—as functions of two key factors: information gap and power asymmetry. For each behavior, we derive theoretical upper bounds on the number of learning iterations required for its emergence, revealing distinct dependencies on the underlying asymmetries. We validate these results in grid-world MARL environments and show that increased power asymmetry accelerates manipulation and agency loss, while increased information asymmetry accelerates paternalism and welfare loss. We further investigate scalability to multiple weak agents, finding that agency loss accumulates approximately linearly, welfare loss can grow superlinearly, and manipulation becomes increasingly uneven across agents. Finally, we examine utility alignment and show systematic relationships between alignment and the emergence of these behaviors. Together, these results provide a quantitative framework for analyzing how unequal information, control, and objectives shape emergent behavior in multi-agent AI systems.