Autonomous Traffic through Self-Play

Traffic rules—lanes, prescribed travel directions, signals, and right-of-way—solve a coordination problem, designed around the limitations of human drivers. We ask what organization a population of autonomous vehicles would discover if these rules were not imposed, and how it would compare with rule-based autonomous traffic. Without rules, vehicles must converge on conventions for passing, merging, and intersection traversal that no one specifies, in continuous two-dimensional road space, with each vehicle acting independently based on local observations. To learn at the required scale, we introduce OpenDrive-GPU, a batched multi-agent driving simulator, and AutoZero, a self-play method combining spatial attention with phasic policy–value learning. Trained on several road networks from hundreds of billions of vehicle transitions without human demonstrations, a single shared policy exhibits recognizable crossing, yielding, merging, and passing behaviors. Evaluated on held-out road networks, rule-free autonomous traffic sustains 1.78$\times$ and 2.63$\times$ the throughput of the best-performing rule-based traffic at medium and high demand, respectively. While these results demonstrate the efficiency potential of self-organized autonomous traffic, they also reveal safety challenges: collisions and road departures increase under heavy demand, and throughput shows sensitivity to failure clearance. These findings identify safe coordination as a central open challenge for rule-free autonomous traffic.