Discovering Safety Spectra in Autonomous Driving via Constrained Adversarial Self-Play

Naturalistic traffic data contain few safety critical interactions, limiting their ability to expose the weaknesses of autonomous driving policies during training. Adversarial scenario generation can target these weaknesses, but it faces two challenges: optimizing only for safety violations may produce implausibly extreme behavior, while scenarios generated against a fixed policy may lose relevance as the protagonist evolves. To address these challenges, we introduce Constrained Self-Play with Adversarial Rationality (Co-SPAR), a closed-loop framework that couples prior-constrained adversary optimization with region-wise protagonist optimization. On the adversary side, a deviation budget around a behavioral prior defines nested adversary classes with different attack strengths. Within each class, the adversary maximizes the constraint decay function (CDF), which measures the protagonist's future violation risk, to identify the worst admissible adversary. On the protagonist side, the resulting robust CDF defines a robust feasible region that certifies safety against every adversary in the class. This region guides the protagonist to reduce violation risk outside it and improve task return inside it while preserving safety. We prove that at each adversarial strength, exact policy iteration monotonically expands the robust feasible region, improves task performance within it, and converges to the optimal CDF. Across strengths, we establish a robust feasible region spectrum and prove that a protagonist optimal at a stronger strength retains its safety certificates against weaker adversary classes. Experiments show Co-SPAR advances the attack–realism Pareto frontier with a controllable adversarial spectrum and learns protagonist policies that outperform adversarial training baselines across nominal and adversarial traffic. Cross-strength evaluations demonstrate the predicted robust feasible region ordering and stronger-to-weaker safety transfer.