Planner-Oriented Scenario Generation: Uncertainty-Guided Generation of Learnable Long-Tail Scenarios

Handling long-tail scenarios is one of the main challenges in autonomous driving. Several recent works have demonstrated that generative models can be a promising way to address this challenge by synthesizing diverse long-tail scenarios. However, they require domain knowledge of what makes a scene critical, such as specific traffic or safety conditions. This domain knowledge is often not available in practice and does not reflect which scenarios the planner has not learned well. Furthermore, these works also need additional components to keep their scenarios learnable. In this paper, we propose Planner-Oriented Scenario Generation (POSG), which generates uncertain and learnable long-tail scenarios based on what the planner has learned without any domain knowledge. We hypothesize that a scene is worth training on if it is uncertain and learnable for the planner, not merely if it is rare or adversarial. To implement our hypothesis, POSG combines the score estimate of the diffusion model with the gradient obtained from the planner's uncertainty. POSG also identifies learnable scenarios by leveraging a generative model that synthesizes the entire scene. The generative model enables us to determine whether the generated trajectories are learnable in the context of the entire scene. Experiments show that scenarios generated by POSG induce a higher planner error than unguided samples from the same generative model, while improving the planner when it is retrained on them.