Bench2Drive-Plan: Bridging Privileged Planner and End-to-End Autonomous Driving under Shared Protocols

Learning-based autonomous driving has developed within two separate communities based on the input information. Privileged planners receive ground-truth object states and HD maps while end-to-end models use raw sensor inputs. Despite sharing many similar modeling ideas, differences in scene inputs, simulation,and metrics make it difficult to compare results and transfer insights across them. In this work, we introduce$\text{Bench2Drive-Plan}$ to bridge this gap by evaluating privileged planners under current end-to-end protocols. On Bench2Drive, we adapt vectorized, diffusion-based, and language-based planning paradigms,including PLUTO, Diffusion Planner and InstructDriver. We also implement two raster-based CNN baselines using RGB semantic BEV images and multi-channel semantic masks, respectively. Comparisons with end-to-end baselines results show that ground-truth scene inputs do not inherently yield a performance advantage for the privileged planners. This finding raises a question about the information needed for learning based driving. Ground-truth boxes and maps provide accurate scene states but may omit useful context from raw sensors. End-to-end models could utilize visual context, but their learned perception may be inaccurate. To examine whether planners could benefit from visual information beyond boxes and maps, we augment privileged planners with camera inputs. On Bench2Drive, extra camera inputs provide no overall benefit. While on NAVSIM, extra camera inputs yield a positive effect, which might indicate that under real world complex environment, scene details beyond boxes could be helpful. Together, these studies show how a shared benchmark can benefit the studies of the two communities and we hope Bench2Drive-Plan could pave the way for future studies about synergistically training planner and end-to-end models.