E2M-Distill: Distilling End-to-End Driving Models into Motion-only Sim Agents for Large-Scale Autonomous Driving Simulation

Widespread deployment of autonomous vehicles (AVs) may introduce unexpected system-level effects that require evaluation of interactions among multiple vehicles, so large-scale autonomous driving simulation is needed to understand their urban-mobility impact. Existing approaches face a trade-off: faithfully simulating real autonomous driving stacks from perception to planning is prohibitively expensive, while motion-only simulation bypasses the perceptual limitations of a target AV stack. We formulate this gap as *AV-Stack-to-Sim-Agent Conversion*: given a target AV stack, derive a motion-only sim agent (consuming only privileged simulator state, without rendering) whose closed-loop rollouts preserve the target stack's behavior. This task exhibits an *Inverted Learner-Expert Asymmetry* (ILEA): the teacher acts on partial sensor perception while the student sees privileged state, so conversion must account for the teacher's perceptual limitations. We propose **E2M-Distill** (E2E-to-Motion-only sim agent Distillation), a distillation framework that addresses ILEA. E2M-Distill places a Perception Contraction Module (PCM) ahead of the retained teacher planning head: PCM contracts the student's privileged simulator-state features toward the teacher's perception-conditioned representation. An interface-alignment loss anchors the contracted features to a teacher-side activation selected by the Perception-Planning Score (PPS). We introduce three closed-loop consistency metrics for this task and demonstrate across the evaluated target stacks that E2M-Distill improves behavioral fidelity while reducing the per-agent computational bottleneck for large-scale AV simulation.