StreamDrive: Streaming Perception for End-to-End Autonomous Driving via Delay-Conditioned Feature Extrapolation
Abstract
End-to-end autonomous driving systems achieve strong offline benchmarks but ignore a real-world constraint: by the time inference completes, the world has moved on and predictions are stale. We present StreamDrive, a streaming perception framework that equips end-to-end driving models with delay-aware feature extrapolation. StreamDrive corrects latent instance features rather than individual task outputs: a single delay-conditioned residual propagates through the entire task stack: detection, motion forecasting, and planning. A timestamp-aware temporal queue replaces frame-indexed storage with wall-clock timestamps, keeping instances matched under variable frame rates and irregular sensor timing. StreamDrive decomposes delay compensation into a constant-velocity baseline that captures first-order motion, plus a FiLM residual that learns non-linear, delay-dependent corrections conditioned on scene context and ego kinematics. On nuScenes, StreamDrive improves streaming detection mAP by pp at real-time () playback over the SparseDrive-S baseline. Streaming planning collision rate drops by 9.4% relative at and 14.2% at playback speed, with consistent gains across all tasks and playback speeds. We also introduce a multi-task streaming evaluation framework for end-to-end driving, including streaming metrics for motion forecasting and the Plan Consistency Score (PCS), a time-adjacent measure of plan stability under latency that complements accuracy metrics. StreamDrive adds less than 1% parameter overhead and less than 1ms of inference latency.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.