WeatherShift: Transitional Weather Video Generative Modeling for Autonomous Driving Perception

Robust perception for autonomous driving requires generalizing beyond discrete weather categories to the continuously evolving conditions encountered on the road. Existing autonomous-driving datasets and perception methods predominantly represent weather as fixed states such as clear, rain, fog, or snow, ignoring the transitions between them, yet real driving sequences pass through intermediate regimes, such as sunny to rainy or rainy to sunny, with continuously varying levels of adversity. Recent weather-transition generation methods primarily operate at the image level and typically assume a static background, limiting their ability to capture the temporal evolution of weather alongside the dynamic scene content and ego-motion inherent to real-world driving videos. To address these limitations, we propose WeatherShift, a video-based framework for generating temporally coherent weather transitions in autonomous-driving scenes. Starting from real-world driving videos in nuScenes, we construct extreme-weather counterparts, namely Rainy-nuScenes, Foggy-nuScenes, and Snowy-nuScenes, by applying weather-specific scene transformations and atmospheric effects. We then generate intermediate weather-transition videos in which weather attributes progressively evolve while scene content, ego-motion, and background elements dynamically change across frames. Building on this framework, we introduce nuScenes-WT (Nuscenes Weather Transition Video Dataset), comprising twenty directional transition categories spanning sunny, rainy, foggy, snowy, and cloudy conditions. We further evaluate the generated videos for downstream object detection and semantic segmentation, establishing a unified generative-to-perceptual benchmark for transitional weather. Experimental results demonstrate that WeatherShift generates visually coherent weather transitions while preserving temporal scene dynamics, and that nuScenes-WT provides a challenging testbed for evaluating perception under continuously changing weather conditions.