HorizonForgeV2: Agentic Scene Editing for Long-Tail Driving Video Generation

Long-tail scenarios, such as unusual road users and unexpected obstacles, remain particularly challenging for autonomous-driving systems and are important for both training and evaluation. They are also extremely rare in real-world data: of over 6.39 million miles of Waymo driving logs, only 0.03% qualify as long-tail scenarios. Collecting long-tail scenarios directly is prohibitively expensive, and some, such as an elephant standing in the middle of the road, cannot be staged safely at all. Driving simulation can instead generate these scenarios on demand. Recent methods combine the precise control of 3D scene representations with the photorealism of video generation. These methods break down, however, when the inserted 3D asset falls outside the video generator’s training distribution: the video generator has rarely seen such objects during training and tends to distort or even remove them. Image-editing foundation models, in contrast, are trained on web-scale data and can insert diverse long-tail objects that fit the surrounding scene. Building on this observation, we present HorizonForgeV2, which moves long-tail appearance generation from the video domain to the image domain: image-editing models synthesize realistic keyframes, and a keyframe-conditioned video diffusion model (VDM) turns them into temporally coherent videos. Importantly, HorizonForgeV2 is trained only on existing driving videos, without collecting any new keyframe data. Edited keyframes can still be inconsistent with the 3D scene, and these errors propagate to the generated video. We therefore employ an agentic framework that inspects the intermediate keyframes and videos, locates failed edits, and iteratively refines generated videos with tool calling. Compared with HorizonForge, HorizonForgeV2 raises the Operation Success Rate (OSR), a six-criterion score on a 1–10 scale, from 5.77 to 7.52 on our long-tail benchmark and from 6.47 to 7.75 on common driving-scene edits.