Beyond Common Scenes: Long-Tailed Accident Anticipation with Scene Memory

Traffic accident anticipation aims to issue early warnings of impending collisions from dashcam video, providing time for preventive action. Yet balanced accident and non-accident samples can conceal substantial imbalance among driving situations, leaving models with limited examples of the early warning cues in rare scenes. We formulate this scene-level long-tail problem and characterize it through complementary analyses of scene frequency, reference-model error, and expert curation, distinguishing statistical rarity, anticipation difficulty, and safety relevance. To support underrepresented scenes with related experience, we propose TailBank, a persistent scene-memory framework. TailBank retains central and peripheral representatives of training scene groups across batches and retrieves relevant knowledge to enhance current scene features. A distance-based fitness mechanism assesses how well the bank covers the current scene, modulates the fused representation, and separately flags insufficient support. This keeps scene familiarity distinct from accident likelihood. Our theoretical analysis establishes sufficient conditions for enhancement to reduce anticipation loss and for known and open scenes to be separated. Experiments on DAD, CCD, and A3D show state-of-the-art accuracy on DAD and A3D and competitive warning timeliness. Further evaluations support TailBank's effectiveness in rare and challenging scenes, generalization to unseen scenes, and identification of difficult cases.