PRED: Predicting Latent Image Features for High-Frequency RGB-Event Object Detection
Abstract
RGB cameras provide rich semantic information, but increasing their frame rate incurs substantial sensing, transmission, and computational costs. Event cameras capture brightness changes with microsecond-level temporal resolution, preserving high-frequency motion while providing limited appearance cues. Their complementary properties make RGB-event fusion promising for high-frequency detection. However, existing asynchronous detectors typically fuse current events with stale image features from the previous RGB frame. This temporal mismatch prevents the fused representation from fully reflecting the current scene and limits detection accuracy between RGB arrivals.We introduce Predictive RGB-Event Detection (PRED), a frame-rate-decoupled framework that predicts the current latent image representation from the previous RGB frame and subsequent events, reducing temporal mismatch before fusion. To provide reliable supervision for this cross-time prediction, we propose two-stage cross-time distillation: a teacher is first trained independently with access to the current RGB frame and then frozen to provide a stable latent target for PRED. We also introduce the High-Frequency Event Dataset (HiFE), constructed through high-fidelity offline rendering and event simulation to provide temporally aligned 500-Hz annotations for millisecond-scale detection evaluation. Experiments on real-world and synthetic event-camera detection benchmarks show that PRED consistently outperforms prior methods across varying query frequencies. In end-to-end wall-clock replay, PRED nearly keeps pace with a 250-Hz offered query stream and delivers predictions at over the rate of prior asynchronous detectors.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.