Event4D: Dense 4D Reconstruction from RGB and Neuromorphic Event Streams

Reconstructing dynamic 3D scenes over time (4D reconstruction) from a single moving camera is important for robotics and AR/VR. However, existing RGBonly feed-forward methods are vulnerable to motion blur and temporal aliasing under rapid camera or object motion. Event cameras asynchronously capture per-pixel brightness changes at microsecond resolution, providing complementary temporal information between RGB frames. We introduce Event4D, the first single-stage feed-forward 4D reconstruction pipeline that fuses events and RGB images. Event4D pairs event voxels at fine-grained reconstruction time steps with the most recently captured RGB frame and fuses their features through a learned RGB-staleness conditioning. Using five benchmark datasets, we evaluate Event4D under three challenging conditions for RGB-based 4D reconstruction: fast object motion, low illumination, and temporal aliasing. Event4D improves threshold accuracy over the state-of-the-art by 10.11% on fast-motion sequences, 59.29% under temporal aliasing, and 11.61–24.77% across low-light sequences.