Velocity 4D Gaussian Splatting: Velocity is All You Need

Novel view synthesis for dynamic scenes remains a significant challenge in computer vision, primarily due to the complexity introduced by the temporal dimension and diverse motion patterns. The additional dimensionality in 4D Gaussians complicates their constraint, resulting in motion inconsistencies and artifacts. Methods that model time independently struggle to decouple temporal and spatial dimensions effectively, while time-slicing techniques fall short in capturing temporal consistency. To address motion inconsistencies, we introduce velocity 4D Gaussian Splatting (V4DGS), a dynamic reconstruction approach that incorporates velocity consistency regularization into 4D Gaussians, leveraging temporal constraints alongside spatial priors. By using velocity to quantify time dimension, we learn transferable temporal features that enable accurate motion prediction at new timestamps. Our velocity consistency regularization framework includes both temporal and spatial components, effectively addressing motion inconsistency by constraining velocity variations within an infinitesimal time interval. V4DGS excels in dynamic scenes with multiple objects and even reconstructing reflective regions. Through extensive evaluations on both monocular and multi-view motion datasets, V4DGS achieves real-time rendering at high resolutions, surpassing existing methods in both quantitative and qualitative metrics. V4DGS substantially improves quality (PSNR gain of 3.31 on the Plenoptic dataset), maintaining comparable rendering speeds and training time to previous methods.