Scene Flow Consistent and Depth-Aware Temporal Regularization for 4D Gaussian Splatting
Abstract
3DGS represents scenes using Gaussian primitives, making it well-suited for a wide range of static scenarios. However, achieving high-quality dynamic video synthesis and physically realistic rendering requires accurate 3D motion modeling and temporal consistency to prevent physically implausible output across time. To address these challenges, we introduce PhysScene4DGS, a physics-based algorithm for 4D dynamic scene generation and rendering. By jointly optimizing 3D scene flow consistency and temporal coherence, PhysScene4DGS enables the physically realistic rendering of dynamic objects. PhysScene4DGS begins by computing 3D scene flow using depth and optical flow. Then, we introduce 4D temporal consistency regularization, which penalizes velocity discrepancies through inverse normalization between velocity and depth, promoting smooth and continuous motion. Furthermore, the scene flow consistency regularization aligns the predicted 3D motion with observed motion derived from back-projecting depth and optical flow, reinforcing physical accuracy. Experimental results demonstrate that PhysScene4DGS ensures both physical plausibility and visual realism in the real-time rendering of complex dynamic scenes, while maintaining comparable training time and FPS.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.