acceptodds
Under review as a conference paper at ICLR 2027

Real-Time Motion Consistency Rendering in 4D Gaussian Splatting

Abstract

Novel view synthesis for dynamic scenes is a key challenge in computer vision, due to the complexities of the time dimension and diverse motion patterns. The added dimension in 4D Gaussian distributions complicates their constraints, hindering the rendering of realistic and consistent appearances. To achieve realistic dynamic appearance reconstruction, we introduce Motion 4D Gaussian Splatting (Motion4DGS), a rendering approach that integrates motion consistency regularization into 4DGS. This regularization incorporates both 4D spatial and temporal consistency, leveraging depth and optical flow information. Furthermore, we extend optical flow constraints to continuous unit time intervals, enhancing the consistency of motion appearance without increasing the point cloud size. Motion4DGS excels in rendering dynamic scenes with multiple objects, including those with complex reflective areas. Comprehensive evaluations on both monocular and multi-view datasets show that Motion4DGS achieves real-time performance at high resolutions, surpassing existing methods in both quantitative and qualitative metrics. It provides a notable improvement in quality, all while maintaining comparable rendering speeds and training times to previous approaches.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Novel View Synthesis, Dynamic Scene, Gaussian Splatting, 4D Gaussian Splatting

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_uh5ZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Motion4DGS, which adds motion consistency regularization to deformable 4DGS. The regularization consists of a spatial term that aligns rendered surface normals with depth gradients and temporal terms based on optical flow. The key idea is to extend these constraints from individual time steps to unit time intervals, penalizing changes in depth, normals, and flow between adjacent time steps. The method is evaluated on the N3V and D-NeRF datasets, where it reports higher rendering quality, shorter training time, and faster rendering than prior methods.

Critical strengths
  1. The paper targets a practical problem (temporal inconsistency such as flickering and ghosting) in dynamic scene reconstruction.
  2. The proposed regularization does not modify the rendering pipeline, so it can be added to existing 4DGS methods with little overhead.
Critical weaknesses
  1. Evaluation is limited to N3V (relatively slow motion) and D-NeRF (synthetic), so the paper’s claims about fast and non-rigid motion cannot be verified. Results on more challenging datasets such as ENeRF-Outdoor [1] or SelfCap [2] are needed.
  2. All baselines date from 2023, and several recent methods such as FreeTimeGS [2] and TRiGS [3] are missing. In addition, the baseline numbers in Table 1 differ from the originally reported values (e.g., 4DGS: 28.42 here vs. 31.15 reported; RealTime4DGS: 29.95 vs. 32.01) without indicating whether they were re-run or cited, and the RealTime4DGS row is identical across Tables 1 and 2. The source and evaluation protocol of these numbers should be clarified.
  3. The proposed losses largely combine existing depth, normal, and flow supervision using off-the-shelf priors, so the novelty mainly lies in extending these constraints to unit time intervals. However, in Table 3, the spatial term alone yields +2.32 dB, while this extension improves PSNR by only about 0.25 dB (Full −- Space), and this is measured only on D-NeRF. Without an ablation isolating this extension on N3DV or other real-world datasets, its contribution is difficult to assess.
  4. Several details required for reproduction are missing, including how depth, normals, and optical flow are rendered from the Gaussians, which estimators provide the priors, the target of Eq. 2, and how λ1\lambda_1-λ3\lambda_3 are learned and what their final values are.
  5. Although the method only adds loss terms, it reports a reduction in training time from 72 to 20 min and an increase in FPS from 40 to 110 over Deformable4DGS. Since additional losses typically increase training cost and the rendering pipeline is unchanged, the source of these gains is unclear; Gaussian counts and whether baselines were re-run on the same hardware should be reported.

[1] ENeRF: Efficient Neural Radiance Fields for Interactive Free-viewpoint Video. Lin et al. (SIGGRAPH Asia 2022)

[2] FreeTimeGS: Free Gaussian Primitives at Anytime and Anywhere for Dynamic Scene Reconstruction. Wang et al. (CVPR 2025)

[3] TRiGS: Temporal Rigid-Body Motion for Scalable 4D Gaussian Splatting. Yeom et al. (arXiv 2026.04)

Questions for authors
  1. Are λ1\lambda_1-λ3\lambda_3 shared across all scenes or determined per scene? If they differ per scene, how does the method perform when they are fixed per dataset?
  2. Could the authors report results on more challenging datasets such as ENeRF-Outdoor or SelfCap, and include comparisons with recent methods such as FreeTimeGS and TriGS?
  3. Since the proposed losses are regularizers, would applying them to other backbones (e.g., Deformable3DGS or STG) yield similar gains? In addition, in Table 3, the time term alone yields +2.13 dB, whereas its marginal gain over the spatial term is only about 0.25 dB, suggesting that much of its effect overlaps with the depth/normal prior. How does the unit-time-interval constraint perform when applied without external priors (i.e., only enforcing consistency between rendered quantities at tt and t+δtt + \delta t)?
1 comment
AuthorsOpenReview ↗

Performance Validation and Comprehensive Experiments

1. Quantitative Results on the Long-sequence Datasets

Table 1 reports quantitative results on three long-sequence datasets, ENeRF-Outdoor [lin2022efficient], MobileStage [xu20234k4d, xu2024relightable], and CMU-Panoptic [joo2015panoptic], using PSNR, SSIM, and LPIPS.

On ENeRF-Outdoor, our method achieves the best performance, exceeding the best competing PSNR by 2.64 dB and the best competing SSIM by 0.0771, while reducing LPIPS by 0.1264. On MobileStage, our method improves PSNR and SSIM by 2.98 dB and 0.0329, respectively, and reduces LPIPS by 0.0871 relative to the best competing results. On CMU-Panoptic, our method outperforms Dy3DGS by 2.73 dB in PSNR and 0.0271 in SSIM, while reducing LPIPS by 0.2133. Across the three datasets, PSNR gains range from 2.64 dB to 2.98 dB, and absolute LPIPS reductions range from 0.0871 to 0.2133, indicating consistent improvements in fidelity and perceptual quality on long sequences.

Overall, our method achieves the best results across all three metrics on all three datasets. These consistent gains support its effectiveness in reconstructing long dynamic sequences.

Table 1. Quantitative Comparison on Long-Sequence Datasets. Higher PSNR and SSIM and lower LPIPS indicate better performance. The best result for each metric within each dataset is shown in bold.

DatasetMethodPSNR ↑SSIM ↑LPIPS ↓
ENeRF-OutdoorOurs28.000.90020.1501
ENeRF-Outdoor4K4D25.360.80800.3795
ENeRF-OutdoorENeRF25.020.78240.3043
ENeRF-Outdoor3DGS24.020.82310.2765
MobileStageOurs31.000.95010.1512
MobileStage4K4D25.900.87880.3872
MobileStageENeRF19.140.74920.4365
MobileStage3DGS28.020.91720.2383
CMU-PanopticOurs27.000.97030.3002
CMU-PanopticDy3DGS24.270.94320.5135

2. Rigorous and Comprehensive Experiments

All our experimental results are reliable and valid, as supported by the findings reported in 4D Rotor-GS. By reimplementing the underlying CUDA kernel functions using C++, we gain improved hardware-level computational performance. As shown in Table 2, Motion4DGS achieves substantially higher computational efficiency than Deformable3DGS. Therefore, when evaluated under an equivalent computational budget, our approach outperforms all Gaussian-based baselines in terms of PSNR. Optical flow is obtained by estimating pixel-wise motion between frames, while depth maps are directly generated through the Gaussian rendering pipeline to capture the underlying 3D scene geometry. We present quantitative ablation results in Table 3 to demonstrate the contributions of individual components of Motion4DGS on the D-NeRF dataset. Furthermore, Figure 6 provides qualitative ablation comparisons on the Plenoptic Video dataset, while Figure 7 offers additional evidence of Motion4DGS's effectiveness by illustrating improvements in optical flow consistency.

Duan, Yuanxing, et al. "4d-rotor gaussian splatting: towards efficient novel view synthesis for dynamic scenes." ACM SIGGRAPH 2024 conference papers. 2024.

Reviewer_eut2OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Motion4DGS, a framework for improving the rendering quality and motion consistency of dynamic scenes with 4D Gaussian Splatting. The authors introduce a motion consistency regularization, which combines spatial consistency based on depth and surface normal, and temporal consistency based on optical flow. Furthermore, the authors extend these constraints from individual time steps to continuous time intervals to improve the motion consistency of Gaussian primitives. Experiments on Plenoptic Video and D-NeRF datasets show improvements in rendering quality and efficiency.

Critical strengths
  1. The proposed method is relatively simple and can be integrated into existing Gaussian Splatting frameworks without introducing significant additional complexity.

  2. The experimental results demonstrate promising improvements in rendering quality and efficiency. The ablation studies also show the effectiveness of different regularization terms.

Critical weaknesses
  1. My major concern lies on the mathematical formulation of the proposed method. In Eq. (1), the authors directly compare surface normals with the gradient of depth maps, while these two quantities are not directly equivalent. Besides, in Eq. (2) and Eq. (4), it is unclear whether the optical flow and depth are predicted from the Gaussian representation or obtained from ground truth. If they are fixed supervision signals, I don't understand how these loss terms can provide gradients to optimize the Gaussian parameters. I believe the authors need to provide a more rigorous formulation of these losses.

  2. In Table 2, Deformable3DGS achieves 39.31 PSNR on D-NeRF, which is significantly higher than the proposed method (35.30). However, the authors claim their method outperforms existing methods. Besides, the reported PSNR improvement in the text (32.99 to 34.37) is inconsistent with Table 2. These inconsistencies make it difficult to understand the actual performance improvement.

  3. The proposed framework mainly builds on existing 4D Gaussian Splatting with additional depth, normal and optical flow regularization. These geometric and temporal constraints have been widely explored in previous dynamic reconstruction methods. I am not fully convinced about the novelty of the proposed regularization, especially the extension to continuous time intervals, which is not clearly explained in the methodology.

  4. The paper misses an important baselines, GaussianFlow (https://arxiv.org/abs/2403.12365), Motion-aware 3D Gaussian Splatting (https://arxiv.org/abs/2403.11447), Dynamic 3D Gaussians (https://arxiv.org/abs/2308.09713), and Edge4DGS (https://ojs.aaai.org/index.php/AAAI/article/view/37847), are not sufficiently discussed and compared. For example, GaussianFlow also explores using optical flow to supervise Gaussian motion and Edge4DGS also proposes temporal consistency regularization over continuous time intervals, with a very similar formulation to Eq. (4).

Questions for authors
  1. Could the authors clarify the inconsistent results in Table 2 and the main text? Why does Deformable3DGS achieve significantly higher PSNR than the proposed method on D-NeRF?

  2. Could the authors add additional comparison results with other related baselines as mentioned above?

2 comments
AuthorsOpenReview ↗

Methodological Novelty and Rigorous Experimental Validation

1. Rigorous and Comprehensive Experiments

All our experimental results are reliable and valid, as supported by the findings reported in 4D Rotor-GS. As shown in Table 2, Motion4DGS achieves substantially higher computational efficiency than Deformable3DGS. Therefore, when evaluated under an equivalent computational budget, our approach outperforms all Gaussian-based baselines in terms of PSNR.

Duan, Yuanxing, et al. "4d-rotor gaussian splatting: towards efficient novel view synthesis for dynamic scenes." ACM SIGGRAPH 2024 conference papers. 2024.

2. Methodological Novelty and Technical Contributions

Recent work improves dynamic Gaussian representations through motion supervision and geometric regularization. GaussianFlow derives differentiable image-space flow from Gaussian dynamics to enable direct optical-flow supervision, while Motion-aware 3D Gaussian Splatting introduces uncertainty-aware flow augmentation. Dynamic 3D Gaussians maintains persistent Gaussian attributes and imposes local rigidity to jointly support dynamic view synthesis and dense 6-DoF tracking. Edge4DGS combines Gaussian primitives with convex hulls and extends flow-guided edge consistency regularization to continuous time intervals, improving boundary sharpness and temporal coherence. Different from these work, the main contributions of Motion4DGS are summarized as follows:

a. Unified Spatiotemporal Motion Consistency Regularization. We propose a unified 4D motion-consistency framework that jointly incorporates spatial and temporal regularization into dynamic Gaussian representations. By explicitly constraining geometric consistency and temporal motion coherence, our formulation facilitates stable dynamic scene reconstruction and high-fidelity appearance rendering.

b. Temporal Multimodal Motion Constraints. We develop a spatiotemporal regularization strategy that jointly leverages depth, surface normals, and optical flow to constrain Gaussian dynamics across temporal intervals rather than relying solely on independent frame supervision. This formulation promotes geometrically consistent and temporally coherent motion representations.

c. Spatiotemporally Guided Gaussian Property Optimization. We introduce a joint spatial–temporal optimization strategy that progressively refines the geometric and appearance properties of 4D Gaussian primitives. By integrating complementary spatial and temporal cues, our approach improves the representation of evolving scene structures and appearance consistency across time.

d. Efficient Differentiable 4D Dynamic Rendering. We develop an efficient, differentiable 4D Gaussian rendering framework that integrates motion-consistency constraints with spatiotemporal Gaussian optimization. Our framework enables high-fidelity dynamic novel-view synthesis with real-time rendering capability and competitive training efficiency, offering a practical balance between reconstruction quality and computational cost.

Thank you for the response. However, I don't think the rebuttal has sufficiently addressed my major concerns.

  1. In Table 2, Deformable3DGS achieves significantly higher PSNR than the proposed method (39.31 vs. 35.30). The authors claim their method performs better under the same computational budget, but no experimental evidence is provided. Simply citing 4D Rotor-GS cannot support this claim.

  2. My concerns about Eq. (1) and Eq. (2) remain unanswered. It is still unclear how the surface normal and depth gradient are directly compared, and why minimizing the product of optical flow and depth would encourage accurate motion. The rebuttal also does not clarify how the gradients propagate.

  3. The authors mainly repeat their high-level contributions without clearly explaining the technical differences from existing works, especially Edge4DGS, which uses a very similar temporal regularization.

I will keep my current score.

Reviewer_ErCgOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper addresses an important problem of the domain which is improving temporal consistency in dynamic Gaussian splatting. The motivation is sound, but the main idea and key formula closely overlap with Edge4DGS (AAAI 2026), which is neither cited nor discussed. Several reported results are also inconsistent, including the Deformable3DGS baseline. These issues raise concerns about possible academic misconduct and undermine the paper’s credibility.

Critical strengths

The paper addresses temporal consistency in dynamic scene reconstruction by combining depth, normal, and optical-flow regularization within a real-time Gaussian splatting framework.

Critical weaknesses
  1. Severe technical overlap with prior work Edge4DGS[1] published in AAAI2026. The representation and rendering pipeline closely follow , including HexPlane-style spatiotemporal encoding and an MLP-based Gaussian deformation field. Morever, the core equation as the core contribution of optical-flow interval regularizer in Eq. (4) presented in this paper has the same core formulation as Eq. (8) of Edge4DGS, with only an additional depth-validity mask.However ,the paper neither discusses this closely related work nor has cited in the references.

2.Multiple inconsistencies in the reported results make the reliability of the experiments and the paper’s conclusions unconvincing.

  • RealTime4DGS in Tables 1 and 2: The results shown in the two table are the ones evaluated on two different datasets Plenoptic Video Dataset and D-NeRF dataset, but the results are exactly same.
  • Table 2 reported that their results on D-NeRF dataset are better than the results of Deformable3DGS which are shown as 32.99db PSNR and 0.05. However, the original results reported in the original paper of Deformable 3DGS should be 39.51 dB PSNR / 0.01 LPIPS.
  • The authors claim that their method outperforms the others methods for perceptual quality: However, the results reported in Table 1 have shown that the proposed method’s LPIPS of 0.10 is worse than K-Planes’ 0.07 and DyNeRF’s 0.08.
  • The D-NeRF result is reported as 34.37 dB in the text but 35.30 dB in Table 2.

[1]Shi, B., Guan, T.N. and Xiaodong, Y., 2026, March. Edge consistency for 4D Gaussian splatting in dynamic scene rendering. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 40, No. 11, pp. 8923-8932).

Questions for authors

see the weakenesses

1 comment
AuthorsOpenReview ↗

Technical Contributions and Computational Efficiency

1. The main contributions of Motion4DGS are summarized as follows:

(1) Unified 4D Motion Consistency Regularization. We propose a unified motion-consistency framework that jointly enforces spatial and temporal coherence in 4D Gaussian representations, enabling consistent modeling of dynamic scene evolution and improving the fidelity of dynamic appearance rendering.

(2) Temporal Spatiotemporal Motion Constraints. We introduce a novel temporal regularization strategy that integrates depth, surface normals, and optical flow to extend motion constraints from discrete timestamps to continuous temporal intervals, promoting temporally smooth and spatially consistent Gaussian dynamics.

(3) Spatiotemporally Guided Gaussian Optimization. We develop an optimization strategy that jointly leverages spatial and temporal information to progressively refine the properties of 4D Gaussian primitives, facilitating accurate modeling of evolving appearance representations across time.

(4) Efficient Differentiable 4D Rendering. We establish an efficient, differentiable 4D rendering framework that integrates motion-consistency regularization with dynamic Gaussian optimization, achieving high-fidelity, real-time dynamic scene rendering while maintaining competitive training efficiency and rendering speed.

2. Rigorous and Comprehensive Experiments

All our experimental results are reliable and valid, as supported by the findings reported in 4D Rotor-GS. As shown in Table 2, Motion4DGS achieves substantially higher computational efficiency than Deformable3DGS. Therefore, when evaluated under an equivalent computational budget, our approach outperforms all Gaussian-based baselines in terms of PSNR.

We have carefully reviewed the manuscript and corrected all numerical issues.

Duan, Yuanxing, et al. "4d-rotor gaussian splatting: towards efficient novel view synthesis for dynamic scenes." ACM SIGGRAPH 2024 conference papers. 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.