acceptodds
Under review as a conference paper at ICLR 2027

One Query, Real Motion: Local-Flow Distillation for Dynamic Gaussian Splatting

Abstract

Dynamic 3D Gaussian Splatting methods usually represents a scene as canonical Gaussians deformed by a time-conditioned network. Such direct-query models enable real-time rendering, but formulate motion as a timestamp-to-state mapping rather than a state-to-state transition cause fast or complex motion is poorly constrained, which often produces motion blur and loss of fine spatial detail. To address this issue, we decompose motion into an exactly composable low-frequency positional semigroup and a state-dependent local flow that captures high-frequency residual dynamics. The local flow is learned through training-time rollouts supervised on keyframes and intermediate frames, and a transition-consistency objective distills the learned rollout dynamics into the direct decoder. At inference, only the direct decoder is used, so rendering remains a single-query operation, while the learned temporal dynamics improve alignment and detail. Experiments on HyperNeRF and N3DV demonstrate consistent improvements over baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Dynamic Reconstruction, Motion Modeling, 4D Gaussian Splatting

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UkZkOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper illustrates the weaknesses of the current dynamic 3DGS: lack of time composability and learning rate degeneration. To mitigate these, this paper decouples direct querying and state transition into two models, and distills the ability of state transition into the direct querying model. According to the experiments, this design shows overall SOTA performance on HyperNeRF and N3DV.

Critical strengths
  • The main highlight of the paper is the decoupling of a direct state-free model and a rollout model, which seems to be brand new. This design enables the capabilities of both time querying and motion transition.
  • Another strength is the distillation strategy. The direct model is matched with the rollout model at every timestamp; as a result, the direct model gains the ability to perform time querying as well as real motion.
  • The experiments and evaluation are reasonably thorough and honestly reported, with three datasets and five representative 4DGS baselines.
Critical weaknesses

I don't doubt the architecture of the method, but the experimental results seem to be less impressive.

  • The central claim is weakly convincing. According to the visualization, the rendering results of ours and 4DGS are very similar. The quantitative results also support that the performance of 4DGS and ours is very close. I wonder whether the advantage of the proposed method would diminish if the dataset has a high frame rate.
  • The trajectory/flow baselines the paper itself cites as addressing long-term motion consistency (SplineGS, TRiGS, MoRGS, SPIN-4DGS) are not compared with, even though they are the closest competitors to the paper's claimed contribution.
  • However, considering motion is still a good aspect. While video timestamps are discrete, motion is continuous. Predicting continuous motion is an intriguing insight.
Questions for authors
  • How much does the rollout model contribute to the final performance? Any explanations from both technical and principled perspectives? Preferably supported by experimental results; see Q2.
  • Can you provide a direct quantitative evaluation of temporal consistency or long-horizon behavior (e.g., an inter-frame warping error, a temporal-coherence metric, or results on substantially longer sequences) that isolates where the rollout and distillation actually help, rather than relying on aggregate PSNR/SSIM/LPIPS?
  • Can the rollout model track the motion of certain Gaussians? For example, there are objects on the desk; can one object be tracked with our model?
Reviewer_XinjOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a dynamic Gaussian Splatting framework that improves motion modeling by separating direct state prediction from state-transition learning. It introduces a lightweight Direct model for single-query inference and a higher-capacity Rollout model that learns state-dependent local dynamics through multi-step training. A transition-consistency distillation objective transfers the learned dynamics to the Direct model, while combining DCT-based low-frequency positional motion with nonlinear local residuals.

Critical strengths
  1. The paper clearly identifies the limitations of direct time-to-state prediction in modeling fast and complex motion, motivating the introduction of state-transition learning and distillation.

  2. This Paper presents the proposed framework in a logical and coherent manner, clearly explaining the roles of the Direct and Rollout models and their interaction during training.

Critical weaknesses
  1. The evaluation could be strengthened by including additional dynamic Gaussian methods that explicitly separate static and dynamic components, particularly mask-based decomposition and anchor-based motion representations. Such comparisons would help clarify the advantages of the proposed transition-based motion modeling.

  2. Although the paper provides ablations on motion representations and transition distillation, further analysis of key training choices, such as keyframe sampling intervals and rollout horizons, would help clarify their influence on reconstruction quality and temporal consistency.

  3. The paper reports training iterations but does not provide sufficient information on total training time, GPU resource consumption, or inference FPS. Reporting these metrics would help assess the computational overhead of the proposed training framework and the efficiency of single-query inference.

Questions for authors
  1. Could the authors compare with additional static-dynamic decomposition methods, particularly mask-based and anchor-based approaches, to clarify the advantages of transition-based motion modeling?

  2. How sensitive is the method to keyframe sampling intervals and rollout horizons? Could the authors provide corresponding ablation results?

  3. Could the authors report total training time, GPU resource consumption, and inference FPS to better assess the efficiency of the proposed framework?

Convincing answers could strengthen my assessment and potentially increase my rating.

Reviewer_rWJ1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a local-flow distillation framework for dynamic Gaussian Splatting. It combines DCT-based positional motion with state-dependent nonlinear transitions and distills training-time rollout dynamics into a Direct model for single-query inference. Experiments on HyperNeRF and N3DV demonstrate improvements in reconstruction quality.

Critical strengths

The framework incorporates state-dependent motion learning without requiring iterative rollout at inference. Separating keyframe and intermediate-frame supervision provides a meaningful approach to learning motion transitions. The method improves perceptual quality and reconstruction performance on real-world dynamic datasets.

Critical weaknesses
  1. Limited novelty: The distinction from TimeFormer (ICCV 2025) and MoE-GS (ICLR 2026), which also leverage training-time temporal modeling or distillation for efficient inference, is insufficiently established.
  2. Missing relevant baselines: Comparisons with TimeFormer (ICCV 2025), Mango-GS (ICLR 2026), MAPo (CVPR 2026), and SPIN-4DGS (ICLR 2026) are absent. Differences in reported 4DGS baseline performance also require clarification.
  3. Incomplete ablations: The contribution of rollout distillation is not isolated from additional supervision or training capacity. Moreover, the DCT composability guarantee applies only to the positional component and is not unique to DCT.

[1] TimeFormer: Capturing Temporal Relationships of Deformable 3D Gaussians for Robust Reconstruction, ICCV 2025

[2] MoE-GS: Mixture of Experts for Dynamic Gaussian Splatting, ICLR 2026

[3] MAPo : Motion-Aware Partitioning of Deformable 3D Gaussian Splatting for High-Fidelity Dynamic Scene Reconstruction, CVPR 2026

[4] SPIN-4DGS: Implicit 4D Gaussian Splatting for Fast Motion with Large Inter-Frame Displacements, ICLR 2026

Questions for authors
  1. What is the specific advantage of state-dependent rollout distillation over TimeFormer's training-time temporal modeling under matched conditions?
  2. Why are recent methods such as Mango-GS, MAPo, and SPIN-4DGS excluded, and are the reported baseline results evaluated under consistent protocols?
  3. How does the method compare with a Direct-only model trained on all frames under matched computational budgets, and what benefit does DCT provide over alternative trajectory bases?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.