- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes PhysMotion4DGS for dynamic scene reconstruction and novel-view synthesis. It constrains Gaussian motion by matching projected 3D velocities to observed optical flow and matching 3D velocities to estimates recovered using flow and depth. Temporal regularization encourages smooth motion, while progressive coarse-to-fine training aims to improve reconstruction and efficiency. Experiments on Plenoptic Video and D-NeRF report 34.00 and 37.00 dB PSNR, with rendering speeds of 45 and 120 FPS, respectively. Component ablations are provided on D-NeRF.
Critical strengths
- Combining image-space motion matching, depth-assisted 3D constraints, and temporal regularization is a reasonable way to reduce motion ambiguity and improve stability. This gives the method a clear practical motivation.
- Table 3 shows that each regularizer improves D-NeRF reconstruction over the stated base model. These results support further investigation of the combined supervision, although the comparative evidence requires clarification.
Critical weaknesses
-
The comparative results do not yet establish the claimed performance advantage. Several Plenoptic Video baseline values are substantially below published results: Table 1 reports 28.42 dB for Wu's 4D-GS, 29.95 for RealTime4DGS, and 30.43 for STG, whereas the corresponding papers report 31.15, 32.01, and 32.05 dB (Wu et al., Table 3, Yang et al., Table 1, Li et al., Table 1). Different protocols could explain some discrepancies, but their origin is not documented. There are also internal contradictions: Deformable3DGS exceeds the proposed method in D-NeRF PSNR (39.31 versus 37.00), and RealTime4DGS exceeds its Plenoptic rendering speed (72.80 versus 45.00 FPS), despite broader superiority claims. Moreover, ablations are limited to D-NeRF, leaving the large Plenoptic gain unexplained. These issues directly affect the paper's principal evidence for significance.
-
The additional contribution over related motion-supervision methods is insufficiently established. GaussianFlow (2024) already connects Gaussian dynamics to image-space optical flow and evaluates this supervision on Plenoptic Video, but is neither discussed nor compared. Later relevant baselines, such as E-D3DGS and Grid4D, are also absent. The paper needs to distinguish the contribution of its depth-assisted velocity constraint and training strategy from existing flow supervision.
Questions for authors
-
Evaluation clarification: Where do the Table 1 baseline results come from, and why do they differ substantially from the published values above? Please specify the scene list, frame range, held-out camera, resolution, training configuration, and metric implementation, and reconcile the superiority claims with Tables 1 and 2.
-
Contribution clarification: What is the substantive methodological advance over GaussianFlow? A comparison under matched settings, including a flow-supervised version of the same backbone, would materially affect my assessment of novelty and significance.
-
Assessment-relevant evidence: Can the 34.00 dB Plenoptic result be supported by per-scene results and same-backbone ablations of the 2D constraint, depth-assisted 3D constraint, temporal regularizer, and progressive training? Is there quantitative temporal or motion evidence demonstrating the claimed improvement beyond per-frame reconstruction quality?
2 comments
I understand the explanation for the lower PSNR values, but my main concerns remain the ablations and compared methods.
Ablations on Plenoptic Video are necessary: D-NeRF is simple, with methods such as Grid4D already exceeding 40+ psnr on most scenes, while 34 psnr on Plenoptic Video is still a strong result. This is worth exploring through ablation studies.
My concern about the baselines is not whether PhysMotion4DGS is equivalent to GaussianFlow or lacks novelty, but that the paper overlooks almost all the works introduced after late 2023. I also could not understand that the absence of LPIPS in Grid4D is a reason to exclude it from comparison. I have provided links to the mentioned papers, although E-D3DGS does not need to be included in the comparison.
PhysMotion4DGS: Methodological Innovation and Comprehensive Experimental Validation
1. Rigorous and Comprehensive Experiments
All our experimental results are reliable and valid, as supported by the findings reported in 4D Rotor-GS. As shown in Table 2, PhysMotion4DGS achieves substantially higher computational efficiency than Deformable3DGS. Therefore, when evaluated under an equivalent computational budget, our approach outperforms all Gaussian-based baselines in terms of PSNR. The ablation experiments on the D-NeRF dataset sufficiently demonstrate the effectiveness of each component of PhysMotion4DGS. Furthermore, Figure 3 provides qualitative visualizations of different ablation configurations of PhysMotion4DGS on the Plenoptic Video dataset.
Duan, Yuanxing, et al. "4d-rotor gaussian splatting: towards efficient novel view synthesis for dynamic scenes." ACM SIGGRAPH 2024 conference papers. 2024.
2. Novelty and Technical Contributions of PhysMotion4DGS
We respectfully disagree that the shared use of optical flow makes PhysMotion4DGS equivalent to GaussianFlow or establishes a lack of novelty. The relevant distinction is the motion formulation and the regularizations imposed on it, rather than whether optical flow is used.
(1) Different motion formulations and supervision targets. GaussianFlow constructs a dense image-space flow from the evolution of projected Gaussian distributions. PhysMotion4DGS instead introduces explicit 3D velocities of Gaussian primitives and constrains them through both image-space projection and depth-supported 3D motion supervision. The distinction is therefore not merely “velocity versus displacement,” but the novel motion representation for four-dimensional space.
(2) Depth-supported 3D velocity supervision is not equivalent to matching a 2D flow field. Our PA term aligns the projected velocity with optical flow. Our PM term additionally constructs a 3D motion target by back-projecting the flow endpoint using the next-frame depth and penalizes its discrepancy from the predicted 3D velocity.
These objectives constrain different quantities. The perspective projection Jacobian maps a 3D velocity to two image-space components and has an unobservable direction along the viewing ray: in camera coordinates. Consequently, image-space flow matching alone cannot distinguish certain different 3D velocities. The depth-supported PM term can constrain this otherwise unobserved component.
GaussianFlow’s covariance-aware 2D transport does not, by itself, provide this explicit depth-lifted 3D velocity target. Thus, PM is not a renaming of its Gaussian-flow loss.
(3) Explicit temporal velocity regularization provides another distinct constraint. PhysMotion4DGS also directly penalizes changes in predicted velocities across temporal intervals, together with a term involving changes in flow-derived velocities. This introduces an explicit temporal prior on velocity evolution. GaussianFlow’s reported formulation supervises Gaussian-induced image-space displacement between frame pairs.
Our contribution lies in the formulation combining explicit 3D velocity modeling, projected 2D motion consistency, depth-lifted 3D velocity consistency, and temporal velocity regularization, integrated into a progressive optimization framework.
Regarding the two additional papers mentioned by the reviewer, we could not locate an accessible record of E-D3DGS, while Grid4D does not report the corresponding evaluation metrics on the Plenoptic Video dataset, especially LPIPS.