- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SD4R, a framework for dynamic 4D Gaussian scene reconstruction and editing based on monocular video without pose information. The method first estimates the camera pose during a camera warm-up phase, then initializes the dynamic Gaussian using the estimated depth and long-range 2D trajectories. Subsequently, it models continuous non-rigid motion using a sparse control field and provides continuous supervision through fixed trajectory identities during the joint optimization process. Furthermore, semantic masks are associated with Gaussian primitives using the same correspondence, enabling object-level 4D editing. The paper achieves competitive new-view synthesis results on seven sequences from the NVIDIA Dynamic Scenes dataset and provides supplementary validation through qualitative reconstruction results and editing examples on the DAVIS dataset. Ablation experiments analyze the roles of the motion modeling and persistent correspondence modules.
Critical strengths
This paper investigates the problem of dynamic scene reconstruction based on monocular video without pose information. This task requires the joint estimation of camera motion and non-rigid deformation, making it highly challenging. The core design proposed in this paper is the decoupling of motion optimization from persistent correspondences, driven by a clear and intuitive research motivation. Maintaining fixed trajectory associations throughout the optimization process provides a promising approach for enhancing temporal supervision, while further integrating the scene reconstruction results with object-level four-dimensional editing.
The method achieved competitive reconstruction results on the NVIDIA Dynamic Scenes benchmark dataset. Across seven test sequences, the average PSNR reached 27.49 dB and the LPIPS reached 0.050, both representing the best average performance among the compared methods in Table 1. Furthermore, ablation experiments preliminarily demonstrated that both the sparse motion control mechanism and persistent correspondence supervision can effectively improve overall reconstruction quality.
Critical weaknesses
Several technical modules of SD4R overlap with existing methods. SC-GS and MoSca have already explored sparse motion control, while MOSAIC-GS also utilizes depth, optical flow, segmentation information, and point trajectories for dynamic scene reconstruction. Although the persistent correspondence mechanism may have some innovative value, the technical distinctions between it and existing trajectory-supervised methods and persistent dynamic representations remain unclear. The authors need to further clarify the innovativeness of this mechanism and its independent technical contributions.
Table 2 primarily evaluates correspondence supervision using rendering metrics such as PSNR, SSIM, and LPIPS; it cannot directly demonstrate a reduction in correspondence drift or an improvement in tracking accuracy. We recommend validating the core mechanism through controlled comparisons with iterative rematching schemes and direct trajectory consistency metrics. Furthermore, Figure 4 shows only a few examples of independent edits and lacks sufficient evidence of temporal and cross-view consistency; relevant experiments should be added to support the claimed advantages of 4D editing.
Questions for authors
Q1. Sparse‑motion‑control designs have been investigated in SC‑GS and MoSca, and Shape of Motion together with MOSAIC‑GS leverage trajectory cues for dynamic reconstruction. What are the core technical distinctions between these prior works and SD4R? An ablation study that disentangles the effect of persistent correspondence from sparse motion field and camera initialization is expected. Head‑to‑head comparisons against closely related approaches would further clarify the paper’s technical novelty.
Q2. Table 2 assesses correspondence supervision mostly via rendering‑quality metrics. Could the authors conduct direct comparisons between fixed track‑identity scheme and iterative correspondence reassignment, and report tracking error or correspondence consistency metrics? Merely counting reduced identity switches is not convincing given the fixed‑by‑design track identities. It also remains unclear how the framework copes with erroneous initial track assignments.
Q3. The editing samples in Figure 4 are isolated cases and cannot fully demonstrate temporal consistency. Results showing the same edited object across multiple frames and novel viewpoints should be supplemented. Ablations disabling the persistent‑trajectory‑to‑Gaussian association or Gaussian‑wise parameter gating are needed to validate their individual contributions. Quantitative measurements for temporal consistency and background preservation are also desirable.
Q4. Section 3.3 requires that trajectories remain visible throughout the entire video sequence in the main experiment, which may result in trajectories affected by occlusion being excluded. Under this setup, how many valid trajectories are actually retained? We recommend evaluating the impact of different visibility requirements, initial trajectory noise, and depth estimation errors on the method’s performance. Additionally, the authors should clarify how the camera’s intrinsic parameters are obtained and specify the loss function used during the camera warm-up phase.
Q5. Table 1 covers seven evaluation sequences, whereas ablation results in Table 2 are computed over only five selected scenes. Which exact scenes are chosen, and what motivates this selection? Per‑scene ablation numbers should be given; ideally, key ablations should run across all seven sequences. Since efficiency is highlighted in the paper, training runtime, total parameter count and peak GPU memory consumption ought to be reported, together with clear breakdowns of preprocessing overhead.