- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PerM4D, a 4D Gaussian Splatting (4DGS) method for progressively reconstructing dynamic scenes from a monocular video stream. The method first initializes frame-wise static and dynamic Gaussians using depth estimates from DA3, optical flow, and motion segmentation. Static Gaussians are accumulated through geometry-aware fusion, while dynamic Gaussians are updated using two types of memory mechanisms. The short-term memory leverages 3D tracking to predict Gaussian displacements and establish motion correspondences. The long-term memory maintains the geometry of a persistent surface representation to verify associations regardless of spatial proximity. The matching results from these memory mechanisms determine whether new observations should be fused with existing Gaussians, initialized as new surfaces, or used to reactivate previously occluded surfaces. Finally, the updated Gaussian representation undergoes a short per-frame optimization. Experimental results demonstrate that the proposed method achieves competitive novel-view synthesis quality with relatively low per-frame processing latency.
Critical strengths
-
The combination of short-term motion memory and long-term representation memory enables the method to handle both occlusions and temporal correspondences effectively. The overall design is conceptually reasonable and well-motivated.
-
The paper demonstrates superior reconstruction quality compared to the existing online methods evaluated in the experiments, while maintaining relatively efficient inference.
Critical weaknesses
-
Important recent streaming/online reconstruction methods, particularly ProDyG (NeurIPS 2025) and 4D-Fly (CVPR 2025), are excluded from the quantitative comparisons. Furthermore, the reported quantitative results for MoVieS differ from those in the original paper (PSNR: 18.46 in the original paper). While this discrepancy may be due to differences in evaluation settings, more comprehensive comparisons with recent methods are needed.
-
Since the experiments use relatively short sequences and ground-truth camera poses, the effectiveness of persistent memory in long-term streaming scenarios remains unclear.
-
The overall framework of the proposed method (e.g., static-dynamic separation and the use of pretrained foundation models) appears to be influenced by several prior works. The main strength of this method lies in the cooperation between the two memory mechanisms and the robustness of the Gaussian representation. However, the paper lacks sufficient analysis of how effectively these methodological contributions work in practice. In particular, the benefits of the proposed method are demonstrated primarily through novel-view synthesis (NVS) performance, without more direct evaluations of its core contributions.
Questions for authors
-
Representation: Does the proposed method support the explicit separation of static and dynamic components in its reconstructed representation?
-
Effectiveness of Long-Term Memory: One of the main objectives of this paper is to preserve surface identities and accurately recover previously observed regions after occlusion. However, the experiments focus exclusively on NVS evaluation. Could the authors provide a more detailed analysis of the effectiveness of long-term memory, such as surface re-identification accuracy under varying occlusion durations, temporal consistency of surface identities, and the impact of these factors on reconstruction performance?
-
Ambiguity in Surface Association: The long-term surface matching cost (Eq. 5) relies on orientation, scale, and color while excluding spatial position. How does this approach distinguish physically distinct surfaces that exhibit similar geometric or visual properties? Could the authors analyze the sensitivity of the proposed association mechanism to false associations, non-rigid deformations, and significant surface rotations?
-
Long-Sequence Training: The experiments use relatively short input videos (12–13 frames). How do the number of stored Gaussians, dormant surface records, GPU memory consumption, and per-frame processing time scale as the video length increases? Could the authors provide evaluations on substantially longer sequences?
-
Fast Motion: Newly observed dynamic objects may require multiple observations before they can be initialized. How well does the proposed method handle the reconstruction of objects undergoing fast motion?