- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes StreamDynamic, a plug-and-play, training-free streaming 3D reconstruction framework designed to address the "historical ghosting" issue in dynamic scenes. Existing methods typically merge predicted point clouds from all frames directly, causing moving objects to leave "dynamic ghosting" at their past locations and previously static objects to leave "static ghosting" at their original locations once moved. To solve these issues, StreamDynamic proposes a prior-anchored epipolar-geometry-based motion segmentation method to remove dynamic ghosting online. It also designs an online backtracking removal method based on sparse-view ghosting templates to eliminate static ghosting at a computational cost independent of the historical sequence length. Furthermore, the authors introduce a novel ghosting-aware evaluation protocol. Experiments demonstrate that this method significantly improves the reconstruction quality of three existing streaming backbones (TTT3R, STream3R, LingBot-Map).
Critical strengths
- The framework does not require retraining the backbone networks on specific dynamic datasets. It can be directly integrated into existing streaming reconstruction models, demonstrating high generalization and practical engineering value.
- To handle objects that move after remaining static for a long time, the authors propose a "Geometry-Verified Sparse View Selection" algorithm. By constructing a 3D ghosting template from only a few sparse views, it avoids the complexity of frame-by-frame autoregressive tracking, ensuring streaming efficiency.
- The paper provides a clear and insightful distinction between "dynamic ghosting" and "static ghosting" in feed-forward streaming reconstruction, addressing a core pain point for downstream online applications like robotic navigation and autonomous driving.
Critical weaknesses
- Strong Reliance on Static Backgrounds: The method uses epipolar geometry to estimate motion masks, which requires sufficient static correspondences to reliably estimate the fundamental matrix. In scenes with dynamic backgrounds, such as swaying trees, moving crowds, or rippling water, RANSAC may fail to identify reliable correspondences, compromising motion segmentation.
- Sensitivity of Mask Completion to Visual Similarity: The method relies on SAM to complete motion masks. When foreground objects share similar colors or textures with the background, SAM may over-segment the object, incorrectly removing static background points from the reconstruction.
- Error Propagation from Pretrained Models: The pipeline depends on off-the-shelf models such as RAFT and SAM. Errors in optical flow, particularly in occluded or textureless regions, and inaccurate mask boundaries can propagate to the 3D reconstruction, resulting in artifacts or missing geometry.
- Latency Spikes During Online Backtracking: Although sparse-view templates improve backtracking efficiency, removing static ghosting when objects suddenly move can still introduce temporary latency spikes, limiting the method's suitability for strict real-time streaming.
Questions for authors
Please refer to the Weaknesses section.
Response to Reviewer vChf
Thanks for your time and comments. We will revise the paper to address the concerns raised. Below, we provide clarifications for the major points.