StreamDynamic: Streaming Dynamic Scene Reconstruction without Historical Ghosting

Streaming 3D reconstruction incrementally predicts the point map of each newly arrived frame from an image stream, and is thus well suited to online applications. However, existing methods suffer from historical ghosting when reconstructing dynamic scenes, as they directly merge the predicted point maps of all frames into an accumulated point cloud. Specifically, the points reconstructed for a moving object at past time instants become dynamic ghosting at the current time instant. Moreover, when a previously static object moves away, its points reconstructed during the static period become static ghosting at its original location. In this work, we take the first step toward feed-forward streaming dynamic scene reconstruction without historical ghosting, and propose StreamDynamic, a novel training-free, plug-and-play framework that identifies moving objects online and removes both their dynamic and static ghosting from the reconstructions of existing streaming backbones. To remove dynamic ghosting, we propose a prior-anchored epipolar-geometry-based motion segmentation method, which estimates the epipolar geometry from the static regions indicated by a coarse motion prior extracted from the backbone predictions. To remove static ghosting accumulated over long histories, we design an online backtracking removal method, which constructs ghosting templates from only a few sparse views and efficiently erases the ghosting within the 3D neighborhoods of the templates. We further introduce a novel ghosting-aware evaluation protocol with ghosting-free ground truth for every time instant and metrics that penalize historical ghosting. Extensive experiments on three dynamic scene datasets show that StreamDynamic substantially reduces the historical ghosting of three representative streaming backbones, consistently improves their reconstruction quality, and outperforms existing methods.