PersistSplat: Streaming Feed-Forward Reconstruction with Persistent Gaussian Memory

Recent feed-forward 3D Gaussian Splatting (3DGS) strive to overcome the limitation of per-pixel Gaussian allocation to build more compact scene representations. However, most follow an offline paradigm that requires all views to be acquired in advance and jointly processed, thus unsuitable for streaming inputs. Existing online compression methods predict dense Gaussians for each frame and subsequently compress redundancy, which severely limits reconstruction quality and efficiency. We propose PersistSplat, a streaming feed-forward 3DGS architecture for constructing compact scene representations. Its core idea is to leverage incoming views continuously refine existing Gaussians, while restricting Gaussian generation to previously unobserved scene content; this allows Gaussian growth to be adaptively determined by scene scale rather than increasing with the view count. Specifically, PersistSplat maintains a Persistent Gaussian Memory for each Gaussian, with a fixed anchor and an updating historical observation set: for regions in the incoming view already covered by existing Gaussians, the model writes the current observations into the corresponding historical observation sets and uses the associated fixed anchors as queries to adaptively retrieve and aggregate historical context to refine existing Gaussians, avoiding redundant generation; for previously unobserved regions, it generates new Gaussians and initializes their anchors and historical observation sets, enabling incremental scene modeling. To ensure stable memory evolution, we further introduce Global Geometry Alignment and Global Feature Alignment to correct geometric drift and feature deviations. Experiments show that PersistSplat effectively reduces redundant Gaussians, allowing it to gradually converge to a scene-scale-dependent upper bound; while matching or outperforming existing offline and online baselines.