ControlGS: Factorized Control for Continual 3D Gaussian Splatting

Continual 3D Gaussian splatting reconstruction updates a model from new images while preserving unchanged content and earlier states.Existing methods apply a uniform end-to-end procedure across different types of scene evolution, without adequately accounting for their distinct effects on scene content and Gaussian model capacity.Such a rigid paradigm is prone to modifying irrelevant attributes, causing unexpected structural changes, and gradually undermining historical recovery as the Gaussian representation evolves.In this paper, we propose ControlGS, which determines which attributes of each Gaussian may change and by how much, where Gaussian capacity may be adjusted, and how each update is recorded and reversed for historical replay.In the attribute update control stage, image differences are combined with visibility, depth, and camera registration to estimate the type and reliability of each change. The estimated change type and uncertainty determine which geometric or appearance attributes may be optimized and by how much, while unrelated attributes are protected from optimizer-state drift.In the structure update control stage, the same evidence localizes the regions affected by scene changes. ControlGS then balances local rendering evidence against model complexity to decide whether Gaussian capacity in each region should change, preventing unreliable differences from triggering structural updates.In the historical replay control stage, every Gaussian receives a stable UID independent of its tensor-row position. Before each update, ControlGS stores the affected pre-update states by UID and records Gaussian births, deaths, and parent-child relations. Full-model updates additionally trigger a complete pre-update snapshot. To recover a target version, ControlGS starts from the current state or the nearest later snapshot and reverses the subsequent records, restoring the corresponding parameters and topology despite intervening structural changes and row reordering.Extensive experiments demonstrate that ControlGS outperforms state-of-the-art methods in reconstruction quality while maintaining real-time rendering. Additional tests verify that irrelevant parameters remain fixed and earlier states are reliably recovered.