Dusting the Phantom: Gaussian Allocation For Fading Dynamic Content

Dynamic Gaussian representations have made high-quality, real-time volumetric video increasingly practical, yet challenging motion can still break their reconstruction: a small fast-moving object may partially disappear, and fine textures in highly dynamic regions are often lost. Surprisingly, simply training longer or using more Gaussians does not reliably resolve these failures. We argue that the key bottleneck is not representational capacity itself, but *where and when that capacity is allocated*. We introduce two complementary strategies for improved spatiotemporal Gaussian allocation. First, *seed injection* introduces new Gaussians directly from multi-view observations, using temporal appearance changes and reconstruction errors to identify under-reconstructed dynamic regions and place new support at geometrically plausible 3D locations. This allows the model to recover content that may have little or no existing Gaussian support. Second, *temporally biased relocation* preserves time-resolved optimization signals and uses them to guide *when and where* relocated Gaussian capacity should be reassigned. Both strategies can be incorporated into existing dynamic Gaussian frameworks without modifying their underlying scene representation. Experiments across multiple dynamic-scene datasets show consistent improvements in reconstruction quality, with particularly pronounced gains, including better recovery of small, rapidly moving objects and fine-grained appearance details and textures.