SGGF: Simulation-Guided Gaussian Field for Relightable Volumetric Effects

Recent advances in agent-assisted workflows are making it increasingly practical to create complex 3D content with substantially less manual scene construction. When coupled with mature physics-based simulators, they provide a promising workflow for creating rich dynamic volumetric effects. Yet the resulting simulated volumes remain expensive to render, especially when repeatedly viewed from different cameras or under changing illumination. We present SGGF, a simulation-guided Gaussian field for efficient rendering and relighting of dynamic volumetric scenes. Trained per sequence from density fields and offline reference images, a shared spatiotemporal field generates Gaussian attributes without explicit cross-frame primitive correspondences. We use simulated density to guide opacity learning and visibility computed from the Gaussians to guide shading under changing illumination. To improve training efficiency, we begin with sparse samples and progressively increase their density, adjusting Gaussian sizes and opacity along the way. Experiments on public and agent-authored volumetric effect scenes evaluate rendering quality and the quality–efficiency tradeoff across sampling budgets. Under a GPU-resident configuration, SGGF renders 128 consecutive frames at 22.40 fps, evaluating the shared field and performing shading and rasterization at each frame.