acceptodds
Under review as a conference paper at ICLR 2027

VFOM-GS: Volumetric Fourier Opacity Map for Relightable Gaussian Smoke Reconstruction

Abstract

Reconstructing dynamic smoke from rendered multi-view videos requires recovering fine-scale geometry while accounting for self-shadowing. Although volumetric Gaussian reconstruction has advanced view synthesis, approaches focused on camera-side transmittance neglect light-side attenuation, baking shadows into primitive colors and limiting relighting. We present VFOM-GS, a shadow-aware 4D Gaussian reconstruction framework that explicitly couples self-shadow formation to the reconstructed density field. Our Volumetric Fourier Opacity Map (VFOM) encodes analytic Gaussian ray profiles in a compact Fourier basis to estimate light-side transmittance. Additionally, the proposed Per-Pixel Depth Normalization (PDN) fits the encoding interval to each light ray's local depth range, improving effective depth resolution. To enable gradient-based optimization of Gaussian parameters, we make both the light and camera passes of our VFOM-based rendering pipeline differentiable. This pipeline models attenuation separately from color while retaining efficient Gaussian rasterization. Experiments on synthetic smoke show that VFOM-GS outperforms the evaluated state-of-the-art baselines in novel-view reconstruction quality and produces consistent self-shadowing under changing illumination. Our Unreal Engine experiments further show that our renderer is nearly twice as fast as the established Niagara system.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Gaussian Splatting, 4D reconstruction, volumetric rendering, relighting, self‑shadow, differentiable rendering, Fourier opacity map, view synthesis

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_feSoOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

VFOM-GS reconstructs dynamic smoke with a volumetric Gaussian density field and explicitly models light-side self-shadowing. Building on Vol3DGS ray-integrated optical thickness, it deposits finite-width Gaussian profiles into a truncated Fourier basis and uses per-pixel depth normalization (PDN) to adapt the encoded interval.

Critical strengths
  1. The Fourier coefficients explicitly incorporate the optical mass and ray-dependent width of each Gaussian. This gives a motivated alternative to treating each occluder as an infinitely thin event, and connects self-shadowing to the reconstructed density instead of relying entirely on baked Gaussian colors.

  2. The system demonstrates shadow-aware reconstruction, changing illumination, smoke/environment shadow interactions and a functioning Unreal Engine implementation. Table 3 reports lower tested render time and storage than the Niagara voxel configuration, showing a concrete potential application.

Critical weaknesses
  1. The broad reconstruction comparison in Table 1 evaluates images under the training light. The quantitative novel-light evaluation in Table 2 primarily contrasts two training stages of the same VFOM renderer rather than competitive relightable Gaussian baselines. Shadow-aware refinement improves PSNR but worsens LPIPS in both shown lighting conditions. GauSmoke is discussed in the appendix but omitted from the main quantitative comparisons.

  2. The first stage explicitly fits shadow-free target images, followed by a shadow-aware stage with frozen colors. This makes appearance/attenuation separation easier than in an uncontrolled real capture. The finite K=7 Fourier approximation and PDN are motivated by analytic curves and a procedural sandglass, but their errors under complex, highly overlapping smoke densities are not quantified against accurate optical-depth integration.

Questions for authors

Q1. Can the authors compare against a volumetric Gaussian smoke renderer with an alternative shadow calculation (and, where feasible, GauSmoke) using matched geometry, illumination conditions and available supervision, with quantitative novel-light metrics?

Q2: Can legacy FOM versus finite-width VFOM, each with and without PDN, be compared on the complete smoke reconstruction task rather than primarily isolated numerical or procedural examples?

Q3: How does the truncated Fourier transmittance compare to high-accuracy integration of the same Gaussian extinction field across optical depths, overlap counts, and Fourier orders? Could the authors discuss how camera-side compositing approximations interact with light-side attenuation during optimization, in relation to analytical volumetric Gaussian rendering work such as 3DGEER and EVER?

[1] 3dgeer: 3d gaussian rendering made exact and efficient for generic cameras.

[2] Ever: Exact volumetric ellipsoid rendering for real-time view synthesis.

1 comment

We sincerely appreciate your time and feedback. Below, we clarify the points raised and describe the added evidence in the revised paper and supplementary material.

W1 — Evaluation scope and metric trade-offs: We agree that refinement improves PSNR but worsens LPIPS in both cases; SSIM slightly improves on the back-lit side but decreases on the light-facing side. Refining the shared extinction field may improve overall shadow intensity while altering local contrast and detail in camera-side rendering. This can reduce pixel-wise error without improving perceptual similarity, so we report a metric trade-off rather than gains across all measures.

Q1–Q2: Supplementary S2 (Table R1) adds a full smoke reconstruction comparison relevant to both questions, using no shadow modeling, VFOM without PDN, FOM+PDN, and VFOM+PDN on Explosion. Without shadow modeling, baked-in dark RGB remains dark when regions face the new light (red dashed boxes). Without PDN, inaccurate shadows hinder recovery of extinction coefficients. With FOM, thin-event encoding increases sharp noise (yellow solid boxes). With VFOM, finite-width encoding preserves Gaussian thickness and a fuller volumetric appearance. The full model gives the best reported PSNR, SSIM, and LPIPS.

Our main goal is compact, low-cost real-time rendering of volumetric assets; ScalarFlow provides a supplementary robustness test rather than our primary reconstruction benchmark. We match GauSmoke's 120 frames, five cameras, and 270×480270\times480 resolution, comparing against its published results because its implementation is unavailable (Table R2). With light directions assumed to align with camera views, we jointly optimize colors and the Gaussian field through VFOM using captured images, without paired shadow-free targets, additional lighting estimation, or real-data-specific model components. Despite lower PSNR than LagrangianSplats, ours has higher SSIM, lower LPIPS, and smaller storage, supporting robustness under this approximation.

Table R1

ConfigurationPSNR ↑SSIM ↑LPIPS ↓
w/o VFOM, PDN23.090.9460.0674
w/o PDN23.270.9400.0716
with FOM, PDN22.730.9510.0772
Full Model27.410.9650.0595

Table R2

MethodPSNR ↑SSIM ↑LPIPS ↓Storage (MB)Train Time (h)GPUFPS ↑
GauSmoke [2]34.830.9550.0168420.8RTX3090114
LagrangianSplats [1]35.970.95540.0954865.6TeslaV10033
VFOM-GS (ours)34.240.98380.004024.45.1TeslaV100645

Q3: Supplementary S4 (Figure S3) uses overlapping Gaussians with different optical thicknesses. Keeping each Gaussian field fixed, we compare FOM and VFOM transmittance curves and their errors against exact integration at K=3,7,11K=3,7,11, testing both the shadow representation and Fourier order. VFOM already has a small discrepancy at K=7K=7 in the tested cases, supporting our default choice; the reconstruction ablation in S2 uses K=3K=3.

Separately, reconstruction PSNRs are close: 27.41 dB at K=3K=3 in S2 and 28.20 dB for the K=7K=7 light-facing result in Experiment 2 (Table 2), a modest 0.79 dB gap.

3DGEER [1] targets exact projection geometry but retains compositing approximations; EVER [2] exactly integrates emission–absorption for constant-density ellipsoids. Our camera pass uses analytic Gaussian line integrals, and VFOM analytically derives Fourier coefficients from Gaussian density profiles. Approximations remain in finite-order shadow reconstruction, depth-ordered color compositing, and lighting sampled at Gaussian centers.

Both passes share Gaussian parameters, so their errors can affect reconstruction. For example, a dark pixel can be fitted by darkening RGB or increasing light-path extinction. On Explosion, joint optimization does not reliably separate shadows from color. We therefore first fit geometry and colors from shadow-free images, then remove dark Gaussians, freeze colors, and refine the extinction field through VFOM using shadowed images. More accurate attenuation and compositing may further improve reconstruction and are valuable directions for future work.

Please see the supplementary material (pages 20–21) and appendix for further details, figures, and quantitative results.

References

[1] Z. Huang et al. 3DGEER: 3D Gaussian Rendering Made Exact and Efficient for Generic Cameras. ICLR, 2026. https://arxiv.org/abs/2505.24053.

[2] A. Mai et al. EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis. arXiv:2410.01804, revised 2025. https://arxiv.org/abs/2410.01804.

Reviewer_3RAqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper fits dynamic volumetric Gaussians to multi-view images rendered from existing synthetic smoke sequences. Training first uses shadow-free images, then refines the model against shadowed images while keeping Gaussian colors fixed. The renderer uses VFOM and per-pixel depth normalization to estimate light attenuation through the fitted density. Experiments compare novel-view image quality and examine self-shadowing under changes in directional lighting. Additional examples show shadow interactions with surrounding objects. A preliminary Unreal Engine test reports lower storage and rendering time than the tested Niagara voxel configuration.

Critical strengths

VFOM-GS achieves the best reported novel-view PSNR, SSIM, and LPIPS on ScalarPurple and Explosion under the training illumination (Table 1). These results support the ability of the complete system to preserve the source smoke's appearance at novel viewpoints.

Critical weaknesses
  1. Input requirements and optimization costs need clarification. The method uses shadow-free and shadowed training images, but does not explain how the shadow-free targets are obtained (§4 and Appendix D.5). This leaves the required access to source data unclear. Optimization time is also unreported, which limits assessment of the trade-off between fitting cost and runtime savings.

  2. The relighting evaluation covers limited lighting conditions. Figures 5 and 8 mainly vary the direction of a directional light. Results for the same reconstructed smoke under colored lights, environment illumination, and multiple light sources would clarify the range of supported lighting conditions.

Questions for authors
  1. How are the shadow-free training images obtained? Could the authors demonstrate the reconstruction pipeline on real captured smoke?

  2. What optimization cost is required to obtain the reported storage and rendering savings? Please specify the time for each stage and the hardware used.

  3. Can the same fitted smoke model be used under colored lights, environment illumination, and multiple light sources?

3 comments

We sincerely appreciate your time and feedback. Below, we clarify the points raised and describe the added evidence in the revised paper and supplementary material.

Q1: We generate Explosion's paired images by turning smoke shadows off/on in UE Niagara. Our goal is to replace costly volumetric assets with compact 4D Gaussians for high-quality real-time rendering, rather than a dedicated real-world capture and reconstruction system. ScalarFlow in supplementary S1 tests robustness beyond synthetic inputs. We set each image's light direction to its camera's viewing direction, approximating the softbox lighting [3], and jointly optimize colors and the Gaussian field through VFOM using captured images. No paired shadow-free targets, additional lighting estimation, or real-data-specific model components are needed. We match GauSmoke's [2] 120 frames, five cameras, and 270×480270\times480 resolution, comparing against its published results because its implementation is unavailable (Table R1). The competitive results support robustness under this lighting approximation.

Table R1

MethodPSNR ↑SSIM ↑LPIPS ↓Storage (MB)Train Time (h)GPUFPS ↑
GauSmoke [2]34.830.9550.0168420.8RTX3090114
LagrangianSplats [1]35.970.95540.0954865.6TeslaV10033
VFOM-GS (ours)34.240.98380.004024.45.1TeslaV100645

Q2: Our ScalarFlow reconstruction takes 5 h 05 min on Tesla V100 and occupies 24.4 MB, versus 5 h 40 min and 86 MB for LagrangianSplats [1]. GauSmoke [2] reports 0.8 h on RTX3090, so the timings are not directly comparable. Reconstruction is performed offline, but the resulting asset is rendered in real time in UE5.8. For Explosion, both renderers were measured in UE5.8 on an NVIDIA GeForce RTX 5060 Ti: ours takes 1.35 ms/frame versus Niagara's 3.0 ms/frame, about 2.2 times faster in the tested configuration.

Q3: Yes. Our fitted model can be rendered under colored directional light without refitting, as shown in supplementary S3. We implement VFOM shadow rendering in UE5.8 and multiply the light's RGB weights with the final shadowed Gaussian RGB output, channel by channel. While our current evaluation focuses on directional lighting, we agree that extending VFOM-GS to multiple lights and environment illumination is an interesting and valuable direction for future research.

Please see the supplementary material (pages 20–21) and appendix for further details, figures, and quantitative results.

References

[1] N. Tao, B. Chen, and M. Chu. LagrangianSplats: Divergence-Free Transport of Gaussian Primitives for Fluid Reconstruction. ACM SIGGRAPH Conference Papers, 2026.

[2] W. Zhang, Y. Cai, L. Huang, D. Ye, J. Guo, and B. Ren. GauSmoke: Hybrid Physics-Optical Gaussian Splatting for Sparse Smoke Reconstruction. ACM SIGGRAPH Conference Papers, 2026. https://doi.org/10.1145/3799902.3811148.

[3] M.-L. Eckert, K. Um, and N. Thuerey. ScalarFlow: A Large-Scale Volumetric Data Set of Real-world Scalar Transport Flows for Computer Animation and Machine Learning. ACM Transactions on Graphics, 38(6), Article 239, 2019. https://doi.org/10.1145/3355089.3356545. Capture setup: https://ge.in.tum.de/?p=1536/.

Thank you to the author for the detailed response and supplementary experiments. Although some of my questions have been addressed, the core concerns I raised in my initial review remain, some of which were also mentioned by all other reviewers (Reviewer eP9i, Reviewer feSo, and Reviewer ftow):

1. Dependence on source data and insufficient cost-effectiveness of the fitting process (W1/Q1/Q2 Reviewer eP9i Q1/Q7/Q8, Reviewer feSo W2, Reviewer ftow W3; )

The authors have set their goal as replacing engine voxel assets rather than dynamically reconstructing real-world smoke. However, given that the underlying 3D voxel source data already exists, the authors first rendered shadow-free target images and multi-view images with shadows in UE by toggling shadow rendering, then spent approximately 5 hours cost on inverse fitting to achieve a speedup of 1.6 ms/frame. This trade-off makes the overall motivation seem shaky, as Reviewer eP9i points out.

2. Limited Relighting Scope (W2/Q3)

The newly added colored lights perform channel-level linear scaling on the output RGB. However, they are essentially still limited to a single parallel light source and do not support multiple light sources or environmental illumination, which limits their practicality as relighting engine assets.

Therefore, I am inclined to maintain the original score.

Thank you for taking the time to revisit our work and share your remaining concerns. Your thoughtful feedback has helped us clarify the intended use case and strengthen our relighting demonstrations.

1. Clarifying the experimental settings for dependence on source data. Our original goal was to convert voxel smoke assets into compact representations for real-time rendering, so the reconstruction and relighting experiments use different training settings. We distinguish two training settings: the reconstruction setting (Sections 4.1, 4.3, 4.5 and 4.6), and the gaming setting (Sections 4.2 and 4.7). In Section 4.1, both ScalarPurple and Explosion use shadowed images without shadow-free supervision, with the same training inputs and evaluation targets as the baselines. Because fitting shadowed images can bake shadows into Gaussian colors, we remove dark-colored Gaussians, freeze the remaining colors, and optimize extinction coefficients. VFOM-GS achieves state-of-the-art novel-view reconstruction quality among the evaluated methods on both datasets without requiring shadow-free images. Our real ScalarFlow experiment also uses captured RGB images without voxel ground truth or paired shadow-free targets. Therefore, our reconstruction method does not require shadow-free images.

The gaming setting instead targets conversion of existing game assets for real-time rendering: we first fit the Gaussian representation to shadow-free images from the source asset, then refine it using shadowed images while keeping colors fixed. Section 4.2 compares the model before and after shadow-aware refinement to demonstrate its benefit for shadow quality. Our original training description detailed the gaming setting but omitted an explicit description of the reconstruction setting, misleadingly suggesting that shadow-free images were always required. We apologize for this omission and have clarified the distinction in the revised paper.

2. Cost-effectiveness of the fitting process. Our goal is visually rich smoke effects for games/AR/VR, prioritizing real-time rendering of reusable assets over one-time offline fitting cost. Although fitting takes approximately five hours, it is performed offline once per asset, whereas rendering cost is incurred repeatedly during deployment. For Explosion, rendering takes 1.35 versus 3.0 ms/frame for Niagara on the same RTX 5060 Ti, with 13.36 versus 85.84 MB of storage. These runtime and storage gains motivate the trade-off for repeatedly used assets, rather than a claim of lower total cost for every use case.

3. Multiple directional lights and GI. Our additional UE experiment applies light shading by heuristically sampling and accumulating direct and indirect illumination, using Lumen irradiance for GI. The visually plausible results with local colored lights and GI demonstrate the potential for richer lighting applications; please see supplementary S3 (pages 20–21). Similarly, environment lighting could be approximated by sampling an environment map and accumulating weighted point-light contributions. Meanwhile, multiple directional lights could be handled by computing one VFOM shadow map per light and summing their shadowed contributions. Under restrictions, we provide still images and will include a demonstration video when releasing the datasets and rendering results, subject to internal review and approval.

Please see the supplementary material (pages 20–21) and appendix for further details and figures. We sincerely appreciate your continued engagement and hope these clarifications and additional results help address your concerns. Subject to internal review and approval, we plan to release the source code, datasets, and JSON configuration files with the full numerical training settings.

Reviewer_eP9iOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the issue of relighting (temporal) scenes (specifically volumetric data like smoke) reconstructed with methods like 3D Gaussian Splatting. For this, they revisit a previously introduced rendering method for shadow estimation in volume rendering (FOM). The authors adapt the method for temporal Gaussian rendering and introduce improvements:

  • per-pixel depth normalization for the light map
  • differentiable FOM to use it in the reconstruction pipeline.

This allows them to change the direct illumination (shadow) after reconstruction during rendering. The authors evaluate their method on synthetic datasets and compare it to previous methods with baked lighting.

Critical strengths

The paper takes an existing method, makes it differentiable, and adapts it for 3D Gaussian image-based reconstruction and relighting. The authors identify key components necessary to adapt the method (e.g., using VolGS) and provide the mathematical derivation. They show that the method improves relighting both qualitatively and quantitatively.

Critical weaknesses

The paper follows an interesting idea, and the proposed adaptation of FOM seems sound, but it fails to properly evaluate and ablate the method and to provide the necessary details to reproduce or reimplement it.

Questions for authors

Here are open questions and things missing from the paper:

Motivation in the introduction is shaky.

In VFX, we do not care too much about rendering speed (more about accuracy) since rendering is usually done offline. (Is backing + rendering really faster than just rendering the smoke?)

Light setup under-specified

The paper gives no light-frustum extents, no light-map resolution, and no description of how the light camera is placed or fitted to the scene.

Missing scene parameterization in paper

The authors only mention in the appendix that they adapt the scene parameterization of STGS. This should be in the main paper (method).

Hardware is missing

What hardware (GPU) was used to measure those timings? What rendering resolution?

PDN Ablation numbers are missing

Qualitatively only.

Differentiating the light pass

The claim that shadow supervision improves results is untested.

Temporal Stability

Results show strong temporal flickering in novel views even with light-aware training. This is not discussed in the paper at all.

''Missing numbers for reconstruction training:** The paper does not give details on:

  • training or reconstruction time, for either stage
  • the number of iterations
  • the learning rates or optimizer settings
1 comment

We sincerely appreciate your time and feedback. Below, we clarify the points raised and describe the added evidence in the revised paper and supplementary material.

Q1: Our goal is to replace storage-intensive, costly-to-render volumetric assets with compact 4D Gaussians for high-quality real-time smoke rendering in games and AR/VR, rather than offline film rendering or a dedicated real-world capture system. We fit the representation offline, then import it into UE for low-cost rendering under changing views and lights. The Niagara comparison measures runtime rendering, not total asset-creation cost; the real ScalarFlow experiment is an additional robustness test.

Q2, Q4: Our light camera uses orthographic projection; training images and light maps in the two-stage setup both use 630×480630\times480 resolution. We use K=7K=7 by default and K=3K=3 in S2. UE5.8 runtime measurements for VFOM-GS and Niagara use an NVIDIA GeForce RTX 5060 Ti. Separately, ScalarFlow reconstruction uses a Tesla V100, with light directions aligned to camera views to approximate its softbox lighting [1]. We match GauSmoke's [5] 120 frames, five cameras, and 270×480270\times480 resolution, comparing against its published results because its implementation is unavailable (Table R2). The light frustum's width, height, and near/far depth bounds will be released in JSON configurations with the source code and datasets after internal review and approval.

Table R2

MethodPSNR ↑SSIM ↑LPIPS ↓Storage (MB)Train Time (h)GPUFPS ↑
GauSmoke [5]34.830.9550.0168420.8RTX3090114
LagrangianSplats [4]35.970.95540.0954865.6TeslaV10033
VFOM-GS (ours)34.240.98380.004024.45.1TeslaV100645

Q3: We adopt STGS's [2] trbf_center, trbf_scale, and velocity parameters to model each Gaussian's temporal center, temporal extent, and motion. For camera-side compositing, we follow Vol3DGS [3] and compute Gaussian optical thickness using analytic ray integrals. Combining these finite-width density profiles with FOM, VFOM derives analytic Fourier coefficients and integrates the truncated series to compute shadow transmittance. This provides an analytic expression for the finite-order shadow approximation.

Q5: Table R1 reports the PDN ablation at K=3K=3. Without PDN, inaccurate shadow estimates hinder recovery of extinction coefficients. Adding PDN improves all three metrics, with the visual comparison shown in supplementary Fig. (b).

Table R1

ConfigurationPSNR ↑SSIM ↑LPIPS ↓
w/o VFOM, PDN23.090.9460.0674
w/o PDN23.270.9400.0716
with FOM, PDN22.730.9510.0772
Full Model27.410.9650.0595

Q6: S2 directly compares reconstruction with and without explicit shadow modeling. The no-shadow baseline fits shadowed images by baking shadows into Gaussian RGB, without optimizing light-side attenuation. In Table R1, the full model achieves 27.41 dB PSNR versus 23.09 dB for this baseline, with higher SSIM and lower LPIPS. The red dashed boxes in the supplementary ablation figure show that baked-in dark regions remain dark when facing the new light, whereas VFOM allows shading to change with illumination. This supports shadow-aware reconstruction relative to baked-color fitting; the comparison changes shadow modeling as well as its gradients.

Q7: We observe shadow flickering in the dense, high-opacity GasWall smoke, particularly in turbulent regions after interaction with a solid obstacle. We attribute this partly to image-based fitting concentrating Gaussians near the visible outer layers, leaving the interior less well reconstructed. Interior errors can be hidden in camera-view rendering under the training illumination, yet affect light attenuation and produce flickering shadows under relighting. This becomes important for our goal of relightable smoke assets, beyond matching camera views. In our initial tests, increasing the Gaussian count reduced this flickering. Better interior reconstruction and temporal stability remain directions for further improvement.

Q8: In our two-stage setup, Stage 1 takes approximately 30 min and Stage 2 approximately 4 h 30 min. Both stages use L1L_1/DSSIM with SSIM weight 0.20.2. Separately, the real ScalarFlow reconstruction takes 5 h 05 min on Tesla V100. Full numerical training settings, including iteration counts, learning rates, and optimizer parameters, will be provided in JSON configuration files. We plan to release these files together with the source code and datasets after internal review and approval.

Please see the supplementary material (pages 20–21) and appendix for further details, figures, and quantitative results.

Reviewer_ftowOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VFOM-GS, which explicitly couples self-shadowing with the reconstructed density field to reconstruct dynamic smoke from multi-view videos. VFOM-GS not only performs novel view synthesis, but also produces consistent self-shadowing under changing illumination. Specifically, it presents the VFOM, which encodes the Gaussian profiles along each light ray in a Fourier basis, coupling the light-side optical depth with the same density field used for camera-side rendering. It also proposes PDN, which fits the Fourier encoding interval to each light ray's depth range to improve the effective depth resolution. Both the light and camera passes are differentiable, so the Gaussian density and geometry can be optimized with shadow supervision for better reconstruction and relighting. Experiments on synthetic smoke datasets show that VFOM-GS outperforms the evaluated baselines in novel-view reconstruction, and its Unreal Engine plugin renders faster than Niagara.

Critical strengths
  1. The proposed method reconstructs multi-view smoke videos into compact and relightable assets, which is practical for applications such as visual effects and real-time engines.
  2. The Unreal Engine plugin shows the potential for deployment, with lower storage and render time than Niagara in the tested configurations.
  3. VFOM and PDN are proposed to better handle self-shadowing in smoke: VFOM encodes the finite thickness of each Gaussian along the light ray to reduce ringing, and PDN adapts the encoding interval to each light ray.
  4. VFOM-GS combines VFOM with Vol3DGS so that the light and camera passes share the same density field, and both passes are differentiable, which allows shadow supervision to update the density and geometry.
  5. The proposed method achieves the best novel-view metrics among the evaluated baselines on two synthetic smoke datasets.
Critical weaknesses
  1. Table 1 shows that the full system outperforms the reconstruction baselines, but it is unclear how much of the improvement comes from the proposed VFOM and PDN, since the system also includes Vol3DGS-based volumetric rendering and a two-stage training scheme with shadow-free images. An ablation that replaces only the shadow model in the same pipeline (e.g., no shadow, legacy FOM, VFOM, and VFOM + PDN) would help clarify the contribution of each component.
  2. A more informative FOM baseline would use the same light-ray optical thickness as VFOM while retaining the Dirac depth profiles. With matched depths and normalization, the coefficient difference is then the Gaussian envelope in Eq. (7). Comparing this baseline with VFOM, both with and without PDN, in the same reconstruction pipeline would isolate the benefit of encoding finite Gaussian thickness.
  3. Only synthetic data is used. Experiments on the original real captures of ScalarFlow, which are also used by recent smoke reconstruction methods such as GauSmoke, could be added to verify the results. Do the two-stage training and the shadow model still work on real captures, where shadow-free images are not available? Even qualitative results would be helpful.
Questions for authors

See above

1 comment

We sincerely appreciate your time and feedback. Below, we clarify the points raised and describe the added evidence in the revised paper and supplementary material.

Q1,2: Table R1 and supplementary Fig. (b) compare four shadow-model configurations on Explosion. Without shadow modeling, shadows are baked into near-black Gaussian RGB, so regions remain dark even when they become light-facing under relighting (red dashed boxes). Explicit light-side attenuation allows shading to respond to the illumination. Without PDN, inaccurate shadow estimates hinder recovery of extinction coefficients. For the finite-width comparison, FOM+PDN and VFOM+PDN use the same PDN normalization and Fourier order. With FOM, each Gaussian is treated as an infinitesimally thin event along the light ray, increasing sharp, spike-like noise during reconstruction (yellow solid boxes). With VFOM, the Gaussian envelope and analytic integration preserve thickness and a fuller volumetric appearance. Table R1 confirms that the full VFOM+PDN model achieves the best results across all reported metrics.

Table R1

ConfigurationPSNR ↑SSIM ↑LPIPS ↓
w/o VFOM, PDN23.090.9460.0674
w/o PDN23.270.9400.0716
with FOM, PDN22.730.9510.0772
Full Model27.410.9650.0595

Q3: Yes, ours supports real-world captures. It's noticed that our primary goal is to replace costly volumetric assets with compact 4D Gaussians for high-quality, low-cost real-time rendering, rather than a dedicated real-world capture and reconstruction system. ScalarFlow in supplementary S1 provides an additional robustness test. We set each image's light direction to its camera's viewing direction, approximating the camera-side softbox lighting [3], and jointly optimize colors and the Gaussian field through VFOM using captured images. This requires no paired shadow-free targets, additional lighting estimation, or real-data-specific model components. We match GauSmoke's [2] 120 frames, five cameras, and 270×480270\times480 resolution, comparing against its published results because its implementation is unavailable. Ours has higher SSIM, lower LPIPS, and smaller storage (24.4 vs. 86 MB) than LagrangianSplats [1], despite lower PSNR (Table R2). These competitive results support robustness under the lighting approximation. Velocity priors can improve sparse-view PSNR but may constrain shape fitting and blur fine-scale turbulence (Fig. (a)); our dense-view setting in Experiment 1 reports higher PSNR than LagrangianSplats.

Table R2

MethodPSNR ↑SSIM ↑LPIPS ↓Storage (MB)Train Time (h)GPUFPS ↑
GauSmoke [2]34.830.9550.0168420.8RTX3090114
LagrangianSplats [1]35.970.95540.0954865.6TeslaV10033
VFOM-GS (ours)34.240.98380.004024.45.1TeslaV100645

Please see the supplementary material (pages 20–21) and appendix for further details, figures, and quantitative results.

References

[1] N. Tao, B. Chen, and M. Chu. LagrangianSplats: Divergence-Free Transport of Gaussian Primitives for Fluid Reconstruction. ACM SIGGRAPH Conference Papers, 2026.

[2] W. Zhang, Y. Cai, L. Huang, D. Ye, J. Guo, and B. Ren. GauSmoke: Hybrid Physics-Optical Gaussian Splatting for Sparse Smoke Reconstruction. ACM SIGGRAPH Conference Papers, 2026. https://doi.org/10.1145/3799902.3811148.

[3] M.-L. Eckert, K. Um, and N. Thuerey. ScalarFlow: A Large-Scale Volumetric Data Set of Real-world Scalar Transport Flows for Computer Animation and Machine Learning. ACM Transactions on Graphics, 38(6), Article 239, 2019. https://doi.org/10.1145/3355089.3356545. Capture setup: https://ge.in.tum.de/?p=1536/.

Comments on the paper

5 comments
AuthorsOpenReview ↗

General Response

We thank all reviewers for their careful reading and constructive feedback. We apologize for the unclear wording and have clarified these points in the revised paper and supplementary material in the end (pages 20–21) .

Four main concerns emerged from the reviews: applicability to real captures, the contributions of individual components, supported lighting conditions, and numerical accuracy. We address these with four supplementary experiments and provide further details in the individual responses.

Our goal is to replace costly volumetric assets with compact, relightable 4D Gaussians for high-quality, low-cost real-time smoke rendering. With this focus, the supplementary experiments evaluate reconstruction quality, relighting, and shadow accuracy:

  • S1 — Real ScalarFlow captures (ftow, eP9i, 3RAq, feSo): Reconstruction from captured images without paired shadow-free targets, under assumed camera-aligned lighting, with quality, storage, and training-cost comparisons.
  • S2 — Explosion reconstruction ablation (ftow, eP9i, feSo): Full reconstruction with no shadow modeling, VFOM without PDN, FOM+PDN, and VFOM+PDN. The no-shadow baseline fits shadowed images by baking shadows into Gaussian RGB, providing a comparison with shadow-aware reconstruction.
  • S3 — Colored directional lighting (3RAq): Rendering the same fitted Explosion asset in UE5.8 without refitting. Multiple lights and environment illumination remain outside the evaluation.
  • S4 — Fourier-order accuracy (feSo): FOM and VFOM transmittance and their errors against exact integration at K=3,7,11K=3,7,11, using the same fields of overlapping Gaussians with different optical thicknesses. VFOM's small discrepancy at K=7K=7 in these tests supports our default setting; S2 uses K=3K=3.

We also provide a revised paper PDF, no more than 10 pages, incorporating the additional experiments, quantitative results, and explanations, with additions and revisions highlighted in red. For easy reference, a separate supplementary file presents the figures, results, and descriptions of the four experiments above. We provide detailed, individual responses to each reviewer's weaknesses and questions below.

Subject to internal review and approval, we plan to release the source code, datasets, and JSON configuration files with the full numerical training settings.

Area_Chair_2rhTOpenReview ↗

Please Participate in the Discussion Phase

Dear Reviewers,

The authors have posted their rebuttal and responses addressing your review comments and questions.

Please carefully read their responses (as well as the updated manuscript, if applicable) and actively engage in the reviewer-author discussion phase. If you have any remaining concerns or follow-up questions, please post them so the authors can address them before the discussion period closes.

Your timely participation and updated assessments are essential to helping us reach a fair and well-informed recommendation for this submission.

Best regards,

Area Chair

AuthorsOpenReview ↗

General Response 2

We thank the reviewers for their continued feedback on supervision requirements and the practicality of relighting. We clarify the training settings and present expanded lighting demonstrations below.

1. Clarifying the experimental settings for dependence on source data. Our original goal was to convert voxel smoke assets into compact representations for real-time rendering, so the reconstruction and relighting experiments use different training settings. We distinguish two training settings: the reconstruction setting (Sections 4.1, 4.3, 4.5 and 4.6), and the gaming setting (Sections 4.2 and 4.7). In Section 4.1, we fit the Gaussian representation from shadowed images without shadow-free supervision, using the same training inputs and evaluation targets as the baselines on both ScalarPurple and Explosion. Because fitting shadowed images can bake shadows into Gaussian colors, we remove dark-colored Gaussians, freeze the colors of the remaining Gaussians, and optimize their extinction coefficients. Under this shared setting, VFOM-GS achieves state-of-the-art novel-view reconstruction quality among the evaluated methods on both datasets without requiring shadow-free images. Therefore, our reconstruction method does not require shadow-free images.

The gaming setting targets the conversion of existing game assets into compact representations for real-time rendering: we first fit the Gaussian representation to shadow-free images rendered from the source asset, then perform shadow-aware refinement using shadowed images while keeping colors fixed. Section 4.2 compares the model before and after this refinement to demonstrate its benefit for shadow quality. Our original training description detailed the gaming setting but omitted an explicit description of the reconstruction setting, misleadingly suggesting that shadow-free images were always required. We apologize for this omission and have clarified the distinction in the revised paper.

2. Expanded relighting demonstrations. We have added a UE experiment that augments Gaussian smoke relighting by heuristically sampling and accumulating direct-light contributions. Direct-light shading uses VFOM transmittance, while indirect illumination is obtained by sampling irradiance from Lumen, Unreal Engine's dynamic global illumination system. The visually plausible results with local colored lights and GI demonstrate the potential of our representation for richer lighting applications as a relightable alternative to voxel assets. Please see supplementary S3 (pages 20–21) for the results and a discussion of possible extensions to environment maps and multiple directional lights, rather than experiments on those extensions. Under rebuttal restrictions, we provide still images and plan to release a demonstration video alongside the datasets and rendering results, subject to internal review and approval.

We sincerely thank all reviewers for their constructive suggestions, which have helped us clarify and strengthen our work, and we have further revised the paper accordingly. Please see the supplementary material (pages 20–21) and appendix for further details and figures. We sincerely appreciate your continued engagement and hope these clarifications and additional results help address your concerns. Subject to internal review and approval, we plan to release the source code, datasets, and JSON configuration files with the full numerical training settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.