- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents LoFT_GS, a locally factorized relighting representation that encodes visibility on a local texture grid within each 2D Gaussian during light-space splatting. This addresses the limitation of using a single visibility value per Gaussian, which cannot represent the spatial distribution of occlusion within its footprint. A range-conditioned shadow formulation is designed to combine the spatially varying local visibility with a Gaussian-level range predictor. Experiments across several scenes demonstrate the improved relighting fidelity over the evaluated baselines.
Critical strengths
- This paper is well written and easy to following. The idea of enhancing the per-Gaussian visibility with local textural representations is practical.
- A range-conditioned shadow formulation is designed to combine texel-level light-space visibility with a Gaussian-level range predictor, ensuring spatially varying shadows.
- Experimental results across 21 scenes show that the proposed method outperforms state-of-the-art methods.
Critical weaknesses
- The idea behind the proposed method is actually quite simple and straightforward. It improves existing relightable 3D Gaussian methods (e.g., GS^3, RNG, and SSD-GS) by incorporating a local texture per-Gaussian. For other shading formulations, they are quite similar to existing methods.
- Like some of other methods, the proposed method can only handle a single point/directional light. This is OK, since relighting 3DGS scenes with a single point/direction light is still challenging. However, I still hope new methods can handle general lighting conditions. I would see some more discussions.
- Why use different numbers of Gaussian primitives for different methods when evaluating runtime performance in Table 3? This seems to make the comparison unfair.
Questions for authors
Why use different numbers of Gaussian primitives for different methods when evaluating runtime performance in Table 3? This seems to make the comparison unfair.
Response to Reviewer 2bPq
We thank the reviewer for their time and careful reading of our paper. We also appreciate your suggestions on lighting conditions and efficiency comparisons, which we address below.
1. More general lighting conditions
Thank you for this suggestion. We have extended LoFT-GS to environment-map relighting and added comparisons with GI-GS and SVG-IR in Table 4 and Figure 6. Specifically, we approximate the environment map as a set of directional lights, render each direction separately using our trained model, and sum the results weighted by the environment light intensities. Therefore, this extension requires no model retraining.
We find that LoFT-GS remains effective under environment lighting. In particular, when the environment map contains strong directional components, LoFT-GS recovers shadow placement, shadow intensity, and specular details closer to the ground truth, as highlighted by the red boxes in Figure 6.
However, per-direction rendering introduces computational overhead. In Supplementary Section D, we show the rendering results under different sample counts: shadow details are well recovered only after increasing the sample count to 512, which requires evaluating many single-light responses. Therefore, although our method can already handle environment lighting in this way, how to support general lighting conditions more efficiently is worth exploring in future work.
2. Efficiency comparison under the same budget
We agree that using the same Gaussian count gives a fairer measure of rendering overhead.
We sampled 200k and 300k Gaussians from the Lego model trained by GS³ as the common input for all four methods, and added the results to Table 3. All methods use the exact same Gaussian positions, scales, orientations, and opacities, along with their respective randomly initialized networks and native rendering pipelines. Although LoFT-GS increases per-frame runtime by about 37% on average compared to GS³, it is still faster than RNG and SSD-GS.
Meanwhile, we also retain the original Table 3 results based on actual training, because reducing the number of Gaussians while maintaining reconstruction quality is itself an important direction for improving efficiency, as also studied in works such as Mini-Splatting, LightGaussian, and Reduced-3DGS (Fang & Wang, 2024; Fan et al., 2024; Papantonakis et al., 2024). Using the same densification settings as GS³, LoFT-GS achieves higher reconstruction quality with 27–50% fewer Gaussians, indicating that our local representation enhances the expressive power of individual Gaussians. As a result, the practical rendering speed of our trained models is comparable to GS³, and even faster on some scenes.
Reference
1. Guangchi Fang and Bing Wang. Mini-Splatting: Representing Scenes with a Constrained Number of Gaussians. In European Conference on Computer Vision (ECCV), 2024.
2. Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, and Zhangyang Wang. LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS. In Advances in Neural Information Processing Systems 37 (NeurIPS), 2024.
3. Panagiotis Papantonakis, Georgios Kopanas, Bernhard Kerbl, Alexandre Lanvin, and George Drettakis. Reducing the Memory Footprint of 3D Gaussian Splatting. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 7(1), 2024.