- Rating
- 3
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents ELM-GS, an efficient 3D Gaussian Splatting framework for inverse rendering. Primitive-wise learnable lighting features are decoded into directional visibilities, from which AO and bent normals are derived, and view-dependent EC to modulate direct lighting. SSAO-derived multi-view occlusion priors supervise AO learning. The authors also introduce Mat-Edit, a synthetic benchmark with ground-truth AO, material annotations, and material-edited reference images.
Experiments show improved intrinsic decomposition and scene editing over GS-IR and GI-GS, with substantially faster rendering. Compared with RadioGS on synthetic scenes, ELM-GS offers higher rendering efficiency but lower novel view synthesis and relighting quality.
Critical strengths
- Clear motivation and efficient design. Fig. 3 intuitively illustrates AO-based lighting ambiguity. ELM-GS models local occlusion through primitive-wise AO and bent normals, while EC compensates for simplified direct lighting. Ablations demonstrate substantial rendering speed gains from learnable AO and improvements in NVS and selected decomposition and editing metrics from EC.
- Effective geometry-based supervision. SSAO-derived multi-view occlusion priors constrain AO learning. Ablations show improved AO estimation and relighting quality with this supervision.
- Useful benchmark contributions. Mat-Edit complements existing evaluations of NVS, albedo, and relighting with ground-truth AO maps, pixel-level material annotations, and material-edited reference images.
Critical weaknesses
-
Limited directional expressiveness. Predicted directional visibilities are reduced to scalar AO and a bent normal for a single diffuse irradiance lookup. This approximation cannot fully capture the interaction between visibility distributions and directional environment lighting. Although EC compensates for local illumination errors under the original lighting, it is omitted during relighting, leaving this approximation uncorrected by EC under novel illumination.
-
Unclear benefit of the visibility parameterization. Deriving AO and bent normals jointly from directional visibilities provides a plausible inductive bias. However, its practical advantage over directly predicting AO and bent normals as a paired output is not evaluated. A comparison between these parameterizations would clarify the benefit of the intermediate directional visibility representation.
-
Potential confounding in material editing evaluation. FlashSplat assigns each Gaussian a single material label, which may be inaccurate for primitives spanning material boundaries or multiple surfaces. Shared initialization helps control comparison conditions, but editing scores may still combine decomposition, representation, and label-association errors. Their respective contributions are not separately quantified.
Questions for authors
-
What advantages does decoding directional visibilities and deriving AO and bent normals offer over directly predicting AO and bent normals? Could the authors provide a comparison of reconstruction quality?
-
In the “SSAO only” and “w/o LAO” variants, is environment irradiance queried using the surface normal or a bent normal derived from SSAO directional visibilities?
10 comments
W1. Limited directional expressiveness. Predicted directional visibilities are reduced to scalar AO and a bent normal for a single diffuse irradiance lookup. This approximation cannot fully capture the interaction between visibility distributions and directional environment lighting. Although EC compensates for local illumination errors under the original lighting, it is omitted during relighting, leaving this approximation uncorrected by EC under novel illumination.
Thank you for highlighting the directional limitations of the AO and bent normal approximation under novel illumination. The limited directional expressiveness is inherent to the AO and bent normal lighting approximation. We identify this limitation in Figure 3 and introduce EC to account for unmodeled shading effects during inverse rendering, reducing their impact on the estimated albedo. Table 5 shows that adding EC improves albedo estimation on both TensoIR and Mat-Edit. Figure 17 further shows that EC reduces differences in estimated albedo across regions of the same material that experience different local shading.
However, we omit EC during relighting since it is specific to the original illumination. In this setting, although the bent normal accounts for the directionality of local occlusion to some extent, the limitation identified by the reviewer remains. Despite this limitation, the simplified lighting model provides substantial efficiency gains. Compared with ray-tracing-based methods, ELM-GS substantially reduces training time and increases rendering speed. This reflects a trade-off between the directional expressiveness of the lighting model and computational efficiency.
W2. Unclear benefit of the visibility parameterization. Deriving AO and bent normals jointly from directional visibilities provides a plausible inductive bias. However, its practical advantage over directly predicting AO and bent normals as a paired output is not evaluated. A comparison between these parameterizations would clarify the benefit of the intermediate directional visibility representation.
Thank you for drawing attention to the role of directional visibility in our method. We follow common practice in computer graphics by using bent normals to guide environment map lookups toward the more visible regions of the hemisphere. Since our method can efficiently estimate visibility in multiple directions, deriving bent normals from those estimates is a natural choice. We adopt bent normals as an established rendering technique, with our main contribution lying in redesigning how per-primitive AO and EC modulate environment lighting to reduce computational cost and improve the performance of AO-based 3DGS inverse rendering.
We agree that scalar AO can be predicted directly. However, directly predicting the bent normal as an unconstrained direction would undermine its physical interpretation. The purpose of using bent normals is to account for the directionality of local occlusion at a surface point when sampling the environment map. By definition, a bent normal represents the average direction of visible incident light. Direct prediction leaves this defining relationship unconstrained and shifts the objective from characterizing the directions from which environment light can reach the point to selecting a sampling direction that best suits the point.
In an additional ablation, we compare the "Full" model with a variant that directly predicts scalar AO and uses the surface normal for the diffuse irradiance lookup, following common practice in AO-based 3DGS inverse rendering.
| Task | Metric | TensoIR w/o BN | TensoIR Full | Mat-Edit w/o BN | Mat-Edit Full |
|---|---|---|---|---|---|
| NVS | PSNR | 35.50 | 35.71 | 36.41 | 36.66 |
| NVS | SSIM | 0.967 | 0.968 | 0.983 | 0.983 |
| NVS | LPIPS | 0.040 | 0.040 | 0.029 | 0.029 |
| Albedo | PSNR | 31.18 | 31.07 | 24.44 | 24.50 |
| Albedo | SSIM | 0.931 | 0.929 | 0.915 | 0.913 |
| AO | PSNR | — | — | 20.76 | 20.89 |
| AO | WMSE | — | — | 0.007 | 0.007 |
| Material Editing | PSNR | — | — | 23.79 | 23.69 |
| Material Editing | SSIM | — | — | 0.939 | 0.937 |
| Material Editing | LPIPS | — | — | 0.073 | 0.074 |
| Relighting | PSNR | 25.98 | 26.03 | 24.69 | 24.85 |
| Relighting | SSIM | 0.896 | 0.896 | 0.917 | 0.917 |
| Relighting | LPIPS | 0.097 | 0.097 | 0.088 | 0.088 |
The full formulation improves relighting performance by 0.05 dB on TensoIR and 0.16 dB on Mat-Edit. These modest gains are consistent with our motivation for using directional visibilities to steer the irradiance lookup toward visible directions. This experiment measures the combined effect of deriving AO from directional visibilities and using a bent normal.
To further assess the effect of directly predicting the diffuse irradiance lookup direction, we conduct an ablation study on Mat-Edit in which an MLP predicts both AO and the lookup direction. As shown in the table below, the "PD" variant improves NVS but substantially degrades albedo estimation, material editing, and relighting. The directly predicted direction lacks a clear physical interpretation, exacerbating the entanglement of lighting and material properties during inverse rendering. It is also strongly coupled to the original learned illumination, leading to inaccurate irradiance estimates and a substantial decline in relighting performance when the environment map is replaced.
| Task | Metric | PD | Full |
|---|---|---|---|
| NVS | PSNR | 37.47 | 36.66 |
| NVS | SSIM | 0.985 | 0.983 |
| NVS | LPIPS | 0.026 | 0.029 |
| Albedo | PSNR | 23.21 | 24.50 |
| Albedo | SSIM | 0.908 | 0.913 |
| AO | PSNR | 20.87 | 20.89 |
| AO | WMSE | 0.007 | 0.007 |
| Material Editing | PSNR | 22.23 | 23.69 |
| Material Editing | SSIM | 0.931 | 0.937 |
| Material Editing | LPIPS | 0.077 | 0.074 |
| Relighting | PSNR | 21.85 | 24.85 |
| Relighting | SSIM | 0.881 | 0.917 |
| Relighting | LPIPS | 0.104 | 0.088 |
In the revised manuscript, we add an ablation study in Section Appendix.I to assess the effect of directional visibility and bent normals.
Thank you for your careful review. We appreciate your recognition of the paper’s "clear motivation and efficient design," "effective geometry-based supervision," and "useful benchmark contributions." In our responses below, we clarify the limitations of the lighting approximation under novel illumination, report an additional ablation on directional visibility and bent normals, and explain the scope of our material editing evaluation and the irradiance lookup used in the SSAO variants. At the end of each response, we describe the corresponding changes and highlight them in the revised manuscript.