- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents an unsupervised post-training framework that adapts pretrained, feedforward 3D Gaussian Splatting models to inverse rendering without training a new network from scratch. By freezing the original reconstruction backbone and analyzing how residual geometry errors propagate into nonlinear PBR shading, the authors replace traditional pre-splat parameter estimation with a post-splat decoding paradigm. In this approach, view-independent material latents are splatted onto the image plane and decoded into PBR parameters conditioned on an aggregated multi-view scene memory. Evaluated across seven feedforward backbones on RealEstate10K, the method adds only a small number of additional parameters while matching or exceeding native novel-view reconstruction fidelity and enabling real-world relighting.
Critical strengths
-
The paper resolves the geometry-to-PBR error propagation bottleneck by rasterizing 64-D material latents rather than blending explicit 8-D PBR parameters before splatting, preserving higher-order distribution moments essential for nonlinear shading, preventing geometry misalignment from degrading material estimation.
-
The framework adapts frozen feedforward 3DGS models to inverse rendering without requiring ground-truth PBR supervision or retraining architectures from scratch. It demonstrates plug-and-play compatibility across seven distinct backbones while introducing only limited parameter overhead.
Critical weaknesses
-
Absence of Quantitative Intrinsic and Relighting Evaluation: The empirical evaluation relies entirely on source-light novel-view synthesis metrics (PSNR, SSIM, LPIPS) on RealEstate10K alongside qualitative crops, offering zero numerical validation on intrinsic material accuracy (e.g., albedo, normal angular, roughness) or ground-truth HDR relighting benchmarks. Because inverse rendering is a heavily ill-posed factorization problem subject to fundamental material-illumination ambiguities, high RGB reconstruction fidelity under the original capture lighting can easily be achieved while baking illumination into albedo or learning unphysical specular shortcuts. Without quantitative verification against ground-truth material datasets or comparisons against prior feedforward inverse rendering pipelines, it is impossible to substantiate whether the estimated decomposition is physically meaningful or merely an over-parameterized appearance fit.
-
Architectural Violations of View-Invariance: Key components of the post-splat and illumination architectures introduce screen-space dependencies that conflict with the view-independent nature of physical scene properties. Applying 8-group Group Normalization across the spatial dimensions of the target-view rasterized feature map forces the decoded albedo, roughness, and normals at any given 3D point to vary based on camera framing, image boundaries, and visible scene context. Similarly, conditioning the global HDR environment cubemap prediction on the rendered target-view feature buffer risks making the estimated scene lighting viewpoint-dependent. These choices compromise the multi-view and temporal consistency inherent to explicit 3D Gaussian representations.
Questions for authors
-
Does evaluating the post-splat decoder along continuous camera trajectories reveal temporal flickering or material drift caused by screen-space GroupNorm and view-dependent feature rasterization?
-
Why does the illumination head ingest target-view rendered feature maps to predict a global scene environment cubemap? Does the predicted cubemap vary when querying different target camera poses within the same scene, and how is cross-view lighting consistency enforced?
-
How does the framework perform quantitatively on established synthetic/real inverse rendering datasets with ground-truth intrinsics and known HDR relighting?
-
Why were external feedforward inverse rendering pipelines (such as MVInverse or MAIR++) excluded from the empirical comparisons, and how does your unsupervised adapter compare to them in terms of inference speed and material accuracy?
-
Because the adapter is trained solely on re10k, how well does the post-trained model generalize zero-shot to completely different scene distributions without retraining or fine-tuning? Have you evaluated cross-dataset transfer to confirm that training on residential interiors does not degrade the zero-shot capabilities of backbones
5 comments
R3.W2 / Q1 / Q2 — Cross-view consistency and illumination prediction
Since Weakness 2 and Questions 1–2 concern the same architectural issues, we address them jointly here, we first evaluate cross-view material consistency through flow-reprojection experiments, then clarify the actual inputs to the illumination head and how the scene-level environment map is shared across target views.
We conducted flow-reprojection experiments following MVInverse [2] on RealEstate10K, DL3DV, Synthetic4Relight, and TensoIR to evaluate cross-view material consistency by comparing material properties at corresponding surface locations across views.
The table reports Albedo / Metallic / Roughness RMSE ↓.
| Dataset | MVInverse | MAIR++ | GI-GS | GS-IR | Ours (AnySplat) |
|---|---|---|---|---|---|
| RealEstate10K | 0.039 / 0.030 / 0.015 | 0.050 / N/A / 0.108 | 0.057 / 0.086 / 0.046 | 0.047 / 0.074 / 0.052 | 0.024 / 0.040 / 0.020 |
| DL3DV | 0.045 / 0.030 / 0.011 | 0.054 / N/A / 0.102 | 0.058 / 0.081 / 0.061 | 0.067 / 0.089 / 0.069 | 0.035 / 0.065 / 0.028 |
| Synthetic4Relight | 0.031 / 0.014 / 0.005 | 0.051 / N/A / 0.081 | 0.018 / 0.017 / 0.030 | 0.020 / 0.014 / 0.028 | 0.013 / 0.039 / 0.017 |
| TensoIR | 0.041 / 0.027 / 0.007 | 0.060 / N/A / 0.123 | 0.021 / 0.022 / 0.028 | 0.023 / 0.022 / 0.026 | 0.018 / 0.054 / 0.022 |
Despite using no intrinsic ground-truth supervision, our method maintains competitive cross-view material consistency on the evaluated datasets.
Illumination-head description and reproducibility. We apologize for the incorrect description of the illumination-head inputs in the manuscript. The text mistakenly identified the rendered target-view feature buffer as its input.
Figure 2 correctly shows the actual inputs: the output tokens of the frozen feed-forward 3DGS backbone and the global scene latent, matching our implementation.
For a fixed set of input images, the scene-level HDR cubemap is shared across target views, and changing the target camera does not change the illumination-head inputs.
We will correct Section 4.2 and Appendix G.5 and subsequently release the implementation code and test and evaluation scripts.
R3.Q3 — Quantitative evaluation on synthetic and real scenes
We report complementary evaluations on TensoIR for ground-truth intrinsic recovery and relighting under known HDR illumination, and on NeRF-OSR for real outdoor scene relighting.
(1) TensoIR.
| Albedo PSNR ↑ | Albedo SSIM ↑ | Albedo LPIPS ↓ | Normal cos. ↑ | Relight PSNR ↑ | |
|---|---|---|---|---|---|
| MVInverse | 18.562 | 0.874 | 0.152 | 0.902 | 18.420 |
| MAIR++ | 21.152 | 0.890 | 0.158 | 0.859 | 21.029 |
| GI-GS | 24.251 | 0.882 | 0.150 | 0.898 | 26.972 |
| GS-IR | 24.181 | 0.880 | 0.155 | 0.912 | 26.549 |
| Ours (AnySplat) | 24.914 | 0.887 | 0.154 | 0.862 | 26.911 |
(2) NeRF-OSR.
| Relight PSNR (dB) ↑ | |
|---|---|
| MVInverse + adapter | 19.857 |
| MAIR++ + adapter | 17.652 |
| GI-GS | 17.378 |
| R3GW | 21.728 |
| LumiGauss | 20.978 |
| Ours (Full) | 19.421 |
The limited training views in our NeRF-OSR evaluation may constrain per-scene optimization methods such as GI-GS and contribute to its weaker relighting performance.
Across synthetic and real-scene evaluations, our method achieves competitive relighting and inverse rendering without intrinsic ground-truth supervision, supporting its applicability across scene domains.
R3.Q4 — Feed-forward baselines and inference efficiency
Following the reviewer's suggestion, we have added quantitative comparisons with MVInverse and MAIR++.
Material accuracy and inference speed. We assess material accuracy against ground-truth intrinsic properties on TensoIR (Question 3) and report inference speed on DL3DV below.
| Time | |
|---|---|
| Ours (AnySplat) | 0.0925 |
| MVInverse | 0.0497 |
| MAIR++ | 0.1025 |
Our method offers competitive material accuracy and inference speed. Its inference cost includes material rasterization and decoding in the deferred rendering pipeline for PBR.
R3.Q5 — Cross-dataset zero-shot transfer
We directly test the AnySplat-based model post-trained on RealEstate10K on DL3DV, without retraining or fine-tuning.
Transfer results for the other backbones are also reported below.
| Backbone | Dense PSNR: Native → PBR (dB) | Dense ΔPSNR (dB) | Sparse PSNR: Native → PBR (dB) | Sparse ΔPSNR (dB) |
|---|---|---|---|---|
| AnySplat | 25.714 → 28.024 | +2.310 | 21.716 → 23.851 | +2.135 |
| DA3 | 22.165 → 24.650 | +2.485 | 11.509 → 11.482 | −0.027 |
| C3G | 21.398 → 21.365 | −0.033 | 20.939 → 20.976 | +0.037 |
| NoPoSplat | 21.599 → 24.355 | +2.756 | 21.326 → 23.327 | +2.001 |
| SPFSplat | 25.151 → 26.437 | +1.287 | 21.518 → 22.926 | +1.408 |
| YoNoSplat | 24.402 → 25.673 | +1.271 | 22.989 → 24.213 | +1.224 |
Our method remains effective on most backbones in these evaluated cases. Entries average five scenes selected after evaluation; DA3-dense evaluates input-view reconstruction.
References
[1] Hongze Chen, Zehong Lin, and Jun Zhang. GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering. ICLR 2025. Appendices H and I.2; authors' response to Reviewer zyyD, Q1.
[2] Xiangzuo Wu, Chengwei Ren, Jun Zhou, Xiu Li, and Yuan Liu. MVInverse: Feed-forward Multi-view Inverse Rendering in Seconds. arXiv:2512.21003, 2025, v2 §§3.2, 4.2, and 7 (Table 5).
[3] JunYong Choi, SeokYeong Lee, Haesol Park, Seung-Won Jung, Ig-Jae Kim, and Junghyun Cho. MAIR++: Improving Multi-view Attention Inverse Rendering with Implicit Lighting Representation. arXiv:2408.06707, v2, 2025, §§IV-D, IV-E, and V-A.
Comment:
Thank you for recognizing our post-splat design for mitigating geometry-to-PBR error propagation and our lightweight adaptation across frozen backbones without PBR supervision. We address the quantitative evaluations, illumination-head description, feed-forward comparisons, and cross-dataset transfer below.
R3.W1 — Quantitative intrinsic recovery and relighting
To address the quantitative evaluation concern, we reproduce below the TensoIR and Synthetic4Relight results reported in our response to Reviewer 1, Weakness 1, including comparisons with MVInverse, MAIR++, GI-GS, and GS-IR.
(1) TensoIR.
(2) Synthetic4Relight.
We use ground-truth intrinsic properties from TensoIR to quantitatively assess material recovery. Compared with feed-forward methods MVInverse and MAIR++, our albedo and relighting PSNR are higher on TensoIR and Synthetic4Relight.
TensoIR is absent from the reported training data of MVInverse, so domain differences may affect its performance [2].
Compared with per-scene optimization methods GI-GS and GS-IR, we achieve higher albedo PSNR and comparable relighting on TensoIR.
Our method uses deferred rendering based on GI-GS. Without intrinsic ground-truth supervision, some illumination may be baked into the estimated intrinsic parameters, an issue also present in GI-GS [1].
Nevertheless, quantitative evaluation against intrinsic ground truth supports competitive material accuracy.