- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents AEGIR, an inverse rendering framework that models local area emitters jointly optimizable with scene geometry and materials from multi-view indoor images. The paper defines emitters as anisotropic ellipsoids and directional emission with a learnable Super-Gaussians. These emitter models are well integrated with 2D Gaussian scene through differentiable deferred rendering with multiple importance sampling and soft visibility to support realistic lighting effects. Experiments on synthetic and real/scanned indoor scenes evaluate novel-view synthesis, material decomposition, lighting reconstruction, and relighting. Controlled lighting evaluations and emitter ablations demonstrate the benefits of the area emitter model.
Critical strengths
-
Clear motivation. The paper addresses limitations in how existing inverse rendering methods represent realistic light sources. This is an important problem that could broaden the applicability of Gaussian-based scene representations for relighting and scene editing.
-
Convincing evaluation of lighting reconstruction. Re-rendering scenes with estimated lighting and ground-truth geometry and materials isolates lighting errors from other reconstruction errors. This directly supports the central contribution beyond novel-view synthesis and albedo reconstruction performance, which is also convincing.
Critical weaknesses
-
Substantial error in recovered light flux. Although AEGIR outperforms the ablation variants in Table 4, its relative flux error remains substantial at 0.464. It is unclear whether this reflects inaccurate emitter reconstruction or the scale ambiguity between albedo and illumination. An analysis of this ambiguity's contribution to the reported error would clarify the accuracy of the recovered light output.
-
Limited analysis of geometry reconstruction. Table S2 and Figure S2 report quantitative and qualitative improvements in normal reconstruction via optimization, but per-scene metrics and broader numerical comparisons with existing works in both synthetic and real benchmarks would better establish AEGIR's benefits. Inaccurate lighting can potentially be absorbed by changes in geometry. Thus, assessing whether the proposed emitter model reduces the normal errors would strengthen the evaluation.
-
Limited analysis of input coverage. The paper does not specify the number of training and test views per scene or the camera selection procedure. As authors mentioned in Appendix E, sparse view inputs may create uncertainty about illumination coming from unobserved regions. Since the current area emitters are directly defined by unprojected points from the input views, unobserved light sources cannot contribute for inverse rendering. A hybrid model combining the proposed area emitters with an environment map could potentially account for distant illumination entering through these regions, but this configuration is neither explored nor evaluated. Clarifying whether the current experiment setting is free from such issue may be helpful.
Questions for authors
- What depth source and depth definition are used to unproject bright pixels during emitter initialization?
- Regarding applicability beyond the indoor setting, can AEGIR handle large-scale outdoor scenes, and under what lighting conditions would its area-emitter representation offer advantages over environment-map-based methods? Any available evidence would help clarify its applicability and computational scalability.
5 comments
Author Response (2/3): Geometry Analysis
W2) Geometry:
Every listed per-scene geometry metric improves after joint optimization, and full AEGIR gives the most accurate geometry among the compared methods.
Table S2 of the submitted supplement (App. D.2) reports geometry before and after joint optimization, when geometry, materials and emitters are optimized together. Geometry is then fixed during the final emitter-only stage. As requested, we now provide per-scene results in Tables R3 and R4 of the revised Appendix D.2, using the saved checkpoints without retraining. Table R3 reports normal and depth errors for the six Hypersim and four FIPT-Synthetic scenes. Table R4 reports depth errors for the four ScanNet++ and two FIPT-Real scenes. To save space in the rebuttal, we show only the four FIPT-Synthetic results below. The remaining scenes are reported in Tables R3 and R4 of the revised PDF.
| Scene (DN-Splatter initialization → AEGIR) | Normal RMSE | Mean angular error | Depth AbsRel | Depth RMSE |
|---|---|---|---|---|
| bathroom | 0.250 → 0.245 | 17.14° → 16.52° | 0.087 → 0.080 | 0.306 → 0.291 |
| bedroom | 0.234 → 0.227 | 16.03° → 15.29° | 0.076 → 0.074 | 0.285 → 0.261 |
| kitchen | 0.176 → 0.170 | 12.34° → 11.41° | 0.060 → 0.054 | 0.216 → 0.200 |
| livingroom | 0.152 → 0.130 | 10.69° → 9.01° | 0.052 → 0.049 | 0.167 → 0.162 |
New in this rebuttal (Table R2), we also compare the methods on the same ten synthetic scenes. GS-ID starts from our DN-Splatter initialization, while IRGS and NeILF++ reconstruct their own geometry. The two AEGIR variants use the same geometry initialization, material model, renderer and training schedule as full AEGIR, but replace our anisotropic emitters with point lights or isotropic area emitters (Table 4).
| Method (10 synthetic scenes) | Normal RMSE | Mean angular error | Depth AbsRel |
|---|---|---|---|
| DN-Splatter initialization (Table S2) | 0.203 | 14.11° | 0.067 |
| GS-ID (App. D.2) | 0.199 | 13.95° | 0.070 |
| IRGS | 0.249 | 19.88° | 0.114 |
| NeILF++ | 0.235 | 18.91° | 0.119 |
| Ours with point lights | 0.198 | 13.87° | 0.069 |
| Ours with isotropic area emitters | 0.191 | 13.05° | 0.066 |
| AEGIR (Table S2) | 0.187 | 12.70° | 0.063 |
AEGIR gives the most accurate geometry on all three metrics. Among variants with the same initial geometry, more accurate light models lead to more accurate geometry. Our emitter model therefore helps reduce geometry errors.
Author Response (3/3): Input Coverage, Depth Initialization, and Outdoor Scenes
W3) Input coverage:
We evaluate hybrid illumination through our baselines, and the uniform-grid ablation shows that initial emitters need not come from projected bright pixels.
Hybrid illumination. GS-ID and SGS-Intrinsic combine an environment map with local point lights (App. C.2). AEGIR outperforms both in novel views on all five benchmarks and in albedo on both synthetic ones (Tables 2 and 3). AEGIR uses a unified emitter representation for local light sources and incoming illumination, with emitters at the windows capturing light entering the scene.
Emitter initialization. Our no-emitter-initialization ablation starts from a uniform grid instead of projected bright pixels and gives nearly the same albedo accuracy (Table S5). Emitter position / size / linear flux errors are likewise comparable (0.088 / 0.065 / 0.468 vs. 0.086 / 0.065 / 0.464 with bright-pixel initialization). This demonstrates that AEGIR can perform inverse rendering without bright-pixel unprojection, so initial emitters need not correspond to directly observed light sources. The emitters are then optimized freely with splitting and pruning (Sec. 3). Even when a light is not directly visible, the brightness and shadows it produces on visible surfaces can help estimate its position and intensity.
View splits. All methods, including every baseline, use the same training and test views of each dataset (Table S1, App. C.1), and the revised App. C.1 now states these splits.
| Dataset | Training views per scene | Test views per scene | Test views chosen as |
|---|---|---|---|
| FIPT-Synthetic | 101–106 | 101–107 (bathroom 6) | official held-out views |
| Hypersim | 93–95 | 4–5 | official held-out views |
| ScanNet++ | 59–137 | 33–76 | every other key frame |
| FIPT-Real | 95–139 | 96–139 | official held-out views |
| Replica | 64 | 64 (separate from training) | uniformly sampled from about 2,000 frames per scene |
Q1) Depth used for initialization:
We use the z-depth predicted by the initial DN-Splatter geometry, in scene units, and unproject each bright pixel with the camera intrinsics and pose. No ground-truth depth is used. The revised Sec. 3 now states this.
Q2) Outdoor scenes:
AEGIR targets indoor scenes lit by local emitters, while an environment map is a more natural fit outdoors, especially in daytime. New in this rebuttal (revised App. E), AEGIR reaches 18.91/0.76/0.34 in novel views on the Mip-NeRF 360 garden scene, a preliminary result showing that AEGIR can also run outdoors. However, a hybrid model, as you suggest, may perform better in these settings. Runtime is the same as indoors (40 minutes of optimization, 55 with initialization, App. C.5) and can grow with the number of emitters.
We hope this addresses your concerns, and we are happy to answer any further questions.
Author Response (1/3): Emitter Flux Error
We thank the reviewer for the constructive suggestions. All additions are now in the revised PDF, in green.
W1) Emitted-power error of 0.464:
Accounting for the shared albedo-light scale lowers flux error from 0.464 to 0.182. Applying the same correction to the next-best primitive preserves the advantage of full AEGIR (0.182 vs. 0.321).
Table 4 of the submitted paper compares emitted power (Flux) with the ground truth directly in absolute linear units, providing the strictest test of a physical quantity. As you suggested, much of the 0.464 reflects the scale ambiguity between albedo and illumination. This comparison is also affected by the brightness and color scale ambiguity between materials and lighting, which LDR images cannot resolve (an albedo darker by a factor k, lit by emitters brighter by k, can produce similar images). Emitter positions and sizes do not depend on this scale and are recovered accurately (Table 4). New in this rebuttal (revised Sec. 4.4, App. C, Table R1), we quantify the ambiguity in two ways.
Rescaling with the albedo scale (0.464 → 0.182), without retraining. For flux evaluation, we estimate the correction from linear RGB albedo. Using the notation in the paper, a denotes the estimated albedo and a^GT the ground-truth albedo. For each scene and color channel c, we fit k_c = Σ a_c a_c^GT / Σ a_c², with the sums taken over all test pixels. We average the three channel factors into one scene factor, k = (k_R + k_G + k_B) / 3, then divide the recovered power of every emitter by k before recomputing the flux error. This applies one brightness correction to the whole scene, reducing our flux error from 0.464 to 0.182 while preserving relative emitter powers and leaving color errors in the metric. The same procedure reduces the flux error of the next-best primitive, Anisotropic Gaussian (ρ = 1), from 0.650 to 0.321. Full AEGIR still achieves the lowest error after rescaling (0.182 vs. 0.321).
Training with HDR inputs. During development, we trained AEGIR on the corresponding HDR observations of the same four FIPT-Synthetic scenes. These observations retain intensity information lost in saturated LDR pixels. HDR reduces the raw flux error from 0.464 to 0.296 and the rescaled error from 0.182 to 0.109. The remaining rescaled error of 0.109 may reflect approximations in our rendering model relative to the Mitsuba reference renderer. Position and size errors remain essentially unchanged at 0.085 / 0.065. We chose LDR inputs to support a wider range of real-world captures. This is also related to reviewer nroG Q4.