acceptodds
Under review as a conference paper at ICLR 2027

AEGIR: Modeling Area Emitters for Indoor Inverse Rendering Using Gaussian Splatting

Abstract

Inverse rendering requires separating illumination from surface materials, which is highly ambiguous due to their tight coupling in observed images. While Gaussian Splatting is efficient for novel view synthesis, existing relightable methods approximate scene lighting using discrete point lights, global environment maps, or implicit representations. By ignoring the physical spatial extent of real-world emitters, these approaches produce inaccurate light attenuation and unrealistic shadows. We present AEGIR (Area Emitters for Gaussian Inverse Rendering), a framework that models compact local area emitters with an anisotropic angular emission profile within a relightable Gaussian Splatting representation. Joint optimization of emitters, materials, and geometry is challenging because the flexible emitter parameterization increases both the number of parameters and the ambiguity between illumination and materials. We address this by introducing a differentiable deferred rendering pipeline that integrates multiple importance sampling with targeted regularization. As a result, AEGIR accurately simulates local light transport and achieves more consistent decomposition. Experiments show that explicit area emitters improve illumination reconstruction and enhance downstream tasks, including novel view synthesis, controlled relighting, and virtual object insertion, particularly in scenes with complex local lighting.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Inverse Rendering, Illumination Estimation, Relightable Gaussian Splatting, Differentiable Rendering

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_Eav3OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents AEGIR, an inverse rendering framework that models local area emitters jointly optimizable with scene geometry and materials from multi-view indoor images. The paper defines emitters as anisotropic ellipsoids and directional emission with a learnable Super-Gaussians. These emitter models are well integrated with 2D Gaussian scene through differentiable deferred rendering with multiple importance sampling and soft visibility to support realistic lighting effects. Experiments on synthetic and real/scanned indoor scenes evaluate novel-view synthesis, material decomposition, lighting reconstruction, and relighting. Controlled lighting evaluations and emitter ablations demonstrate the benefits of the area emitter model.

Critical strengths
  1. Clear motivation. The paper addresses limitations in how existing inverse rendering methods represent realistic light sources. This is an important problem that could broaden the applicability of Gaussian-based scene representations for relighting and scene editing.

  2. Convincing evaluation of lighting reconstruction. Re-rendering scenes with estimated lighting and ground-truth geometry and materials isolates lighting errors from other reconstruction errors. This directly supports the central contribution beyond novel-view synthesis and albedo reconstruction performance, which is also convincing.

Critical weaknesses
  1. Substantial error in recovered light flux. Although AEGIR outperforms the ablation variants in Table 4, its relative flux error remains substantial at 0.464. It is unclear whether this reflects inaccurate emitter reconstruction or the scale ambiguity between albedo and illumination. An analysis of this ambiguity's contribution to the reported error would clarify the accuracy of the recovered light output.

  2. Limited analysis of geometry reconstruction. Table S2 and Figure S2 report quantitative and qualitative improvements in normal reconstruction via optimization, but per-scene metrics and broader numerical comparisons with existing works in both synthetic and real benchmarks would better establish AEGIR's benefits. Inaccurate lighting can potentially be absorbed by changes in geometry. Thus, assessing whether the proposed emitter model reduces the normal errors would strengthen the evaluation.

  3. Limited analysis of input coverage. The paper does not specify the number of training and test views per scene or the camera selection procedure. As authors mentioned in Appendix E, sparse view inputs may create uncertainty about illumination coming from unobserved regions. Since the current area emitters are directly defined by unprojected points from the input views, unobserved light sources cannot contribute for inverse rendering. A hybrid model combining the proposed area emitters with an environment map could potentially account for distant illumination entering through these regions, but this configuration is neither explored nor evaluated. Clarifying whether the current experiment setting is free from such issue may be helpful.

Questions for authors
  1. What depth source and depth definition are used to unproject bright pixels during emitter initialization?
  2. Regarding applicability beyond the indoor setting, can AEGIR handle large-scale outdoor scenes, and under what lighting conditions would its area-emitter representation offer advantages over environment-map-based methods? Any available evidence would help clarify its applicability and computational scalability.
5 comments
AuthorsOpenReview ↗

Author Response (1/3): Emitter Flux Error

We thank the reviewer for the constructive suggestions. All additions are now in the revised PDF, in green.

W1) Emitted-power error of 0.464:

Accounting for the shared albedo-light scale lowers flux error from 0.464 to 0.182. Applying the same correction to the next-best primitive preserves the advantage of full AEGIR (0.182 vs. 0.321).

Table 4 of the submitted paper compares emitted power (Flux) with the ground truth directly in absolute linear units, providing the strictest test of a physical quantity. As you suggested, much of the 0.464 reflects the scale ambiguity between albedo and illumination. This comparison is also affected by the brightness and color scale ambiguity between materials and lighting, which LDR images cannot resolve (an albedo darker by a factor k, lit by emitters brighter by k, can produce similar images). Emitter positions and sizes do not depend on this scale and are recovered accurately (Table 4). New in this rebuttal (revised Sec. 4.4, App. C, Table R1), we quantify the ambiguity in two ways.

  • Rescaling with the albedo scale (0.464 → 0.182), without retraining. For flux evaluation, we estimate the correction from linear RGB albedo. Using the notation in the paper, a denotes the estimated albedo and a^GT the ground-truth albedo. For each scene and color channel c, we fit k_c = Σ a_c a_c^GT / Σ a_c², with the sums taken over all test pixels. We average the three channel factors into one scene factor, k = (k_R + k_G + k_B) / 3, then divide the recovered power of every emitter by k before recomputing the flux error. This applies one brightness correction to the whole scene, reducing our flux error from 0.464 to 0.182 while preserving relative emitter powers and leaving color errors in the metric. The same procedure reduces the flux error of the next-best primitive, Anisotropic Gaussian (ρ = 1), from 0.650 to 0.321. Full AEGIR still achieves the lowest error after rescaling (0.182 vs. 0.321).

  • Training with HDR inputs. During development, we trained AEGIR on the corresponding HDR observations of the same four FIPT-Synthetic scenes. These observations retain intensity information lost in saturated LDR pixels. HDR reduces the raw flux error from 0.464 to 0.296 and the rescaled error from 0.182 to 0.109. The remaining rescaled error of 0.109 may reflect approximations in our rendering model relative to the Mitsuba reference renderer. Position and size errors remain essentially unchanged at 0.085 / 0.065. We chose LDR inputs to support a wider range of real-world captures. This is also related to reviewer nroG Q4.

AuthorsOpenReview ↗

Author Response (2/3): Geometry Analysis

W2) Geometry:

Every listed per-scene geometry metric improves after joint optimization, and full AEGIR gives the most accurate geometry among the compared methods.

Table S2 of the submitted supplement (App. D.2) reports geometry before and after joint optimization, when geometry, materials and emitters are optimized together. Geometry is then fixed during the final emitter-only stage. As requested, we now provide per-scene results in Tables R3 and R4 of the revised Appendix D.2, using the saved checkpoints without retraining. Table R3 reports normal and depth errors for the six Hypersim and four FIPT-Synthetic scenes. Table R4 reports depth errors for the four ScanNet++ and two FIPT-Real scenes. To save space in the rebuttal, we show only the four FIPT-Synthetic results below. The remaining scenes are reported in Tables R3 and R4 of the revised PDF.

Scene (DN-Splatter initialization → AEGIR)Normal RMSEMean angular errorDepth AbsRelDepth RMSE
bathroom0.250 → 0.24517.14° → 16.52°0.087 → 0.0800.306 → 0.291
bedroom0.234 → 0.22716.03° → 15.29°0.076 → 0.0740.285 → 0.261
kitchen0.176 → 0.17012.34° → 11.41°0.060 → 0.0540.216 → 0.200
livingroom0.152 → 0.13010.69° → 9.01°0.052 → 0.0490.167 → 0.162

New in this rebuttal (Table R2), we also compare the methods on the same ten synthetic scenes. GS-ID starts from our DN-Splatter initialization, while IRGS and NeILF++ reconstruct their own geometry. The two AEGIR variants use the same geometry initialization, material model, renderer and training schedule as full AEGIR, but replace our anisotropic emitters with point lights or isotropic area emitters (Table 4).

Method (10 synthetic scenes)Normal RMSEMean angular errorDepth AbsRel
DN-Splatter initialization (Table S2)0.20314.11°0.067
GS-ID (App. D.2)0.19913.95°0.070
IRGS0.24919.88°0.114
NeILF++0.23518.91°0.119
Ours with point lights0.19813.87°0.069
Ours with isotropic area emitters0.19113.05°0.066
AEGIR (Table S2)0.18712.70°0.063

AEGIR gives the most accurate geometry on all three metrics. Among variants with the same initial geometry, more accurate light models lead to more accurate geometry. Our emitter model therefore helps reduce geometry errors.

AuthorsOpenReview ↗

Author Response (3/3): Input Coverage, Depth Initialization, and Outdoor Scenes

W3) Input coverage:

We evaluate hybrid illumination through our baselines, and the uniform-grid ablation shows that initial emitters need not come from projected bright pixels.

Hybrid illumination. GS-ID and SGS-Intrinsic combine an environment map with local point lights (App. C.2). AEGIR outperforms both in novel views on all five benchmarks and in albedo on both synthetic ones (Tables 2 and 3). AEGIR uses a unified emitter representation for local light sources and incoming illumination, with emitters at the windows capturing light entering the scene.

Emitter initialization. Our no-emitter-initialization ablation starts from a uniform grid instead of projected bright pixels and gives nearly the same albedo accuracy (Table S5). Emitter position / size / linear flux errors are likewise comparable (0.088 / 0.065 / 0.468 vs. 0.086 / 0.065 / 0.464 with bright-pixel initialization). This demonstrates that AEGIR can perform inverse rendering without bright-pixel unprojection, so initial emitters need not correspond to directly observed light sources. The emitters are then optimized freely with splitting and pruning (Sec. 3). Even when a light is not directly visible, the brightness and shadows it produces on visible surfaces can help estimate its position and intensity.

View splits. All methods, including every baseline, use the same training and test views of each dataset (Table S1, App. C.1), and the revised App. C.1 now states these splits.

DatasetTraining views per sceneTest views per sceneTest views chosen as
FIPT-Synthetic101–106101–107 (bathroom 6)official held-out views
Hypersim93–954–5official held-out views
ScanNet++59–13733–76every other key frame
FIPT-Real95–13996–139official held-out views
Replica6464 (separate from training)uniformly sampled from about 2,000 frames per scene

Q1) Depth used for initialization:

We use the z-depth predicted by the initial DN-Splatter geometry, in scene units, and unproject each bright pixel with the camera intrinsics and pose. No ground-truth depth is used. The revised Sec. 3 now states this.

Q2) Outdoor scenes:

AEGIR targets indoor scenes lit by local emitters, while an environment map is a more natural fit outdoors, especially in daytime. New in this rebuttal (revised App. E), AEGIR reaches 18.91/0.76/0.34 in novel views on the Mip-NeRF 360 garden scene, a preliminary result showing that AEGIR can also run outdoors. However, a hybrid model, as you suggest, may perform better in these settings. Runtime is the same as indoors (40 minutes of optimization, 55 with initialization, App. C.5) and can grow with the number of emitters.

We hope this addresses your concerns, and we are happy to answer any further questions.

Reviewer_nroGOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework to perform inverse rendering and relighting for indoor scenes, using Gaussian Splatting (GS) as the backbone and jointly optimizing the color, geometry, material and lighting (emitter) of the scene via differentiable rendering. Specifically, this paper focuses on the problem of light sources (emitters) within the indoor scene, which is crucial to lighting reconstruction and existing illumination models (point lights, environment maps, etc.) cannot effectively handle. To solve this, this paper proposes a new anisotropic Gaussian-based area emitter representation, parameterizing several emitter properties, including angular spread and falloff. The emitter parameters are initialized and optimized jointly during the GS training. Experimental results show superior intrinsic decomposition and rendering results to baseline methods.

Critical strengths

The problem of emitter representation in indoor scene inverse rendering is real and significant. Indoor scene illumination exhibits strong spatial variation and complex effects like soft shadows that point emitters or environment maps cannot capture effectively. The proposed area-emitter representation is a reasonable solution to this issue, and its design explicitly parameterizes several emitter properties, enabling it to capture more complex lighting effects.

Critical weaknesses

I list the main weaknesses here, while minor weaknesses go to "Questions":

  1. The bright-pixel-guided emitter initialization is not robust and is prone to errors, with counterexamples from both sides: (i) False positive: There are lots of bright regions in the image that are not caused by emitters, but by strong direct lighting and reflections, e.g., a white wall directly lit by a light. In LDR images, these regions may be overexposed and have a (255, 255, 255) RGB value, which will be falsely considered as emitters if "bright pixel" is the criterion; (ii) False negative: There may be some lights that do not have a large emission intensity, e.g., night lights or some occluded light sources, resulting in pixels without a bright RGB value. That said, I acknowledge that there is a correlation between pixel brightness and the emitter, but the correlation may be weak and unreliable to serve as a criterion.
  2. An important existing paper on the same task of indoor scene inverse rendering using a similar approach, I^2-SDF [1], is not cited or mentioned. That paper also explicitly represents area emitters within the scene, using a similar "unproject to 3D + clustering" (see Sec. 4.2 of their paper) to identify emitters, although they require an explicit emitter mask as input rather than using the bright-region criterion, and do not represent as rich emitter properties as this paper. However, this still challenges this paper's originality and novelty.
  3. The Gaussian-based emitter representation can only represent elliptical-shaped emitters, while having large errors for polygonal-shaped emitters that are also very common in indoor scenes. Sec. 5 of this paper has already acknowledged this, but I still posit it as a major weakness here.

[1] Zhu, Jingsen, et al. "I 2-SDF: Intrinsic indoor scene reconstruction and editing via raytracing in neural SDFs." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023.

Questions for authors
  1. Relighting is one of the most important applications of inverse rendering, but the paper only includes results in the appendix and does not provide comparisons against baselines. Can the authors include such comparisons during the rebuttal?
  2. Is the NVS part within the framework essentially identical to the vanilla 2DGS? What is this paper's contribution? This paper's contribution should focus on inverse rendering and relighting, while NVS is less informative.
  3. This method is a per-scene, optimization-based method. How does it compare to zero-shot or few-shot feed-forward methods based on large Transformer-based foundation models?
  4. How does this method handle the LDR-HDR problem? Lighting reconstruction typically requires HDR intensities, while input images are LDR RGB values. This is also related to my Critical Weakness #1. A simple inverse-gamma correction may not be sufficient.
7 comments
AuthorsOpenReview ↗

Author Response (1/3): Initialization and Relation to I²-SDF

We thank the reviewer for the careful review. All additions are now in the revised PDF, in green.

W1) Bright-pixel initialization:

A uniform-grid initialization changes albedo by only 0.03 dB, and adding false emitters or removing a dim source leaves accuracy nearly unchanged.

The submitted supplement evaluates initialization without bright-pixel detection in Table S5 (App. D.6, “W/o Emitter Initialization” row). Starting from 27 spheres on a uniform grid instead of bright pixels gives almost the same albedo (17.30 vs. 17.33 dB) and similar emitter position / size / flux errors (0.088 / 0.065 / 0.468 vs. 0.086 / 0.065 / 0.464). Bright pixels are only a starting point (Sec. 3). Candidates must stand out locally and agree across views, so view-dependent reflections rarely survive. All emitter parameters are then optimized, with splitting and pruning adjusting the emitter count. A false emitter on a bright wall is not supported by the shading across views, so it tends to be dimmed and pruned. New in this rebuttal, we perturbed the initialization directly on FIPT-Synthetic (Table R8).

InitializationNovel views P/S/LAlbedo P/S/LPositionPower (Flux)# emitters
Ours (Table 4)22.53/0.86/0.1917.33/0.80/0.210.0860.46412.8
+ 3 false emitters on bright surfaces22.51/0.86/0.1917.34/0.81/0.210.0870.46612.8
− dimmest detected emitter22.61/0.86/0.1917.31/0.80/0.210.0830.46512.6

The injected false emitters are removed and the emitter count is unchanged. Removing the dimmest detected emitter changes novel views by only +0.08 dB, within seed variance (±0.14 dB, App. C.5).

W2) Relation to I²-SDF:

AEGIR jointly optimizes independent explicit area lights with their own geometry and angular emission profile, without requiring ground-truth emitter masks or restricting emitters to reconstructed scene surfaces.

We acknowledge I²-SDF as closely related work that also models emission. Its emission MLP predicts whether a 3D point belongs to an emitter, with supervision from ground-truth or annotated emitter masks. These points are clustered, with one radiance value per cluster. However, our contribution is the joint optimization of independent emitter geometry and angular emission from multi-view images. Each emitter has its own learnable position, spatial extent, orientation and angular profile, separate from the scene surface representation. This added expressiveness makes optimization more challenging, which we address through our regularization and staged optimization. These explicit light sources require no emitter masks, are optimized freely, and can be reused for controlled relighting and editing.

I²-SDFAEGIR (ours)
InputLDR images, depth and normal priors, ground-truth or annotated emitter masksLDR images, DN-Splatter initialization (App. C.4), material priors as soft regularizers, no emitter masks
Representationimplicit neural SDFexplicit 2D Gaussians (Sec. 3)
Emittersurface clusters with one radiance value per clusterindependent explicit area light with learnable geometry and an angular emission profile
Emitter discoveryMLP supervised by emitter masks, followed by clusteringbright-pixel initialization, followed by optimization, splitting and pruning (Sec. 3)

In the controlled ablation, our emitter model gives the best albedo, emitter and novel-view metrics among all tested emitter models (Table 4), supporting better material-lighting disentanglement. AEGIR also outperforms the more recent EAG-PT and IRIS, which also use mask-based surface emission, on every reported synthetic albedo and controlled relighting metric (Tables 2 and S4).

Comparison using the released I²-SDF reconstruction code. The I²-SDF paper does not report quantitative material or lighting metrics, so we compare novel views and geometry using its released reconstruction code. The released I²-SDF code fits an implicit SDF and a view-dependent appearance network, producing geometry and novel views under the captured lighting. It does not include the full material and lighting decomposition needed for relighting. We evaluated it on the same four FIPT-Synthetic scenes with ground-truth depth, normals and emitter masks, using the same view and normal metrics as AEGIR. It reaches 21.46/0.82/0.39 in novel-view PSNR/SSIM/LPIPS versus 22.53/0.86/0.19 for AEGIR, with normal mean angular error of 19.01° versus 13.06°. AEGIR renders its novel views through physically based rendering of recovered materials and emitters, which also support relighting and editing. We agree that I²-SDF should be cited and discussed, and have added it to the revised Sec. 2 and App. C.2.

AuthorsOpenReview ↗

Author Response (2/3): Polygonal emitters, Relighting, and NVS

W3) Polygonal emitters:

We agree that a single ellipsoid cannot exactly reproduce polygonal corners. On rectangular and thin-strip lights, AEGIR outperforms the other emitter models on every metric reported in Table 4.

This limitation is noted in the submitted App. E. However, all FIPT-Synthetic emitters are rectangles, including thin strips. The axes of each ellipsoid are optimized independently, so it can flatten into a panel or stretch into a strip, and splitting covers larger lights. The main approximation is at the corners, which mostly affect the soft penumbra away from the light. On these rectangles, our emitters recover light size more accurately than spheres, flat disks and point lights (Table 4). One possible extension is to use our recovered geometry and emitters to initialize and fit superellipsoid lights (revised App. E).

Q1) Relighting comparison:

AEGIR leads all six baselines on all three relighting metrics, and the added color editing tests remain close to edited ground truth.

This comparison is reported in the submitted supplement (Table S4, App. D.5) and referenced in the main paper (Sec. 4.3). All six baselines are relit with the same moving light and scored against ground-truth Mitsuba renders, and AEGIR is best on all three metrics (21.93 dB vs. 21.54 for EAG-PT and 20.12 for IRIS). We have also added visual comparisons against ground truth for bedroom and livingroom scenes after Table S4 (Fig. R2, revised App. D.5). New in this rebuttal (Table R7), we further evaluate color editing of the recovered emitters against ground truth on FIPT-Synthetic, without retraining. This is also related to reviewer KskL Q2. With the same warm or cool tint on the ground-truth and our emitters, AEGIR reaches 21.74/0.84/0.20 (warm) and 21.79/0.85/0.20 (cool), less than 0.8 dB below the original lighting. We agree that relighting deserves more prominence and will move Table S4 into the main paper for the camera-ready version.

Q2) Is novel-view synthesis just 2DGS?

No. Novel views are physically rendered from recovered materials and lights, so they test material-lighting disentanglement as well as rendering quality.

We render each held-out view from the recovered materials and emitters, including traced shadows (Sec. 3.2, App. A.3). Thus, the evaluation tests the recovered scene decomposition. We agree that inverse rendering is the core problem. Our contribution is the explicit emitter model and a stable optimization for disentangling materials and lighting, evaluated by material accuracy (Table 2), emitter errors (Table 4), lighting accuracy with ground-truth geometry and materials (Sec. 4.2) and relighting against ground truth (Table S4).

AuthorsOpenReview ↗

Author Response (3/3): Feed-forward Models and LDR-HDR

Q3) Feed-forward transformer models:

The paper compares with DiffusionRenderer, and the added MVInverse comparison shows competitive albedo recovery against these large pretrained models.

The submitted paper compares with DiffusionRenderer, a feed-forward model built on the Cosmos video diffusion transformer that provides our material prior, and our optimization improves albedo recovery (Table S3, App. D.3). New in this rebuttal (Table R5), we add MVInverse, a recent feed-forward multi-view transformer for inverse rendering, on the scenes and test views of Table 2 (albedo P/S/L, using AEGIR checkpoints without retraining).

MethodFIPT-SyntheticHypersim
DiffusionRenderer15.41 / 0.80 / 0.2214.54 / 0.72 / 0.30
MVInverse15.21 / 0.79 / 0.2421.14 / 0.78 / 0.20
AEGIR (ours)17.33 / 0.80 / 0.2120.55 / 0.78 / 0.19

AEGIR is best or tied on every metric except Hypersim albedo PSNR, where MVInverse leads by 0.59 dB. These feed-forward models predict per-image materials, not editable 3D lights.

Q4) LDR vs. HDR:

We deliberately use LDR to handle a wider range of real-world captures, mapping rendered HDR to LDR only when computing the loss against LDR observations. Our method also supports HDR inputs.

We use captures such as ScanNet++ directly, without converting the inputs to HDR. Rendering uses unbounded emitter intensities and linear HDR light transport. For LDR supervision, we then apply the sRGB curve and clipping before the loss (revised App. A.3, App. B.1). A saturated pixel cannot reveal how bright the light really is. Instead, illumination on well-exposed surfaces across views constrains emitter intensity, while material priors constrain surface reflectance. HDR methods can degrade on LDR inputs. For example, FIPT, which is designed for HDR inputs, reaches 17.06/0.71/0.34 in novel views on LDR FIPT-Synthetic vs. our 22.53/0.86/0.19 (Sec. 4.3).

However, we agree that HDR preserves more intensity information. During the development of our method, we trained with HDR inputs on the four FIPT-Synthetic scenes with all other settings unchanged (Table R1).

InputAlbedo P/S/LEmitter position / size / power error
LDR (paper, Tables 2 and 4)17.33 / 0.80 / 0.210.086 / 0.065 / 0.464
HDR20.05 / 0.84 / 0.170.085 / 0.065 / 0.296

Emitter position and size remain essentially unchanged. HDR inputs better preserve intensity scale and dynamic range, reducing power error from 0.464 to 0.296 and improving albedo PSNR from 17.33 to 20.05 dB.

We hope this addresses your concerns, and we are happy to answer any further questions.

Reviewer_KskLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AEGIR, a Gaussian Splatting framework for indoor inverse rendering that jointly recovers geometry, materials, and explicit local area emitters from multi-view images. The emitters are modeled separately from the scene’s 2D Gaussian surface primitives, using anisotropic 3D ellipsoids and a Super-Gaussian angular emission profile. AEGIR combines differentiable deferred rendering, multiple importance sampling, diffusion-based material priors, targeted regularization, and a progressive optimization schedule. Experiments evaluate lighting estimation, intrinsic decomposition, novel view synthesis, controlled relighting, and virtual object insertion.

Critical strengths

Clear motivation and coherent method design. The paper connects the limitations of simplified illumination models to inaccurate indoor lighting patterns and residual illumination in estimated materials. Explicitly optimizing emitter extent and angular emission is a well-motivated approach to addressing these issues.

Compact and expressive emitter representation. Separately parameterizing spatial extent and angular emission provides interpretable controls over local illumination. The primitive ablation supports the effectiveness of the complete formulation, which achieves better rendering, material recovery, and emitter estimation with fewer emitter primitives. The comparison between fixed and learned falloff power also demonstrates the benefit of learning ρe\rho_e.

Well-motivated optimization strategy. Diffusion-based material priors and targeted regularization provide explicit constraints for the ambiguous joint estimation of geometry, materials, and illumination. The final emitter-only refinement stage allows lighting to be further optimized while keeping the recovered geometry and materials fixed.

Critical weaknesses

The source of the improvements over EAG-PT could be clarified. EAG-PT already represents finite-area emitters using emissive surface Gaussians and supports physically based relighting. The full-method comparisons additionally differ in BRDF models, emitter-mask requirements, material priors, and optimization strategies. Although these results support AEGIR as a complete system, they do not directly establish how much of its improvement over EAG-PT comes specifically from the independent emitter parameterization.

The contributions of spatial geometry and angular emission are not fully separated in the primitive ablation. Table 4 provides useful same-pipeline comparisons, but the transition from Diffuse 2DGS disks to Anisotropic Gaussian emitters changes both the spatial primitive and the angular emission model. The results demonstrate the benefit of the combined formulation and separately evaluate learning ρe\rho_e, but an intermediate variant using anisotropic 3D ellipsoids with direction-independent emission would further clarify the contribution of the complete angular emission profile.

Questions for authors

Could Table 4 include an intermediate variant using anisotropic 3D ellipsoids with direction-independent emission, while retaining the remaining pipeline? This would help distinguish the contribution of ellipsoidal geometry from that of the angular emission profile parameterized by σe\sigma_e and ρe\rho_e.

Figure S8 demonstrates color editing of the recovered emitters. Could matched ground-truth renderings and quantitative results be provided for this setting on synthetic scenes? This would complement the existing lighting-recovery evaluations and the novel-source replacement experiment in Table S4 by directly assessing the accuracy of editing the recovered original emitters.

4 comments
AuthorsOpenReview ↗

Author Response (1/1): Comparison to EAG-PT, Ellipsoid Ablation, and Color Editing

We thank the reviewer for the helpful suggestions. All additions are now in the revised PDF, in green.

W1) Where the gain over EAG-PT comes from:

Within the same framework, our emitter model improves novel views by 2.5 dB and albedo by 2.4 dB over diffuse surface disks.

Table 4 of the submitted paper (Sec. 4.4) isolates this contribution. Its diffuse 2DGS disk row is a representative variant of the emitter model in EAG-PT (flat emissive 2D Gaussians that emit equally in all directions) inside our system, with the same BRDF, material priors, optimization schedule and no emitter masks. It is not a reimplementation of EAG-PT, whose emitters are mask-supervised surface Gaussians. Our model also lowers all emitter errors and uses 12.8 instead of 1165.9 emitters. Thus, this controlled emitter comparison complements the full-system results against EAG-PT (Tables 2, 3 and S4).

W2/Q1) Ellipsoids with direction-independent emission:

The new ellipsoid variant separates the two gains, with shape improving compactness and materials, while angular emission further improves rendering and power accuracy.

New in this rebuttal, we trained this intermediate variant and added it to the revised Table 4, with the rest of the pipeline unchanged. It splits the step from diffuse disks to our emitters into a change of spatial primitive and a change of angular emission. This is also related to reviewer j5Pk W1.

Emitter (Table 4 row)Novel views P/S/LAlbedo P/S/LPositionSizePower (Flux)# emitters
Diffuse 2DGS disk20.01/0.73/0.3314.97/0.71/0.380.1080.1730.7921165.9
Ellipsoid, uniform (new, "Anisotropic, direction-independent")20.94/0.79/0.2916.65/0.76/0.290.1290.1020.66119.1
Ellipsoid, Gaussian profile (ρ = 1)21.43/0.80/0.2817.15/0.79/0.220.1060.0720.65014.3
Full AEGIR (learned ρ)22.53/0.86/0.1917.33/0.80/0.210.0860.0650.46412.8

Changing only the primitive (disk to ellipsoid, both uniform) mainly improves emitter compactness and material recovery (19.1 instead of 1165.9 emitters, +1.68 dB albedo, lower size and power errors). Position error rises because 1165.9 small disks cover the true sources densely. The angular profile then adds 1.59 dB in novel views and 0.68 dB in albedo and lowers size / power errors to 0.065 / 0.464, with the learned falloff ρ giving most of the novel-view and power gains. Both components contribute, with angular emission providing the larger reduction in power error.

Q2) Color editing against ground truth:

The color editing tests show close agreement with ground-truth renderings under the same emitter edits.

The submitted Fig. S8 (App. D.5) shows qualitative color editing. The revised figure adds matching ground-truth renders. Table R7 also reports quantitative results on the four FIPT-Synthetic scenes, using the same warm or cool tint on ground-truth and recovered emitters, without retraining.

LightingPSNR / SSIM / LPIPS
Original22.53 / 0.86 / 0.19
Warm tint21.74 / 0.84 / 0.20
Cool tint21.79 / 0.85 / 0.20

Recoloring lowers PSNR by less than 0.8 dB and SSIM by at most 0.02, while increasing LPIPS by at most 0.01. The edited scene therefore remains close to the edited ground truth, showing accurate emitter recovery.

We hope this addresses your concerns, and we are happy to answer any further questions.

Thank you for the clarification and additional experiments. The new ablations clarify the complementary roles of emitter shape and angular emission, and the added ground-truth renders strengthen the relighting evaluation. Table R7 primarily evaluates global color changes, so quantitative validation of individual-emitter editing remains limited.

Overall, the revisions address most of my concerns. I maintain my rating of 3 (Weak acceptance).

Thank you for the follow-up and for your positive assessment.

We want to clarify one more point. Our description of Table R7 was unclear. The edits are applied to individual emitters, not globally. As you requested, our primary goal was to quantify how changing the colors of individual emitters affects the re-rendering metrics against ground truth.

Each emitter is recolored separately with either a warm or a cool tint, with the same edit applied to the matching ground truth emitter. This follows the same procedure as the edit shown in Fig. S8, where the emitters are recolored separately and compared with the matching ground truth render. We have clarified this in Sec. 4.3, App. D.5 and the caption of Table R7 in the revised PDF.

We apologize for the confusion on this point and are happy to elaborate further.

Thank you for clarifying the editing protocol. The individual-emitter edits with matching ground-truth edits address my remaining question about Table R7. I have no further concerns to raise and maintain my rating of 3 (Weak acceptance).

Reviewer_j5PkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AEGIR, an indoor inverse rendering framework that jointly estimates geometry, materials, and compact local area emitters from posed multi-view images. Its main contribution is an emitter representation with independently optimized spatial extent and a learnable Super-Gaussian angular emission profile. The framework combines deferred rendering, Gaussian ray tracing, material priors, and staged optimization. The experiments demonstrate competitive rendering and material recovery.

Critical strengths

The emitter representation is well motivated. Separating source geometry from surface Gaussians provides compact and interpretable controls over light position, extent, and emission. The primitive ablation supports the practical value of the complete representation.

The evaluation goes beyond novel-view synthesis. The controlled Mitsuba experiment fixes ground-truth geometry and materials, allowing estimated illumination to be assessed separately. Source recovery metrics and replacement-light relighting provide complementary evidence.

The appendix contains useful diagnostics. The stage-wise analysis shows that optimized materials substantially outperform the raw diffusion prior. The paper also reports geometry refinement, component ablations, seed variability, and limitations. These analyses strengthen the empirical contribution.

Critical weaknesses
  1. Insufficient Separation of Spatial and Angular Contributions. The paper motivates finite emitters through their ability to represent near-field illumination and soft shadows. Although Table 4 supports the benefits of finite area, the transition from isotropic to anisotropic emitters changes both spatial shape and angular emission. Consequently, the contribution of each component to the overall improvement remains unclear.

  2. Limited Justification for the Final Emitter-Only Optimization Stage. Freezing geometry and materials during the final stage may reduce further illumination baking. However, this strategy may also force emitters to compensate for residual geometry or material errors. Table S3 demonstrates improved rendering after refinement, but does not establish whether freezing is preferable to continued joint optimization. A comparison under the same total optimization budget on existing synthetic scenes would strengthen the justification for this design. The comparison should assess material and emitter recovery alongside image quality.

  3. Potential Conflict between RGB-Guided Material Smoothing and Illumination Removal. The edge-aware material TV loss uses RGB gradients to preserve material boundaries. However, RGB gradients also reflect illumination changes, including shadow boundaries. The resulting weaker smoothing at such boundaries may preserve illumination artifacts in the estimated materials. Although the existing ablation supports the overall benefit of TV, it does not resolve this potential conflict. Visualizing RGB edges, ground-truth albedo, and predictions with and without TV on existing synthetic scenes would provide more targeted evidence.

  4. Unclear Trade-off between Emitter Compactness and Physical Accuracy. The scale penalty discourages excessively large emitters, but smaller emitters are not necessarily more physically accurate. This constraint may bias size estimates and encourage emission parameters to compensate. The current ablation removes emitter regularization as a whole, leaving the specific effect of the scale penalty insufficiently characterized. A focused ablation of this penalty on the existing FIPT-Synthetic scenes would clarify the trade-off. Comparing emitter size and flux errors alongside rendering quality would help assess whether compactness improves physical recovery.

  5. Limited Per-Scene Quantitative Analysis. The paper reports aggregate quantitative results without a per-scene breakdown of the main comparisons. These averages make it difficult to assess whether improvements are consistent across individual scenes. The overall gains could be concentrated in a subset of scenes, potentially obscuring challenging cases. Providing per-scene novel-view synthesis metrics, together with material recovery metrics where ground truth is available, would clarify the consistency of the reported advantages. Representative visualizations from existing scenes with large and small gains would further clarify the method’s applicability.

Questions for authors

All questions are detailed in the weaknesses above.

3 comments
AuthorsOpenReview ↗

Author Response (1/3): Spatial vs. Angular Contributions and Emitter-only Stage

We thank the reviewer for the constructive suggestions. All additions are now in the revised PDF, in green. We ran all four requested ablations and added per-scene results.

W1) Spatial vs. angular contributions:

The completed ablations separate shape from angular emission and show that full AEGIR gives the best rendering, albedo, and emitter recovery.

Table 4 of the submitted paper (Sec. 4.4) compares point lights, spheres, flat disks, Gaussian-profile ellipsoids and full AEGIR in the same framework. As you suggested, we trained the two missing variants (revised Table 4), which complete a 2×2 comparison of shape (sphere / ellipsoid) and angular emission (uniform / our profile).

Emitter (Table 4 row)Novel views P/S/LAlbedo P/S/LPositionSizePower (Flux)# emitters
Sphere, uniform ("Isotropic area")20.89/0.77/0.3516.98/0.78/0.270.1780.1910.78353.4
Sphere, our profile (new, "Isotropic area, learned profile")20.91/0.77/0.3216.99/0.78/0.270.1700.1870.76153.6
Ellipsoid, uniform (new, "Anisotropic, direction-independent")20.94/0.79/0.2916.65/0.76/0.290.1290.1020.66119.1
Ellipsoid, Gaussian profile ("Anisotropic Gaussian")21.43/0.80/0.2817.15/0.79/0.220.1060.0720.65014.3
Ellipsoid, our profile (full AEGIR)22.53/0.86/0.1917.33/0.80/0.210.0860.0650.46412.8

Shape and angular emission are complementary. With uniform emission, ellipsoids improve localization and compactness over spheres (position / size errors 0.178 / 0.191 → 0.129 / 0.102, 53.4 → 19.1 emitters), while novel views stay similar and albedo drops by 0.33 dB. Adding our angular profile to ellipsoids then improves novel-view and albedo PSNR by 1.59 and 0.68 dB and lowers power error from 0.661 to 0.464. Adding the profile to spheres alone changes little.

W2) Final emitter-only stage:

For the same number of iterations, emitter-only refinement gains 0.99 dB versus 0.34 dB for continued joint optimization and lowers every emitter error.

Table S3 of the submitted supplement (App. D.3) shows that this stage raises novel-view PSNR by about 1 dB on the ten synthetic scenes with materials fixed. New in this rebuttal (Table R9), we compare it with continuing joint optimization for the same total budget (20k iterations) on FIPT-Synthetic.

Final stageNovel views P/S/LAlbedo P/S/LRoughness MSEPosition / size / power
After 15k joint iterations (shared start)21.54/0.82/0.2217.33/0.80/0.210.0920.092 / 0.075 / 0.481
+ 5k joint iterations (new)21.88/0.83/0.2217.29/0.80/0.210.0920.095 / 0.076 / 0.484
+ 5k emitter-only iterations (ours)22.53/0.86/0.1917.33/0.80/0.210.0920.086 / 0.065 / 0.464

Starting from the same checkpoint, our emitter-only stage raises novel-view PSNR by 0.99 dB and lowers all emitter errors, while continuing joint optimization gains only 0.34 dB, slightly worsens the emitter errors and lowers albedo by 0.04 dB. The 0.65 dB advantage exceeds the reported seed variability of ±0.14 dB (App. C.5). With materials fixed and all emitter errors decreasing, these results do not indicate compensation for material errors in the tested scenes. Both branches resume the same checkpoint and optimizer state, and the joint branch holds the geometry and material learning rates at 10⁻⁶ and 10⁻⁴, one tenth of their initial values. We therefore keep the emitter-only stage.

AuthorsOpenReview ↗

Author Response (2/3): TV and Compactness Ablations

W3) Image-guided TV and shadow edges:

TV reduces shading leakage, and the edge guide ablation shows comparable material accuracy when using input images or material priors as the guide.

As described in the submitted paper (Sec. 3.3, App. B.3), the material TV term mainly suppresses color inconsistencies across views in the material diffusion priors (Fig. S3), and Table S5 and Fig. S12 in the submission evaluate its effect. New in this rebuttal (Table R10), to address shadow edges in the input image, we keep the TV term and change only its edge guide.

Material TVAlbedo P/S/L
None (Table S5)15.96/0.75/0.25
No edge guide (new)16.51/0.77/0.23
Guided by the shadow-free prior albedo (new)17.35/0.80/0.21
Guided by the input image (ours, Table S5)17.33/0.80/0.21

Guiding the TV term matters, as smoothing without a guide also blurs true material boundaries and lowers albedo PSNR by 0.82 dB. Using the material priors as the guide changes PSNR by only 0.02 dB. We therefore retain the input image, which is available for every scene and shows no measurable aggregate penalty from shadow edges. The requested figure is now in the revised appendix (Fig. R1). In the shown examples, TV suppresses shadow and shading artifacts in the albedo. Without TV, some shading remains, including a dark band across the bed in the bedroom scene.

W4) Compactness vs. accuracy:

Our chosen area penalty gives the best image metrics and the lowest size and power errors across the tested settings.

As you note, Table S5 removes all emitter regularization at once. New in this rebuttal (Table R11), we vary only the size penalty λ_area on FIPT-Synthetic.

λ_areaNovel views P/S/LAlbedo P/S/LSize errorPower errorLargest emitter (% of scene diagonal, as in Table S6)
019.25/0.72/0.3517.25/0.79/0.230.1850.48818.61%
0.0521.19/0.81/0.2517.28/0.79/0.220.0970.4719.12%
0.5 (ours, Table 4)22.53/0.86/0.1917.33/0.80/0.210.0650.4643.75%
520.45/0.80/0.2517.21/0.79/0.220.0890.4761.92%

Without the penalty, emitters grow, size error nearly triples and novel-view PSNR drops by 3.3 dB. A strong penalty shrinks them and raises size and power errors. Across settings, power error ranges from 0.464 to 0.488 and is lowest at our chosen penalty. Thus, better size accuracy does not come at the expense of power accuracy.

AuthorsOpenReview ↗

Author Response (3/3): Results Per Scene

W5) Per-scene results:

AEGIR improves albedo on all ten synthetic scenes and novel-view PSNR on 20 of 24 scenes.

As you requested, we provide per-scene results for the 24 scenes listed in Table R6 of the revised appendix, computed from the saved submission checkpoints without retraining. Each column compares AEGIR with the baseline that has the best dataset average (named in the average rows). To save rebuttal space, we show Hypersim and FIPT-Real below and report FIPT-Synthetic, Replica, and ScanNet++ in Table R6.

SceneNovel views, ours / best baselineAlbedo, ours / best baseline
ai_001_00421.76 / 21.5820.48 / 17.95
ai_004_00220.42 / 20.2519.18 / 16.25
ai_010_00622.15 / 22.0121.84 / 18.78
ai_015_00320.95 / 20.8119.45 / 16.88
ai_032_00122.54 / 22.3422.03 / 19.14
ai_034_00221.18 / 21.0520.32 / 17.02
Hypersim average21.50 / 21.34 (EAG-PT)20.55 / 17.67 (GS-ID)
classroom (FIPT-Real)21.99 / 21.98—
conferenceroom (FIPT-Real)22.15 / 21.94—
FIPT-Real average22.07 / 21.96 (IRIS)—

Against the best baseline by dataset average PSNR for each metric, AEGIR leads novel views on 20 of 24 scenes and albedo on all ten synthetic scenes. On FIPT-Synthetic, IRIS leads novel-view PSNR on the four scenes with a small average margin, which we attribute to IRIS baking residual lighting into its albedo (Sec. 4.3, Fig. S4), while AEGIR leads albedo PSNR on all four against EAG-PT (Tables 2 and R6). Qualitative comparisons are provided in the submitted paper (Figs. S5 and S6).

We hope this addresses your concerns, and we are happy to answer any further questions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of our response and revision

We thank all reviewers for their careful review and constructive suggestions, which have helped us improve the paper and its presentation.

The revised PDF includes the additional experiments and analyses, with additions marked in green (Tables R1–R11 and Figures R1–R2). The main points are:

  • Shape vs. angular emission (j5Pk, KskL). Two new variants separate their contributions. Direction-independent ellipsoids improve compactness and localization over uniformly emitting spheres. Adding our angular profile to ellipsoids then improves novel view synthesis and albedo PSNR.

  • Relation to I²-SDF (nroG). We now cite and discuss I²-SDF and compare against its released reconstruction-only code. I²-SDF uses emitter masks to identify and cluster emitters restricted to reconstructed surfaces, assigning one emission value per cluster. AEGIR instead jointly optimizes independent, editable lights with their own geometry and angular emission profiles, without emitter masks or restricting emitters to scene surfaces.

  • Initialization robustness (nroG, Eav3). A uniform-grid initialization, injected false emitters and a removed dim emitter all leave accuracy nearly unchanged, supporting recovery without bright-pixel unprojection. We use bright-pixel initialization primarily to speed up optimization.

  • Power error and geometry (Eav3). We retain absolute-scale flux evaluation as the strictest test. However, accounting for the shared albedo-light brightness scale lowers the flux error from 0.464 to 0.182, with AEGIR remaining best after alignment (next-best primitive 0.321). Every listed per-scene geometry metric also improves after our joint optimization.

We also included per-scene results, HDR experiments, relighting visual comparisons after Table S4, ablations of the final optimization stage, TV guidance and the area penalty, color editing against ground truth, and a comparison with MVInverse.

We thank the area chair and all reviewers for their time and consideration. We would be happy to address any further questions during the discussion period.

AuthorsOpenReview ↗

Summary of the discussion and revision

We thank the Area Chair and all reviewers for their time and engagement during the discussion period. We briefly summarize the discussion and changes incorporated into the revised PDF, with updated text marked in green.

  • Requested ablations (j5Pk, KskL, nroG). Two additional variants in Table 4 separate spatial shape from angular emission and clarify their complementary contributions. We also added final stage, TV guide and area penalty ablations, initialization robustness analyses, and per-scene results for all 24 scenes.

  • Relation to I²-SDF and positioning (nroG). The revision discusses I²-SDF and compares against its released reconstruction-only code. We more clearly distinguish our joint optimization of independent emitter geometry and angular emission without emitter masks, while clarifying the contribution scope and limitations.

  • Emitted power and geometry (Eav3). Accounting for the shared albedo-light brightness scale lowers flux error from 0.464 to 0.182, with AEGIR remaining best after alignment. Every reported per-scene geometry metric also improves after joint optimization.

  • Relighting (nroG, KskL). AEGIR leads all six baselines on the three controlled relighting metrics (Table S4). The revision adds visual comparisons (Fig. R2) and quantitative individual emitter color editing against matching ground truth (Table R7).

We also clarified evaluation terminology, HDR inputs, MVInverse comparison, view splits, initialization depth, and outdoor applicability.

We appreciate the reviewers’ follow-up feedback. Reviewers Eav3 and KskL confirmed that their concerns were addressed. Moreover, Reviewer nroG stated that their major concerns were addressed and raised their rating.

We thank the Area Chair and all reviewers for their constructive feedback, which has strengthened the paper, and remain happy to clarify any remaining point.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.