acceptodds
Under review as a conference paper at ICLR 2027

Uncertainty- and Parallax-Aware Gaussian Splatting for Indoor Object-Centric Reflective Scenes

Abstract

Specular reflections remain a major challenge for novel view synthesis, requiring both reliable localization of reflective regions and accurate modeling of their view-dependent appearance. However, reflective regions are difficult to localize consistently across views, while existing reflection models often assume distant environment lighting. This assumption breaks down indoors, where finite-distance scene geometry induces strong reflection parallax, leading to baked-in appearance and view-inconsistent artifacts. To address these challenges, we leverage learned photometric uncertainty to derive metallic supervision through Uncertainty-Guided Metallic Supervision (UGMS), enabling stable diffuse-specular decomposition, and incorporate Parallax-Corrected Cubemaps (PCC) to account for finite-distance environment lighting. We further introduce IndoorRef12, a synthetic benchmark featuring near-field reflection parallax and cluttered backgrounds. It comprises 12 diverse indoor environments with varying scene layouts, reflective-object configurations, and background complexity. Extensive experiments demonstrate substantial improvements in reflection reconstruction, particularly in challenging indoor scenes with strong near-field parallax. We will release our code and IndoorRef12 dataset.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Novel View Synthesis, Reflection Modeling, Inverse Rendering, Specular Reflection, Indoor Scenes, Uncertainty Estimation, Benchmark Dataset

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ncs9OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper mainly studies how to reconstruct indoor scenes with highly reflective objects using Gaussian splatting. The paper proposes a method that combines learned uncertainty, visibility filtering, and SAM segmentation to generate masks for reflective metallic objects. These masks supervise the metallic parameters in a subsequent rendering stage, which uses parallax-corrected cubemap sampling to model near-field reflections. The authors also introduce the IndoorRef12 dataset and report improvements on both synthetic and real scenes.

Critical strengths
  1. The paper designs a detailed pipeline for metallic object localization and mask generation. The combination of uncertainty estimation, visibility filtering, spatial candidate selection, and SAM segmentation is carefully considered. The experiments show that the generated masks provide useful supervision and improve reconstruction quality in the evaluated scenes.

  2. The paper evaluates not only overall image quality but also reflective-region rendering and surface normals. These results help in understanding how the method improves the reconstruction of reflective objects.

  3. The paper includes a detailed discussion of its limitations. The authors discuss the ambiguity of uncertainty, inaccurate material boundaries in object-level masks, dependence on COLMAP and SAM, and limitations of a single cubemap in handling complex reflections. They also consider cases without strong reflectors and cases with changing illumination. This analysis helps clarify the conditions under which the method may fail.

Critical weaknesses
  1. The introduction states that “most methods assume an infinite distance environment map” and motivates PCC as a way to address near-field reflection shifts. The limitation of direction-only environment maps is reasonable. However, SpecTRe-GS already uses secondary ray tracing to capture nearby reflected objects, and Ref-GS models spatially varying specular appearance. Therefore, near-field reflection modeling has already been considered in prior work. Methods that explicitly trace rays through scene geometry also account for local geometry and visibility more directly than a proxy box, although this does not necessarily mean better reconstruction results. Since PCC itself is an established technique, it remains unclear to me what advantage its use here provides over these existing solutions.

  2. Table 4 shows the benefits of UGMS and PCC within the proposed pipeline, while Table 6 compares different mask generation methods. However, these experiments do not sufficiently distinguish whether the improvement over other baselines mainly comes from better masks, stronger metallic supervision, or the reflection rendering method. In particular, it remains unclear whether a ray-tracing-based reflection model, such as those used in SpecTRe-GS or MaterialRefGS, could achieve similar or better results under the same masks, supervision, and otherwise matched settings. If the advantage mainly comes from more accurate masks and stronger supervision, the evidence for the reflection rendering formulation as a separate methodological contribution would be limited. The paper’s contribution would then rest primarily on the mask generation and supervision pipeline.

  3. The mask generation pipeline relies on a heuristic connection between high uncertainty and reflective regions. However, uncertainty is learned from reconstruction error and is not specific to reflections. The visibility-based weighting is intended to suppress unreliable candidates that appear in only a few views, but it does not distinguish the causes of high uncertainty. In real captures, exposure or white balance variations, changing illumination or shadows, and persistent errors in camera poses or geometry can also produce reconstruction inconsistencies. These effects may occur in regions that remain visible across many views and therefore would not necessarily be suppressed by the visibility weighting. It remains unclear to me how reliably the selected prompts identify reflective objects under these conditions. Although the authors acknowledge related limitations, the current evaluation provides limited direct evidence of mask accuracy and robustness across more diverse real scenes. Together with W2, this becomes particularly relevant: if the performance gains mainly come from the masks and their associated supervision, the reliability of this heuristic mask generation pipeline would be central to the paper’s main contribution.

Questions for authors
  1. Regarding Weakness 1, I would suggest revising the introduction to acknowledge existing work on near-field reflections, including SpecTRe-GS and Ref-GS. It would help to distinguish the limitations of direction-only environment maps from the capabilities of methods that already model spatially varying reflections, and clarify what remains unresolved by these approaches.

  2. Regarding Weakness 2, could the authors compare PCC with a scene-based ray-tracing alternative while keeping the UGMS masks, metallic supervision, geometry representation, material parameterization, and training settings fixed where possible? This experiment would help determine whether the advantage comes primarily from the masks and supervision or from the reflection rendering formulation.

  3. Regarding Weakness 3, additional mask generation results on a broader range of real captures would strengthen the paper, particularly in scenes with competing sources of reconstruction inconsistency. Qualitative mask visualizations and, where annotations are available, quantitative mask accuracy would help assess the robustness of the uncertainty-guided selection beyond the current examples.

4 comments
AuthorsOpenReview ↗

Response to Reviewer ncs9 (1/2, W1 and Q1)

The authors would like to thank the reviewer for the constructive comments. We address each comment below.

All figures and videos referred to below are available on our anonymous rebuttal page (https://iclr2027-rebuttal-paper53498.netlify.app).

(W1-1, Q1-1) Acknowledge existing works on near-field reflections in the introduction

As per the reviewer’s suggestion, the authors will revise the introduction to distinguish the limitations of direction-only environment maps from the capabilities of methods that already model spatially varying reflections. We will also clarify what remains unresolved by these approaches and explain why we use PCC to address it. The revision is as follows.

(Before)

Second, most methods assume an infinite distance environment map, where lighting depends only on direction. While this is reasonable outdoors, it breaks down indoors because dominant reflectors such as walls and furniture lie at finite distances. As a result, direction only lighting fails to capture spatially varying near-field reflection shifts, leading to flattened highlights and misaligned reflection anchoring across views (Fig. 1: Right).

(After)

Second, indoor reflections often come from nearby structures such as walls and furniture. Therefore, rays leaving different surface points in the same direction can hit different parts of the scene. A direction-only environment map misses this variation, which can lead to misaligned reflections across views (Fig. 1: Right). Recently, SpecTRe-GS and MaterialRefGS model near-field reflections by tracing secondary rays through the reconstructed scene, so geometry errors along those rays can misplace reflected content. Ref-GS represents spatially varying specular appearance through learned features. The remaining challenge is to model near-field reflection shifts explicitly with less reliance on detailed scene geometry. We address this by integrating parallax-corrected cubemaps (PCC) into Gaussian inverse rendering, using a coarse proxy of the surroundings to adjust the cubemap sampling direction as the reflection point moves.

We will incorporate this revision into the introduction of the final version.

(W1-2, Q1-2) What remains unresolved by existing approaches and Advantage of PCC

As stated in the revision above, the remaining challenge is to predict where nearby structures appear in the reflection as the reflection point moves, with less reliance on detailed scene geometry.

SpecTRe-GS and MaterialRefGS trace secondary rays through the reconstructed scene to evaluate local geometry and visibility more directly. If a surface along the reflection path is missing or inaccurately reconstructed, the ray can miss that surface or intersect it at the wrong location.

Ref-GS captures spatially varying specular appearance through learned features, while its far-field lighting is sampled by reflection direction and roughness. The relationship between reflector distance and reflection shift is therefore implicit in the learned appearance.

The advantage of PCC in our Gaussian inverse renderer is that it models this relationship explicitly from a coarse representation of the surroundings. Two rays with the same direction can intersect the proxy at different locations when they originate at different surface points. PCC uses those intersections to adjust the cubemap sampling direction and approximate where reflected content should appear.

For qualitative comparisons, in the indoor scenes shown in Video 3 on our anonymous rebuttal page LINK, our method better preserves the placement of reflected details across views, while the evaluated ray tracing baselines produce blurred or misplaced reflections. In addition, SpecTRe-GS takes a binary mask of the highly specular region as input. On the other hand, our method localizes the reflective regions automatically.

AuthorsOpenReview ↗

Response to Reviewer ncs9 (2/2, W2, W3, Q2, and Q3)

(Continued from the previous comment.)

(W2, Q2) Controlled comparison of PCC with scene-based ray tracing

We compared PCC with the scene-based ray tracing formulation used in MaterialRefGS within our Gaussian inverse rendering pipeline. All variants used the same UGMS masks, metallic supervision, geometry representation, material parameterization, and training settings.

On IndoorRef12, PCC (Ours) achieved 33.72 dB on full images and 23.82 dB on reflective objects. Scene-based ray tracing (MaterialRefGS) achieved 31.14 dB and 20.82 dB, respectively (Table R6). This comparison shows that the reflection rendering formulation contributes to the improvement under same masks and supervision.

Video 4 on our anonymous rebuttal page LINK shows that some reflected details in the ray tracing results are blurred or misplaced. One possible explanation is that secondary rays hit floaters or inaccurately reconstructed surfaces before reaching the intended geometry.

Table R6. Controlled comparison of reflection models within our Gaussian inverse rendering pipeline on IndoorRef12. Note that all three reflection models are evaluated using the same UGMS masks, loss weights, and training protocol.

Reflection modelFull image PSNR (dB) ↑Reflective object PSNR (dB) ↑
Direction-only environment map33.1222.70
Secondary ray tracing (MaterialRefGS)31.1420.82
PCC (Ours)33.7223.82

(W3-1) Uncertainty and the limits of visibility weighting

The authors agree that competing sources of reconstruction inconsistency can produce high uncertainty across many views and lead to prompts on non-reflective regions. In our controlled test, small camera pose perturbations had a limited effect on mask accuracy (Table R8). Larger pose errors and persistent illumination changes remain limitations (Appendix K.1, K.2).

On IndoorRef12, high uncertainty was 12.5 times as frequent on reflective objects as in the background, and all 96 selected prompts fell on reflective objects (Table S3). In the following (W3-2, Q3-1), we examine prompt selection and mask accuracy on a broader range of real captures with competing sources of reconstruction inconsistency.

(W3-2, Q3-1) Additional mask generation results on a broader range of real captures

We evaluated mask generation on three new real-world captures containing metallic and dielectric reflective objects. We used automatic exposure and white balance and estimated camera poses with COLMAP. Fig. 2 on the LINK shows the uncertainty maps, selected prompts, and resulting masks for two of the three captures.

As shown in Table R7, the generated masks achieved IoUs of 0.874 for Desk, 0.907 for TV, and 0.963 for Helmet, with a mean IoU of 0.915.

Table R7. UGMS mask IoU on three real captures. Mask IoU is measured against manual annotations on sampled test views.

SceneReflective objectsMaterialMask IoU ↑
DeskChrome, bellMetal0.874
TVTVDielectric0.907
HelmetHelmet shellDielectric0.963
Mean0.915

(W3-3, Q3-2) Quantitative mask accuracy for the robustness of the uncertainty-guided selection

We perturbed the training data for four IndoorRef12 scenes, testing ±20% exposure and white balance variations and 0.05° camera rotation noise separately (Table R8). We measured how well the uncertainty separates reflective objects from the background using AUROC. Exposure and white balance variations reduced AUROC from 0.948 to 0.906, while pose noise yielded an AUROC of 0.945. Mask IoU was 0.898 with the original data, 0.901 with exposure and white balance variations, and 0.884 with pose noise. Fig. 3 on our anonymous rebuttal page LINK shows the uncertainty maps and the resulting masks under each perturbation. Exposure and white balance variations raised the uncertainty almost uniformly over the whole image, and pose noise raised it along edges. In both cases, the uncertainty remained highest on the reflective objects, and the resulting masks were nearly unchanged.

Table R8. Uncertainty separation (AUROC) and mask accuracy of UGMS under exposure, white balance, and camera pose perturbations on four IndoorRef12 scenes (Rabbit, Chick, Apple, Knight).

ConditionAUROC ↑Mask IoU ↑
Original0.9480.898
Exposure and white balance (±20%)0.9060.901
Pose noise (0.05°)0.9450.884
Reviewer_ncs9OpenReview ↗

Comment by Reviewer ncs9

Thank you for your thorough response. My concerns have been adequately addressed, and I appreciate the additional clarifications and experiments. I am raising my score to 3 accordingly.

We sincerely thank the reviewer for carefully considering our response and for raising the score. We will incorporate the clarifications and additional experiments into the final version.

Reviewer_ouDjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a Gaussian Splatting method for highly reflective objects in indoor scenes, addressing inaccurate reflective-region localization and near-field reflection parallax. The authors introduce UGMS, which uses learned photometric uncertainty to generate metallic supervision, and PCC, which corrects environment-map sampling directions using proxy geometry. The paper also introduces the IndoorRef12 dataset and reports promising reconstruction results on both synthetic and real-world datasets.

Critical strengths
  1. The paper addresses two practical challenges in indoor reflective scenes. Using uncertainty to guide reflective-region localization and parallax correction to improve near-field reflections is a reasonable and well-motivated design.
  2. The experimental results are strong, particularly in image reconstruction quality, normal estimation, and reflective-region recovery. The paper also provides component ablations, material-swap experiments, challenging-scene evaluations, and extensive visual comparisons.
Critical weaknesses
  1. In Appendix C.2, UGMS achieves higher PSNR than supervision with ground-truth material masks even when its mask IoU is only 0.23. This suggests that coarse object-level supervision may already be sufficient to improve reconstruction, making the independent contribution of learned uncertainty unclear.
  2. Although Table 4 demonstrates improvements from PCC, the dedicated Near-field Furniture experiment does not include a comparison without PCC. Since parallax-corrected cubemaps are an existing technique, the paper would also benefit from more direct quantitative comparisons with related near-field reflection methods such as SpecTRe-GS.
  3. The experiments mainly evaluate reconstruction quality using PSNR, normal errors, and visual comparisons, without directly measuring the accuracy of diffuse/specular decomposition or recovered material properties. Since IndoorRef12 is synthetic, quantitative evaluation against ground-truth material attributes would provide stronger evidence for the claimed improvements in physical decomposition.
  4. Appendix D reports 30,000 iterations for Stage 1 uncertainty learning, while Table S6 in Appendix E reports 20,000 iterations. This inconsistency makes it unclear which training configuration was used for the reported performance and efficiency measurements.
Questions for authors
  1. Provide a controlled comparison between UGMS and accurate object-level mask supervision to clarify the independent contribution of learned uncertainty.
  2. Include an ablation without PCC in the Near-field Furniture experiment and evaluate its benefits at different reflector distances. Also provide a more direct comparison with SpecTRe-GS or other relevant near-field reflection methods.
  3. Report quantitative errors for recovered material parameters and diffuse/specular components using the ground-truth information available in synthetic scenes.
  4. Clarify the actual number of Stage 1 training iterations and the configurations used for the main experiments and runtime measurements.
6 comments
AuthorsOpenReview ↗

Response to Reviewer ouDj (1/2, W1 and Q1)

The authors would like to thank the reviewer for the constructive comments. We address each comment below.

(W1, Q1) Independent contribution of learned uncertainty

The authors compared UGMS with accurate object-level mask supervision on the same partially metallic scenes as in Appendix C.2. We replaced the UGMS mask with the ground-truth mask of the entire object and kept the reconstruction settings fixed. UGMS achieves 36.63 and 37.20 dB on the Half-metallic and Interleaved scenes, compared with 36.78 and 37.30 dB using ground-truth object masks (Table S4). On IndoorRef12, the corresponding scores are 33.72 and 33.90 dB (Table 6).

Object-level supervision can be sufficient to improve reconstruction. However, in practice, the reflective object must first be located from the input images. UGMS uses learned uncertainty to select point prompts for segmentation. SAM2 achieves an object mask IoU of 0.039 with Top-K prompts and 0.898 with uncertainty-guided prompts (Ours). Marigold and MaterialRefGS also achieve low IoUs of 0.263 and 0.040, respectively. Despite the low mask IoUs, both methods can still obtain relatively high PSNR and SSIM because the reflective objects occupy only a small part of the image.

Table S4. Results on partially metallic objects. Oracle is trained with the ground-truth mask of the entire object (object-level) or of the metallic region only (material-level). Mask IoU is measured against the metallic region. Refl. PSNR is computed on the ground-truth mask of the object.

GuidanceHalf-metallic Mask IoU ↑Half-metallic Full PSNR ↑Half-metallic Refl. PSNR ↑Interleaved Mask IoU ↑Interleaved Full PSNR ↑Interleaved Refl. PSNR ↑
None (w/o UGMS)–28.9221.89–28.0121.04
Ours (w/ UGMS)0.4536.6326.720.2337.2027.66
Oracle (object-level)0.4736.7826.800.2237.3027.77
Oracle (material-level)1.0036.7327.111.0036.6127.69

Table 6. Comparison of mask sources on IndoorRef12. The IoU is measured against the ground-truth object masks. Refl. PSNR is computed on the ground-truth mask of the reflective object.

Mask sourceIoU ↑PSNR ↑SSIM ↑LPIPS ↓Refl. PSNR ↑
SAM2 (Top-K)0.03924.380.9130.26320.11
Marigold0.26332.090.9570.19521.40
MaterialRefGS0.04031.270.9530.20420.54
Ground-truth object mask1.00033.900.9680.17323.95
UGMS-based object mask (Ours)0.89833.720.9630.17623.82
AuthorsOpenReview ↗

Response to Reviewer ouDj (2/2, W2 to W4 and Q2 to Q4)

(Continued from the previous comment.)

(W2-1) PCC ablation in the Near-field Furniture experiment

The authors added a comparison with and without PCC to the Near-field Furniture experiment (on Living Room–Near-field furniture (Appendix I.2)). For the ablation “without PCC”, PCC is replaced with an infinitely distant environment map and all other settings are fixed.

In the Near-field Furniture scene, PCC improves full-image PSNR from 33.36 to 33.91 dB and PSNR on the reflective object from 20.42 to 21.56 dB (Table 5).

Table 5. Results on the Near-field Furniture scene. Refl. PSNR is computed on the ground-truth mask of the reflective object.

LayoutMethodPSNR ↑SSIM ↑LPIPS ↓Refl. PSNR ↑
Near-field FurnitureRef-Gaussian30.310.9520.13118.63
Near-field FurnitureMaterialRefGS28.760.9410.14017.90
Near-field FurnitureRef-DGS33.110.9620.11519.21
Near-field FurnitureOurs (w/o PCC)33.360.9630.11320.42
Near-field FurnitureOurs (w/PCC)33.910.9660.10921.56

(W2-2) PCC benefit at different reflector distances

The authors evaluated PCC at three distances (near, mid, far) between the furniture and the metallic vase (Table R4).

Near is the original distance between the furniture and the metallic vase in Living Room–Near-field Furniture (Appendix I.2). Mid and Far are two and four times that distance, respectively.

Compared with the results without PCC, enabling PCC increased PSNR within the mask of the reflective object (i.e., the vase) from 20.42 to 21.56 dB at Near, 20.63 to 21.76 dB at Mid, and 20.71 to 21.79 dB at Far (Table R4).

Table R4. PSNR (dB) in the Near-field Furniture scene at three furniture distances. Full and Refl. denote the full image and the reflective object.

Reflection modelNear FullNear Refl.Mid FullMid Refl.Far FullFar Refl.
MaterialRefGS28.7617.9028.4818.0228.9017.99
Ours (w/o PCC)33.3620.4233.4020.6333.4220.71
Ours (w/PCC)33.9121.5633.9421.7634.1021.79

(W2-3, Q2) Direct comparison with a near-field reflection method

As per the reviewer’s comment, parallax-corrected cubemaps are an existing technique. They model how reflections of nearby objects change with the reflection point on the surface. Our contribution is to incorporate this geometric relationship into differentiable Gaussian inverse rendering. Thus, our method can explicitly model near-field parallax when learning scene appearance from the input images.

SpecTRe-GS’s code and evaluation scenes (both synthetic and real-world) are not publicly available.

To provide direct comparison with a near-field reflection method, the authors evaluated MaterialRefGS, a publicly available near-field reflection method based on ray tracing, to the same scenes at all three reflector distances (Table R4).

As seen in Table R4, our method achieved higher PSNR within the vase mask than MaterialRefGS at each distance, with 21.56 vs. 17.90 dB at Near, 21.76 vs. 18.02 dB at Mid, and 21.79 vs. 17.99 dB at Far.

(W3, Q3) Quantitative evaluation of material properties and diffuse/specular decomposition against GT

The authors rendered ground-truth roughness and metallicity maps and diffuse and specular components in Blender, then quantitatively evaluated the corresponding predictions on the reflective objects in IndoorRef12 (Table R5). Our method outperformed both Ref-Gaussian and MaterialRefGS on all four measurements. MSEs for roughness and metallicity were 0.0790 and 0.0192, respectively. PSNRs for diffuse and specular components reached 29.76 dB and 30.15 dB, respectively, while both baselines scored below 22 dB on both components.

Table R5. Material and decomposition accuracy on the reflective objects in IndoorRef12. Ground-truth reference maps and components were rendered in Blender.

MethodMaterial Roughness (MSE ↓)Material Metallic (MSE ↓)Decomposition Diffuse (PSNR ↑)Decomposition Specular (PSNR ↑)
Ref-Gaussian0.28980.741521.3720.62
MaterialRefGS0.17370.158821.2820.80
Ours0.07900.019229.7630.15

(W4, Q4) Inconsistent Stage 1 iteration count

The authors would like to thank the reviewer for pointing out this typo. The authors will correct the Stage 1 iteration count in Table S6 from 20,000 to 30,000 to match Appendix D.

Thank you for the detailed rebuttal and additional experiments. The new comparisons and quantitative evaluations have satisfactorily addressed most of my concerns.

I have one remaining clarification regarding W4/Q4: Could the authors confirm that the reported runtime and efficiency measurements were also obtained using the corrected 30,000 Stage 1 iterations?

Overall, I appreciate the authors' efforts and maintain my positive assessment of the paper.

Reviewer_SseVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces a system for improved specular reflection fitting in 3D Gaussian splatting. This is a heavily researched area; the improvements introduced in this work are a better localization of metallic objects, a cubemap approximation to incoming illumination that handles a certain amount of parallax from finite depth, and a new small dataset of synthetic indoor test scenes.

Critical strengths

The focus of the paper is certainly valuable: highly specular objects are indeed an important and unsolved issue in Gaussian splatting or any novel view synthesis methods.

The new dataset will be publicly released and is likely to be useful by further research.

The numerical metrics reported outperform previous methods, though only slightly.

Critical weaknesses

I feel the main problem with the paper is its low generality. The specular NVS field is full of competing methods that solve a subset of the problem. In my opinion, this paper is no different.

First, the GlossyReal dataset is from a 3 years old paper, and has only 5 objects, very clearly separated from their surroundings. I feel this is insufficient; showing results on complex real-world captures, indoor and outdoor, should be a requirement.

Second, the cubemap parallax solution is certainly an improvement on infinitely distant environments, but will be insufficient in any complex scenes: imagine an office environment with multiple cubicles and multiple specular objects on desks separates by cubicle walls, or multiple cars in a parking lot with trees in between parking spaces. A solution for localized incoming illumination is not explored.

Finally, the focus on metallicity is limiting. The focus on "Uncertainty-Guided Metallic Supervision" ("Localization" would be a more accurate term) misses the fact that many (if not most) specular objects in realistic scenes are not metallic; cars being a prime example. Appendix C.3 admits the limitation to clearly metallic objects.

Questions for authors

Would the method, with minor modifications, work on captures of highly polished dielectric objects (e.g. cars, TV/monitor screens)?

2 comments
AuthorsOpenReview ↗

Response to Reviewer SseV (1/2, W1 and W2)

The authors would like to thank the reviewer for the constructive comments. We address each comment below.

All figures and videos referred to below are available on our anonymous rebuttal page (https://iclr2027-rebuttal-paper53498.netlify.app).

(W1) Low generality and distinction from competing specular NVS methods

The authors believe the scope of our approach is broader than the indoor setting emphasized in the paper. Because the proposed method estimates the surrounding geometry from each scene, it can accommodate different layouts and reflective object configurations. Therefore, our approach could extend to even outdoor settings such as open-air stadiums and enclosed courtyards, where nearby structures can produce similar reflection parallax and can be approximated by a coarse proxy.

Our contribution is to model reflection parallax from nearby structures without requiring a detailed reconstruction of those structures.

Prior specular NVS methods often model environmental illumination as distant. This assumption can miss the parallax caused by nearby structures. Some methods consider this parallax by tracing reflected rays through the reconstructed scene, which makes the accuracy of the reflections depend on how well those structures are reconstructed.

In this work, we use coarse scene geometry to model how reflections shift across viewpoints and learn their appearance from the input images. This allows us to approximate reflection parallax even for objects whose geometry is not explicitly represented in the proxy.

(W2) Results on complex real-world captures, indoor and outdoor

We evaluated the effectiveness of our method on three additional complex real-world captures, both indoor and outdoor (Table R1). Our two new indoor captures, Desk and Cabinet, contain multiple reflective objects surrounded by everyday clutter. The outdoor scene capture is the Audi scene from EnvGS (Xie et al. 2025), a parking lot with several cars.

Our method obtains higher PSNR and SSIM than both baselines on all three scenes. It also obtains lower LPIPS on the Desk and Audi scenes and matches Ref-DGS on Cabinet (Table R1). We provide visual comparisons for both indoor and outdoor scenes on our anonymous rebuttal page in Fig. 1 and Video 2, respectively LINK.

Table R1. Quantitative comparison on new real captures.

MethodIndoor Desk PSNR ↑Indoor Desk SSIM ↑Indoor Desk LPIPS ↓Indoor Cabinet PSNR ↑Indoor Cabinet SSIM ↑Indoor Cabinet LPIPS ↓Outdoor Audi PSNR ↑Outdoor Audi SSIM ↑Outdoor Audi LPIPS ↓
MaterialRefGS31.600.9600.10930.380.9460.14127.840.8650.164
Ref-DGS33.590.9690.09333.170.9560.12328.020.8670.181
Ours34.200.9730.08733.980.9610.12328.870.8740.140
AuthorsOpenReview ↗

Response to Reviewer SseV (2/2, W3, W4, and Q1)

(Continued from the previous comment.)

(W3-1) Complex scenes with multiple reflectors separated by occluders

The Office scene that we captured covers an entire office with multiple reflective objects in cubicles separated by partitions. On this scene, our method achieves higher PSNR and SSIM and lower LPIPS than MaterialRefGS and Ref-DGS (Table R2 and Video 1). The outdoor example is the same Audi parking lot scene from EnvGS provided in W2 (Table R1 and Video 2)

Table R2. Quantitative comparison on the complex real scene, Office.

MethodPSNR ↑SSIM ↑LPIPS ↓
MaterialRefGS26.670.8940.231
Ref-DGS26.810.8960.227
Ours33.880.9550.201

(W3-2) Localized incoming illumination

In Appendix K.2, the authors mentioned a position-dependent lighting representation as a natural extension of the shared cubemap. In the Office scene, partitions block different parts of the surroundings for reflective objects in different cubicles. To deal with that, we apply the same PCC formulation separately to each object, using a cubemap to model its local incoming illumination and a proxy box fitted to its surroundings. We separate the objects in 3D using UGMS masks from multiple views, so this setup requires no additional annotations.

(W4) Material scope and terminology

The uncertainty signal is used to localize regions whose appearance changes consistently across views. In Appendix C.3, replacing the metallic targets with smooth dielectrics weakened this signal, and the prompts no longer reliably localized the targets. The real captures in Table R3 show that the same signal can also localize strongly reflective dielectric surfaces.

Therefore, as per the reviewer’s suggestion, we will use “Uncertainty-Guided Reflective Region Localization (UGR²L)” for this step and describe separately how the resulting masks supervise the metallicity parameter.

(Q1) Highly polished dielectric objects

Yes. Without any modifications, our method worked on real captures of a TV screen, a glossy helmet shell, and painted car bodies, using the same configuration as for the metallic scenes.

Our method achieved higher PSNR and SSIM and lower LPIPS than MaterialRefGS and Ref-DGS on all three scenes (Table R3 for TV screen and Helmet, and Table R1 in W2 for Audi).

Please see qualitative results (Fig. 2) on our anonymous rebuttal page: LINK

Table R3. Quantitative comparison on real captures of highly polished dielectric objects.

MethodTV Screen PSNR ↑TV Screen SSIM ↑TV Screen LPIPS ↓Helmet PSNR ↑Helmet SSIM ↑Helmet LPIPS ↓
MaterialRefGS31.550.9440.15126.570.9180.201
Ref-DGS32.170.9500.13228.940.9400.176
Ours33.090.9550.11930.650.9480.174

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.