acceptodds
Withdrawn from ICLR 2027

4DRefGS: Reflective 4D Gaussian Splatting for Multi-View Dynamic Scene Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reconstructing dynamic reflective scenes from multi-view videos is challenging because physical motion and reflected appearance are strongly entangled. Our ob- servation is that surface-attached and localized virtual reflections exhibit different dynamic behaviors: the former remain tied to physical surface, while the latter may exhibit independent apparent motion. Modeling both with the same dynamic primitives can therefore entangle geometry, motion and reflected appearance. We introduce 4DRefGS, a role-aware framework for reconstructing dynamic reflec- tive scenes. First, to disentangle physical surface motion from reflection-induced apparent motion, we employ two asymmetric 4D Gaussian fields: primary planar Gaussians reconstruct the physical surface, whereas auxiliary Gaussians indepen- dently follow localized virtual-reflection cues without contributing to geometry or diffuse appearance. Second, we model reflective appearance through a reflection- aware deferred shading framework, where scene-shared directional lighting and time-varying surface-attached features predict the global reflection, while auxil- iary features provide spatially localized corrections. To facilitate evaluation, we further collect SelfCap, a synchronized multi-view dataset featuring fast motion and substantial time-varying reflections. Experiments on SelfCap and Neural3DV demonstrate that 4DRefGS faithfully reconstructs dynamic reflective appearance, substantially improving rendering quality on challenging reflective scenes while remaining competitive on general dynamic scenes.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Computer Vision

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ok6xOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces 4DRefGS, a 4D reconstruction pipeline for dynamic reflective scenes. Motivated by prior works, 4DRefGS introduces a dual 4D Gaussian representation to separate physical-surface from reflections. Evaluation on the self-captured dataset shows improved rendering quality. However, the evaluation is not thorough enough. There is no visualization to support reflective disentanglement.

Critical strengths
  • This paper stuides dynamic reflective reconstruction, which is a challenging yet important problem.

  • On SelfCap, 4DRefGS outperforms all the baselines in rendering.

Critical weaknesses
  • 4DRefGS is more like a system design, which combines many modules from existing works, such as Ref-DGS and FreeTimeGS. Thus, the novelty is limited.

  • Evaluation is not thorough.

    • More baselines are required. FreeTimeGS should be compared.
    • The diversity of evaluation benchmark is limited. I suggest adding evaluation on NeRF-DS [1], which focuses on dynamic specular objects.
    • On Neural3DV, PSNR and SSIM and not explicitly improved. Authors should visualize the rendering results of different methods, especially in the regions with reflections.
    • SelfCap is a small dataset that only contains two scenes. More videos with different scene types should be added to test the robustness of the pipeline. As shown in Figure 2, the reflection is mainly diffuse reflection and the reflective objects are flat floors. In addition, the video corresponding to the first row does not contain changing illumination as claimed in the paper. I would suggest adding the evaluation on more challenging settings, such as sharp mirror reflection and specular highlight. In addition, reflective objects with different shapes could be added, e.g. balls.
    • Geometry should be visualized, such as normal map.
  • Ablation study is not convincing. SSIM and LPIPS keep the same after removing modules. In addition, the experiments of different modules are conducted on different sequences.

  • To better support reflective disentanglement, it would be helpful to visualize the rendering results of both global branch and local branch.

[1] Yan et al. Nerf-ds: Neural radiance fields for dynamic specular objects. CVPR 2023.

Questions for authors

see ''weaknesses''

Reviewer_FP3cOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes 4DRefGS for 4D reconstruction with time-varying reflections. The authors proposes 1. separate Gaussian for physical surface and auxiliary Gaussians localized virtual-reflection cues. 2. the reflective appearance is model through a reflection-aware deferred shading framework that the primary branch predicts surface-conditioned reflections, and the auxillary branch supplies localized corrections.

Critical strengths
  1. The motivation of allowing reflection-specific motion while restricting which primitives generate geometry is reasonable for the reflective dynamic reconstruction.
  2. The proposed method achieves competitive performance on Neural3DV dataset and self-captured dynamic reflective dataset.
Critical weaknesses
  1. The method builds on linear dynamic trajectories in FreeTimeGS and flat-top temporal support in SharpTimeGS, but the comparisons are missing.
  2. There is no quantitative depth, normal, geometry, or motion evaluations.
  3. SelfCap contains five short sequences with dense camera coverage and only one test camera. More test views and qualitative samples on both datasets are needed for better comparison.
  4. The ablation of w/o temporal dimension split in Table 3 brings marginal differences.
Questions for authors

Please see comments above.

Reviewer_HDd7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents 4DRefGS for dynamic reflective scene reconstruction from multi-view videos. It uses separate Gaussian fields for physical surfaces and reflections, allowing them to move independently. Experiments on Neural3DV and SelfCap show improved novel-view rendering quality.

Critical strengths
  1. The method addresses a practical challenge in dynamic reflective reconstruction. Giving surface and reflection Gaussians different roles is a reasonable way to model reflected content whose apparent motion differs from the physical surface.

  2. The method improves rendering quality on SelfCap, achieving 27.68 dB PSNR compared with 26.50 dB for UBS. The supplementary video also shows better preservation of reflected structures than FreeTimeGS in the presented examples.

  3. The method remains competitive on general dynamic scenes. On Neural3DV, it achieves the highest PSNR 31.51 dB and lowest LPIPS 0.1035 among the compared methods.

Critical weaknesses
  1. The paper overclaims the extent of geometry–reflection disentanglement. Not directly outputting geometry does not mean having no effect on geometry. The auxiliary Gaussians do not directly contribute to depth or normals, which is a useful structural constraint. However, the final reflection still depends on the normals, roughness, and features produced by the primary Gaussians. The image reconstruction loss can therefore still change the primary surface. This design may reduce reflection-induced geometry errors, but does not guarantee their removal. The contribution can be enhanced if some more tests can be added to show :
  • Depth, normal, or surface reconstruction errors;
  • The accuracy of physical motion trajectories;
  • Whether auxiliary Gaussians actually track reflected virtual images;
  • Whether the reconstructed geometry remains stable under changing illumination.
  1. Some key baselines are missing. The method adopts the motion model of FreeTimeGS and the flat-top temporal window of SharpTimeGS, but both of them do not appears in the main comparison tables. This makes it difficult to separate the gains from the dynamic representation from those of the proposed reflection model.

  2. The ablations showing in Table3 do not isolate the benefit of independent reflection motion. Removing the auxiliary reflection branch decreases PSNR by 0.34 dB and SSIM by 0.0003, and increases LPIPS by 0.0018 on SpecFloor2. It shows that adding the branch is useful, but does not approve that the improvement comes from independent reflection motion. Some more comparisons could include to show shared vs independent motion, static vs dynamic auxiliary fields, and Ref-DGS with the same dynamic backbone.

Questions for authors

see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.