acceptodds
Under review as a conference paper at ICLR 2027

DualGS: Unified Geometry and Appearance Modeling with Dual Gaussian Fields

Abstract

Recent advances in Gaussian Splatting have shown remarkable progress in neural scene representation and rendering. However, existing methods often face an inherent trade-off between geometric accuracy and photometric fidelity. While geometry-oriented approaches such as 2DGS focus on reconstructing thin and precise surfaces, they often suffer from limited rendering expressiveness. Conversely, appearance-oriented methods like 3DGS achieve photorealistic novel view synthesis but tend to produce blurred or inconsistent surfaces. In this work, we present DualGS, a unified framework that bridges this gap through a coarse-to-refine dual-branch design. Specifically, we introduce two specialized Gaussian fields, one dedicated to surface reconstruction and the other to appearance refinement, which are jointly optimized through edge-aware multi-view consistency constraints. The surface branch is guided by edge-aware photometric and feature consistency losses that leverage cross-view geometry cues, while the appearance branch enhances visual fidelity via rendering supervision. Furthermore, we introduce a cross-branch consistency mechanism, allowing the appearance field to provide auxiliary supervision for stabilizing surface optimization. Extensive experiments on standard benchmarks demonstrate that our method significantly improves reconstruction accuracy and rendering realism over state-of-the-art methods, achieving precise surfaces and high-fidelity appearance simultaneously. We will release our source code.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Gaussian Splatting, 3D Scene Reconstruction, Novel View Synthesis, Dual-Branch Optimization, Multi-View Consistency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3aobOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DualGS, a coarse-to-fine training paradigm for both geometry and appearance modeling. It aims to tackle the problem that traditional methods perform well only in a single task, such as surface reconstruction or appearance modeling. The authors propose a dual-branch Gaussian splatting framework that initializes with a strong surface reconstruction baseline, like 2DGS, to provide accurate geometry initialization. They also propose a cross-branch supervision strategy to encourage the Gaussians to learn geometry and appearance modeling more effectively. Experiments over several datasets, including surface reconstruction and novel view synthesis tasks, demonstrate the effectiveness of the proposed method.

Critical strengths
  1. This paper is well presented and easy to follow.

  2. The idea is easy to understand and effective to deal with the problem that previous methods only focus on a single field.

  3. The proposed method shows superior results to previous methods.

Critical weaknesses
  1. The idea is straightforward. Training a two-branch of Gaussian Splatting method means we need to train the Gaussians two times. The trainig iterations is doubled. Although the results demonstrate it is useful, I don't think the idea is novel and over the bar of ICLR.

  2. In line 240, how do the authors obtain the normal nrn_r? Do the authors compute the normal first from some pretrained methods or just from the dataset itself, or using the rendered normal? What is the difference between this normal and NN methioned on the above?

  3. I strongly suggest the authors move the ablation studies in the main paper. It would be better for readers to understand which part of the proposed method is significant.

  4. I do not think the proposed scheme is elegant, since it needs to switch to a specific part of Gaussians for better performance. How to combine the advantage of the proposed two-branch gaussians is deserved for research.

Questions for authors
  1. The multi-view consistency loss has already been proposed by MVGS[1]. I am curious whether this can further improve the rendering quality of the proposed method.

  2. Have the authors tried other tasks different from surface reconstruction, like relighting [2] and reflective object reconstruction [3]

[1] Mvgs: Multi-view regulated gaussian splatting for novel view synthesis

[2] GS3: Efficient Relighting with Triple Gaussian Splatting

[3] Ref-gs: Directional factorization for 2d gaussian splatting

Reviewer_it1BOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes DualGS, a dual-branch Gaussian Splatting framework for surface reconstruction and novel-view synthesis. The method first optimizes coarse Gaussians with geometric constraints, including normal regularization and multi-view photometric and feature consistency. It then separates the Gaussians into two branches: a 2D Gaussian branch for surface reconstruction and a 3D Gaussian branch for appearance modeling. Cross-branch supervision is introduced to guide surface refinement using appearance information. The method is evaluated on DTU, Tanks and Temples, Mip-NeRF 360, and Deep Blending datasets.

Critical strengths
  1. The paper is well written and easy to follow, with a clear motivation and straightforward framework.
  2. Separating geometry and appearance into two Gaussian branches is an intuitive way to address the trade-off between reconstruction accuracy and rendering quality.
  3. The method reports competitive reconstruction and rendering results with relatively short training times.
Critical weaknesses
  1. In Table 7, the ablation results remain surprisingly strong even when major components or an entire branch are removed. For example, removing the 3D branch still achieves results comparable to some SOTA methods. Does this mean most of the improvement actually comes from the geometric constraints in the coarse stage rather than the dual-branch design? Is there any unfairness in the comparison?
  2. According to Table 6, the appearance branch uses only about 111k Gaussians, much fewer than standard 3DGS, yet achieves competitive rendering quality. Why can such a small number of Gaussians work so well?
  3. Some experimental settings and results need further clarification. In Table 7, the authors report a Chamfer Distance for the "w/o 2D branch" setting, although the surface is supposed to be reconstructed from the 2D branch. How is this result obtained? Also, the rendering results of the full model differ between Table 3 and Table 7. Are these results evaluated under the same training and testing settings?
  4. The novelty compared with existing Gaussian-based methods could be better explained. The method mainly combines 2DGS, 3DGS, and several existing geometric consistency losses. Similar ideas of combining 2D and 3D Gaussians have also been explored in previous works such as MGSR. What is the key technical difference here? Given the relatively small gains from some cross-branch losses in Table 7, it is unclear how much the proposed interaction mechanism actually contributes.
Questions for authors
  1. Why do the ablation models in Table 7 still perform so well after removing major components? How much improvement comes from the coarse geometric initialization rather than the dual-branch refinement?
  2. How is the Chamfer Distance computed when the 2D branch is removed? Why do the full-model rendering results differ between Table 3 and Table 7?
  3. What is the main difference between DualGS and existing hybrid 2D/3D Gaussian methods such as MGSR? Could the authors better demonstrate the specific benefits of the proposed cross-branch supervision?
Reviewer_dYLUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents DualGS, a method that uses two Gaussian fields to improve surface reconstruction and image rendering. It first optimizes a coarse geometry representation, then splits it into a surface branch and an appearance branch. The surface branch uses normal and multi-view consistency constraints, while the appearance branch focuses on image quality and provides additional supervision to the surface branch. Experiments on DTU, Tanks and Temples, Mip-NeRF 360, and Deep Blending show competitive geometric accuracy and rendering quality.

Critical strengths
  • The paper addresses a useful problem: improving surface geometry while maintaining good rendering quality.
  • The two branches have clear roles, and the design is straightforward.
  • The method achieves strong surface reconstruction results on DTU and TNT with reasonable training time.
  • Experiments cover several standard datasets, with ablations on the main components.
Critical weaknesses
  • This paper does not clearly explain what the proposed design adds beyond MGSR. In MGSR, it already uses a 2DGS branch for geometry and a 3DGS branch for appearance, with cross-branch supervision and alternating optimization. Besides, a closely related work EGGS[1] also combines 2D and 3D Gaussians to balance geometry and appearance, but it is missing in the references.

  • Some experimental results are not coherent. For example, GS-Pull’s indoor PSNR on Mip-NeRF 360 is 30.78 in Table 4, but the corresponding scene values in Table 9 is 28.10. The full model’s DTU PSNR are also different between Table 3 and Table 7. These differences make it difficult to judge whether the results are convincing.

  • The gains from cross-branch supervision are small in Table 7. Removing both cross-branch losses changes CD from 0.46 to 0.48 and appearance PSNR from 35.59 to 35.46. Removing the neighbor variation loss alone leaves CD unchanged. The paper argues that these losses mainly help boundaries and thin structures, but it does not provide experimental results on those regions.

  • The results in Table 3 are measured on training views, while the test-view results are reported in Tables 13–14 of the appendix. However, the test-view results does not also show the consistent improvements: On TNT, DualGS achieves a lower PSNR of 24.58 compared with 25.10 for 3DGS. On TNT, both methods have an SSIM of 0.851.

  • The paper describes the branches as helping each other, but the equations and appendix show one-way supervision from appearance to geometry. Since the appearance output is detached, it is unclear why removing losses applied only to the surface branch changes the appearance branch’s results.

[1] Zhang, Y., Sun, G. and Chen, C., 2026. EGGS: Exchangeable 2d/3d gaussian splatting for geometry-appearance balanced novel view synthesis. Advances in Neural Information Processing Systems, 38, pp.147665-147691.

Questions for authors

see the weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.