acceptodds
Under review as a conference paper at ICLR 2027

DC-DLO: Decoupling Local Repair and Pose Discovery for 3D Gaussian Splatting

Abstract

3D Gaussian Splatting (3DGS) has shown strong performance in novel-view synthesis, but few-view reconstruction remains highly sensitive to which observations are acquired under a limited budget. Active view selection offers a natural way to address this sensitivity by sequentially choosing additional views. However, applying sequential selection to 3DGS is complicated by its evolving representation, as optimization, densification, and pruning continually change the underlying Gaussian primitives. As a result, selection states indexed by the current primitives lack a stable spatial reference across acquisition rounds. To address this issue, we propose DC-DLO, a decoupled active view selection framework that separates model-based local repair from pose-only discovery. We first introduce World-Space Local Repair (WSLR), which rebuilds local observability on a fixed world-space voxel grid at every round to avoid dependence on persistent Gaussian identities. Within each occupied cell, a compact \(3\times3\) matrix encodes directional support from the acquired cameras. Candidate views are evaluated by the marginal log-determinant gain of these local matrices, and an adaptive Top-K rule aggregates the largest local gains to form the repair score. Then, to preserve pose diversity independently of current model support, we introduce Pose Discovery, a pose-only gate that constrains the feasible candidate set without modifying the repair score. Candidate RGB is never accessed before selection and is revealed only after a view is chosen. We evaluate DC-DLO on the full nine-scene Mip-NeRF 360 benchmark, the seven-scene Extended NBV benchmark, and the four-scene Light-Field benchmark. When results are aggregated over scenes within each benchmark, DC-DLO achieves the highest PSNR and SSIM and the lowest LPIPS among the evaluated selectors on all three benchmarks. Matched ablations further support the benefit of rebuilding the local scoring state in world space.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
active view selection, next-best-view selection, 3D Gaussian Splatting, sparse-view reconstruction, neural rendering, local observability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over one review, above 27% of the others.
Reviewer_3NoLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses active view selection for 3D Gaussian Splatting (3DGS) under few-view budgets, where the underlying primitive set evolves due to densification/pruning, destabilizing per-primitive acquisition states across rounds. The authors propose DC-DLO, which decouples reconstruction-dependent local repair from pose diversity. World-Space Local Repair rebuilds a per-round repair state on fixed voxel cells, storing a 3×3 directional support matrix per occupied cell and scoring candidates by marginal log-determinant gain aggregated via an Adaptive Top-K scheme; Pose-Only Discovery enforces pose novelty through a pool-relative rank gate independent of the repair utility.

Critical strengths
  • Explicitly frames spatial indexing of acquisition evidence as a first-class design choice for sequential 3DGS and demonstrates a robust alternative to per-primitive indexing via fixed world-space cells.

  • Proposes a simple, effective Adaptive Top-K aggregation to prevent candidates from winning via many tiny local gains.

Critical weaknesses
  • The world-space discretization introduces dependencies on voxel size, grid orientation, and scene scale that may affect stability across scenes

  • The coarse visibility and normal-estimation steps rely on current 3DGS geometry and ad hoc thresholds; failure modes early in reconstruction (or under severe occlusion) are not deeply analyzed.

Questions for authors

What is the exact definition and range of the facing/visibility weight αℓ(j), and how is the coarse visibility test implemented (z-buffer resolution, depth tolerance)? How do these choices affect early-round stability?

8 comments
AuthorsOpenReview ↗

Response to Reviewer 3NoL

Thank you for your specific questions about facing weights, coarse visibility, and the reliability of early geometry. We have clarified the implementation of facing weights and visibility, and conducted frozen-state perturbation experiments to quantify how these geometric tests and grid configurations affect candidate selection. We report the experimental setup, results, and implications below.

1. Response to Weakness 2 and Question 1: Facing Weights, Normal Estimation, and Coarse Visibility

Let rt,l,jr_{t,l,j} denote the unit direction from the cell representative to camera jj, and let nt,ln_{t,l} denote the oriented unit normal. The front-facing quality is

Qt,l(j)=max⁡(0,nt,l⊤rt,l,j).Q_{t,l}(j)=\max\left(0,n_{t,l}^{\top}r_{t,l,j}\right).

The visibility-gated facing weight is defined as

Zt,l(j)=Z~t,l(j)⋅1[Qt,l(j)≥qmin⁡],Z_{t,l}(j)=\widetilde{Z}_{t,l}(j)\cdot \mathbb{1}\left[Q_{t,l}(j)\geq q_{\min}\right],

αt,l(j)=Zt,l(j)Qt,l(j).\alpha_{t,l}(j)=Z_{t,l}(j)Q_{t,l}(j).

Here, Z~t,l(j)\widetilde{Z}_{t,l}(j) is the binary result of the coarse z-buffer visibility test, and Zt,l(j)Z_{t,l}(j) combines this visibility result with the front-facing threshold, where qmin⁡=0.1q_{\min}=0.1. Consequently, αt,l(j)∈{0}∪[0.1,1]\alpha_{t,l}(j)\in\{0\}\cup[0.1,1]. The visibility and facing tests determine whether an observation contributes support, while αt,l(j)\alpha_{t,l}(j) determines its continuous facing weight.

Normals are estimated by PCA over the KNN-30 neighborhood of each current cell representative, using the eigenvector associated with the smallest eigenvalue. They are oriented according to the aggregate viewing direction of the acquired supporting cameras.

Coarse visibility uses a point z-buffer. Image width and height are reduced to one quarter of their original resolution, with camera intrinsics scaled accordingly. Each projected point updates the minimum axial depth within a disk of radius 3 pixels. A candidate point is classified as occluded when zcandidate>zbuffer+0.01,z_{\mathrm{candidate}}>z_{\mathrm{buffer}}+0.01,where the tolerance is in native scene units. This test uses only the current reconstruction and candidate pose; candidate RGB is not accessed.

2. Response to Weakness 2 and Question 2: Parameter Sensitivity at Early and Late Acquisition Stages

To examine how facing-based filtering and coarse-visibility tests affect view selection at different reconstruction stages, we froze rounds 1–4 and 13–16 from 18 reference trajectories (9 scenes × 2 seeds), yielding 144 states. For each frozen model and candidate pool, we perturbed the facing threshold and depth tolerance by ±20% relative to their default values and compared the resulting Top-1 views with those selected under the default settings.

Table 1. Top-1 view change rates under perturbations of the facing threshold and depth tolerance
ParameterDefaultPerturbedEarly-stage change rateLate-stage change rate
Facing threshold0.100.0815.28%8.33%
Facing threshold0.100.1213.89%11.11%
Depth tolerance0.0100.0082.78%15.28%
Depth tolerance0.0100.0121.39%16.67%

The default Top-1 view was retained in 515 of 576 comparisons (89.4%). Of the 61 changed selections, 47 (77.0%) occurred when the score difference between the default top two candidates was no greater than 1%. Thus, under these local parameter perturbations, the selected view usually remained unchanged, and changes occurred mainly when candidate scores were close. The early- and late-stage groups each contain 72 states.

The stage-wise results further show that perturbing the facing threshold changes the selected view in 13.89%–15.28% of early-stage states and 8.33%–11.11% of late-stage states. Perturbing the depth tolerance changes the selected view in 1.39%–2.78% of early-stage states and 15.28%–16.67% of late-stage states. Facing-based filtering therefore has a greater effect on early-stage candidate ranking, whereas the depth-occlusion boundary is more influential at later stages.

3. Response to Weakness 1 and Question 3: Sensitivity to Grid Scale, Rotation, and Translation

The voxel-width experiment shows that the spatial aggregation scale affects reconstruction quality. On the three-scene subset, reducing the default width from 0.25 to 0.125 changes PSNR by −0.040 dB, whereas increasing it to 0.5 reduces PSNR by −0.755 dB. The larger loss with the coarser grid indicates that spatial scoring units need to preserve sufficient local discrimination.

To examine the effect of grid placement and orientation on candidate selection, we tested 10 fixed grid-rotation and translation configurations on 144 frozen states from 9 scenes and 2 seeds, giving 1,440 comparisons. The Top-1 view changed in 521 comparisons (36.18%); 242 of these changes (46.45%) occurred in near-tie states, where the relative score difference between the default top two candidates was no greater than 1%. The median relative repair cost of changed selections was 2.02%, and the 95th percentile was 9.06%. These results quantify the effect of grid placement and orientation on candidate ranking and the repair cost of changed selections.

Table 2. Selection changes and repair costs under grid-rotation and translation perturbations
DiagnosticResult
Frozen states and perturbation configurations144 states; 10 grid perturbations; 1,440 comparisons
Top-1 selection changes521/1,440 (36.18%)
Changed selections in near-tie states242/521 (46.45%)
Relative repair cost of changed selectionsMedian 2.02%; 95th percentile 9.06%

Note: A near-tie is defined as a relative score difference of no more than 1% between the default top two candidates.

Comments on the paper

15 comments
AuthorsOpenReview ↗

Response to Reviewer WQFu

Thank you for the constructive comments on the method design, experimental attribution, and computational efficiency. These comments helped us clarify the scope of the method and add component-mechanism analyses, controlled comparisons, and end-to-end timing.

1. Response to Question 1: Four-view Initialization and Input Information

The shared initial 3DGS checkpoint is initialized using geometry triangulated exclusively from the four acquired RGB images. No SfM geometry reconstructed from additional images is used. The initial model is optimized for 1K iterations using only these four images, and the resulting checkpoint is shared by all methods for the same scene and seed.

2. Response to Weakness 1 and Question 2: Applicability of Pose-Only Discovery and the Directional Matrix

1. Pose-Only Discovery

Under the matched 4+16 protocol, the two-seed Full−No-Discovery PSNR differences are +0.1580 dB on Light-Field and +0.08054 dB on Extended NBV, with both 95% CIs above zero; the Mip-NeRF 360 mean is −0.0634 dB (Table 1).

Table 1. Pose-Only Discovery ablation: Full − No-Discovery
DatasetΔPSNR (95% CI)ΔSSIMΔLPIPS
Mip-NeRF 360−0.0634 [−0.3807, +0.2075]−0.004483+0.004111
Light-Field+0.1580 [+0.0120, +0.2932]+0.00763−0.00628
Extended NBV+0.08054 [+0.00007, +0.16873]+0.00047+0.00013

An additional two-seed Mip-NeRF 360 ablation with six initial views and 14 acquisitions (the same 20-view total) gives favorable Full−No-Discovery changes: +0.07210 dB PSNR, +0.00896 SSIM, and −0.00436 LPIPS (Table 2).

Table 2. Full−No-Discovery on Mip-NeRF 360 under 6+14 (two seeds)
ProtocolΔPSNR ↑ (dB)ΔSSIM ↑ΔLPIPS ↓
6+14 (Full−No-Discovery)+0.07210+0.00896−0.00436

The scene-level results further illustrate the effect of initialization. Under 6+14, Counter changes from −0.23325 dB under 4+16 to +0.50748 dB, while the PSNR drop on Bonsai is substantially reduced from −1.06591 dB to −0.27633 dB. These results indicate that additional initial views can provide a stronger geometric basis for pose-based exploration, making novel viewpoints more likely to contribute useful complementary observations and reducing the adverse effects of Discovery in challenging scenes.

We will add this limitation to the Limitation Section:

The effectiveness of Pose-Only Discovery depends on scene geometry and initialization conditions. Its benefits are most evident when pose-novel viewpoints provide complementary observations. When such complementarity is limited, the gains from exploration may be modest compared with prioritizing immediate local repair.

2. Directional Support Matrix

The full nine-scene, two-seed terminal ablation shows a mean PSNR advantage of +0.16498 dB for Matrix over Scalar, but does not establish a consistent quality advantage across metrics. Nevertheless, Matrix improves two-seed mean PSNR on Bonsai, Treehill, Kitchen, and Flowers by 1.211, 0.532, 0.520, and 0.224 dB, respectively, with corresponding SSIM improvements and LPIPS reductions.

These gains suggest that directional modeling is beneficial when existing observations provide concentrated or uneven viewing-direction support. Bonsai illustrates the benefit of complementary observations under concentrated historical viewing directions, while Treehill and Flowers show favorable cases under different initializations.

Across nine Mip-NeRF 360 scenes and 16 acquisition rounds, Matrix adds approximately 0.224 s per round in candidate scoring, accounting for only 0.59% of the total acquisition-and-training pipeline time.

Therefore, we propose to retain Matrix as an optional direction-aware scoring component within WSLR. It provides additional directional information that can improve view selection under favorable scene and initialization conditions, while Scalar offers a lower-cost alternative.

3. Response to Weakness 2 and Question 3: Spatial Aggregation and Attribution

The controlled comparison shows that Density-normalized and World-space achieve similar reconstruction quality, with PSNR gains of +0.44597 dB and +0.46434 dB over Matched Gaussian, respectively (Table 3A). Their direct difference is only +0.01838 dB, indicating that density normalization accounts for much of the observed quality gain. However, World-space achieves comparable reconstruction quality with substantially lower computational cost, reducing full CPU candidate-scoring time from 88.27 s to 6.71 s (Table 3B).

Table 3A. Reconstruction-quality comparison of spatial scoring representations
ComparisonΔPSNR ↑ΔSSIM ↑ΔLPIPS ↓
World-space − Matched Gaussian+0.46434+0.01277−0.01113
Density-normalized − Matched Gaussian+0.44597+0.01426−0.00971
World-space − Density-normalized+0.01838−0.00149−0.00142
Table 3B. CPU candidate-scoring time at a late reconstruction stage (seconds)
Timing stageDensity-normalizedWorld-space
Local-matrix stage85.50 s6.63 s
Aggregation stage2.53 s0.052 s
Other overhead0.24 s0.028 s
Full candidate scoring88.27 s6.71 s

Planned Changes to the Manuscript

We will revise the manuscript to attribute the observed quality gains to spatial aggregation and density normalization, while framing fixed world-space indexing primarily in terms of computational efficiency and a stable spatial reference, rather than attributing the gains solely to cross-round Gaussian-identity stability.

1. Section 4.3 — Controlled Spatial Indexing

Current:

Table 3 matches the source Gaussians and all non-indexing components while changing only the indexing domain.

Planned revision:

Table 3 matches the source Gaussians and the remaining selection pipeline while replacing individual Gaussian scoring units with spatially aggregated world-space cells.

2. Appendix A.3 — Controlled Spatial-Indexing Diagnostic

Current:

Because all non-indexing components are matched, this experiment isolates spatial indexing as an empirical design variable in sequential 3DGS view selection.

Planned revision:

Because the remaining selection components are matched, this experiment evaluates the world-space scoring design, including spatial aggregation, in sequential 3DGS view selection. The additional density-normalized Gaussian control indicates that density normalization accounts for much of the observed quality improvement, while world-space scoring substantially reduces CPU candidate-scoring cost.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.