acceptodds
Under review as a conference paper at ICLR 2027

NPM-GS: 3D-Native PatchMatch for Sparse-View Gaussian Splatting

Abstract

Gaussian Splatting achieves high-quality novel-view synthesis with dense observations but degrades severely in sparse-view settings, producing floaters, distorted geometry, and strong train-view overfitting. Existing depth priors, regularization, and feed-forward initialization alleviate these failures, yet residual geometric errors can still survive subsequent Gaussian optimization. We identify a key cause: under sparse observations, geometry can co-adapt with appearance, opacity, scale, and densification, allowing different 3D configurations to explain the same training images. In contrast, cross-view matching directly tests whether a geometric hypothesis is supported by multiple observed views. We further find that this signal is highly geometry-sensitive but effective only within a finite capture range, motivating a bounded discrete search rather than unconstrained continuous optimization. Based on these observations, we propose NPM-GS, a 3D-native hypothesis search and verification framework that directly refines Gaussian centers using discrete cross-view matching. A learned geometric prior defines the search region, while multi-view and spatial agreement identify reliable Gaussians whose geometry is propagated to ambiguous regions through plane-snap and superpixel-based propagation. A subsequent reliability-aware training stage refines appearance while preserving the verified structure. NPM-GS consistently improves sparse-view reconstruction on DTU and Mip-NeRF 360 across PSNR, SSIM, and LPIPS.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D gaussian splatting, sparse view

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GXCTOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces NPM-GS, an approach that investigates sparse-view 3D Gaussian Splatting (3DGS) from a geometric perspective. The authors argue that under sparse-view conditions, the various parameters of 3DGS can become entangled, resulting in similar RGB fitting but incorrect geometry. The method employs DA3 to generate a dense initialization and samples candidate points for GS centers along each ray; it then assesses the reliability of these candidates using NCC calculated via reference-to-target view back-projection. Finally, a reliability-aware GS optimization strategy is implemented, incorporating depth supervision, multi-view color regularization, and a dropout mechanism to enhance accuracy.

Critical strengths

1.The paper provides a compelling diagnosis of the problem and employs multiple experiments to demonstrate the issue of geometric perturbation in few-view 3DGS reconstruction.

Critical weaknesses
  1. The distinctions and boundaries of innovation regarding existing work have not been sufficiently articulated: using VFM for initialization is already common practice in prior studies (e.g., DNGaussian, NexusGS); the "drop" mechanism originates from DropGaussian; and supervision based on depth estimation is also frequently employed (e.g., FatesGS, Sparse2DGS). The authors need to clearly delineate the differences between this paper and these existing works to establish the paper's specific innovations and contributions.
  2. Insufficient experimentation: Although the primary issue addressed in this paper is geometric error, the final experimental evaluation lacks an assessment of geometric accuracy. This is a significant shortcoming; metrics such as surface reconstruction accuracy (e.g., Chamfer Distance/CD, commonly used with the DTU dataset) or the accuracy of depth maps rendered by the Gaussian Splatting model (common in RGB-D datasets) should be included.
Questions for authors
  1. The paper employs DA3 for dense initialization; are the camera parameters encoded into the network? Furthermore, since DA3 is essentially a feed-forward network for unconstrained images, could directly using the predicted depth maps (even with official alignment) and the actual camera parameters result in a failure to accurately align the multi-view point clouds? The authors could compare rendered images from the test views (using only the initialization) against the actual images to check for misalignment.
  2. The DTU images in Figure 3 do not have the background removed, yet the DTU dataset provides background masks. Were the backgrounds removed for the evaluation data presented in the tables? If so, please update the DTU images in Figure 3; if not, the comparison may be unfair due to background discrepancies.
3 comments

We thank the reviewer for recognizing our diagnosis of geometric perturbation under sparse views.

W1: Differences from prior work

We differ from these works not in using priors, but in how the final geometry is decided. Dense priors, depth supervision, and dropout are common, and we do not claim them as contributions: DA3 only initializes and bounds the search (Sec. 4.2), our depth loss uses depth corrected by reliable Gaussians, not raw DA3 depth (Eq. 4), and for dropout we already cite DropGaussian.

DNGaussian regularizes with monocular depth, NexusGS initializes from optical flow and epipolar geometry, FatesGS regularizes with monocular depth ranking and multi-view feature consistency, and Sparse2DGS initializes from learning-based MVS with geometry-prioritized constraints. In all of them, the prior is an initial value or a constraint target, and the final geometry is still set by continuous joint optimization. In Tables 1–2, NPM-GS leads DNGaussian/NexusGS by 7.61/3.96 dB PSNR on DTU (3 views) and 4.82/1.54 dB on Mip-NeRF 360.

Sec. 3 and App. E show why continuous optimization cannot reliably recover geometry from sparse views: (1) depth supervision pulls geometry toward the prior without checking it against the images, so prior errors persist (Fig. 1(b)); (2) optimizing geometry before appearance stays well below NPM-GS (19.86, or 19.97 dB with depth supervision, vs. 23.19 dB; DTU scan 105, 4 views; Table 7); (3) the rendering loss barely reacts to geometric perturbation, while cross-view NCC reacts strongly but flattens beyond about 13% of scene scale, giving no gradient toward distant correct positions (Figs. 1(c), 1(d), 7). NPM-GS thus uses the prior only to bound the search and accepts only discrete candidates supported by multiple views.

Our contributions are (i) the analysis of geometry–appearance coupling (Sec. 3), (ii) discrete cross-view verification on the final Gaussians (Sec. 4.3), and (iii) reliability-guided propagation and training (Secs. 4.4–4.5). The revision will discuss FatesGS and Sparse2DGS and add a table comparing these methods by prior source, prior use, and whether geometry is checked against the input images.

W2: Geometric accuracy

Our task is sparse-view novel view synthesis, so we only report its standard metrics (PSNR, SSIM, LPIPS), as do DNGaussian and NexusGS. Here, geometry is the means rather than the target: NPM-GS fixes geometry by discrete search to resolve the coupling in Sec. 3, which improves rendering. Sec. 3 and App. A also measure geometry directly in controlled tests, as the displacement of perturbed geometry from a reference reconstruction.

As suggested, we are computing Chamfer Distance on DTU for all methods with one pipeline (TSDF fusion of rendered depths as in 2DGS, then the official DTU script) and will add the results to the revision.

Q1: DA3 initialization alignment

Yes, DA3 takes the dataset cameras as input, and its depths are unprojected with the same cameras (Sec. 4.2). As the reviewer expects, the fused depths are still clearly misaligned across views, forming layers especially in weakly textured planar regions. This is the "dense initialization with misalignment" in Fig. 2, a key problem NPM-GS addresses, and why we treat planar regions separately (Sec. 4.4).

The suggested check matches the "DA3 Init" row of Table 3, which uses the DA3 initialization without our NPM optimization: it gets only 19.77 dB (19.91 dB with DA3 depth supervision) vs. 21.62 dB for the full NPM-GS. This shows that the misalignment has a large impact. The revision will clarify this row and its setting in Sec. 5.3 and the Table 3 caption, and add its test-view renderings to the appendix.

Q2: DTU background

The background was neither removed nor kept in full. As stated in Sec. 5.1, we evaluate pixels visible from the training views: we reproject reference depths of the 3-view training set into each test view (App. C.1) and keep pixels seen by at least one of them, plus the DTU object mask. This keeps visible background such as the table, where sparse-view failures like floaters often occur and an object mask would hide them. In our supplementary videos, the compared methods show many floaters in these regions. PSNR uses only pixels in the mask; for SSIM and LPIPS, pixels outside it are filled with ground truth and add no error for any method. All methods and the 3/6/9-view settings share one mask, so the background causes no unfairness.

The black regions in Fig. 3 are such unseen, excluded pixels, which NPM-GS does not fill (see our reply to Reviewer rN7k, W3). As our description was too brief, the revision will state in the main text that pixels outside the mask are excluded, show the masks, update Fig. 3, add these details to App. C.1, and release all masks.

We thank the reviewer again; the clarifications and new experiments will be included in the revision.

AuthorsOpenReview ↗

Results Update

We thank Reviewer GXCT again. Following up on our previous response, we have uploaded the revised manuscript with all promised experiments, visualizations, and clarifications:

1. Chamfer Distance Evaluation on DTU (W2) We evaluated 3D geometric accuracy on DTU using Chamfer Distance (mm ↓\downarrow) via TSDF fusion of rendered depths and the official DTU script, and added the results to Sec. 5.2 and Appendix G (Table 9):

Method3 Views ↓\downarrow6 Views ↓\downarrow9 Views ↓\downarrow
SparseGS4.22144.56694.3511
DNGaussian4.26203.55683.1225
CA-3DGS3.10292.20642.4766
NexusGS1.62731.29371.4813
NPM-GS (Ours)0.95910.90660.8201

2. Visualizations & Manuscript Clarifications

  • DA3 initialization (Q1): Clarified the DA3 camera input in Sec. 4.2 and the "DA3 Init" baseline in Sec. 5.3 and Table 4, and added visualizations of the misaligned initialized point cloud and test-view renderings in Appendix F.
  • DTU evaluation masks (Q2): Updated Fig. 3, detailed the visibility mask protocol and unobserved regions in Sec. 5.2 and Appendix C.1, and added mask visualizations in Appendix H.
  • Distinctions from prior work (W1): Expanded Sec. 2 to discuss FatesGS and Sparse2DGS and clarify our differences in how priors are used and verified.

We thank the reviewer again for the valuable feedback.

AuthorsOpenReview ↗

Looking forward to hearing from you

Dear Reviewer GXCT,

We sincerely appreciate your time and effort in reviewing our manuscript and providing valuable feedback.

In response to your insightful comments, we have made the following revisions and clarifications:

  • We have expanded Sec. 2 (including FatesGS and Sparse2DGS) to clarify our boundary of innovation: our core contribution lies not in learned-prior initialization or depth supervision, but in the subsequent test-time optimization and multi-view verification.
  • We have added 3D geometric accuracy evaluation on DTU using Chamfer Distance across 3, 6, and 9 input views in Sec. 5.2 and Appendix G (Table 9), demonstrating sub-millimeter surface accuracy and confirming that strong novel-view RGB metrics under continuous optimization do not guarantee accurate 3D geometry.
  • We have clarified in Sec. 4.2 that dataset camera parameters are provided to DA3, detailed the "DA3 Init" baseline in Sec. 5.3 and Table 4, and added Appendix F (Fig. 8) visualizing both the cross-view misalignment in the unprojected DA3 point cloud and the corresponding test-view rendering comparisons.
  • We have updated the DTU visualizations with evaluation mask contours in Fig. 3 and Appendix H (Fig. 9), and detailed our shared visibility mask protocol and handling of unobserved regions in Sec. 5.2 and Appendix C.1 to ensure fair comparisons across all methods.

We hope these revisions adequately address your main concerns regarding our boundary of innovation relative to prior work and the evaluation of 3D geometric accuracy.

We look forward to actively participating in the Author-Reviewer Discussion session and welcome any additional feedback you might have on the manuscript or the changes we have made.

Once again, we sincerely thank you for your contributions to improving the quality of this work.

Best regards, The Authors of Submission 25601

Reviewer_MP86OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces NPM-GS, a 3D-native hypothesis search and verification framework designed to tackle the severe geometric distortion and train-view overfitting issues inherent in sparse-view 3D Gaussian Splatting (3DGS). The authors identify that under sparse observations, 3D geometry can easily co-adapt with appearance and opacity, rendering standard continuous optimization objectives insufficient for recovering the correct underlying structure. To address this ambiguity, NPM-GS replaces unconstrained gradient optimization with a discrete, PatchMatch-style geometric verification directly on the Gaussian primitives, evaluating candidate depth hypotheses using a geometry-sensitive multi-view normalized cross-correlation (NCC) cost. Furthermore, to handle textureless or ambiguous regions where NCC struggles, the framework propagates reliable, verified geometry to unverified areas through plane-snapping and superpixel-based warping mechanisms. Finally, a reliability-aware training stage refines the appearance attributes while preserving the validated 3D structure, allowing NPM-GS to significantly outperform existing sparse-view 3DGS baselines on DTU and Mip-NeRF 360 by effectively eliminating floaters and preserving high-fidelity details.

Critical strengths
  • Geometry–Appearance Co-adaptation: The paper identifies geometry–appearance co-adaptation as a key challenge in sparse-view 3DGS: lower rendering loss does not necessarily imply more accurate geometry.
  • 3D-Native Discrete Verification: NPM-GS directly verifies 3D Gaussian centers using multi-view evidence, separating geometric verification from appearance optimization.
  • Reliability-Aware Structural Propagation: The method propagates reliable geometric information to ambiguous regions through plane snapping and superpixel warping.
Critical weaknesses

Major Weakness: Outdated Baselines and Unclear Value Proposition Against Modern Feed-Forward Models

Sparse-view 3D reconstruction has increasingly embraced feed-forward models and large reconstruction models (LRMs). While NPM-GS presents an interesting 3D-native PatchMatch optimization strategy, the paper does not adequately justify the need for a 13.5-minute per-scene optimization pipeline compared with modern feed-forward approaches.

The evaluation includes FLARE as its only feed-forward baseline, overlooking more recent methods such as YoNoSplat and TTT-LRM. These models offer an alternative paradigm by generating 3D representations through feed-forward inference, potentially reducing reconstruction time substantially.

Without comparisons against stronger and more recent feed-forward models, it is difficult to determine whether NPM-GS's lengthy optimization delivers sufficient gains in reconstruction quality to justify its computational cost. If modern LRMs achieve comparable novel-view synthesis quality with significantly lower latency, the practical advantage of the proposed pipeline remains unclear.

Questions for authors
  1. Missing Recent Baselines: Why does the evaluation include only FLARE as a feed-forward baseline? Could the authors compare NPM-GS with recent methods such as YoNoSplat and TTT-LRM on DTU and Mip-NeRF 360, reporting PSNR, SSIM, LPIPS, and runtime under comparable settings?
  2. The Role of Per-Scene Optimization: Given the progress in feed-forward reconstruction, what advantages justify the 13.5-minute optimization time? Does NPM-GS offer substantial improvements in geometric accuracy, fine-grained appearance details, or generalization to challenging scenes that existing feed-forward methods cannot achieve?
  3. Potential Synergy with LRMs: The proposed pipeline uses Depth Anything 3 (DA3) for dense initialization. Have the authors considered applying 3D-native PatchMatch verification to an initial 3D Gaussian representation generated by an LRM, such as YoNoSplat? Could this combination improve geometric reliability and reconstruction quality beyond what either method achieves independently?
5 comments

We thank the reviewer for recognizing our analysis and the 3D-native PatchMatch.

We first clarify the relationship between NPM-GS and feed-forward models, and then address each concern. We see NPM-GS as complementary to, rather than a replacement for, feed-forward models: it builds on one (DA3) for initialization and for bounding the search (Sec. 4.2).

A feed-forward model is optimal on average over its training distribution but does not check its predictions against the input views of a given scene; NPM-GS performs exactly this check, so the final geometry is decided by cross-view verification (Sec. 2.2). Recent works such as TTT-LRM and YoNoSplat also adapt to each scene at inference time, which confirms the value of per-scene adaptation; unlike them, NPM-GS adapts the explicit Gaussians used for rendering rather than latent features or network parameters.

Moreover, feed-forward models are typically good at predicting pure geometry such as depth, but less suited to directly producing Gaussians; this is also why we use DA3 only for depth and let NPM-GS build and verify the Gaussians. In the sparse-view setting, feed-forward methods such as FLARE, YoNoSplat, and TTT-LRM still fall clearly behind NPM-GS and are prone to artifacts such as ghosting. We report quantitative sparse-view results in our response to Q1 & Q2, and visual comparisons will be provided with the revised version.

Q1 & Q2: Feed-forward baselines and the need for per-scene optimization

We evaluated YoNoSplat (DL3DV checkpoint) and TTT-LRM on DTU with 3, 6, and 9 views; FLARE and NPM-GS are taken from Table 1. Each cell reports PSNR↑/SSIM↑/LPIPS↓.

Method3-view6-view9-view
FLARE19.54/0.736/0.42420.11/0.740/0.43220.78/0.746/0.437
YoNoSplat18.72/0.724/0.48519.43/0.725/0.49820.64/0.743/0.495
TTT-LRM17.41/0.721/0.45922.12/0.775/0.40422.95/0.782/0.393
NPM-GS24.84/0.826/0.28726.79/0.857/0.26828.07/0.878/0.256

In every setting, NPM-GS exceeds the best feed-forward result by 4.67–5.30 dB in PSNR and 0.08–0.10 in SSIM, and lowers LPIPS by about 0.14. With sparse inputs, both feed-forward methods show noticeable ghosting, which we attribute to cross-view inconsistency among per-view predictions: structures predicted from different views are misaligned when fused. This is exactly the inconsistency that the cross-view verification of NPM-GS detects and corrects (see Q3).

Feed-forward models therefore suit applications that need instant results, while per-scene refinement on top of their outputs suits high-quality reconstruction.

Q3: Combination with LRMs

We thank the reviewer for this suggestion, which matches the future direction stated in our conclusion.

NPM-GS only requires a dense Gaussian initialization aligned with the input views, so pixel-aligned Gaussians from an LRM such as YoNoSplat can directly replace DA3. Since the target of NPM-GS, cross-view consistency with the input views, does not depend on the initialization, a better prior only helps: more Gaussians fall within the NCC capture range (Fig. 1(d)) and can be verified with fewer iterations, and non-planar regions, which keep the prior's local shape (Sec. 4.4), directly benefit from a more accurate prior. Conversely, NPM-GS corrects residual cross-view inconsistencies in LRM predictions, so the combination pairs fast initialization with verification against the input images.

We thank the reviewer again; the additional experiments will be included in the revision manuscript.

AuthorsOpenReview ↗

Results Update

We thank Reviewer MP86 again. Following up on our previous response, we have uploaded the revised manuscript with the promised evaluations and revisions:

1. Quantitative Comparisons with YoNoSplat and TTT-LRM (Major Weakness & Q1) We evaluated YoNoSplat and TTT-LRM across DTU (Table 1, updated under the unified evaluation protocol), Mip-NeRF 360 (Table 3), and LLFF (Table 2):

DTU (Table 1; PSNR ↑\uparrow / SSIM ↑\uparrow / LPIPS ↓\downarrow):

Method3 Views6 Views9 Views
FLARE19.54 / 0.736 / 0.42420.11 / 0.740 / 0.43220.78 / 0.746 / 0.437
YoNoSplat20.58 / 0.749 / 0.47319.76 / 0.731 / 0.49720.26 / 0.737 / 0.505
TTT-LRM17.41 / 0.721 / 0.45922.12 / 0.775 / 0.40422.95 / 0.782 / 0.393
NPM-GS (Ours)24.84 / 0.826 / 0.28726.79 / 0.857 / 0.26828.07 / 0.878 / 0.256

Mip-NeRF 360 (4 Views, Table 3):

MethodPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrow
FLARE21.460.5360.265
YoNoSplat16.280.4400.563
TTT-LRM20.020.5350.472
NPM-GS (Ours)24.070.7410.254

LLFF (3 Views, Table 2):

MethodPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrow
FLARE14.700.2920.556
YoNoSplat18.270.5490.387
TTT-LRM15.700.5890.406
NPM-GS (Ours)21.900.7660.157

Qualitative results in revised manuscript reveal noticeable artifacts, distortions, and jitter under sparse views, which are significantly alleviated as the number of input views increases. This is consistent with YoNoSplat's paper, where ghosting is substantially reduced from 32 to 128 views. A possible explanation lies in task complexity: predicting depth alone from learned features is relatively straightforward, whereas predicting 3D Gaussian representations requires jointly estimating multiple coupled attributes, making the task substantially more challenging and prone to artifacts. This further highlights the necessity of per-scene optimization for achieving high-quality reconstruction under sparse-view settings.

2. Visual Comparisons & Related Work Discussion (Q2 & Q3)

  • Qualitative comparisons: Added visual comparisons against FLARE, YoNoSplat, and TTT-LRM in Fig. 3 and Appendix H.
  • Related work & positioning: Expanded Sec. 2 to discuss YoNoSplat and TTT-LRM and clarify the complementary relationship between feed-forward priors and per-scene verification.

We thank the reviewer again for the helpful comments and suggestions.

AuthorsOpenReview ↗

Looking forward to hearing from you

Dear Reviewer MP86,

We sincerely appreciate your time and effort in reviewing our manuscript and providing valuable feedback.

In response to your insightful comments, we have made the following revisions and clarifications:

  • We have analyzed why per-scene optimization remains essential alongside strong feed-forward models: under sparse views, feed-forward 3DGS methods perform suboptimally due to the task complexity of jointly predicting coupled 3D Gaussian attributes, whereas directly predicting dense geometric priors (e.g., depth) is much more tractable. Leveraging dense priors within our per-scene verification and optimization framework achieves substantially better reconstruction quality.
  • We have added quantitative and qualitative comparisons with YoNoSplat and TTT-LRM across all three benchmarks: DTU in Table 1, LLFF in Table 2, and Mip-NeRF 360 in Table 3, Fig. 3 and Appendix H (Figs. 9–11).
  • We have expanded Sec. 2 to discuss YoNoSplat and TTT-LRM and position our verification-based optimization as orthogonal to feed-forward priors.
  • We have expanded our sparse-view benchmark evaluation to forward-facing LLFF scenes in Table 2 and Sec. 5.2.

We hope these revisions adequately address your concerns regarding comparisons with modern feed-forward models and the practical value of per-scene optimization.

We look forward to actively participating in the Author-Reviewer Discussion session and welcome any additional feedback you might have on the manuscript or the changes we have made.

Once again, we sincerely thank you for your contributions to improving the quality of this work.

Best regards, The Authors of Submission 25601

Reviewer_rN7kOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes NPM-GS for sparse-view Gaussian Splatting, motivated by geometric ambiguity under joint geometry–appearance optimization. Starting from DA3 depth initialization, it directly refines Gaussian centers through coarse-to-fine discrete search along viewing rays, using cross-view NCC to evaluate candidate positions. Reliable Gaussians guide plane-snap and superpixel-based propagation into ambiguous regions, followed by reliability-aware training to improve rendering while preserving the refined structure. Experiments on DTU and Mip-NeRF 360 report improvements over the evaluated baselines in PSNR, SSIM, and LPIPS

Critical strengths
  • Strong empirical performance. The method substantially outperforms the evaluated baselines, particularly in PSNR.

  • A coherent refinement pipeline. The method integrates discrete cross-view verification, reliability identification, structural propagation, and subsequent reliability-aware training into a coherent framework

Critical weaknesses
  • Reliable anchors supporting a fitted plane do not necessarily establish planarity throughout the region, especially when unsupported areas contain curved surfaces or fine structures.

  • The reliability-identification step in Fig. 2 is underspecified. The label “stable” does not convey the cross-view consensus, spatial consensus, and minimum view-support criteria described in the text.

  • Figs. 3 and 9 exhibit visible holes that are absent in the compared methods, but the paper does not discuss these artifacts.

  • Limited evaluation across datasets. The experiments cover DTU and Mip-NeRF 360 but omit LLFF, a commonly used benchmark for sparse-view novel-view synthesis. Results on LLFF would strengthen the evidence that the method generalizes to forward-facing scenes.

Questions for authors
  • PGSR also uses cross-view NCC for continuous geometry optimization. How does the proposed discrete Gaussian-position search compare with this approach in geometric accuracy?

  • Could multiple Gaussians along the same reference ray converge to the same NCC minimum, causing redundant clustering?

  • How does plane-snap propagation avoid flattening such geometry, and are the propagated positions subsequently verified across views?

3 comments

We thank the reviewer for the positive assessment and the detailed questions.

W1 & Q3: Plane snapping and verification

Planarity is decided not by the anchors but by the dense DA3 depth over the whole region, including unanchored parts; the anchors (cross-view verified Gaussians) only fit the parameters of identified planes. Specifically, we segment each view’s DA3 depth into SLIC superpixels, label a superpixel planar only if its RANSAC plane fit has an inlier ratio ≥ 0.95 (threshold: 0.2% of scene scale), merge adjacent planar superpixels with normals within 5° that are coplanar with the merged plane, and refine boundaries with SAM2. Structures deviating from the plane thus lower the inlier ratio and prevent the superpixel from being labeled planar. Non-planar regions, including textureless curved surfaces, are never fitted with planes: superpixel warping keeps the local DA3 shape and only corrects its scale and offset with reliable anchors (Sec. 4.4; see also our supplementary videos). We will add this procedure to the revision.

As for verification, propagated Gaussians stay unreliable and unfrozen (Alg. 1, line 7) and re-enter the NCC search in the next iteration, where positions in textured regions are confirmed or corrected. In textureless regions, where NCC is not discriminative, the plane is constrained by anchors jointly fitted across views rather than a single-view prior, a limitation noted in Sec. 6.

W2: Reliability labels in Fig. 2

We used “stable” in Fig. 2 to convey the overall pipeline without technical details, but we agree that this single label does not reflect the three criteria in Sec. 4.3. We will revise the figure accordingly.

W3: Holes in Figs. 3 and 9

The black regions are areas not observed by any training view, either outside all training frusta or occluded. NPM-GS initializes Gaussians only from observed pixels and suppresses floaters rather than filling such space with oversized Gaussians, so these regions show the background color. Other methods often place large or low-opacity Gaussians there, which hide the holes but lack geometric support and visibly float as the viewpoint changes in our supplementary videos. These regions are excluded from evaluation for all methods: as stated in Sec. 5.1 and Table 1, DTU is evaluated only on pixels visible from the training views, using the same mask for all methods and settings (App. C.1). However, we did not visualize the masks or relate them to the black regions. The revision will add this discussion, a mask visualization and update Fig. 3.

W4: LLFF

We agree that LLFF is a representative forward-facing benchmark. Experiments under the standard sparse-view protocol are in progress, and the results will be included in the revision.

Q1: Comparison with PGSR

Both methods use cross-view NCC, but in different roles: PGSR uses it as a differentiable loss that corrects geometry locally via gradients within joint optimization, whereas NPM-GS uses it as a verifier that evaluates the full cost curve over discrete candidates. Two differences follow:

Coupling. PGSR optimizes the NCC loss jointly with the rendering loss, so geometry updates stay coupled with appearance and opacity; NPM-GS freezes appearance and updates only Gaussian centers during the search (Sec. 4.3), directly targeting the coupling identified in Sec. 3.

Reliability. Discrete evaluation yields one cost curve per source view, so we can check whether the per-view minima agree, whether the minimum lies inside the search interval, and whether the curve is flat or multi-modal, e.g., in weakly textured regions (App. A.4). This gives the per-Gaussian reliability labels that propagation relies on, whereas a gradient step only provides a descent direction and cannot tell whether an update is supported by multiple views.

We are also comparing PGSR and NPM-GS on DTU under identical settings and will include the results in the revision.

Q2: Redundant clustering

We address two possible readings; please let us know if we misunderstood.

(a) Merging surfaces along a ray. In each NPM iteration, visibility is determined by comparing each Gaussian’s projected depth with every view’s rendered depth (Sec. 4.2), and only Gaussians visible in the current reference view join its search, so Gaussians on occluded surfaces are not pulled onto the foreground; the initial warm-up establishes these occlusion relationships.

(b) Redundant stacking on one surface. Candidates are sampled along each Gaussian’s own reference ray, so its reference-view pixel stays essentially fixed (Sec. 4.3) and Gaussians projecting to different pixels cannot converge to one point. Only Gaussians on the same ray and visible surface may coincide; they come from different training views observing that point, so their number is bounded by the view count and the search adds no redundancy.

We thank the reviewer again; the additional experiments and figures will be included in the revision.

AuthorsOpenReview ↗

Results Update

We thank Reviewer rN7k again. Following up on our previous response, we have completed all promised experiments and uploaded the revised manuscript. Below we summarize the new experimental results and point to the corresponding revisions.

1. Sparse-View Evaluation on LLFF (W4) We evaluated NPM-GS and the baselines on forward-facing LLFF scenes under the standard 3-view protocol (added to Table 2 and Sec. 5.2, with qualitative comparisons in Fig. 3 and Appendix H):

MethodPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrow
FLARE14.700.2920.556
TTT-LRM15.700.5890.406
YoNoSplat18.270.5490.387
PGSR13.800.3260.487
SparseGS15.300.4010.414
DNGaussian18.590.5870.302
NexusGS20.590.7190.192
CA-3DGS21.860.7670.160
NPM-GS (Ours)21.900.7660.157

NPM-GS achieves the highest PSNR and lowest LPIPS while matching CA-3DGS in SSIM, confirming that our discrete 3D verification generalizes effectively to forward-facing scenes.

2. Quantitative Comparison with PGSR on DTU (Q1) We compared PGSR and NPM-GS on DTU (3, 6, and 9 views) under identical camera parameters and visibility masks (added to Table 1 and discussed in Sec. 2, with additional LLFF/Mip-NeRF 360 results in Tables 2–3 and visual comparisons in Fig. 3 and Appendix H; each cell reports PSNR ↑\uparrow / SSIM ↑\uparrow / LPIPS ↓\downarrow):

Method3 Views6 Views9 Views
PGSR17.51 / 0.6955 / 0.380620.75 / 0.7574 / 0.342323.88 / 0.8230 / 0.3037
NPM-GS (Ours)24.84 / 0.8259 / 0.286726.79 / 0.8574 / 0.267528.07 / 0.8778 / 0.2558

NPM-GS outperforms PGSR across 3, 6, and 9 views. Under sparse inputs, optimizing cross-view NCC as a continuous gradient loss still suffers from narrow local capture basins and appearance co-adaptation, whereas our decoupled discrete hypothesis search and reliability-guided propagation recover substantially cleaner geometry and novel-view renderings.

3. Summary of Other Manuscript Revisions

  • Planar region extraction & verification (W1 & Q3): Added full implementation details in Sec. 4.2 and Appendix C.2.
  • Reliability identification in pipeline diagram (W2): Revised Fig. 2(c) and its caption (along with Sec. 4.3) to replace the shorthand "stable" label with an explicit consensus check reflecting our reliability criteria.
  • Unobserved regions & evaluation masks (W3): Detailed the DTU visibility mask protocol and unobserved black regions in Sec. 5.2 and Appendix C.1, and updated the qualitative results and visibility mask visualizations in Fig. 3 and Appendix H.

We thank the reviewer again for the constructive feedback, which helped substantially strengthen our paper.

AuthorsOpenReview ↗

Looking forward to hearing from you

Dear Reviewer rN7k,

We sincerely appreciate your time and effort in reviewing our manuscript and providing valuable feedback.

In response to your insightful comments, we have made the following revisions to our manuscript:

  • We have added sparse-view quantitative and qualitative evaluation results on the LLFF dataset in Table 2, Sec. 5.2, Fig. 3, and Appendix H (Fig. 11).
  • We have included quantitative and qualitative comparisons with PGSR in Tables 1–3, Fig. 3, and Appendix H (Figs. 9–11), along with a methodological discussion in Sec. 2.
  • We have added full implementation details on planar region detection, coplanar merging, and SAM2 boundary refinement in Sec. 4.2 and Appendix C.2.
  • We have updated the pipeline diagram in Fig. 2(c) to explicitly reflect the consensus-based reliability identification criteria in Sec. 4.3.
  • We have added explanations of unobserved black regions and the DTU visibility mask protocol in Sec. 5.2 and Appendix C.1, along with evaluation mask visualizations in Fig. 3 and Appendix H (Fig. 9).

We hope these revisions adequately address your concerns.

We look forward to actively participating in the Author-Reviewer Discussion session and welcome any additional feedback you might have on the manuscript or the changes we have made.

Once again, we sincerely thank you for your contributions to improving the quality of this work.

Best regards, The Authors of Submission 25601

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank our reviewers for their time and effort in providing thorough and constructive feedback on our paper.

We are encouraged to see that our reviewers recognize this work as a well-motivated and effective framework for sparse-view 3D Gaussian Splatting:

  • Reviewers MP86 and GXCT highlight that our paper provides a "compelling diagnosis of the problem" through controlled experiments and identifies "geometry–appearance co-adaptation as a key challenge in sparse-view 3DGS".
  • Reviewers rN7k and MP86 recognize our "coherent refinement pipeline", noting that NPM-GS "directly verifies 3D Gaussian centers using multi-view evidence, separating geometric verification from appearance optimization".
  • Reviewers rN7k and MP86 commend our "strong empirical performance", noting that NPM-GS "substantially outperforms the evaluated baselines, particularly in PSNR", while "effectively eliminating floaters and preserving high-fidelity details".

We summarize the positioning and significance of our work as follows:

  • From Continuous Joint Fitting to 3D-Native Discrete Verification: Continuous optimization permits geometry–appearance co-adaptation and pulls toward unverified priors, while cross-view NCC flattens outside a local capture basin. NPM-GS instead decides geometry via discrete multi-view verification directly on 3D Gaussians before refining appearance.
  • Equipping Fast Feed-Forward Models with Strict Consistency Verification: Data-driven feed-forward models offer fast inference from learned distributions, yet lack strict scene-specific multi-view consistency checks—causing cross-view misalignment and ghosting under sparse inputs. NPM-GS bridges this gap via test-time optimization with explicit cross-view verification.
  • Orthogonality to Stronger Future Feed-Forward Models: NPM-GS is strictly orthogonal to, rather than competing with, feed-forward reconstruction models. Stronger future depth or Gaussian priors can directly initialize tighter search basins, placing more primitives within the NCC capture range for fewer search iterations and higher reconstruction quality.

We highlight our contributions as follows:

  • Coarse-to-fine 3D-native PatchMatch verification that scores discrete ray-wise Gaussian hypotheses via triangulation- and incidence-weighted multi-view NCC.
  • Dual structural propagation combining cross-view RANSAC plane snapping and anchor-guided superpixel depth warping.
  • Reliability-aware training with refined-depth supervision, outlier pruning, and planar cross-view color regularization.

As suggested by the reviewers, we have revised our manuscript as follows:

Additional Results:

  • Added sparse-view evaluation results on the LLFF dataset in Table 2 and Sec. 5.2, as suggested by Reviewer rN7k.
  • Added quantitative comparisons with YoNoSplat and TTT-LRM in Tables 1–3 and qualitative visual comparisons in Fig. 3 and Appendix H, as suggested by Reviewer MP86.
  • Included quantitative comparisons with PGSR in Tables 1–3, as suggested by Reviewer rN7k.
  • Added 3D geometric accuracy evaluation using Chamfer Distance on DTU in Sec. 5.2 and Appendix G, as suggested by Reviewer GXCT.
  • Added visualizations of the misaligned DA3 initialized point cloud and test-view renderings in Appendix F, as suggested by Reviewer GXCT.
  • Updated qualitative results and added DTU visibility mask visualizations in Fig. 3 and Appendix H, as suggested by Reviewers rN7k and GXCT.

Other Modifications:

  • Expanded the discussion of prior work (including FatesGS, Sparse2DGS, PGSR, YoNoSplat, and TTT-LRM) in Sec. 2 to clarify our differences, as suggested by Reviewers GXCT, MP86, and rN7k.
  • Updated Fig. 2(c) to explicitly reflect the reliability identification criteria in Sec. 4.3, as suggested by Reviewer rN7k.
  • Added implementation details on planar region extraction in Appendix C.2, as suggested by Reviewer rN7k.
  • Clarified the DA3 initialization baseline in Sec. 5.3 and Table 4, and detailed the DTU visibility mask protocol and unobserved regions in Sec. 5.2 and Appendix C.1, as suggested by Reviewers rN7k and GXCT.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.