- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces GeoGSplat, a geometry-grounded feed-forward 3D Gaussian Splatting (3DGS) framework that operates in a non-pixel-aligned paradigm. While existing pixel-aligned methods suffer from a primitive count that scales linearly with input resolution and view count (), recent non-pixel-aligned methods predict a fixed number of Gaussians via learnable tokens but discard explicit ray/depth geometric constraints, leading to poorly placed Gaussian centers and severe rendering artifacts under large viewpoint changes.
Critical strengths
The core diagnosis of why non-pixel-aligned 3DGS methods (like TokenGS) fail under large viewpoint changes—namely, photometric overfitting due to underconstrained Gaussian centers—is insightful and empirically well-substantiated (especially in Section 4.3, Figures 5 & 6, and Appendix A.1). By coupling fixed-budget Anchor queries with continuous Triplane feature fields, GeoGSplat maintains a constant primitive count () regardless of whether 2, 4, or 12 views are input, while achieving superior geometric accuracy (Chamfer Distance reduced from to on NRGBD compared to TokenGS) and significantly better extrapolation to distant target views.
A common pitfall in large-scale feed-forward 3D vision papers is confounding architectural gains with dataset scaling or supervision differences. The authors proactively address this in Appendix A.1 (Table 4) by retraining TokenGS under the exact same five-dataset mixture, resolution, and explicit Chamfer Distance geometry loss (TokenGS-R + Geometry loss). Showing that GeoGSplat still outperforms TokenGS-R + Geometry loss by PSNR on DTU and reduces CD from to convincingly proves that the hierarchical Anchor-Triplane architecture itself—not merely the addition of GT 3D supervision—drives the performance leap.
Critical weaknesses
Joint self/cross-attention over Anchor queries and Triplane queries across 12 shared attention blocks. Two stages of Triplane upsampling up to with 3D-to-2D cross-plane fusion and image-conditioned cross-attention at high spatial resolutions (Eq. 18). Computing symmetric Chamfer Distance and Farthest Point Sampling (FPS) on points during training (requiring 128 NVIDIA L20 GPUs). And The paper only reports the number of Gaussians (NUM) in Table 1, completely omitting feed-forward inference latency (ms), peak GPU memory usage, and rendering FPS compared to MVSplat, DepthSplat, AnySplat, and TokenGS. Without wall-clock runtime and memory metrics, the claim of "computational efficiency" remains only half-proven (rendering is fast, but encoding/decoding might be a bottleneck).
Unlike purely photometric feed-forward 3DGS methods (e.g., pixelSplat, MVSplat, TokenGS) that can be trained on posed RGB videos alone (such as RealEstate10K or ACID), GeoGSplat collapses without explicit ground-truth 3D point cloud supervision (w/o Geometry Losses in Table 3 drops PSNR from to on DTU and CD degrades from to ). This restricts its training exclusively to datasets with accurate RGB-D or MVS depth. Training Only on Input Views: In Section 3.4 and 4.1, the authors note that during training, is computed only by rendering at the 4 input camera views without sampling held-out novel target views. In 3DGS, supervising only on context views notoriously encourages view-dependent overfitting or degenerate flat Gaussians. Although regularizes the 3D positions, it is unclear why standard novel-view photometric supervision was omitted and how much performance is lost/gained by this choice.
Questions for authors
Could you provide a quantitative breakdown of: (a) encoder-decoder feed-forward inference time (ms), (b) rasterization time (ms), and (c) peak GPU memory consumption across 2, 4, and 12 input views compared to MVSplat, DepthSplat, AnySplat, and TokenGS? Specifically, how expensive is the image-conditioned cross-attention (Eq. 18) at the and Triplane resolutions?
Equations (4) and (7) bound the Anchor centers and Triplane queries to via . In Appendix B.5, you mention that during geometric evaluation, "the ground-truth point cloud determines its scale" to normalize the scene into . During inference (before GT geometry is known), how is the scene coordinate frame and scale defined solely from the input camera poses so that unbounded scenes (like Mip-NeRF 360 in Appendix A.3) fit properly inside the Triplane bounds without clipping background geometry?
Why did you choose to compute only on the 4 input views during training rather than held-out target views (which is standard in feed-forward NVS)? Does supervising on novel target views further improve PSNR/LPIPS, or allow the model to rely less on GT depth supervision ()?
Notation in Eq. (8): In Eq. (8), . Are the learnable offset tokens shared across all Anchors, or distinct per Anchor? Moreover, since already has a 1-to-1 correspondence with the sampled local reference points , why use an all-to-all cross-attention with learnable queries instead of a point-wise MLP or local self-attention directly on
3 comments
Official Response by Authors (2/3)
W2: Dependence on geometry supervision and input-view training (Q3)
Summary: GeoGSplat's improvements are enabled by our architecture, which is closely integrated with geometry supervision, rather than by geometry supervision alone. Simply adding geometry loss without a design that effectively leverages geometry is insufficient. Although geometry supervision is important to our method, GeoGSplat maintains stable NVS performance even with imperfect depth annotations. Moreover, additional novel-view photometric supervision does not yield significant improvements over input-view-only supervision.
Dependence on geometry supervision. As demonstrated by our additional TokenGS retraining experiments in the General Response, simply adding geometry loss is insufficient to match GeoGSplat. Even with its original novel-view photometric supervision, visibility loss, and additional geometry loss, TokenGS still underperforms GeoGSplat, which achieves 52.6% lower CD on DTU and 41.4% lower CD on NRGBD.
Unlike TokenGS, which directly predicts Gaussians from tokens, GeoGSplat explicitly models intermediate geometry through Anchors and dense points, complemented by a Triplane representation that encodes fine-grained high-dimensional scene features queryable at arbitrary 3D coordinates. This design enables more effective use of geometry, while the geometry supervision makes these intermediate structures geometrically meaningful.
We acknowledge that geometry supervision is important for our explicit intermediate geometry. However, GeoGSplat does not require perfectly accurate or complete GT depth. Experiments with noisy and missing depth using our lightweight model show PSNR changes of at most 0.05 dB, demonstrating stable NVS performance despite some degradation in CD under noisy depth.
| Training setting | DTU PSNR ↑ | DTU CD ↓ | NRGBD PSNR ↑ | NRGBD CD ↓ |
|---|---|---|---|---|
| Original | 19.79 | 0.019 | 23.70 | 0.019 |
| Noisy depth | 19.74 | 0.030 | 23.75 | 0.025 |
| Missing depth | 19.76 | 0.019 | 23.67 | 0.020 |
Furthermore, Tables 3 and 4 support the effectiveness of our design. Component ablations consistently degrade performance, while the controlled comparison in Table 4 shows higher PSNR and lower CD under matched training conditions and geometry supervision.
Input-view-only photometric supervision. Novel-view photometric supervision alone cannot resolve all geometric ambiguities. As shown in Fig. 6, TokenGS still produces geometric outliers, even after opacity filtering, while points unprojected from rendered depth remain displaced from GT geometry. These errors may have limited impact near input views but become increasingly apparent under large viewpoint changes (Fig. 5).
Moreover, we also tested novel-view photometric supervision on GeoGSplat, but did not observe a significant improvement over input-view-only supervision. Similarly, AnySplat adopts context-view-only photometric supervision.
Official Response by Authors (3/3)
Q2: Scene normalization and unbounded scenes
Summary: GeoGSplat does not require GT geometry for inference-time scene normalization and maintains stable NVS performance with approximate scene scales. It also achieves strong NVS and geometric reconstruction performance on large-scale scenes without scene contraction or background masking.
Inference-time normalization. GeoGSplat does not require GT geometry to estimate scene normalization parameters at inference time. The scene center can be estimated from input camera poses using either camera-ray intersection or mean camera centering, depending on the dataset configuration. Similar to TokenGS, GeoGSplat then normalizes camera translations using a dataset-dependent fixed scale factor.
To verify robustness to scene-scale estimation, we additionally evaluated GeoGSplat on DTU using a fixed scale factor independent of GT geometry. GeoGSplat still outperforms all baselines in Table 1 without GT scale, even though these baselines are evaluated with GT scale.
| Normalization strategy | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|
| GT scale | 20.48 | 0.771 | 0.137 |
| Fixed scale (w/o GT geometry) | 20.35 | 0.759 | 0.148 |
Unbounded scenes. GeoGSplat achieves the highest PSNR (21.59 dB) and lowest CD (0.015) on Mip-NeRF360 without scene contraction or background masking. For distant backgrounds such as skies, finite Gaussians approximate their appearance rather than physically accurate infinite-depth geometry. Nevertheless, these results demonstrate that GeoGSplat remains effective on large-scale scenes with unbounded backgrounds.
Q4: Anchor-to-Point cross-attention
Summary: Local cross-attention enables more effective aggregation of geometric features within each anchor, improving both NVS quality and geometric reconstruction compared with pointwise MLP prediction.
Offset queries. The same set of K = 16 learnable offset queries is shared across all anchors.
Cross-attention design. Cross-attention enables each offset query to adaptively aggregate local Triplane features, allowing point offsets to be predicted using the overall local geometry rather than individual point features. To validate its effectiveness, we replaced cross-attention with a pointwise MLP, resulting in a 0.30 dB PSNR decrease and a 0.002 increase in CD.
Official Response by Authors (1/3)
We thank the reviewer for the constructive feedback. We address the main weaknesses and their related questions below.
Summary: The efficiency emphasized in our paper is the representation efficiency of non-pixel-aligned methods over pixel-aligned methods. Our additional experiments demonstrate that this representation efficiency translates into substantial practical computational gains while maintaining superior NVS and reconstruction quality, without significant overhead from high-resolution triplane cross-attention.
Computational profiling. GeoGSplat's representation efficiency translates into substantial practical computational gains while achieving superior NVS and reconstruction quality. For feed-forward 3DGS, practical efficiency is primarily reflected in reconstruction speed and rendering throughput. We therefore profile GeoGSplat (351.9M parameters) against AnySplat across 12–64 input views, reporting inference latency, FLOPs, rendering throughput, and peak GPU memory. The computational advantages become more pronounced as the number of input views increases.
Parentheses indicate GeoGSplat's resource usage as a percentage of AnySplat's for Gaussians, inference latency, FLOPs, and peak GPU memory, and relative rendering throughput (×) for FPS.
High-resolution triplane cross-attention. The two high-resolution triplane cross-attention stages are lightweight. We pool image key/value features to 64 tokens before cross-attention. We further measured the latency of the two high-resolution triplane refinement stages.