acceptodds
Under review as a conference paper at ICLR 2027

SampleSplat: Efficient Feed-Forward 3D Gaussian Splatting via Scene-Aware Spatial Sampling

Abstract

Feed-forward 3D Gaussian Splatting enables rapid scene reconstruction, but pixel-aligned Gaussian allocation couples representation capacity to input resolution and view count, preventing adaptive allocation of the Gaussian budget according to scene complexity. To address this limitation, we present SampleSplat, a feed-forward framework that decouples Gaussian generation from the input pixel grid through scene-aware probabilistic sampling. Given 3D anchors constructed from image features and estimated geometry, our sampling module jointly learns anchor selection and spatial offsets, enabling Gaussians to be adaptively allocated and positioned under a shared budget. The module is trained end-to-end using only reconstruction supervision, without handcrafted complexity measures or auxiliary allocation labels. As a result, SampleSplat supports variable-view inputs, including monocular input, and flexible input resolutions within a single model, while enabling adaptive allocation under a controllable Gaussian budget Experiments on DL3DV-10K demonstrate state-of-the-art novel view synthesis performance across these settings, with monocular reconstruction taking  ms at the native resolution of .

Primary area:
foundation or frontier models, including LLMs
Keywords:
3D Gaussian Splatting, Feedforward GS

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GscZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SampleSplat, a feed-forward 3DGS framework from uncalibrated input images. Instead of predicting fixed number of Gaussians, SampleSplat predicts a categorical smapling probability and a local 3D Gaussian offset distribution per anchor, drawing anchors with replacement to place primitives where scene complexity is high. SampleSplat achieves state-of-the-art performance.

Critical strengths

[S1] The visualizations of the learned sampling probability are intuitive and show that the model naturally learns where to concentrate on without heuristics.

[S2] The results are strong and the qualitative result shows sharp novel-view synthesis compare to other baselines.

Critical weaknesses

[W1] Missing adaptive / compact 3DGS baselines: Figure 2 highlights F4Splat and Off The Grid, but they are not included in the related work or compared in any experiment. The paper also misses C3G [1], which compresses pose-free feed-forward 3DGS to ~2K Gaussians, and closely related designs like Scaffold-GS (anchor + offset decoding) [2].

[W2] In Table 4, learned sampling improves PSNR by only about 0.6, whereas GS Depth head and point transformer drive most of the gain.

[W3] Because the sampling probability receives its gradient through each Gaussian's opacity, an anchor's probability only increases when making its existing Gaussians more opaque lowers the rendering error. However, making one Gaussian more opaque is not the same as needing more Gaussians in that location (for example, a blurry edge covered by an already-opaque Gaussian needs to be split into several smaller Gaussians, not made more opaque). Also, anchors that are not sampled in a step get no gradient at all even if their rendering error is huge, while anchors sampled multiple times accumulate larger gradients.

Questions for authors

[Q1] How does SampleSplat performs against F4Splat, Off The Grid, and C3G under matched # Gaussians budgets?

3 comments

Dear Reviewer GscZ,

Thank you for taking the time to review our work. Your comments helped us clarify what learned sampling contributes and how the opacity surrogate works. We have marked all revisions in blue.

  • Q1 and W1. Sections 4.1–4.2 add baselines and report default and matched-budget comparisons in Tables 1–2 and 4–5.
  • W2. Section 4.2 (Table 6) and Section C.6 (Table 12) report the sampling ablations and matched-budget allocation results.
  • W3. Section 3.3 and Section C.7 explain the opacity surrogate and how unsampled anchors can receive indirect gradients. Table 13 compares the surrogate with Gumbel-ST and REINFORCE.

We would welcome any further feedback. If you feel that our response and revisions address your concerns, we would be grateful if you would reconsider your rating.

Best regards,
The Authors

Thank you for the detailed rebuttal and for running the additional experiments. However, two concerns still remain regarding the comparisons.

  1. In the 256x256 matched comparison, SampleSplat degrades monotonically as the number of Gaussians increases, with a similar degradation at 518x518 comparison. Nearly 1dB drop suggests that SampleSplat does not scale to larger Gaussian budgets.
  2. Although C3G cannot retrained, comparing it with SampleSplat at 2K Gaussians would be a fair comparison. Since SampleSplat supports a controlalble Gaussian budget, a direct evaluation at a matched 2K budget is still missing.

Given these remaining concerns, I will maintain my score.

[Q1] Reconstruction quality as the Gaussian budget increases

Thank you for pointing out how reconstruction quality changes as the Gaussian budget increases.

  1. At 256×256, PSNR decreases at the highest tested budget, while SampleSplat retains its advantage at every matched budget. From the best-performing budget to the largest tested budget, PSNR decreases by 0.98 dB for two input views and 0.90 dB for six input views. At the largest tested budget, SampleSplat still leads the strongest baseline at that budget by 2.54 dB and 4.57 dB, respectively. Across the comparisons with matched input FOV, resolution, and Gaussian count, SampleSplat outperforms the corresponding baselines on all four quality metrics, with PSNR gains of 2.37–6.23 dB.
  2. At 518×518, increasing the budget improves reconstruction quality over part of the tested range. With two input views, increasing the budget from 44K to 77K raises PSNR from 25.9987 dB to 26.7233 dB and improves the other three metrics. Increasing the budget further to 88K lowers PSNR by only 0.025 dB. With six input views, PSNR likewise peaks at the intermediate tested budget of 230K. The best-performing budget therefore depends on the input configuration.
  3. The other models have a fixed upper limit on their Gaussian budgets. For a given input configuration, EcoSplat, F4Splat, and OffTheGrid each have a fixed maximum Gaussian budget, and their training covers the corresponding full-budget settings.
  4. SampleSplat is trained with a single Gaussian budget setting, and its performance at all other budgets in the reported sweeps reflects test-time generalization from the same checkpoint. Its sampling-based design imposes no fixed architectural cap on the Gaussian count, so users can choose a budget at inference based on the trade-off between reconstruction quality and computational cost.

Nevertheless, SampleSplat outperforms the evaluated baselines on all four reconstruction metrics at matched Gaussian budgets representative of those commonly used by existing models. A single SampleSplat checkpoint supports different input resolutions, aspect ratios, and view counts, while allowing users to specify the Gaussian count at inference to fit their resource constraints without retraining.

[Q2] Direct comparison with C3G at a matched 2K Gaussian budget

To address the reviewer’s suggestion, we fine-tuned the existing SampleSplat checkpoint for only 0.4K steps at C3G’s input resolutions, setting the total number of sampled Gaussians to 2K to match C3G’s budget. With this brief fine-tuning, SampleSplat outperforms C3G on all four reconstruction metrics at every tested view count. C3G uses a dedicated checkpoint for inference at N=2N=2, while SampleSplat uses the same fine-tuned checkpoint for all view counts reported below.

2 views6 views8 viewsAverage
MethodInput ResolutionPSNR↑SSIM↑LPIPS↓DISTS↓#GSPSNR↑SSIM↑LPIPS↓DISTS↓#GSPSNR↑SSIM↑LPIPS↓DISTS↓#GSPSNR↑SSIM↑LPIPS↓DISTS↓#GS
C3G (GT Tgt. Pose)224×224/256×25615.53360.43310.56630.28552K15.18110.44330.62240.37112K15.21020.44370.62110.37032K15.30830.44000.60330.34232K
SampleSplat (GT Tgt. Pose)224×224/256×25619.29200.51240.40930.24432K17.90800.45670.49910.29552K17.98800.46170.49530.29412K18.39600.47690.46790.27802K
SampleSplat (GT Tgt. Pose)540×96025.94710.80880.25070.1015113K22.53890.69940.30710.1121259K22.82960.70890.29300.1055324K23.77190.73900.28360.1064232K
16 views24 views32 viewsAverage
MethodInput ResolutionPSNR↑SSIM↑LPIPS↓DISTS↓#GSPSNR↑SSIM↑LPIPS↓DISTS↓#GSPSNR↑SSIM↑LPIPS↓DISTS↓#GSPSNR↑SSIM↑LPIPS↓DISTS↓#GS
C3G (GT Tgt. Pose)256×25614.67600.43920.64440.41662K13.42840.40850.68160.51052K13.18980.40300.68830.52692K13.76470.41690.67140.48472K
SampleSplat (GT Tgt. Pose)256×25617.63100.44370.51540.30932K17.47100.43700.54750.33842K16.69400.40440.55710.33712K17.26530.42840.54000.32832K
SampleSplat (GT Tgt. Pose)540×96022.55940.69180.27300.0962648K21.11380.64320.31200.1133972K20.99450.64030.31240.11331296K21.55590.65850.29910.1076972K

We hope the additional experiments and clarifications address your concerns, and we would appreciate it if you would consider revisiting your score in light of these results.

Reviewer_9UDdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SampleSplat, a feed-forward method that learns how to allocate and position Gaussians according to scene characteristics. It builds 3D anchors from image features and geometry estimated by pretrained DA3, then uses a Point Transformer to aggregate spatial and multi-view information. The model predicts selection probabilities and offset distributions for the anchors and samples Gaussians within a specified budget. An opacity-based gradient surrogate enables learning despite the discrete anchor selection. A single model supports both single-image and multi-view inputs. Reconstruction quality is evaluated on DL3DV and RealEstate10K.

Critical strengths
  • Jointly learning anchor selection probabilities and 3D offset distributions is a sensible design. It allows the model to direct a given Gaussian budget toward regions that need more capacity without requiring complexity annotations or separate allocation supervision. Figures 6 and 8 show how the learned distributions respond to object boundaries and texture variations.
  • A single model supports varied input conditions while producing compact Gaussian representations. It is trained with up to 8 input views and evaluated with up to 32. In the single-image setting of Table 5, it reconstructs a scene from a 540 × 960 image using 64,800 Gaussians, with a reported processing time of 90.97 ms. The evaluation on RealEstate10K also tests generalization to a dataset not used for training.
Critical weaknesses
  • The additional benefit of the learned allocation scheme is not sufficiently established. Off The Grid [1] already learns Gaussian positions and varies their number across image regions. The paper discusses the design differences but provides no direct performance comparison. The Random Sampling ablation in Table 4 is also unclear. It is not specified whether this variant only replaces the learned anchor probabilities with a uniform distribution or also changes how the offsets are learned. It is also unclear whether the Gaussian count is matched across variants. The current results support the sampling module as a whole but do not clearly isolate the contribution of the learned allocation probabilities.
  • Differences in evaluation conditions make the reconstruction quality comparisons difficult to interpret. Appendix B.1 states that baselines requiring cropping are evaluated against matching ground-truth crops, whereas SampleSplat uses the full original field of view. The methods may therefore be evaluated on different image regions as well as using different input resolutions. Tables 1 and 2 separate the target pose settings, but Figure 1 does not identify the setting used for each method in its quality and Gaussian count comparison. Results on a common field of view under the same pose protocol would help clarify how much of the reported quality difference comes from the reconstruction methods.
Questions for authors
  • Please clarify what changes in the Random Sampling and w/o Sampling variants in Table 4, including whether the Gaussian count is held fixed. Does Random Sampling replace only the anchor selection probabilities with a uniform distribution while retaining the learned offsets? Could you also explain why Off The Grid [1] was not included in the comparisons and share any existing results under comparable conditions?
  • Which target pose setting is used for each method in Figure 1? If results are available for representative baselines evaluated on a common field of view under the same pose protocol, please provide them.

[1] Moreau, Arthur, et al. "Off the grid: Detection of primitives for feed-forward 3d gaussian splatting." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

1 comment

Dear Reviewer 9UDd,

Thank you for taking the time to review our work. We appreciate your questions about the sampling ablations and the fairness of the comparisons. We revised the manuscript in response and marked all changes in blue.

  • Q1 and W1. Section 4.2 (Table 6) clarifies the sampling variants and their Gaussian budgets. Tables 1–2 and 5 report the default and matched-budget comparisons with OffTheGrid, respectively.
  • Q2 and W2. Section 4.2 and Appendix B.2 clarify the target-pose settings underlying Figure 1. Section C.4 (Tables 9–10) reports the configurations and results under matched fields of view.

We would welcome any further feedback. If you feel that our response and revisions address your concerns, we would be grateful if you would reconsider your rating.

Best regards,
The Authors

Reviewer_GqsYOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

SampleSplat decouples Gaussian count from the pixel grid in feed-forward 3DGS. It builds 3D anchors on a frozen DA3 backbone, predicts a categorical distribution over anchors plus per-anchor offset Gaussians, and samples centers under a budget M = ρ(N)·L. Anchor selection is trained via an opacity-based gradient surrogate. A single model handles 1–32 views at native resolution, with strong DL3DV results (e.g., 22.01 dB monocular PSNR with 64.8K Gaussians in 91 ms).

Critical strengths

Allocation is learned purely from reconstruction loss. I love this idea. The opacity surrogate is simple, keeps the forward pass unchanged, and needs no handcrafted complexity measure or auxiliary target, unlike Off The Grid, F4Splat, or ATSplat. The learned allocation also responds to appearance (e.g., reflections), not just geometry.

One model covers monocular to dense views at native resolution, with large margins and far fewer Gaussians than the baselines.

Critical weaknesses

The most related adaptive-allocation works are neither discussed nor compared: SparseSplat (CVPR’26) uses probabilistic sampling with a point-cloud decoder, but its sampling is entropy-based. EcoSplat (CVPR’26) offers explicit control of the primitive count. SplatWeaver (arXiv 2605.07287) learns per-location cardinality.

All quantitative baselines are pixel- or voxel-aligned methods at unmatched budgets, so the central claim that learned sampling beats heuristic allocation is not directly tested.

Budget control and the surrogate are under-analyzed. ρ(N) is fixed, and there is no quality-vs-budget curve. The surrogate is biased, yet there is no sensitivity study for β/η and no comparison with Gumbel-ST, REINFORCE, or top-k.

Questions for authors

How does SampleSplat compare with SparseSplat, EcoSplat, and SplatWeaver (or at least one of them) under the same Gaussian budget and resolution?

Can M be changed freely at test time, independent of ρ(N)? Please report PSNR vs. #Gaussians for SampleSplat over a range of budgets, ideally against a baseline pruned or controlled to the same counts.

1 comment

Dear Reviewer GqsY,

Thank you for taking the time to review our work and for your constructive suggestions. Your feedback helped us strengthen the evaluation of Gaussian budget control. We have marked all revisions in blue.

  • W1. Sections 4.1–4.2 add adaptive-allocation baselines and report their results in Tables 1–2.
  • Q1, Q2, and W2. Section 3.2 explains how to set MM directly or adjust ρ\rho(N) at inference. Tables 4–5 report matched-budget comparisons, and Figure 1(b) shows PSNR versus Gaussian count.
  • W3. Figure 1(b) illustrates the quality–budget trade-off. Sections C.5–C.7 (Tables 11–13) report sensitivity to β\beta and η\eta, matched-budget allocation results, and comparisons with alternative gradient estimators, alongside a discussion of top-kk.

We hope these revisions address your questions, and we would welcome any further feedback.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.