acceptodds
Withdrawn from ICLR 2027

Gauss What You Need: Compact Gaussian Splatting Across Scene Scales

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

3D Gaussian Splatting reconstructs a scene as a collection of Gaussian primitives from a set of posed photographs called the capture. The number of primitives used to represent the scene affects reconstruction quality, storage, and rendering cost. How to select this number automatically across capture scales remains unresolved: configurations effective on standard benchmarks can leave larger captures with too few Gaussians to reconstruct fine details. We observe that the surface to represent, given by the capture's extent and resolution, is known before training, whereas its content complexity becomes apparent during training, through the reconstruction quality on the training views. We introduce TangoGS, which combines capture-derived model sizing with training-based adaptation: the capture determines the scale of the model, and training feedback determines its final size within that scale. Before training, TangoGS derives a learning allowance for model growth from the capture's total pixels after discounting views that re-observe the same scene points. During training, reconstruction quality guides how many Gaussians to add and remove. On 13 standard benchmark scenes, TangoGS matches the mean PSNR of the best-performing evaluated baseline, LeGS, with 48% fewer Gaussians. On eight large captures, the same configuration automatically scales to larger models when necessary, achieving the highest mean PSNR among evaluated methods: dB above the runner-up with as many Gaussians. Together, capture-derived learning allowances and training-quality guided density control enable a state-of-the-art quality–size compromise across scene scales without retuning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Density control

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_krPrOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces TangoGS, a density control framework for 3D Gaussian point clouds designed to automatically adapt the number of Gaussian primitives across different acquisition scales. The method first estimates the learning margin by adjusting the total input pixels and the average SfM trajectory length. During optimization, the PSNR of training views is used to regulate the densification target and pruning intensity. Finally, a sensitivity-based pruning step determines which Gaussian primitives to retain.

The proposed method is evaluated on 13 standard benchmark scenes and 8 larger-scale scenes, compared against 13 published baseline methods. Results show comparable quality-size trade-offs on standard scenes, while achieving improved reconstruction quality on larger-scale scenes—all without requiring scene-specific re-tuning.

Critical strengths
  1. This article employs a simple approach to achieve better control over the number of Gaussians, by using all the pixels and the track length to determine the initial upper limit for optimizing the number of Gaussians.

  2. The concept of capacity allocation under capture conditions and adaptive allocation based on training quality is clearly defined and easy to integrate into the existing 3DGS pipeline. This paper includes component-level reduction experiments, comparison of matching counts with LiteGS, alternative capture statistical methods, redundant view experiments, and portability tests for another implementation. These experiments provide certain support for the practical application of the proposed controller.

Critical weaknesses
  1. This article aims to enable the automatic adjustment of the optimization amount of Gaussian quantities, thereby avoiding the need for manual setting of the Gaussian quantity in the optimization process. However, the adopted method will automatically introduce other parameters that require manual setting in Equation (2) (Line 140)

  2. The relationship between the proposed control signal and the required representational capacity lacks sufficient justification. This method relies on two heuristic mappings: there is a square root relationship between the trajectory adjustment pixels and the learning tolerance, and there is a linear relationship between the training PSNR and the retention rate. Although these rules have empirical basis, the paper does not adequately explain why they should be able to be generalized to images under different observation quality, geometric structure, or photometric consistency.

  3. The images and tables in the article make it difficult to distinguish the author's contributions. For instance, in Figure 4, the purple F dots and the star of TangoGS are always very close to each other, making it hard to tell from this ablation study whether the proposed module in the article can have a significant effect. Moreover, in the table in the supplementary materials, the lack of bold font annotations makes it difficult to discern the quality of the data.

  4. This paper reads like an incremental improvement from previous paper without significant insights and looks like not enough for ICLR venue.

Questions for authors
  1. In Figure 1 (a), at the bottom of "Learning allowance", there is a text "sublinear mapping". What does this mean?
  2. In line 147, you used the statistics of 13 scenarios to set T_ref and T_lin. How do you ensure their generalization ability in other standard scenarios? Can their generalization ability be demonstrated through ablation experiments? And are the hyperparameters in Equation (3) also empirical values? For example, s = 1.7, c = 0.0075, P = 20 dB, etc.
  3. In your ablation study figure 4, does point F indicate that your method has degenerated into LiteGS? If so, based on the data in the graph, in the standard scenes, the two are very close. For large scenes, your method needs to achieve approximately a 0.6dB improvement with about 2 times the number of Gaussians, while SSIM and LPIPS hardly change. Is this because you used the PSNR signal to supervise the learning allowance? Why did you choose PSNR for supervision and can other metrics be used for supervision instead? What kind of changes would there be if different metrics are used for supervision?
1 comment

We thank the reviewer for their feedback!

  • Introducing new constants in TangoGS

    The goal of our technique is to remove the dependency of 3DGS techniques on constants that require retuning per scene (such as gradient thresholds) or per capture scale and replace them with a technique that is demonstrably transferable across scenes and across capture scales, since there cannot be a technique that is completely free of parameters. We will make this clear in the paper.

  • Justification of functional forms (X~\sqrt{\tilde{X}} and linear ρ(P)\rho(P))

    Our intuition for these two design choices is to use the lowest-order monotonic parameterizations so that a single calibration on standard scenes transfers without tuning across capture scales:

    • Why ρ(P)\rho(P) is linear in PSNR (Eq. 3): Because PSNR=−20log⁡10(RMSE)\text{PSNR} = -20\log_{10}(\text{RMSE}), a linear function of PSNR is a logarithmic function of reconstruction error (RMSE), reducing capacity proportionally to the orders of magnitude of remaining error as convergence progresses. The floor P∗=20 dBP^{\ast} = 20\text{ dB} corresponds directly to RMSE=0.1\text{RMSE} = 0.1 (1010\\% error), below which terminal pruning is disabled.

    • Why TcapT_{\text{cap}} scales with X~\sqrt{\tilde{X}} (Eq. 2): Because 3D Gaussians are multi-scale primitives (one large Gaussian can represent thousands of pixels on smooth surfaces while fine textures require dense primitives), required capacity grows sublinearly with observed surface X~\tilde{X}. Taking the geometric mean Tcap=TrefTlinT_{\text{cap}} = \sqrt{T_{\text{ref}} T_{\text{lin}}} (β=0.5\beta = 0.5) is the parameter-free midpoint between a constant budget (β=0\beta = 0, e.g., LiteGS) and a linear budget (β=1\beta = 1).

    We will add the intuition behind these choices, in addition to parameter ablations, to the paper to justify the specific design decisions.

  • Figure 4: What point F measures

    F disables both proposed components: it uses a fixed 1M allowance and LiteGS's density-control rules, while keeping the shared training profile and the timing of densification and pruning fixed across the Figure 4 variants. It is therefore a controlled removal of both components; the released LiteGS baseline is evaluated separately.

  • Figure 4: Close results on standard scenes and value of quality improvement

    The metrics are indeed close for standard scenes where the quality saturation point is already 1M (the preset budget used by point F in the ablation). For the transfer experiment on large scenes (lower part of Fig. 4), TangoGS gains 0.62 dB PSNR and 0.031 SSIM and lowers LPIPS by 0.044 over F. This change in SSIM and LPIPS is quite meaningful for observed visual quality. We reference a few published works [1–3] that have smaller quality improvements as their main gain.

  • Why PSNR feedback, and could other metrics replace it?

    Training PSNR is already computed in training, so it is a natural choice for our work. Our experiments show that a single linear mapping to the keep ratio provides useful capacity feedback across the evaluated scenes without per-scene tuning. We additionally tested a combination of SSIM and PSNR as guiding feedback and found that we can achieve the same results with a simpler approach using only PSNR. LPIPS would be an interesting option to test in a context where the computational overhead can be comfortably afforded. We will add to the paper more justification for our design decisions in addition to insights regarding which versions of the feedback components failed and why. Comparing additional alternative feedback metrics would be an interesting direction for future work.

  • Meaning of “sublinear mapping” (Question 1)

    This is referring to the square-root conversion Tcap=TrefgX~T_{\mathrm{cap}}=\sqrt{T_{\mathrm{ref}}g\tilde X}.

  • Generalization to other standard scenes and choice of constants

    We designed TangoGS to perform as well as the best SOTA methods on the standard benchmarks, and this guided the choice of TrefT_{\mathrm{ref}} and the empirically chosen constants s=1.7s=1.7 and c=0.0075c=0.0075, while the main contribution is to show the transferability of these parameters across scales. For generalization, we rely on the representativeness of the benchmark results as an indication of how our technique performs on other captures of the same scale.

  • The lack of bold font annotations in supplementary tables

    We will fix this. Thank you for the suggestion.

[1] Zheng Zhang et al. Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splatting. ECCV 2024. Table 1 reports +0.008 SSIM and −0.026 LPIPS on Mip-NeRF 360.

[2] Haosen Yang et al. Improving Gaussian Splatting with Localized Points Management. CVPR 2025. Table 1 reports +0.011 SSIM and −0.013 LPIPS on Mip-NeRF 360.

[3] Yanzhe Lyu et al. ResGS: Residual Densification of 3D Gaussian for Efficient Detail Recovery. ICCV 2025. Table 1 reports +0.013 SSIM and −0.017 LPIPS on Mip-NeRF 360.

Reviewer_RxiDOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes TangoGS, an adaptive density-control framework for 3D Gaussian Splatting that automatically determines the appropriate number of Gaussian primitives across different scene scales. The method combines two components: (1) a capture-derived learning allowance, which estimates the required Gaussian capacity based on image resolution, the number of input views, and overlapping observations measured through SfM tracks; and (2) a training-quality-guided adaptation mechanism, which uses training-view PSNR to dynamically adjust Gaussian densification and pruning. Experiments on 13 standard benchmark scenes and eight large-scale captures demonstrate competitive reconstruction quality with adaptive Gaussian counts. On standard benchmarks, TangoGS achieves comparable mean PSNR to LeGS while using approximately 48% fewer Gaussians.

Critical strengths
  1. Well-motivated problem formulation. The paper addresses an important practical challenge in 3D Gaussian Splatting: determining an appropriate number of Gaussian primitives across scenes of different scales and complexities. Existing density-control methods often rely on fixed thresholds or manually specified budgets, which may not generalize well to larger scenes. The proposed approach provides a reasonable solution by combining capture-dependent information with reconstruction-quality feedback.

  2. Comprehensive experimental evaluation. The authors evaluate their approach on 13 standard benchmark scenes and eight larger captures, covering datasets including Mip-NeRF 360, Tanks & Temples, Deep Blending, Mill19, UrbanScene3D, and Zip-NeRF. The evaluation includes comparisons with 13 existing methods, quality–size trade-off analyses, cross-scale generalization experiments, and controlled ablation studies. These experiments provide substantial evidence for the effectiveness of the proposed density-control strategy.

  3. Simple and practical integration. The proposed density-control strategy can be incorporated into existing 3DGS optimization frameworks without requiring substantial modifications to the rendering architecture. The authors also demonstrate its portability by applying the controller to a reference 3DGS implementation, suggesting that the approach has practical applicability beyond its original LiteGS-based implementation.

Critical weaknesses
  1. Limited methodological novelty. The proposed method primarily combines existing density-control techniques with two relatively simple heuristic rules. The capture-derived learning allowance is computed from the total number of input pixels and the average SfM track length, followed by a manually designed square-root mapping. Similarly, the quality-guided adaptation mechanism adjusts Gaussian counts through a predefined linear function of training PSNR. The underlying densification, sensitivity-based pruning, Gaussian representation, and rendering pipeline are largely inherited from existing methods, particularly LiteGS. Although the resulting framework demonstrates practical benefits, the methodological contribution appears relatively incremental. It remains unclear whether the proposed formulation introduces a sufficiently substantial technical advance beyond existing adaptive Gaussian density-control strategies.

  2. Heavy Reliance on Handcrafted Heuristics and Insufficient Justification. The proposed method relies heavily on manually designed formulas and heuristic assumptions. For example, the learning allowance is determined by a predefined square-root relationship with the track-adjusted pixel count, while Gaussian densification and pruning are controlled by a fixed linear function of training PSNR. The choices of these functional forms and their associated hyperparameters appear largely empirical, without sufficient theoretical justification or comprehensive ablation studies. Moreover, the assumption that lower training PSNR necessarily indicates a greater need for Gaussian primitives is not always valid, as reconstruction quality can be affected by factors unrelated to scene complexity. The authors should provide stronger justification for these design choices and more extensive sensitivity analyses to demonstrate the robustness and necessity of the proposed formulations.

  3. Limited evidence for generalization across capture configurations. Although the authors demonstrate that the proposed controller transfers from standard benchmarks to larger captures without additional hyperparameter tuning, the evaluation primarily considers two groups of datasets with different scene scales. It remains unclear whether the same density-control strategy generalizes to different capture configurations, such as varying input resolutions, camera densities, view overlap, and SfM reconstruction quality. Since the proposed learning allowance explicitly depends on these capture statistics, controlled experiments varying these factors would provide stronger evidence for the claimed cross-scale generalization.

  4. Insufficient justification for using training PSNR as an indicator of scene complexity. A central assumption of the proposed method is that lower training PSNR indicates greater scene complexity and therefore justifies allocating more Gaussian primitives.

Questions for authors

please refer to the weaknesses

1 comment

We thank the reviewer for their feedback!

  • Use of existing 3DGS techniques and methodological novelty

    TangoGS aims to better guide densification and pruning of existing 3DGS density controllers. This means that it is designed to work with existing pruning techniques and to use existing 3DGS infrastructure that is not related to density control, such as the Gaussian representation and rendering pipeline. We show in our work that our solution is intentionally simple, yet, to our knowledge, no existing technique combines capture-derived model sizing with training-quality feedback to achieve the same quality–size trade-off across scene scales.

  • Justification of functional forms (X~\sqrt{\tilde{X}} and linear ρ(P)\rho(P))

    Our intuition for these two design choices is to use the lowest-order monotonic parameterizations so that a single calibration on standard scenes transfers without tuning across capture scales:

    • Why ρ(P)\rho(P) is linear in PSNR (Eq. 3): Because PSNR=−20log⁡10(RMSE)\text{PSNR} = -20\log_{10}(\text{RMSE}), a linear function of PSNR is a logarithmic function of reconstruction error (RMSE), reducing capacity proportionally to the orders of magnitude of remaining error as convergence progresses. The floor P∗=20 dBP^{\ast} = 20\text{ dB} corresponds directly to RMSE=0.1\text{RMSE} = 0.1 (1010\\% error), below which terminal pruning is disabled.

    • Why TcapT_{\text{cap}} scales with X~\sqrt{\tilde{X}} (Eq. 2): Because 3D Gaussians are multi-scale primitives (one large Gaussian can represent thousands of pixels on smooth surfaces while fine textures require dense primitives), required capacity grows sublinearly with observed surface X~\tilde{X}. Taking the geometric mean Tcap=TrefTlinT_{\text{cap}} = \sqrt{T_{\text{ref}} T_{\text{lin}}} (β=0.5\beta = 0.5) is the parameter-free midpoint between a constant budget (β=0\beta = 0, e.g., LiteGS) and a linear budget (β=1\beta = 1).

    We will add the intuition behind these choices, in addition to parameter ablations, to the paper to justify the specific design decisions.

  • Generalization across capture configurations

    Our controlled experiment (Fig. 6, Table F.3) tests increased view count and redundant observations by duplicating every image and its SfM observations across all 21 captures. The six tested datasets cover indoor and outdoor scenes, handheld walkthroughs, and aerial surveys. The transition to larger captures also changes image resolution, view count, camera trajectories, and overlap.

  • Why use training PSNR specifically as the feedback signal for scene complexity?

    Training PSNR is already computed in training, so it is a natural choice for our work. Our experiments show that a single linear mapping to the keep ratio provides useful capacity feedback across the evaluated scenes without per-scene tuning. We additionally tested a combination of SSIM and PSNR as guiding feedback and found that we can achieve the same results with a simpler approach using only PSNR. LPIPS would be an interesting option to test in a context where the computational overhead can be comfortably afforded. We will add to the paper more justification for our design decisions in addition to insights regarding which versions of the feedback components failed and why. Comparing additional alternative feedback metrics would be an interesting direction for future work.

  • When low training PSNR reflects factors beyond scene complexity

    Any feedback signal derived from training images (including the local image gradients and photometric errors used by standard 3DGS, LeGS, and CDC-GS) is affected by capture imperfections such as exposure variations, motion blur, or pose noise, which means this is a shared issue for all density controllers. We attempt to empirically test robustness against capture imperfections through our benchmarks, which cover a wide variety of illumination shifts, motion blur, and pose noise and in which allowing more Gaussians did help the output quality. We will look for and investigate cases where adding Gaussians in the case of a low PSNR further hurts the quality.

Reviewer_4Ph4OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes TangoGS, a 3DGS density control method that automatically determines the number of Gaussians across scene scales. Before training, it derives a learning allowance from SfM track lengths while discounting repeated observations; during training, PSNR-based quality feedback linearly scales the densification target. Without per-scene retuning, it matches LiteGS quality on standard scenes with fewer Gaussians and achieves the highest PSNR on large-scale captures, attaining a better quality–size trade-off.

Critical strengths
  1. The paper distinguishes surface scale from content complexity, and suggest to combine capture-derived model sizing with training-based adaptation. The observation at Line 36 that per-scene Gaussian counts range from 87K to 7.4M and span 5 dB supports the motivation.
  2. The evaluation is good: 13 standard scenes are compared under a common implementation with COLMAP rerun, supplemented by transfer experiments (hyperparameters applied to large-scale captures), a four-quadrant ablation (Fig. 4), and a portability experiment embedding the controller into the Taming codebase.
Critical weaknesses
  1. Comparing the ablations only against LiteGS is a reasonable control-variable design, as it cleanly attributes quality differences to the two components, namely capacity and feedback. However, it is insufficient to support generality. It is necessary to apply TangoGS to at least two substantially different backbones (MCMC and vanilla 3DGS), even if only on 3–5 representative scenes, and report which constants (s, c, g, P*, and the growth interval) must be recalibrated upon porting.
  2. The feedback relies solely on training-view PSNR, with no held-out validation. Since high training PSNR may reflect overfitting, Eq. 3, which allocates smaller capacity as PSNR rises, risks misdirecting the capacity decision.
Questions for authors

How is the proposed method influenced by the quality of sfm tracks? especially for weak-texture scenes.

1 comment

We thank the reviewer for the feedback!

  • Applying TangoGS to at least two substantially different backbones.

    We agree that testing our technique on different backbones is important. This is why we apply TangoGS to both LiteGS and vanilla 3DGS (through Taming's codebase), without changing ss, cc, gg, P∗P^{\ast}, and the growth interval. On the 13 standard scenes, mean PSNR is only 0.12 dB lower than Taming with roughly 3× fewer Gaussians. On the eight large captures, mean PSNR is 0.77 dB higher than Taming with 2.2× as many Gaussians. These results demonstrate the benefit of our technique and its transferability without retuning parameters. The experiment is described in Supplementary §H of the original submission, and we will highlight it more in our revision.

  • How is TangoGS affected by overfitting?

    When training PSNR is high, reducing capacity can remove redundancy in a well-reconstructed scene or act as regularization when the model overfits. Consider the two scenarios:

    • When high training PSNR reflects a well-reconstructed, low-complexity scene (quality saturation), fewer Gaussians are needed to represent the geometry and appearance. This is the main argument we highlight in the paper.

    • If high training PSNR reflects overfitting, excess Gaussians would be fitting view-specific noise or floaters rather than shared scene structure, which makes more aggressive pruning and lower capacity (through a lower ρ\rho) work as a regularizing action. Furthermore, because pruning ranks primitives by sensitivity accumulated across all training views in an epoch, Gaussians that model view-specific noise contribute little globally and are more likely to be removed first. We will add an analysis of this behaviour to the paper.

  • How is the proposed method influenced by the quality of SfM tracks, especially in weak-texture scenes?

    TangoGS is designed to be more resilient to sparse or noisy SfM tracks in weak-texture scenes for three reasons:

    • Invariance to feature density (Eq. 1): Weak textures reduce the number of triangulated points MM and observations OO, which starves methods that scale budgets directly from MM. Because TangoGS uses only their ratio (the mean track length tˉSfM=O/M\bar{t}_{\text{SfM}} = O/M) to discount total pixels (X~=X/tˉSfM\tilde{X} = X/\bar{t}_{\text{SfM}}), MM and OO scale down together and leave X~\tilde{X} more stable. We observe this on the indoor Zip-NeRF captures (Alameda, London, NYC, Berlin), which contain large low-texture walls and doors (Fig. A.1): SfM-point allowances drop to 0.5×to 0.8×0.5\times\text{to }0.8\times of ours (Sec. 3.6, Table F.2), whereas TangoGS maintains capacity and outperforms point-budgeted Taming 3DGS by 1.0–2.1dB1.0\text{–}2.1\text{dB} (Tables D.3–D.4).

    • Reduced sensitivity (Eq. 2): The square-root mapping Tcap∝X~0.5T_{\text{cap}} \propto \tilde{X}^{0.5} makes the allowance less sensitive to changes in mean track length tˉSfM\bar{t}_{\text{SfM}}. For the same input images, a 10% decrease in mean track length increases the allowance by approximately 5%.

    • Training feedback (Eq. 3): Even if fragmented tracks slightly raise the initial allowance TcapT_{\text{cap}}, weak-texture surfaces are photometrically easy to fit, yielding higher training PSNR that automatically lowers ρ\rho and prunes excess Gaussians.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.