acceptodds
Under review as a conference paper at ICLR 2027

Store Sparse, Render Dense: Neural Gaussian Mixture Splatting

Abstract

We introduce Neural Gaussian Mixture Splatting (NGMS), a compact, unified representation for static and dynamic scenes. Our core insight is that dense geometry, view-dependent appearance, and motion can be generated from a sparse set of stored Gaussians through a shared residual field. NGMS stores complete base Gaussians and expands each into local components through learned residuals in position, rotation, scale, opacity, and spherical-harmonic (SH) appearance. A single nonlinear field query per base jointly predicts all components, reducing nonlinear evaluations by a factor of relative to independent per-component prediction. Only diffuse base Gaussians and shared network weights are stored, avoiding independent storage of every rendered primitive and its higher-order SH coefficients. Factorized view and time branches capture appearance and deformation within the same formulation, while the generated components retain compatibility with standard Gaussian rasterization. Across four static and dynamic benchmarks, NGMS achieves the smallest serialized models among the evaluated methods, with competitive static reconstruction quality and a  dB PSNR gain over 4DGS on D-NeRF. Dataset-average storage is reduced by – relative to scaffold methods and by – relative to explicit 3D/4DGS, reaching two orders of magnitude on dynamic scenes while maintaining real-time rendering.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Radiance Field, Gaussian Splatting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_R9NAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents NGMS, a compact framework for static and dynamic scene reconstruction. It stores a small set of base Gaussians and uses a shared residual field to generate multiple local Gaussians, reducing storage and nonlinear computation while remaining compatible with standard Gaussian rasterization. Experiments on four benchmarks show that NGMS achieves the smallest model sizes among evaluated methods, with competitive reconstruction quality and real-time rendering.

Critical strengths
  1. NGMS substantially reduces model size while maintaining reasonable reconstruction quality and real-time rendering performance, which is beneficial for deploying Gaussian Splatting on edge devices.
  2. The residual mixture design provides a unified approach to modeling geometry, appearance, and dynamics while remaining compatible with the standard Gaussian rasterization pipeline.
Critical weaknesses
  1. Insufficient comparison with recent state-of-the-art methods. The experiments mainly compare against established or relatively older methods, without sufficient evaluation against recent state-of-the-art approaches in compact Gaussian representations and dynamic scene reconstruction. This makes it difficult to determine whether NGMS provides a meaningful improvement over the current state of the art.
  2. Limited reproducibility and experimental presentation. The code link provided in the paper is currently inaccessible, making it difficult to verify the implementation and assess the reproducibility of the reported results. In addition, the paper provides relatively few qualitative visualizations and experimental figures, which limits the reader’s ability to visually assess reconstruction quality and the practical advantages of the proposed method. For a computer vision paper, stronger qualitative and visual evidence would be expected.
Questions for authors
  1. For static scenes, how does NGMS compare with recent compression methods such as HAC++: Towards 100X Compression of 3D Gaussian Splatting and Optimized Minimal 3D Gaussian Splatting? These methods can achieve comparable or even better compression and reconstruction performance. Could the authors clarify what the key advantage of NGMS is over these approaches under comparable storage and rendering constraints?
  2. For dynamic scenes, how does NGMS compare with recent methods such as Light4GS: Lightweight Compact 4D Gaussian Splatting Generation via Context Model? Light4GS appears to achieve comparable or even better performance in terms of compression and reconstruction quality. Could the authors provide a detailed comparison and clarify what advantages NGMS offers over such recent approaches?
  3. Could the authors provide more qualitative visual results to better support the claimed advantages of NGMS? The current paper contains relatively few visualization results, making it difficult to assess the visual quality of reconstruction and the effectiveness of the proposed representation, particularly for fine geometry, view-dependent appearance, and dynamic deformation.
3 comments
AuthorsOpenReview ↗

Response to Reviewer R9NA

We thank the reviewer for the thoughtful and constructive suggestions. The requested comparisons, additional compression experiments, and upgraded qualitative figures have strengthened the experimental support for NGMS and helped us explain its practical advantages more clearly.

We have submitted a revised manuscript with expanded discussion and coverage of recent related work (Section 2), improved qualitative comparisons with matching zoom-in crops for fine detail and view-dependent effects, together with error maps (Figures 3 and 4), and a new compression integration experiment demonstrating the benefit of NGMS's Gaussian-native design (Section 4.4). Extended quantization additionally uses INT8 scales and network weight rows; combining it with G-PCC position coding (voxel step 0.0010.001) reduces storage by a further 49-51% relative to default quantization, with only 0.03-0.19 dB additional mean PSNR loss across the four datasets.

1. Additional comparisons. We summarize the requested comparisons with HAC++ (low rate), OMG-M, and Light4GS below, including an additional full-resolution NGMS run on D-NeRF. For the main-text experiments, all methods use the same evaluation protocol, including dataset splits, image resolutions, and metric definitions. In the appendix comparisons, we clearly distinguish results taken from published papers from those obtained in our own runs. Appendix E ("Broader Quality-Storage Comparisons") provides the broader comparisons. In the tables below, baseline values are published results; NGMS values are our runs.

NeRF Synthetic

MethodPSNR (dB)Size (MB)
HAC++ (low rate)33.030.88
NGMS33.581.9

Mip-NeRF 360

MethodPSNR (dB)Size (MB)
HAC++ (low rate)27.608.34
OMG-M27.215.31
NGMS27.448.0

D-NeRF, full resolution

To match Light4GS's full-resolution setting, we trained NGMS at full resolution. Light4GS values are its published low-rate results; Light4GS was not retrained.

MethodPSNR (dB)Size (MB)
Light4GS32.582.20
NGMS39.151.64

These results show that NGMS provides competitive reconstruction quality with compact storage across both static and dynamic scenes. On NeRF Synthetic, NGMS achieves 33.58 dB with a 1.9 MB model. On Mip-NeRF 360, it achieves 27.44 dB at 8.0 MB, remaining within 0.16 dB of HAC++ at comparable storage. On full-resolution D-NeRF, NGMS exceeds the reported Light4GS result by 6.57 dB while reducing storage from 2.20 to 1.64 MB. NGMS achieves these results through a unified base-Gaussian and residual-mixture formulation, with a time branch enabled for dynamics.

2. Orthogonality and integration benefits. The reviewer's suggestions also motivated us to examine an integration benefit of our design. NGMS reduces learnable parameters within this unified formulation: complete diffuse bases and a shared field generate multiple residual Gaussians per base. Appendix B.2 reports the base/field storage breakdown. Quantization and coding can then compress the remaining bases and field weights, creating an opportunity to benefit from the compression techniques highlighted by the reviewer.

We use default quantization because it substantially reduces storage while changing mean PSNR by only about 0.01-0.02 dB across all four datasets. This setting uses INT8 color, opacity, and rotation, with FP16 scales and field weights. Extended quantization additionally uses INT8 scales and network weight rows.

Retaining complete, explicit base Gaussian attributes supports compatibility with standard Gaussian rasterizers and attribute-based processing tools. We integrate G-PCC position coding at voxel step 0.001, using only the geometry coding stage of HAC++, as post-processing without retraining. The HAC++ scene-specific hash-grid context model and rate loss are not applied. This integration illustrates the practical benefit of our Gaussian-native design.

The following additional post-compression experiment covers all four datasets. Each cell reports PSNR (dB) / size (MB). Dynamic experiments use the paper's 2x downsampling.

NGMS settingNeRF SyntheticMip-NeRF 360D-NeRFN3V
Before post-compression33.56 / 4.47727.44 / 19.62740.53 / 3.46131.38 / 24.605
Default quantization33.55 / 1.86827.43 / 7.98540.51 / 1.50031.37 / 10.003
G-PCC only33.42 / 3.83827.39 / 16.37640.43 / 3.15131.36 / 20.552
Default quantization + G-PCC33.41 / 1.28427.38 / 5.03440.41 / 1.21531.35 / 6.339
Extended quantization + G-PCC33.40 / 0.91827.37 / 4.03040.32 / 0.76731.34 / 5.050

On these NGMS checkpoints, default quantization achieves higher PSNR and approximately half the file size of G-PCC geometry coding alone on all four datasets. This supports its use for the main results: it preserves quality while providing substantial storage savings.

For more aggressive compression, combining extended quantization with G-PCC achieves the smallest checkpoints among the tested settings. It reduces storage by a further 49-51% relative to default quantization, with only 0.03-0.19 dB additional mean PSNR loss across the four datasets. NGMS therefore offers both a quality-preserving default and a stronger compression option, enabled by combining learnable parameter reduction with post-compression.

3. Code and reproducibility. We acknowledge that the code is not currently accessible. As stated in our reproducibility statement, we will release the code and data after acceptance. Sections 3-4 and Appendix A provide implementation-level specifications to support reproduction, including network architecture and encodings, residual composition, initialization, optimization and densification schedules, evaluation protocol, and checkpoint format. Because NGMS retains standard Gaussian attributes and rasterization, these details support independent reimplementation within the established open-source Gaussian Splatting ecosystem.

4. Qualitative results. Following the reviewer's suggestion, we have upgraded Figures 3 and 4 with matching zoom-in crops and absolute RGB error maps. Figure 3 highlights repeated lego studs, thin bonsai branches, and view-dependent reflections against 3DGS and Scaffold-GS. Figure 4 highlights the trex ribcage and hook boundaries against 4DGS and 4D Scaffold-GS. Both include GT and consistent crop regions and error scales. These additions expose localized detail and appearance errors, with NGMS preserving clearer articulated structures and lower errors in the highlighted dynamic regions. Appendix B.1 provides complementary per-scene SSIM and LPIPS results.

Ahead of the discussion deadline, we kindly invite you to check our response and revision.

We have carefully addressed your concerns and added the requested experiments. We remain available if any further clarification is needed.

Thank you again for your time and helpful reviews.

Reviewer_oFFkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Neural Gaussian Mixture Splatting (NGMS) to achieve compact representation for static or dynamic 3D scenes. NGMS stores N complete base Gaussians and expands each into K local components using a shared residual field. The paper reports substantially smaller serialized checkpoints than the selected 3DGS, Scaffold-GS, 4DGS, and 4D Scaffold-GS baselines, while retaining competitive static quality and strong D-NeRF performance.

Critical strengths
  1. The central representation is well motivated and is implemented in a renderer-compatible manner. This method can retain the original primitive-based representation after expansion without requiring an additional specialized neural renderer. In addition, the paper is well organized. The progression from conventional Gaussian rendering to residual parameterization, to densification, and to experiments is coherent.

  2. This paper effectively improves a meaningful storage-quality trade-off in scene representation. Specifically, Tables 1 and 2 consistently show very small NGMS checkpoint sizes. Table 1 reports average sizes of 1.9 MB on NeRF Synthetic and 8.0 MB on Mip-NeRF 360, versus 59.3/613.5 MB for 3DGS and 17.3/174.2 MB for Scaffold-GS. Table 2 similarly reports an average of 1.5 MB for D-NeRF and 10.0 MB for N3V. These are large reductions, and the authors report end-to-end FPS rather than only neural-field query throughput. Moreover, most of the ablation studies are also well-designed. For example, Figure 5 also gives a useful non-monotonic mixture-size sweep, showing that larger K is not automatically beneficial once the growing prediction heads and reduced base count are jointly considered.

Critical weaknesses
  1. I see the central idea of NGMS shares similarity with the anchor-plus-shared-decoder pattern of Scaffold-GS, and the paper does not isolate what is gained by its particular replacement of anchors with complete Gaussian bases. Scaffold-GS already jointly predicts K candidate Gaussians from a sparse entity using a shared MLP. NGMS’s meaningful distinctions are that its stored entities are complete diffuse Gaussians, residuals are composed around those bases, and it uses mixture-aware densification. However, the stated “factor of K” reduction in nonlinear queries is mainly relative to a hypothetical independent-per-component decoder, not relative to the actual strongest scaffold-style alternative. A more informative ablation would compare: (i) a scaffold decoder with complete base-Gaussian attributes and (ii) NGMS without mixture-aware densification. This would reveal which part of the design choices drives the gains.

  2. Limited evaluation under rapid temporal variation. Although the paper evaluates dynamic scenes on D-NeRF and N3V and reports temporal coherence metrics, it does not explicitly analyze how performance changes with increasing motion speed or temporal complexity. Consequently, robustness to rapid non-rigid deformation, large inter-frame displacements, and frequent changes in occlusion remains insufficiently established. Given the reliance on sparse Gaussian bases and shared time-conditioned residuals, it is unclear whether the reported advantages in quality, compactness, and efficiency persist under these more demanding conditions. Additional comparisons on selected high-motion sequences would strengthen the evaluation. Relevant candidates include sports and dance sequences from DNA-Rendering [1] and paper-windmill and spin from DyCheck iPhone [2].

[1] DNA-Rendering: https://dna-rendering.github.io/inner-download.html [2] DyCheck iPhone: https://github.com/KAIR-BAIR/dycheck/blob/main/docs/BENCHMARK.md

Questions for authors
  1. For every entry in Tables 1 and 2, were all methods retrained by the authors under the same image downsampling, train/test splits, iteration budgets, and hardware? Please provide the exact source or configuration of every baseline result, including whether it was reproduced or taken from a published report.

  2. In Equation (8), what precisely is \hat{o}_i? Is it \bar{o}_i, the base opacity logit from Equation (2), or another quantity? Why is sigmoid(*)/K preferred over a mixture-opacity parameterization that preserves total alpha in the identical-component case?

  3. Since the view branch predicts geometry as well as SH appearance, can the authors show generated Gaussian centers/covariances for a fixed base across multiple test cameras? Does NGMS retain geometry under nearby views, or is view-conditioned geometry essential to the reported gains?

  4. Please report the average NK and visible rendered Gaussian counts per dataset, together with field, expansion, and rasterization timing. This would clarify whether the reported FPS gains derive from the learned sharing, lower primitive counts after densification, or both.

  5. Can the authors include main-paper aggregate SSIM, LPIPS, and other metrics, rather than relegating these metrics to the appendix? This is important for determining whether small checkpoints preserve perceptual and temporal quality.

3 comments
AuthorsOpenReview ↗

Response to Reviewer oFFk

We thank the reviewer for the careful assessment and for recognizing the representation's motivation, compactness, and renderer compatibility. The constructive suggestions helped us clarify the relationship to Scaffold-GS and motivated additional controls on decoder design and view-conditioned geometry.

We have submitted a revised manuscript with expanded discussion of recent related work (Section 2), improved qualitative comparisons with zoom-in crops highlighting fine detail and view-dependent effects, and absolute RGB error maps (Figures 3 and 4). Section 4.4 also demonstrates the compression benefits of NGMS's Gaussian-native design for static and dynamic scenes: extended quantization additionally uses INT8 scales and network weight rows; combined with G-PCC position coding (voxel step 0.0010.001), it reduces storage by a further 49-51% relative to default quantization, with only 0.03-0.19 dB additional mean PSNR loss. This integration uses only HAC++'s geometry coding stage, without its context model or rate-loss training.

1. Complete bases and the requested Scaffold-style controls.

Thank you for suggesting these controls. A key distinction is that NGMS uses only the base Gaussian's location as its network input, without additional learned feature vectors. View direction and optional time provide conditioning; the remaining Gaussian attributes enter through residual composition. We agree that Scaffold-GS also shares queries across KK components, so our factor-of-KK statement is relative to independent per-component prediction.

For (i), we trained a Scaffold-style decoder on drums and bonsai, replacing learned anchor features with the 14 activated base-Gaussian attributes plus viewing direction. The control retains the official width-32 Scaffold heads, direct attribute decoding, and learned local offsets. Both use the NGMS training/densification strategy, K=4K=4, 30K iterations, matched splits/resolutions, and the same INT8/FP16 codec.

SceneMethodPSNR (dB)SSIMSize (MiB)Bases NN
drumsScaffold-style control26.110.9497.396156,519
drumsNGMS26.230.9492.42890,034
bonsaiScaffold-style control30.800.93715.101340,002
bonsaiNGMS32.560.9468.137355,393

NGMS improves two-scene PSNR with less storage. On bonsai, storage falls despite a slightly larger base count.

For (ii), Section 4.4 reports the requested ablation at fixed K=4K=4: mixture-aware densification improves the four-scene mean from 32.65 to 32.86 dB, reduces storage from 12.5 to 5.5 MB, and raises FPS from 95 to 190. Thus, allocating bases according to mixture contributions is important for converting expanded capacity into compact storage and faster rendering.

2. Rapid temporal variation.

We thank the reviewer for highlighting this valuable direction for future work. Our D-NeRF and N3V results support temporal modeling on the evaluated benchmarks, while faster motion and frequent occlusion changes warrant broader evaluation. Adapting NGMS and the baselines to additional datasets requires validating camera and distortion conventions, scene normalization, temporal sampling, and evaluation masks and splits to ensure fair comparisons. We welcome the suggested benchmarks and would be happy to extend the evaluation, with the hope of including additional results in the camera-ready version.

3. Protocol and result sources (Q1).

All entries in main-text Tables 1 and 2 are our runs, with matched dataset splits, image resolutions, and metric definitions; training time and FPS are measured on one RTX A6000. Section 4.1 specifies full resolution on NeRF Synthetic, 2x downsampling on D-NeRF, N3V, and indoor Mip-NeRF 360, and 4x on outdoor Mip-NeRF 360. Appendix comparisons also clearly distinguish our runs from published results.

4. Opacity in Eq. (8) (Q2).

Thank you for asking. The quantity is the base opacity logit oˉi\bar{o}_i from Eq. (2).

For identical coincident components, preserving target alpha at a pixel requires component alpha 1−(1−a)1/K=a/K+O(a2)1-(1-a)^{1/K}=a/K+O(a^2). Thus, our 1/K1/K scaling is a first-order approximation for it. We tried the alpha-preserving alternative, but occasionally observed numerical instability during training, so we retained the approximation.

5. View-conditioned geometry (Q3).

To examine its contribution, we disabled only view-branch position/rotation/scale residuals, retaining view appearance, opacity, and all time outputs. Inference-only removal lowers PSNR from 26.18 to 22.15 dB on drums and from 43.16 to 29.54 dB on bouncingballs. We also retrained both scenes with these outputs fixed to zero, using the same 30K-step protocol and paired encoded-checkpoint evaluation:

SceneModelPSNR (dB)SSIMSize (MiB)
drumsFull NGMS26.200.9492.428
drumsRetrained without view geometry25.960.9463.001
bouncingballsFull NGMS43.190.9961.785
bouncingballsRetrained without view geometry43.030.9961.703

On drums, removal costs 0.24 dB and uses 23.6% more storage. On bouncingballs, the retrained model is 43.03 dB at 1.703 MiB, against 43.10 dB at 1.785 MiB for the full model. View-conditioned geometry therefore slightly helps visual fidelity.

6. Primitive counts and rendering efficiency (Q4).

Thank you for asking us to clarify the source of efficiency. Sections 3.3 and 4.3 describe two complementary mechanisms:

  • Fewer base primitives. Mixture-aware densification reduces the number NN of bases processed by the field. At fixed K=4K=4, enabling it increases end-to-end speed from 95 to 190 FPS while reducing storage from 12.5 to 5.5 MB (Section 4.4).
  • Shared backbone computation. The nonlinear backbone (spatial trunk and active conditioning branches) is evaluated once per base; lightweight linear heads then jointly predict all KK residual components. Neural backbone evaluation therefore operates on NN base sites while the rasterizer renders NKNK Gaussians, avoiding an independent backbone evaluation for each component.

At K=4K=4, dataset-average counts and end-to-end speeds are:

DatasetBases NNExpanded NKNKFPS
NeRF Synthetic64,516258,064371
Mip-NeRF 360348,1911,392,76483
D-NeRF31,486125,944329
N3V427,4041,709,61645

Counts are before visibility culling; FPS includes field evaluation, expansion, and rasterization. Appendix B.2 provides storage breakdowns.

7. Aggregate perceptual quality (Q5).

Thank you for this helpful suggestion. For the camera-ready version, we will consider replacing the main paper's per-scene tables with dataset-average comparisons to include more recent methods and restore SSIM/LPIPS alongside PSNR, while keeping per-scene details in the appendix. We provide the current dataset means below. Method order is 3DGS / Scaffold-GS / NGMS for static scenes and 4DGS / 4D Scaffold-GS / NGMS for dynamic scenes.

DatasetSSIMLPIPS
NeRF Synthetic0.970 / 0.967 / 0.9680.031 / 0.034 / 0.035
Mip-NeRF 3600.826 / 0.825 / 0.8000.202 / 0.206 / 0.255
D-NeRF0.975 / 0.961 / 0.9930.037 / 0.047 / 0.008
N3V0.942 / 0.946 / 0.9440.140 / 0.131 / 0.137

These show strong D-NeRF gains and a clearer perceptual trade-off on Mip-NeRF 360. Revised Figures 3 and 4 complement the metrics with matched crops and error maps for lego studs and reflections, bonsai branches, trex ribs, and hook boundaries.

As the discussion period is approaching its end, we wanted to send a brief reminder regarding our rebuttal and updated revision.

Thank you again for your constructive feedback. We have carefully addressed your questions and added the requested clarifications and experiments. Please let us know if anything remains unclear before the deadline.

We look forward to hearing your thoughts and continuing the discussion.

Reviewer_sfksOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Neural Gaussian Mixture Splatting (NGMS), which stores a sparse set of base Gaussians and uses a shared neural residual field to generate multiple Gaussian components at rendering time. View- and time-conditioned branches support static and dynamic scenes within a common formulation. The reported results suggest substantial storage savings while retaining real-time rendering. The compact representation is practically relevant. However, the evaluation in the supplied manuscript does not sufficiently establish its competitiveness against recent methods or its effectiveness in challenging deployment settings.

Critical strengths

The method combines compact storage with standard Gaussian rasterization, making the design potentially useful in existing rendering pipelines. The paper evaluates both static and dynamic scenes and reports reconstruction quality, storage, and rendering speed. The mixture-aware densification ablation provides useful evidence that base allocation contributes to the storage and efficiency gains.

Critical weaknesses
  1. The dynamic-scene evaluation needs stronger baselines and more challenging settings.

The main comparisons use 4DGS and 4D Scaffold-GS. Although the reported improvements on D-NeRF are substantial, these comparisons do not sufficiently establish competitiveness against more recent dynamic reconstruction frameworks. A comparison with a recent method such as DualGS would help clarify the contribution, provided that the input assumptions and evaluation protocols are compatible. The current evaluation includes real-world multiview sequences from Neural 3D Video, which is valuable. Nevertheless, it remains unclear how the representation behaves under larger motions, complex nonrigid deformation, and substantial occlusion. These conditions are particularly relevant because the method generates dynamic components around persistent base Gaussians. Evaluation in a challenging multiview setting would help establish whether the compact representation retains sufficient motion capacity.

  1. The static compression comparisons are insufficient.

Static Gaussian compression is an extensively studied area. Comparing primarily against uncompressed 3DGS and Scaffold-GS establishes savings relative to these representations, but does not establish competitiveness with dedicated compression methods. ContextGS is a particularly relevant baseline; comparisons with methods such as HAC or HAC++ would also help position the work. Moreover, NGMS combines a compact representation with attribute quantization and reduced-precision network weights. The evaluation should distinguish the contribution of the representation from that of its encoding scheme. Quality–storage curves, together with rendering speed under clearly specified measurement conditions, would be more informative than a single operating point. The manuscript refers to broader comparisons in Appendix E, but that appendix is absent from the supplied PDF, so I could not assess whether it resolves this concern.

  1. Detail preservation and scalability remain insufficiently demonstrated.

NGMS can be understood as a lossy representation: scene-specific information is encoded through fewer independently stored primitives and a shared field. This may constrain the representation of fine structures and high-frequency appearance. The quality reduction on Mip-NeRF 360 makes this trade-off worth examining directly, although it does not by itself demonstrate a particular failure mechanism.

The paper should provide detailed comparisons on thin structures, repeated textures, and complex appearance, including localized crops or error maps. Its practical scalability would also benefit from evaluation on a substantially larger scene, reporting reconstruction quality, file size, peak GPU memory, training cost, and end-to-end rendering speed. Small serialized models alone do not establish scalability, since rendering still requires generating and processing the expanded Gaussian set.

Questions for authors

How does NGMS compare with recent dynamic methods such as DualGS under compatible input and evaluation settings?

How does its quality–storage trade-off compare with ContextGS and other dedicated compression methods?

4 comments
AuthorsOpenReview ↗

Response to Reviewer sfks

We thank the reviewer for the thoughtful suggestions on recent baselines, representation versus encoding, detail preservation, and scalability. These comments motivated additional comparisons, compression experiments, and upgraded qualitative figures that clarify NGMS's practical benefits.

We have submitted a revised manuscript with expanded discussion and coverage of recent related work (Section 2), improved qualitative comparisons with matching zoom-in crops for fine detail and view-dependent effects, together with error maps (Figures 3 and 4), and a new compression integration experiment demonstrating the benefit of NGMS's Gaussian-native design (Section 4.4). Extended quantization additionally uses INT8 scales and network weight rows; combining it with G-PCC position coding (voxel step 0.0010.001) reduces storage by a further 49-51% relative to default quantization, with only 0.03-0.19 dB additional mean PSNR loss across the four datasets.

For the main-text experiments, all methods use the same evaluation protocol, including dataset splits, image resolutions, and metric definitions. In the appendix comparisons, we clearly distinguish results taken from published papers from those obtained in our own runs.

1. Dynamic baselines and motion capacity. We considered the suggested DualGS comparison. However, it evaluates foreground-matted human-performance sequences with frame-by-frame optimization and per-frame storage and doesn't report results on those four commonly used benchmarks. Following the reviewer's qualification about compatible input assumptions and evaluation protocols, we focus on a recent dynamic compression method with an overlapping benchmark: Light4GS, also suggested by Reviewer R9NA. We trained NGMS at full D-NeRF resolution to align with its published setting; Light4GS values are its published low-rate results.

Method, full-resolution D-NeRFPSNR (dB)Size (MB)
Light4GS (published, low rate)32.582.20
NGMS (our run)39.151.64

At full resolution, NGMS achieves 6.57 dB higher PSNR with 25.5% less storage than the reported Light4GS result.

Time-conditioned residuals move and deform the expanded Gaussians; the stored bases provide their reference geometry. The existing 2x D-NeRF runs provide scene-level deformation evidence: NGMS reaches 37.71, 37.73, and 36.83 dB on trex, jumpingjacks, and hook, compared with 29.77, 30.29, and 31.97 dB for our 4DGS runs. On real multiview N3V, average tPSNR is 49.04 versus 43.28 for 4DGS and 46.55 for 4D Scaffold-GS; flow-warping residual is 0.092 versus 0.175 and 0.114 (Appendix C). These support nonrigid-motion modeling and temporal coherence on the evaluated sequences.

2. Dedicated static compression methods. The following baseline values are published low-rate results; NGMS values are our runs. This provides the requested context for ContextGS, HAC, and HAC++ without relying on access to Appendix E.

Mip-NeRF 360

MethodPSNR (dB)Size (MB)
ContextGS27.6212.68
HAC27.5315.26
HAC++27.608.34
NGMS27.448.0

NeRF Synthetic

MethodPSNR (dB)Size (MB)
HAC33.241.18
HAC++33.030.88
NGMS33.581.9

On Mip-NeRF 360, NGMS remains within 0.18 dB of ContextGS and 0.16 dB of HAC++ while using less storage. NGMS provides this compact representation through a unified static/dynamic formulation, enabling the time branch for dynamics. Published rows are contextual comparisons rather than matched reproductions.

3. Separating representation gains from encoding gains. We address this distinction through two complementary experiments. First, the mixture-size sweep holds the checkpoint encoding fixed. The four-scene means from Section 4 and Appendix D are:

Mixture size KPSNR (dB)Size (MB)FPS
132.7812.2138
232.887.7187
432.865.5190
832.704.8186
1632.575.1126

This shows the representation's quality-storage-speed trade-off under one encoding. At fixed K=4, mixture-aware densification further reduces storage from 12.5 to 5.5 MB and raises speed from 95 to 190 FPS, with PSNR changing from 32.65 to 32.86 dB. Appendix B.2 separates base-attribute and shared-field storage.

Second, the following post-compression experiment holds the learned model fixed and varies its encoding. Default quantization uses INT8 color, opacity, and rotation, with FP16 scales and field weights. Extended quantization additionally uses INT8 scales and network weight rows. G-PCC denotes position coding at voxel step 0.001, using the geometry coding stage of HAC++. Each cell reports PSNR (dB) / size (MB); dynamic scenes use the paper's 2x downsampling.

NGMS settingNeRF SyntheticMip-NeRF 360D-NeRFN3V
Before post-compression33.56 / 4.47727.44 / 19.62740.53 / 3.46131.38 / 24.605
Default quantization33.55 / 1.86827.43 / 7.98540.51 / 1.50031.37 / 10.003
G-PCC only33.42 / 3.83827.39 / 16.37640.43 / 3.15131.36 / 20.552
Default quantization + G-PCC33.41 / 1.28427.38 / 5.03440.41 / 1.21531.35 / 6.339
Extended quantization + G-PCC33.40 / 0.91827.37 / 4.03040.32 / 0.76731.34 / 5.050

Default quantization changes mean PSNR by about 0.01-0.02 dB across the four datasets, supporting our choice for the main results. It also achieves higher PSNR and approximately half the file size of geometry coding alone. Combining extended quantization with G-PCC reduces storage by a further 49-51% relative to default quantization, with 0.03-0.19 dB additional mean PSNR loss. These are within-NGMS operating points, separate from the published baseline points above.

Retaining explicit base Gaussian attributes makes this integration straightforward while preserving NGMS's model structure and training recipe. The HAC++ hash-grid context model and rate loss are not applied. The experiment illustrates how learnable parameter reduction can benefit from further coding.

4. Fine detail and appearance. We have upgraded Figures 3 and 4 with matching zoom-in crops and absolute RGB error maps, using consistent regions and error scales across methods. Figure 3 compares NGMS with 3DGS and Scaffold-GS on repeated lego studs, thin bonsai branches, and view-dependent reflections. Figure 4 compares NGMS with 4DGS and 4D Scaffold-GS on the trex ribcage and hook boundaries. Ground truth is included in both figures. These additions expose local detail and appearance errors; NGMS retains clearer articulated structures with lower errors in the highlighted dynamic regions.

Appendix B.1 reports per-scene SSIM and LPIPS. On NeRF Synthetic, NGMS reaches SSIM 0.968 and LPIPS 0.035, close to 3DGS at 0.970 and 0.031. On Mip-NeRF 360, NGMS has SSIM 0.800 and LPIPS 0.255 versus 0.826 and 0.202 for 3DGS. The upgraded visual comparisons complement these metrics and make the local quality-storage trade-offs visible.

5. Large scene scalability. We acknowledge that, similar to 3DGS, NGMS remains subject to VRAM limits as the base Gaussian counts grow, with additional memory required for rasterization buffers and training states.

Retaining standard Gaussian attributes also provides a concrete route to integrate large-scene memory management, analogous to the coding integration in Section 3. LODGE (NeurIPS 2025) combines spatial chunking, view-dependent loading, and level of detail (LoD); A LoD of Gaussians (SIGGRAPH 2026) supports out-of-core training and rendering through hierarchical LoD selection and streaming. A corresponding NGMS extension would spatially index base-mixture groups, keep the shared field resident on the GPU, stream potentially visible groups from host memory, and expand only selected groups before standard rasterization. This would combine learnable parameter reduction with a smaller active GPU working set. This integration and peak-memory evaluation on substantially larger scenes remain future work.

As the discussion period is approaching its end, we wanted to send a brief reminder regarding our rebuttal and updated revision.

Thank you again for your constructive feedback. We have carefully addressed your comments in our response and updated the revision accordingly. We hope these changes clarify the concerns you raised and would greatly appreciate any further feedback when you are available.

We look forward to hearing your thoughts and continuing the discussion.

Thank you for the authors' response. However, for 4DGS, it appears that the more widely adopted commercial applications are currently based on approaches such as DualGS and human-body & large motion reconstruction (e.g., the ZJU-3DV public dataset). I suggest that the authors consider 4DGS from a commercialization perspective, as this would create more opportunities for practical applications of the technology.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.