acceptodds
Under review as a conference paper at ICLR 2027

RATE-DISTORTION ADAPTIVE PRIMITIVE SELECTION FOR OMNIDIRECTIONAL GAUSSIAN SPLATTING

Abstract

Learned image codecs (LICs) achieve high reconstruction quality, but their decoding speed is often insufficient for immersive virtual reality (VR). Gaussian splatting (GS) codecs render much faster, yet still lag in reconstruction quality and typically decide primitive allocation without considering the coding cost of each primitive. We introduce OIC-GS, an omnidirectional GS codec with a new hierarchical HEALPix primitive grid representation. Gaussian primitives are anchored at predefined spherical locations, eliminating explicit coordinate coding. Finer levels refine their coarser ancestors, naturally supporting coarse-to-fine reconstruction and layered transmission. The predefined grid also enables efficient viewport decoding by selecting only view-relevant primitives. We further introduce a lightweight entropy model for quantized primitives and optimize the codec under a spherical rate-distortion objective. Primitives with insufficient rate-distortion benefit are automatically removed when their quantized opacity becomes zero, allowing OIC-GS to adapt both primitive density and level of detail without a fixed primitive budget. A single bitstream supports full-sphere, viewport-dependent, and progressive decoding. The first viewport reaches final quality after decoding only 52% of the bitstream, and is then rendered at 1,270 FPS. On a 100-image omnidirectional benchmark, OIC-GS outperforms all evaluated GS codecs, reducing WS-PSNR BD-rate by 49.6% over GaussianImage++ and 68.6% over SGI, which uses a learned entropy model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
2D Gaussian splatting, image compression

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3D9HOpenReview ↗
Rating
3
Confidence
1
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OIC-GS, an omnidirectional Gaussian splatting codec based on a hierarchical HEALPix primitive grid. Primitives are anchored at predefined spherical locations, so coordinates need not be coded, and a lightweight entropy model estimates the rate of each primitive. A rate-distortion adaptive primitive selection mechanism removes primitives whose coding cost is not justified, and a rendering-conditioned context model improves entropy coding. A single bitstream supports full-sphere, viewport-dependent, and progressive decoding. Experiments show significant BD-rate reductions over evaluated GS codecs, with real-time viewport rendering.

Critical strengths

The paper presents a well-motivated approach with convincing results. The proposed OIC-GS achieves significant BD-rate reductions over all evaluated GS codecs, and the overall pipeline is logically coherent. The combination of HEALPix-based hierarchical representation, rate-distortion adaptive primitive selection, and rendering-conditioned context is technically sound, and the experimental validation on both the main benchmark and SUN360 supports the main claims. The viewport-dependent and progressive decoding capabilities are practically relevant for immersive VR applications.

Critical weaknesses

Neither the abstract nor the introduction explicitly states "omnidirectional image compression" or "GS-based image compression." Given that the paper's core task is compression, this omission weakens the positioning and may mislead readers into thinking this is a rendering/representation paper.

Weak connection between Figure 1 and Figure 2. Figure 1 lacks any variable or visual cue linking it to Figure 2. For example, pass 2 outputs h, but h does not appear in Figure 1, and there is no arrow or annotation guiding the reader from Figure 1 to Figure 2. It is difficult to quickly establish the correspondence between the two figures.

In Figure 2 caption. RCC stands for "rendering-conditioned context," not "rendering-conditioned entropy model.". In the "spherical Gaussian rendering" part of Figure 1, the last row should be F6

From Equation (9) onward, it is hard to connect the equations to the content of Figure 2, and it is unclear which part of the pipeline these equations correspond to. The paper would benefit from clearer textual linking between equations, figures, and the encoding/decoding workflow.

Questions for authors

How did you come up with Equation (10)? Is there any reference or rationale behind it?

I'm curious whether RCC and RDPS need to be enabled at all levels. Table 14 shows level 4 has a selection ratio of 71.9–78.6% while level 6 only has 6.3–18.2%, and Table 3 shows removing level 4 from the checkerboard only costs +0.4%. This suggests different levels play very different roles. Could you discuss the trade-off of not enabling RCC or RDPS at the coarser levels? An ablation would help.

Reviewer_51C2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes OIC-GS, an omnidirectional GS codec with hierarchical HEALPix primitive grid representation. It combines fixed Spherical location anchoring, quantized features and opacities, rate-distortion adaptive primitive selection, and a rendering-conditioned entropy model. The resulting bitstream supports full-sphere, viewport-dependent, and progressive decoding with a lower BD-rate.

Critical strengths

The proposed OIC-GS combines the advantages of HEALPix grid representation, entropy coding, and partial decoding. It achieves substantial improvements over the evaluated GS image codecs on two datasets. Reporting actual bitstream sizes, including auxiliary information, and verifying bit-exact viewport decoding are valuable aspects of the evaluation.

Critical weaknesses

Major

a) More evidence about the superior of combining GS with HEALPix-based spherical representation. The HEALPix-based spherical representation and omnidirectional compression have already been explored, notably in OSLO. Thus, the authors should verify the contribution of introducing GS. Since the primitive positions are fixed and kernel widths are shared within each level, the possible baseline could be a non-Gaussian interpolation scheme using hierarchical spherical latent grid with local kernel interpolation, and same entropy model, and selection mechanism.

Besides, since the HEALPix-based representation nearly uniformly samples the locations on the spherical surface, the comparison with ERP-based merhods, i.e., GaussianImage, GaussianImage++, should consider the difference of training objective (ERP is a distorted projection format) for better showing where the performance improvement comes from, introducing HEALPix-based representation or just changing the training objective.

b) The RDPS ablation should better isolate the entropy-aware selection. As shown in Eq.9 and Eq.11, the current penalisation will continously encourage the GSs' opacities to be lower and the primitives to be with lower bits. The ablation in Appendix E.2 replaces the default linear opacity weighting with a saturated function, which will remove the penalisation (the derivative to the opacity will be zero rather than the feature rate). One reasonable ablation is to retain the same opacity relaxation and gradient path, but replace the per-primitive coding costs with a shared constant in the opacity penalty, while retaining entropy-model training. This might help verify whether using each primitive’s individual coding cost provides benefits beyond a uniform sparsity penalty.

c) The benefits for real VR use need stronger evidence. The appendix includes progressive decoding and simulated head turns, but no user study with a VR headset. As authors also recoginize this work is more for applications, the related user study should be provided.

Minor

a) More comparison experiments are required. Currently, the authors only compare the speed with ElIC, MLIC++; however, the Rate-distortion performance comparison should be provided to confirm the advantages of OIC-GS on quality and bit rate. Meanwhile, low-complexity per-image methods, COOL-CHIC and C3 should be not only introduced but also compared.

b) Quality and speed should be measured using the same rendering process. The main quality results are computed after converting HEALPix to ERP, while the fast renderer generates viewports directly from HEALPix. The quality of this faster output should be reported more fully. Also, the reported 1,270 FPS applies to a fixed view; re-rendering during head turns runs at about 314 FPS on average.

c) One type error. No author information for "2d gaussian splatting for image compression".

Questions for authors

Since the rating mechanism is differently from the previous one, I rated with Weak rejection. But I encourage the authors could provide more experimental resutls, i.e., the rate-distortion performance comparison with EIIC and MLIC and the comparison with COOL-CHIC and C3, as said in Weaknesses. If the authors could solve my concerns, I will change me rating.

Reviewer_oZXzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces OIC-GS, a per-image codec for omnidirectional images. It represents an image using a hierarchy of fixed HEALPix locations, quantized features and opacities, and a small color network. An opacity-weighted rate surrogate encourages primitive selection under a spherical distortion objective. A rendering-conditioned entropy model uses previously decoded levels and a two-pass spatial context. The resulting representation supports full-sphere reconstruction, coarse-to-fine previews, and tile-based viewport access. Experiments on two 100-image datasets show substantial gains over the evaluated Gaussian image codecs, with actual coded bitstreams and additional analyses of decoding, selection and resolution.

Critical strengths
  1. A coherent codec design with a useful spherical representation. Fixed HEALPix positions avoid coordinate transmission and ERP sampling imbalance. Coarse-level inheritance, rendering-conditioned coding, and spatially addressable streams fit together naturally. While HEALPix compression and rate-distortion-optimized Gaussian representations have prior precedents, their integration here into an omnidirectional codec with selective viewport access is a meaningful technical contribution.

  2. Strong empirical support within the stated GS comparison scope. Tables 1 and 7 and Figs. 3–4 report sizeable improvements over several Gaussian image codecs on 100 main-benchmark and 100 SUN360 images. Appendix A.3 describes public-code refitting and explicitly accounts for SGI’s complete bitstream. The authors use written file sizes, report four spherical/viewport quality measures, and repeat ablations on both datasets. The gains therefore have substantially better support than a single-dataset or estimated-rate-only evaluation.

  3. Careful verification and transparent limitations. Appendix D checks bit-exact decoding and viewport agreement across 500 fitted models and describes the 1.2% average random-access storage overhead. Appendix F distinguishes representation capacity from matched-rate performance and evaluates an additional level, yielding a 7.7% WS-PSNR BD-rate reduction on 20 images. The paper also acknowledges JPEG2000’s superior rate-distortion performance and reports the slower moving-viewport runtime rather than hiding it.

Critical weaknesses
  1. The ablation does not isolate the headline contribution of primitive-specific rate-aware selection. The main RDPS ablation sets the active-state rate weight to one. As explained in Appendix E.2, this removes the rate gradient with respect to opacity above the threshold; it consequently retains 6.1 times as many primitives at the same multiplier. The resulting +31.2% rate difference is evidence for the usefulness of a differentiable selection surrogate, but does not establish how much comes from using each primitive’s own predicted code length rather than a generic differentiable sparsity penalty. The paper would be substantially stronger with a constant-cost opacity penalty and a fixed-budget or matched-count selection baseline under the same HEALPix representation, entropy model and fitting schedule. This also matters for novelty: the paper already acknowledges learnable Gaussian masks and rate-distortion optimization in prior work, and its distinguishing claim should be demonstrated directly.

  2. The GS comparison changes both representation and distortion allocation. OIC-GS is optimized with uniform spherical distortion (Eq. 13), whereas Appendix A.3 describes fitting the planar baselines using their official configurations on ERP images. The evaluation appropriately emphasizes spherical and viewport quality, but the reported gaps can combine the proposed representation/selection gains with the advantage of optimizing a better-aligned distortion objective. At least GaussianImage++ and SGI should also be evaluated with a latitude-weighted reconstruction loss, keeping their coding mechanisms intact. The existing off-the-shelf comparison remains useful; it does not by itself isolate the benefit of the new selection mechanism or spherical representation.

  3. The empirical rate-distortion-complexity trade-off is not established against the LICs used to motivate the application. Table 2 times ELIC, MLIC++ and other LICs, but Tables 1 and 7 provide no corresponding quality-rate results. JPEG2000 is already appreciably better in Table 1. Moreover, a static LIC reconstruction can also be cached and projected to changing viewports; comparing repeated GS rendering with repeated LIC synthesis does not establish an equivalent difference in interactive display speed. A shared caching and viewport protocol is needed. The restriction to GS image codecs is clearly stated in Appendix A.3 and is reasonable, but the broader argument about a practical alternative for immersive viewing needs matched-quality or matched-rate first-viewport comparisons.

Questions for authors
  1. Can you compare the primitive-specific code-cost penalty with a constant per-primitive penalty and matched-count/fixed-budget selection in the same representation? How much of the RDPS gain survives these controls?

  2. How much of the gain over GaussianImage++ and SGI remains when those baselines optimize a spherical/latitude-weighted loss? This would help distinguish improved distortion allocation from improvements in representation and coding.

  3. What is the relationship between the training surrogate and the complete written rate over the operating range? Appendix E.1 validates the masked feature rate of Eq. 9, whereas training uses the opacity-weighted Eq. 11. Appendix D.1 reports substantial mask, opacity and header costs. Please report the Eq. 11 estimate, full-rate breakdown, and actual files together, and clarify which rate-dependent terms receive an opacity gradient. This is a request to substantiate the optimization interpretation, not a claim that the file-size results are invalid.

  4. Can the existing LICs and JPEG2000 be placed on a matched-quality first-viewport latency/transfer plot, including all startup costs, and can viewport transfer ratios be summarized across the benchmark and multiple orientations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.