acceptodds
Under review as a conference paper at ICLR 2027

Rate-Adaptive Context Evolution for Progressive 3D Gaussian Coding

Abstract

Progressive 3D Gaussian Splatting (3DGS) compression decodes a hierarchy of refinement levels from a single bitstream. However, existing progressive codecs largely rely on entropy contexts whose content or spatial granularity does not adapt to the level-dependent distribution of active Gaussians, limiting probability estimation. We introduce Rate-Adaptive Context Evolution (RACE), which treats the entropy context as a causal state that evolves with decoded content. RACE first splats previously decoded Gaussian attributes onto directional hint maps to update a shared triplane prior. It then selects a directional granularity tuple using encoder-side rate estimates and transmits the selected indices, allowing the decoder to reconstruct the same context without repeating the search. Finally, cross-level attention uses the preceding-level anchor state to condition the aggregation of the current-level spatial neighborhood. Built on the progressive framework of PCGS, RACE produces multiple operating points from a single bitstream. Across 19 scenes from four benchmarks, RACE achieves the lowest aggregate compression rank among the compared methods with complete four-dataset coverage, while keeping decoding time close to PCGS. These results demonstrate the benefit of adapting context content, spatial scale, and cross-level interaction for progressive 3DGS entropy modeling.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, 3DGS Compression, Progressive Coding, Rate-Adaptive Context Modeling, Entropy Coding, Rate-Distortion Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Uw2zOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on entropy context modeling in progressive 3DGS compression and proposes RACE. The paper points out that existing methods typically reuse static spatial priors across layers, so newly activated anchors lack cross-layer information, and the context granularity is also difficult to match the per-layer Gaussian distributions. Mechanisms such as generating a hint map from the attributes of Gaussians decoded in the previous layer and updating a shared triplane prior improve task performance, and this is validated on multiple datasets.

Critical strengths

The paper's problem motivation is supported by data. The number of new anchors is small, and refined anchors account for most of the payload, so updating the shared prior is a targeted design. The method preserves causality, the granularity index precedes attribute transmission, the decoder does not need to perform repeated searches, and the decoding overhead is close to that of PCGS. Experiments show that the method achieves improvements on multiple datasets.

Critical weaknesses
  1. The paper lacks an asymptotic decoding comparison with PCGS.

  2. Current 3DGS compression methods generally require per-scene optimization except for the FCGS family, so does the encoding time include training time? It is recommended to report the relevant metrics, because encoding time that excludes training time lacks practical significance for this type of problem.

Questions for authors

Answer the questions raised by the critical weakness.

1 comment
AuthorsOpenReview ↗

Response to Reviewer Uw2z

Response to Reviewer Uw2z

We thank the reviewer for requesting a clearer account of decoding complexity and end-to-end optimization cost. We have added both theoretical and empirical measurements.

W1: Asymptotic decoding comparison with PCGS

Answer. We have added both measured decoding time and asymptotic analysis; RACE and PCGS have nearly identical decoding time and the same O(Ns)O(N_s) decoding order.

Table 3 now reports an average decoding time of 27.827.8 s for RACE and 27.427.4 s for PCGS over the same 19 scenes, corresponding to only a 1.5%1.5\% increase. This small difference follows from the decoder design. Let NsN_s be the number of active anchors, KK the fixed neighborhood size, and P=3P=3 the directional planes. Hint splatting, neighborhood attention, and probability prediction cost O(PNs)O(PN_s), O(KNs)O(KN_s), and O(Ns)O(N_s), respectively. Since PP and KK are constants, RACE retains the O(Ns)O(N_s) decoding order of PCGS and only adds a linear context-construction term. The candidate-granularity search is encoder-only; the decoder reads the selected tuple directly and therefore has no ∣Ss∣|\mathcal{S}_s| search factor.

Revision. Section 4.4, Table 3 and the new paragraph “Decoder complexity” (p. 10).

W2: Does encoding time include training?

Answer. No; the reported encoding time measures post-training bitstream coding and excludes per-scene optimization.

We now state this explicitly in the text and Table 3 caption. We additionally report matched optimization measurements on five scenes: PCGS versus RACE averages are 66.466.4 versus 122.6122.6 minutes for training and 11.1711.17 versus 23.2523.25 GiB for peak GPU memory. Presenting these values separately prevents Table 3 from being interpreted as end-to-end training cost.

Revision. Section 4.4, revised Table 3 caption and the new matched training-cost table (p. 10).

Reviewer_7JM8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Rate-Adaptive Context Evolution (RACE) for progressive 3D Gaussian Splatting compression. Built on PCGS, it retains the progressive masking and quantization framework while modifying entropy-context modeling through decoded-information-guided triplane updates, rate-based directional resolution selection, and cross-level attention.

Critical strengths

The method preserves the practical benefit of obtaining multiple reconstruction qualities from a reusable progressive bitstream. The paper includes rate–distortion curves and reports improvements over its underlying progressive baseline, PCGS. These results provide evidence that the proposed context modifications can improve compression within this framework.

Critical weaknesses
  1. The incremental novelty over existing progressive compression methods needs clearer justification.

LOD representations, progressive coding, and cross-level context modeling already have substantial prior work. The paper explicitly builds on PCGS, which already supports progressive refinement and uses previously decoded information for probability prediction. I recognize that RACE proposes additional context-modeling mechanisms rather than claiming the progressive framework itself as new. However, I am not yet convinced that the distinction from existing approaches is sufficiently substantial. The contribution should be articulated more precisely around what the proposed context evolution adds beyond PCGS and related context-based codecs.

  1. The comparisons do not sufficiently establish a broad rate–distortion advantage over strong competing methods.

Table 1 compares selected operating points with different sizes and reconstruction qualities. Although the authors describe the aggregate rank as a summary statistic rather than their primary RD evidence, this rank cannot establish superiority at matched rates or matched quality.

The RD comparison is more informative, but some important baselines are sparsely sampled. Table 4 provides only two HEMGS operating points on Mip-NeRF 360. Its higher-rate point achieves 27.940 dB at 20.03 MB, compared with RACE’s 27.927 dB at 21.53 MB. Thus, HEMGS is smaller and has slightly higher PSNR at these measured points. The PSNR difference is tiny and should not be interpreted as a meaningful perceptual advantage, but these results do not support uniform superiority of RACE.

A single-rate codec can, in principle, be evaluated at multiple operating points by adjusting its rate-control settings and training or encoding separately. Denser, matched-rate comparisons with HEMGS would help determine whether RACE’s apparent intermediate-rate advantage persists. The reported 27.37% BD-rate reduction over PCGS on Mip-NeRF 360 also covers a narrow common PSNR interval, 27.793–27.852 dB, and should not be interpreted as a full-range improvement.

Overall assessment

The results suggest a useful improvement over PCGS, but I remain unconvinced that the current novelty argument and comparative evaluation establish a sufficiently strong contribution. Clearer positioning and denser matched-rate comparisons would be important for resolving these concerns.

Questions for authors
  1. What is the clearest technical distinction from PCGS and the closest context-based compression methods?
  2. Why are only two HEMGS operating points included on Mip-NeRF 360? Can additional points be evaluated near RACE’s operating rates?
  3. Can the authors more clearly separate the benefits of a progressive bitstream from claims about RD efficiency against single-rate codecs?
1 comment
AuthorsOpenReview ↗

Response to Reviewer 7JM8

Response to Reviewer 7JM8

We thank the reviewer for the constructive questions about technical novelty and comparative evaluation. We have clarified the motivation, scope, and evidence of our claims as follows.

W1/Q1: Technical distinction from PCGS and context-based codecs

Answer. RACE makes the entropy context evolve with the level-dependent anchor distribution, thereby reducing the context mismatch that arises when progressive refinement introduces new anchors and changes existing ones.

This motivation distinguishes RACE from both PCGS and single-rate context codecs. PCGS conditions an existing anchor on its own previous-level value, but this anchor-local history is unavailable to newly activated anchors and does not summarize the updated spatial distribution. RACE therefore projects all decoded anchors into a spatial prior shared by refined and new anchors, selects an anisotropic XY/XZ/YZXY/XZ/YZ granularity tuple that matches the current level, and uses the previous-level feature as the query to condition current-level neighborhood aggregation before distribution prediction. These operations provide broader cross-level coverage, level-appropriate spatial evidence, and direct interaction between long- and short-term dependencies. The resulting probability model assigns fewer bits to current-level attributes, as supported by the component ablations in Table 2 and the consistent R-D improvement over PCGS.

Revision. Section 2.1, final paragraph (p. 3); Section 3 overview and Sections 3.1--3.3 (pp. 3--5).

W2/Q2: Only two HEMGS operating points

Answer. Only two points are shown because HEMGS has not released its source code, and these are the only Mip-NeRF 360 results available in its paper appendix.

We extracted both published points directly from the appendix; they were not selected from a denser curve. Without the implementation, we cannot retrain HEMGS or generate additional operating points under matched rate-control settings. We agree that two points cannot establish uniform superiority: near 2020--2222 MB, HEMGS is slightly better than RACE, and the revised text states this directly. We therefore use HEMGS only as a published single-rate reference; the controlled progressive comparison is against PCGS under the same framework and one reusable bitstream.

Revision. Section 4.1, “Baselines,” and Section 4.2, paragraph following Fig. 3 (p. 6).

Q3: Progressive-bitstream benefit versus single-rate R-D claims

Answer. We now separate the reusable progressive-bitstream claim from comparisons with independently trained single-rate codecs.

RACE's progressive benefit is that one causal bitstream exposes multiple reconstruction levels, which is evaluated directly against PCGS. Comparisons with single-rate codecs only indicate where these reusable operating points lie and are no longer described as uniform matched-rate superiority. The compression rank is retained as an aggregate summary, not as the primary R-D evidence.

Revision. Section 4.2 and the discussion surrounding Table 1 (pp. 6--7).

Reviewer_aS6uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a progressive 3DGS compression framework in which the entropy contexts adapt to the level-dependent distribution of active Gaussians.Building upon PCGS (Chen et al., 2026), whose entropy model is shared across levels and conditioned only on a level-specific scalar, the authors introduce an updatable hint-map and a hierarchical context attention module to model intra-level context and capture spatial structure. In addition, they propose a rate-adaptive granularity mechanism that selects the projection granularity of the spatial prior by estimating coding rate.

Critical strengths
  1. Addressing the key limitations of prior progressive 3DGS compression methods by using a hint-map preserves spatial relationships among decoded anchors.

  2. This method achieved R-D performance comparable to single-rate codecs and shifted backbone method PCGS (Chen et al., 2026) toward a lower rate that uses only previously decoded Gaussians as context, with only a modest increase in encoding time.

Critical weaknesses
  1. The R-D improvement over PCGS is limited. At the three lowest rate points, neither PSNR nor size improves much, while encoding time increases. The gain does not clearly justify the added complexity.

  2. Compared with PCGS, the method shows a large FPS drop on BungeeNeRF datasets, but the paper does not explain why. Since the entropy model should not affect rendering speed, this may suggest that more Gaussians are kept, which makes the R-D comparison harder to interpret.

Questions for authors
  1. At the first level, no anchors have been decoded yet. How is the hint-map initialized?

  2. In Table 3, the FPS on BungeeNeRF is much lower than that of PCGS. What causes this drop?

  3. This paper does not report training time. Could the authors provide the training time and memory usage compared with PCGS?

1 comment
AuthorsOpenReview ↗

Response to Reviewer aS6u

Response to Reviewer aS6u

We thank the reviewer for recognizing the motivation of our hint-map design and the consistent R-D improvement over PCGS. We have revised the manuscript and respond point by point below.

W1: Low-rate improvement and added complexity

Answer. RACE consistently improves both reconstruction quality and compressed size at the low-rate points, while the added rate-selection search is confined to the encoder and does not alter the asymptotic decoding complexity.

At the three lowest Mip-NeRF 360 operating points, RACE improves PSNR by 0.0800.080--0.0830.083 dB and simultaneously reduces size by 2.7%2.7\%--5.3%5.3\% relative to the corresponding PCGS points. These jointly improved operating points are obtained from one reusable progressive stream rather than separately optimized single-rate models. The decoder only reads three small granularity indices and reconstructs the selected context once; therefore, the search incurs no decoder-side search, and the measured decoding-time increase over PCGS is only 1.5%1.5\%. For precise interpretation, we now state that the reported 27.37%27.37\% BD-rate is evaluated over the common PSNR interval [27.793,27.852][27.793,27.852] dB.

Revision. Section 4.2, paragraph following Fig. 3 (p. 6); Appendix B, opening paragraph (p. 12); Section 4.4, “Decoder complexity” (p. 10).

W2/Q2: BungeeNeRF FPS

Answer. The apparent large FPS drop was caused by an FPS extraction error, not by entropy-context inference during rendering.

The submitted Table 3 inadvertently mixed intermediate training-time evaluations with post-decoding rendering measurements for RACE on BungeeNeRF. Recomputing FPS from the final decoded representations gives 184.1184.1 FPS rather than 169.9169.9 FPS. At the complete reconstruction endpoint, RACE gives 217.2217.2 FPS versus 234.5234.5 FPS for PCGS. Because the entropy model is not invoked during rendering, the remaining endpoint difference reflects the decoded representation and renderer measurement. We corrected the table and now define the FPS protocol explicitly.

Revision. Section 4.4 and Table 3, BungeeNeRF FPS column (p. 10).

Q1: First-level hint-map initialization

Answer. No hint map is used at level 1; the first hint map is constructed after level 1 has been decoded.

At the first progressive level, RACE uses the reconstructed base triplanes directly because no anchor is yet available. The decoded level-1 anchors then form the hint maps for level 2, where context evolution begins. This procedure is causal and identical at the encoder and decoder.

Revision. Section 3.1, second paragraph (p. 4).

Q3: Training time and memory

Answer. RACE requires more per-scene optimization time and memory than PCGS, while the Enc./Dec. times in Table 3 exclude this training stage.

We ran matched PCGS and RACE optimization on five representative scenes using RTX 5880 Ada GPUs, identical iteration schedules, and one-second memory sampling. Averaged over these scenes, training time is 66.466.4 versus 122.6122.6 minutes and peak memory is 11.1711.17 versus 23.2523.25 GiB for PCGS and RACE, respectively (+84.7%+84.7\% time and +108.2%+108.2\% memory). Across individual scenes, time overhead ranges from 64.2%64.2\% to 100.5%100.5\%, and memory overhead from 40.6%40.6\% to 198.5%198.5\%. The added cost comes from directional context planes, candidate-rate evaluation, and cross-level attention. We now report all per-scene measurements separately from post-training coding and decoding.

Revision. Section 4.4, paragraph following Table 3 and the new matched training-cost table (p. 10).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.