acceptodds
Under review as a conference paper at ICLR 2027

FeCoSplat: Feedback-Guided Compression for Feed-Forward 3D Gaussian Splatting

Abstract

Feed-forward 3D Gaussian Splatting (3DGS) enables efficient novel-view synthesis from sparse multi-view images, yet its representations remain costly to store and transmit. Existing approaches compress either the input images, incurring heavy receiver-side reconstruction, or the reconstructed Gaussian primitives, which are difficult to compress due to their heterogeneous and irregular attributes. We instead compress compact intermediate features, providing a better balance between compression efficiency and receiver-side complexity. Based on this paradigm, we propose FeCoSplat, a feedback-guided compression framework for feed-forward 3DGS. FeCoSplat first compresses multi-view features to obtain an intermediate 3DGS, whose rendered views are used as feedback to guide a second-stage compression for further refinement. The resulting bitstreams are decoded into a compact implicit state, from which the final Gaussian primitives are reconstructed with a lightweight predictor. Experiments demonstrate that FeCoSplat achieves favorable rate–distortion performance, particularly at low bitrates, while requiring only 3.45M parameters for receiver-side Gaussian reconstruction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feedforward 3DGS, compression

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Ts4EOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper tackles the problem of feedforward 3dgs compression and studies how to send a 3dgs scene built by a feed forward model to a weak device (which is memory constrained) using as few bits as possible. Sending the input images directly forces the receiver to run the whole feedforward network whereas sending the full set of gaussians is far too costly in terms of the communication bits. The authors main contribution is to send the networks intermediate features instead and they do it in 2 rounds. In the first round the sender compresses the features and decodes them itself just as the receiver would and then it builds the 3d scene from this decoded version and renders it back at the input views so it can compare the renders with the real images. This lets the second round send only what the first round missed and the recevier combines the two decoded pieces and turns them into gaussians with a small network. Across 3 standard scene datasets the method matches or slightly beats prior compression approaches.

Critical strengths
  1. The paper frames feed forward 3dgs compression in terms of the split of computation between sender and receiver -> It compares image compression and gaussian compression and feature compression under one setup (Fig 1 and Table 1) and also evaluates zeroshot transfer to ACID. Under this framing the decoder is very lightweight. It has only 3.45M parameters and requires no intermediate rendering since the feedback loop runs entirely at the encoder (Sec 3.3). This is about 13x smaller than the CodecSplat decoder and far smaller than running DepthSplat or ReSplat at the receiver. This is a meaningful advantage in a one2many setting where a single sender serves many resource constrained receivers.

  2. The component ablation (Fig 5 left) directly tests the core design choice -> When the second stage extractor receives only the input image without the rendered view it loses about ~0.6 dB PSNR at high rates. This indicates that the rendering feedback carries information beyond what a second encoding pass of the same image provides.

Critical weaknesses
  1. The better RD claim holds only partly -> On RE10K and ACID FeCoSplat nearly ties CodecSplat in PSNR and SSIM. On DL3DV it is below CodecSplat at every rate and only FeCoSplat* wins by using CodecSplat's frozen backbone and 150K extra fine-tuning steps. Elsewhere the backbones differ (DepthSplat small vs base). The LPIPS gains may partly come from the LPIPS training loss and there is no BD rate or variance to judge the gain. The claim should be "a much lighter receiver at similar RD" but not "better RD".

  2. The novelty of the feedback stage is overstated -> Decoding at the encoder and then coding only what the reconstruction still misses is a long standing idea in learned compression. Toderici et al. (ICLR 2016 and CVPR 2017) already code residuals iteratively with RNNs whose recurrent decoder state is updated by each new bitstream which is essentially the S0 -> S1 -> S2 state with two stages. Progressive codecs such as DPICT (CVPR 2022) and DCVC style conditional coding on decoded context (NeurIPS 2021) follow the same pattern and none of this work is cited.

Questions for authors
  1. Please report BD rate in PSNR and LPIPS against CodecSplat on all three datasets and run the CodecSplat backbone variant on RE10K and ACID as well. This would show whether the RD gain holds beyond DL3DV and beyond the backbone change, and the claims in the abstract and Sec 4.3 should then be aligned with the results.

  2. Please show the two stage results on top of a strong single-stage model (for example the FeCoSplat* backbone with one stage) so that the feedback is not measured against a weak base.

  3. Please discuss the closed-loop residual and conditional coding literature (Toderici et al. 2016 and 2017, DPICT, DCVC) and state clearly what is new relative to it.

5 comments

W1&Q1: Rate--Distortion Performance and Model Comparability.

We thank the reviewer for raising concerns about RD comparisons and backbone differences. To further investigate the impact of model capacity, we conducted additional experiments with three FeCoSplat variants (S, B, and L) at different parameter scales. For each variant, we trained separate models on RE10K and DL3DV for 300K iterations each. We evaluated the RE10K-trained models on ACID without fine-tuning. It should be noted that FeCoSplat-B/L follow the same training strategy as CodecSplat: the pretrained multi-view feature extractor is frozen, while the remaining modules are trained for 300K iterations, consistent with the training schedule specified in the official CodecSplat GitHub repository. This corresponds to the second stage of CodecSplat's two-stage training strategy described in the original paper. Importantly, FeCoSplat-B/L are trained using the pretrained multi-view feature extractor rather than fine-tuned from a fully trained CodecSplat model. We have also added detailed RD results and PSNR-/LPIPS-based BD-Rate comparisons in Appendices B.1 and B.6.

Table 1: Comparison of rate–distortion performance and model complexity. BD-Rate is computed relative to CodecSplat and negative values indicate better compression efficiency.

MethodSender Params (M) ↓Receiver Params (M) ↓RE10K PSNR BD-Rate (%) ↓RE10K LPIPS BD-Rate (%) ↓ACID PSNR BD-Rate (%) ↓ACID LPIPS BD-Rate (%) ↓DL3DV PSNR BD-Rate (%) ↓DL3DV LPIPS BD-Rate (%) ↓
ELIC→DepthSplat-B26.45144.81-1.76+53.51+28.11+137.43+232.54+248.17
ELIC→ReSplat-B26.45246.20-6.30+42.71-11.52+133.48-52.12+188.33
CodecSplat181.7324.900.000.000.000.000.000.00
FeCoSplat-S37.283.45+13.85-12.28+6.22-17.17+17.39+15.52
FeCoSplat-B126.113.45-4.69-18.46+2.47-22.62-22.36-14.87
FeCoSplat-L154.8023.60-26.13-35.74-15.23-37.91-49.03-44.43

FeCoSplat-B/L achieve competitive rate–distortion performance compared with CodecSplat while requiring fewer model parameters, demonstrating favorable compression performance across different model capacities. Gaussian-compression baselines are excluded from BD-Rate calculations because their higher operating bitrates provide insufficient RD-curve overlap, making extrapolation unreliable. Nevertheless, the RD curves and complexity comparisons show that FeCoSplat-S achieves better compression efficiency with lower overall model complexity and coding latency than these Gaussian-compression-based pipelines. We have also revised the relevant discussions of our experimental results to better reflect the trade-offs among rate–distortion performance and model complexity, providing a more comprehensive analysis from a rate–distortion–complexity (RDC) tradeoff.

Regarding perceptual supervision, FeCoSplat uses the same pixel-wise and LPIPS reconstruction losses as CodecSplat. Thus, the improvements cannot be attributed solely to differences in training objectives. Furthermore, FeCoSplat demonstrates stronger zero-shot generalization across input-view counts, spatial resolutions, and datasets (Section 4.4, Appendix B.7), supporting its effectiveness beyond standard evaluation settings.

W1&Q1: Rate--Distortion Performance and Model Comparability (Continued from the Previous Comment).

We also provide a controlled CodecSplat w/ Feedback experiment in Appendix B.2.2. It demonstrates that rendering-based feedback improves BD-PSNR by +0.08 dB on RE10K and +0.19 dB on DL3DV compared with the original CodecSplat, with larger gains under zero-shot settings (+4.02 dB for view-count generalization and +1.01 dB for resolution generalization). This provides additional evidence that the improvements are not solely attributable to differences in feature extraction backbones or training strategies.

Table 2: Effectiveness of rendering-based feedback on a frozen pretrained CodecSplat baseline. Due to the noticeable differences in PSNR between the two methods, we report BD-PSNR instead of BD-Rate, which measures the average PSNR gain at equivalent bitrates. BD-PSNR improvements are measured relative to the original CodecSplat, with higher value indicating a stronger performance.

MethodRE10K BD-PSNR ↑DL3DV BD-PSNR ↑DL3DV (2 ctx) BD-PSNR ↑DL3DV (256×256) BD-PSNR ↑Sender Params (M) ↓Receiver Params (M) ↓Sender Latency (s) ↓Receiver Latency (s) ↓
CodecSplat0.000.000.000.00181.7324.900.1890.138
CodecSplat w/ Feedback+0.08+0.19+4.02+1.01183.9827.180.2730.195

W2&Q3: Novelty of Rendering-Based Feedback.

We thank the reviewer for highlighting the connection to progressive compression. We acknowledge that multi-stage coding and recurrent state updates are established techniques, and have expanded Appendix C.2 to clarify the distinctions between our approach and related methods in progressive image and video compression.

Our contribution lies in task-driven progressive feature compression through rendering-based feedback for 3D scene reconstruction. Unlike conventional progressive coding methods that progressively refine image reconstructions or latent representations toward predefined reconstruction targets, feed-forward 3DGS reconstruction relies on indirect supervision through rendered 2D images, as ground-truth Gaussian primitives are generally unavailable and the learned multi-view features lack predefined reconstruction target. Inspired by this observation, FeCoSplat reconstructs intermediate 3D Gaussians and renders them to guide the extraction of additional scene information, incorporating this rendering-based feedback into the second-stage coding process.

To validate this distinction, we introduce a residual feature coding baseline (Appendix B.2.1, Figure A3, and Table A3), which extends the residual coding strategy of the residual autoencoder mentioned in Toderici et al. (ICLR 2016) [1] to multi-view features in our framework:

Table 3: Comparison of rendering-based feedback and residual feature coding. Due to the noticeable differences in PSNR between the two stages, we report BD-PSNR instead of BD-Rate, which measures the average PSNR gain at equivalent bitrates. BD-PSNR is measured relative to Stage 1 of feedback-guided coding, with higher value indicating a stronger performance.

MethodStage 1 BD-PSNR (dB) ↑Stage 2 BD-PSNR (dB) ↑Sender Params (M) ↓Receiver Params (M) ↓
Feedback-Guided Coding (Ours)0.00+0.7437.283.45
Residual Feature Coding-0.31-0.2235.494.46

Rendering-based feedback achieves a higher improvement from Stage 1 to Stage 2 compared with residual feature coding.

Together with the feedback-enhanced CodecSplat, these experiments support the effectiveness of rendering-based feedback in progressively improving 3D scene reconstruction, while distinguishing our contribution from conventional progressive feature refinement.

[1] Toderici et al., “Variable Rate Image Compression with Recurrent Neural Networks,” ICLR 2016.

Q2: Effectiveness of Two-Stage Coding with a Strong Backbone.

We agree that feedback should be evaluated on top of a strong single-stage model. We have added stage-wise rate--distortion comparisons for FeCoSplat-B/L, which share CodecSplat's pretrained feature extraction backbone (Appendix B.3, Figure A6). Both variants improve PSNR, SSIM, and LPIPS from Stage 1 to Stage 2 on both context and target views, with similar improvements under zero-shot generalization from eight to two input views (Figure A7). Moreover, the CodecSplat w/ Feedback mentioned above demonstrates that adding rendering-based feedback to a strong pretrained single-stage model yields further improvements.

Reviewer_6vSUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the problem of scene compression in 3DGS feed-forward reconstruction. It encodes intermediate deep features instead of input images or Gaussians. This allows it to store/transfer compact features that can be decoded into 3DGS with a specialized decoder model. The method has two stages: Stage 1: This stage turns multi-view features into a latent that updates an implicit state, which is decoded into intermediate Gaussians. Stage 2: The intermediate Gaussians are rendered back into the input view. A feedback extractor compares the render with the originals, and the resulting features are computed conditioned on the Stage-1 state. (Two feature sets are produced) The decoder entropy-decodes both latents, updates the state twice, and runs a Gaussian predictor to compute the decompressed reconstruction.

Contributions:

  1. A framework that avoids heavy receiver-side reconstruction (image compression) and heterogeneous Gaussian attributes (Gaussian compression) by utilizing compact intermediate features.
  2. A feedback-guided two-stage coding strategy
  3. Improved RD performance over existing feed-forward 3DGS compression, with high-quality novel views and a light receiver.
Critical strengths

Clear problem framing: The paper is well-motivated, and the sender-receiver asymmetry is a realistic argument for coding in the feature domain.

Feedback-free decoding: Rendering and comparison occur only at the encoder, so the decoder incurs no extra cost and requires no loop (Sec. 3.3). This seems like a very good novel design choice to me.

Extensive evaluation and experiments show improved RD on multiple backbones.

Critical weaknesses

The "consistently state-of-the-art" claim is stronger than the evidence.

  • Appendix B.2 shows ELIC (ReSplat-Base) beating FeCoSplat in PSNR and SSIM at mid and high rates on ACID and DL3DV, by roughly 1–2 dB on DL3DV. The main text instead describes image compression as rapidly degrading and dominated.
  • There are no BD-rate numbers, seeds, variance or per-scene statistics
  • Both the headline result and the efficiency story rest on this evidence. The efficiency edge over CodecSplat is modest (0.117 vs 0.138 s receiver), and FCGS has a smaller receiver (2.05M). A reader can't tell whether the RD gain is real or a metric- and rate-dependent artifact.
Questions for authors
  • Does Stage 2 help novel views, or mainly the input views it can see? Please report the gain separately on input and target views.
  • How were the baselines trained or adapted? Were ELIC, FCGS, and DepthSplat/ReSplat retrained or fine-tuned on compressed inputs, or used off the shelf? This determines whether the low-rate wins reflect the method or handicapped baselines. -Does the reported rate include camera parameters (B_cam)?
  • On what hardware were the latencies measured?
4 comments

W1: Rate--Distortion Performance and Quantitative Evidence.

We thank the reviewer for this observation. We acknowledge that ELIC$\rightarrow$ReSplat-B achieves higher PSNR and SSIM at certain operating points. We have therefore refined our claims to emphasize the rate--distortion--complexity trade-off and added PSNR- and LPIPS-based BD-Rate comparisons in Appendix B.1.

Table 1: Comparison of rate–distortion performance and model complexity. BD-Rate is computed relative to CodecSplat and negative values indicate better compression efficiency.

MethodSender Params (M) ↓Receiver Params (M) ↓RE10K PSNR BD-Rate (%) ↓RE10K LPIPS BD-Rate (%) ↓ACID PSNR BD-Rate (%) ↓ACID LPIPS BD-Rate (%) ↓DL3DV PSNR BD-Rate (%) ↓DL3DV LPIPS BD-Rate (%) ↓
ELIC→DepthSplat-B26.45144.81-1.76+53.51+28.11+137.43+232.54+248.17
ELIC→ReSplat-B26.45246.20-6.30+42.71-11.52+133.48-52.12+188.33
CodecSplat181.7324.900.000.000.000.000.000.00
FeCoSplat-S37.283.45+13.85-12.28+6.22-17.17+17.39+15.52
FeCoSplat-B126.113.45-4.69-18.46+2.47-22.62-22.36-14.87
FeCoSplat-L154.8023.60-26.13-35.74-15.23-37.91-49.03-44.43

Consistent with prior studies on rate--distortion--complexity (RDC) optimization [1, 2] and model scaling in learned image compression [3], our results reveal the trade-off between model capacity and compression performance. FeCoSplat-B achieves improved PSNR and LPIPS BD-Rate on RE10K and DL3DV with only 3.45M receiver parameters, while scaling to FeCoSplat-L yields further RD gains with comparable receiver complexity to CodecSplat (23.60M vs. 24.90M). These results highlight the scalability of our framework and its favorable RDC trade-off. The relatively lower PSNR and SSIM at higher bitrates may partly reflect differences in reconstruction capacity, as ELIC$\rightarrow$ReSplat-B employs a larger receiver (246.20M parameters).

Moreover, FeCoSplat demonstrates favorable zero-shot rate--distortion performance compared with CodecSplat across different input-view counts, spatial resolutions, and datasets (Section 4.4 and Appendix B.7), further supporting its generalization capability under unseen input configurations.

[1] D. Minnen and N. Johnston. Advancing the Rate-Distortion-Computation Frontier for Neural Image Compression. IEEE International Conference on Image Processing (ICIP), 2023.

[2] Y. Gao et al. Exploring the Rate-Distortion-Complexity Optimization in Neural Image Compression. Journal of Visual Communication and Image Representation, 2024.

[3] Y. Li et al. Scaling Learned Image Compression Models up to 1 Billion. arXiv preprint arXiv:2508.09075, 2025.

W2: BD-Rate Evaluation and Detailed Numerical Results.

We have added PSNR- and LPIPS-based BD-Rate comparisons in Table 1 and Table A2 (Appendix B.1), together with detailed numerical rate--distortion results at individual operating points in Table A7 (Appendix B.6). Per-scene metrics are additionally provided for representative examples in our qualitative comparisons in Appendix D.

W3&Q4: Receiver-Side Complexity, Decoding Efficiency, and Evaluation Hardware.

We acknowledge that the original GPU decoding latency improvement over CodecSplat is relatively modest and that FCGS has fewer receiver parameters. It is also worth noting that Gaussian-compression-based pipelines operate at higher bitrates and incur longer decoding latency, despite having relatively few receiver-side parameters. Compared with CodecSplat, FeCoSplat-B reduces receiver parameters by 86.1% (3.45M vs. 24.90M), while achieving comparable RD performance and stronger zero-shot generalization (Appendix B.7, Figure A9). It also requires fewer receiver parameters than image-compression-based pipelines such as ELIC$\rightarrow$DepthSplat-B (144.81M) and ELIC$\rightarrow$ReSplat-B (246.20M).

To further investigate decoding efficiency, we provide a detailed latency breakdown and comparisons across different hardware configurations in Appendix B.4. Under the original CompressAI backend on an RTX 4090, FeCoSplat-B achieves a total decoding latency of 0.1165 s, compared with 0.1359 s for CodecSplat. Specifically, FeCoSplat-B requires 0.0227 s for network inference and 0.0938 s for entropy decoding, whereas CodecSplat requires 0.0568 s and 0.0791 s, respectively. Entropy decoding accounts for 80.5% of FeCoSplat-B's total decoding latency partly due to the computational overhead of the four-step autoregressive quadtree context model, thereby limiting the overall latency reduction achieved by its lightweight network architecture. When considering the subsequent network inference alone, CodecSplat requires 2.50$\times$ the inference time of FeCoSplat-B.

We further evaluate both methods using the more efficient torch_ans implementation across hardware platforms with progressively decreasing computational capabilities, including the NVIDIA RTX 4090, GTX 1080 Ti, and AMD EPYC 7Y43 CPU. On the RTX 4090, FeCoSplat-B achieves 0.0628 s versus 0.0783 s for CodecSplat, representing a 19.8% reduction in total decoding latency. On the GTX 1080 Ti, FeCoSplat-B achieves 0.0964 s versus 0.1240 s, corresponding to a 22.3% reduction in total decoding latency. Under CPU-only execution, FeCoSplat-B requires 1.638 s versus 2.391 s, achieving a 31.5% reduction in total decoding latency. These results consistently demonstrate the efficiency of our lightweight receiver architecture.

Except for the latency breakdown reported above, all other latency measurements are conducted on the same hardware, equipped with an AMD EPYC 7Y43 48-Core CPU and NVIDIA RTX 4090 GPU, under the RE10K two-view setting at 256×256 resolution, as specified in Section 4.1. We have refined our efficiency claims to emphasize lightweight receiver architecture and competitive decoding latency, without claiming the smallest receiver or fastest decoding across all methods.

Q1: Effectiveness of Stage 2 on Novel Views.

We have added stage-wise rate--distortion comparisons on both context and target views of DL3DV (Appendix B.3, Figure A6). Stage 2 consistently improves PSNR, SSIM, and LPIPS on both sets of views, and additional two-view zero-shot experiments (Appendix B.3, Figure A7) further confirm these improvements and demonstrate the generalization to reduced input-view configurations.

Q2: Training and Adaptation of Baselines.

Following CodecSplat's evaluation protocol, all image- and Gaussian-compression baselines use off-the-shelf pretrained models without additional fine-tuning, as detailed in Appendix A.5. While task-specific fine-tuning could improve their RD performance, it would not directly address the computationally expensive Gaussian reconstruction in image-compression-based pipelines or the Gaussian attribute transmission overhead in Gaussian-compression-based pipelines.

Q3: Inclusion of Camera Parameters in Bitrate.

Yes. All reported bitrates include the camera parameter bitstream BcamB_{\mathrm{cam}} and necessary metadata. As detailed in Appendix A.4, we adopt a more compact camera coding scheme than the original CodecSplat implementation by exploiting structural redundancy in camera matrices and applying lossless zlib compression. This reduces the average camera overhead on RE10K from approximately 1.2 KiB to 0.17 KiB. For fair comparison, the same scheme is applied to CodecSplat and all image-compression baselines, and their RD results are re-evaluated accordingly.

Thanks for the rebuttal and the revision. The clarifications answer my questions, and the claims are now more balanced. Remaining concerns:

  • The results are still single runs with no variance, even though many of the differences are small.
  • ELIC->ReSplat-B, which beats FeCoSplat on DL3DV in PSNR, appears only in the appendix and should be in the main comparison.
  • The clear RD gains come from FeCoSplat-L, not from the lightweight-receiver variant. The text should separate these claims.
  • Feedback adds little in-distribution (+0.08 / +0.19 dB on CodecSplat) and costs extra decoding latency.

I keep my rating of Weak Accept.

Thank you for your thoughtful follow-up and constructive suggestions. We appreciate your positive assessment and will carefully consider the remaining concerns in our further revisions to improve the clarity and rigor of our work.

Reviewer_FysMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents FeCoSplat, a two-stage feedback-guided feature compression framework for feed-forward 3D Gaussian Splatting (3DGS). The method first compresses geometry-aware multi-view features to obtain an intermediate Gaussian representation, whose rendered views are then used to guide a second-stage compression of complementary feedback features. A shared feature codec and an implicit Gaussian state integrate information across the two stages, enabling lightweight receiver-side reconstruction without intermediate Gaussian rendering. Experiments on RealEstate10K and DL3DV, together with cross-dataset evaluation on ACID, show favorable rate–distortion performance, particularly at low bitrates, compared with image-, Gaussian-, and feature-based compression baselines. Ablation studies investigate the contributions of the proposed components, while complexity analysis demonstrates a lightweight receiver with only 3.45M parameters.

Critical strengths
  1. The paper introduces a two-stage feature compression framework that leverages intermediate Gaussian renderings to guide the encoding of feedback-derived features. The ablation results support the benefit of rendering-based feedback for improving reconstruction fidelity, providing evidence for the effectiveness of this design.

  2. By restricting intermediate Gaussian reconstruction and feedback extraction to the encoder, the method avoids the need for intermediate rendering at the receiver. Compared with CodecSplat, FeCoSplat substantially reduces receiver-side model size while achieving slightly lower decoding latency, making it potentially attractive for resource-constrained applications.

  3. The paper evaluates rate–distortion performance on RealEstate10K and DL3DV, with additional cross-dataset evaluation on ACID, demonstrating favorable perceptual quality at low bitrates. The comparisons against different compression paradigms provide further evidence for the effectiveness.

Critical weaknesses
  1. The proposed method builds on existing feature compression techniques, with its main contribution being the incorporation of rendering-based feedback into a two-stage coding framework. Although the paper discusses its differences from iterative reconstruction methods, comparisons with simpler two-stage or residual-feature coding alternatives would help better establish the novelty.

  2. The comparison with CodecSplat involves differences in feature extraction backbones and Gaussian prediction architectures, making it difficult to isolate the gains attributable to the proposed compression strategy. Although the authors provide additional experiments using CodecSplat's feature extractor, the subsequent coding and reconstruction architectures remain different, limiting the conclusiveness of the comparison.

  3. Despite the substantial reduction in receiver-side parameters, the decoding latency improvement over CodecSplat is relatively modest (0.117s vs. 0.138s). The absence of evaluations on actual resource-constrained devices further limits the evidence supporting the practical deployment benefits of the lightweight decoder.

Questions for authors

See Critical Weaknesses

4 comments

W1: Comparisons with simpler two-stage or residual-feature coding alternatives.

We thank the reviewer for this suggestion. To evaluate rendering-based feedback against simpler alternatives, we introduce a residual feature coding baseline that encodes the residual between the original and reconstructed multi-view features in Appendix B.2.1, and compare it with the proposed feedback-guided coding.

Table 1: Comparison of rendering-based feedback and residual feature coding. Due to the noticeable differences in PSNR between the two stages, we report BD-PSNR instead of BD-Rate, which measures the average PSNR gain at equivalent bitrates. BD-PSNR is measured relative to Stage 1 of feedback-guided coding, with higher value indicating a stronger performance.

MethodStage 1 BD-PSNR (dB) ↑Stage 2 BD-PSNR (dB) ↑Sender Params (M) ↓Receiver Params (M) ↓
Feedback-Guided Coding (Ours)0.00+0.7437.283.45
Residual Feature Coding-0.31-0.2235.494.46

On RE10K, the Stage 2 result with rendering-based feedback achieves a higher BD-PSNR than residual feature coding. This demonstrates the effectiveness of using intermediate 3D Gaussian renderings to extract previously uncaptured features for novel-view synthesis. We have also expanded Appendix C.2 to discuss prior progressive and conditional coding methods, highlighting our distinct contribution of rendering-based feedback for 3D scene reconstruction.

W2: Controlled Comparison with CodecSplat.

We agree that architectural differences may affect rate--distortion performance. To isolate the contribution of feedback, we introduce CodecSplat w/ Feedback in Appendix B.2.2, which freezes the original pretrained CodecSplat components and trains only the newly added feedback modules for 50K iterations.

Table 2: Effectiveness of rendering-based feedback on a frozen pretrained CodecSplat baseline. Due to the noticeable differences in PSNR between the two methods, we report BD-PSNR instead of BD-Rate, which measures the average PSNR gain at equivalent bitrates. BD-PSNR improvements are measured relative to the original CodecSplat, with higher value indicating a stronger performance.

MethodRE10K BD-PSNR ↑DL3DV BD-PSNR ↑DL3DV (2 ctx) BD-PSNR ↑DL3DV (256×256) BD-PSNR ↑Sender Params (M) ↓Receiver Params (M) ↓Sender Latency (s) ↓Receiver Latency (s) ↓
CodecSplat0.000.000.000.00181.7324.900.1890.138
CodecSplat w/ Feedback+0.08+0.19+4.02+1.01183.9827.180.2730.195

Rendering-based feedback improves BD-PSNR by +0.08 dB on RE10K and +0.19 dB on DL3DV, with larger gains under zero-shot settings (+4.02 dB for view-count generalization and +1.01 dB for resolution generalization). These results demonstrate that feedback can improve an existing strong feature compression framework, providing more direct evidence of its effectiveness beyond architectural differences.

W3: Decoding Efficiency and Claims Regarding Resource-Constrained Devices.

We thank the reviewer for this observation. We provide a detailed breakdown of decoding latency to further clarify the efficiency gains of FeCoSplat-B. Under the original CompressAI implementation on an RTX 4090, FeCoSplat-B achieves a total decoding latency of 0.1165 s, compared with 0.1359 s for CodecSplat. Specifically, FeCoSplat-B requires 0.0227 s for network inference and 0.0938 s for entropy decoding, whereas CodecSplat requires 0.0568 s and 0.0791 s, respectively. Notably, entropy decoding accounts for 80.5% of FeCoSplat-B's total decoding latency, partly due to the computational overhead of the four-step autoregressive quadtree context model, which limits the overall latency reduction achieved by our lightweight network architecture. When considering the subsequent network inference stage, FeCoSplat-B achieves a 2.50× speedup over CodecSplat.

We further evaluate both methods using the more efficient torch_ans implementation across hardware platforms with progressively decreasing computational capabilities, including the NVIDIA RTX 4090, GTX 1080 Ti, and AMD EPYC 7Y43 CPU. On the RTX 4090, FeCoSplat-B reduces total decoding latency to 0.0628 s, compared with 0.0783 s for CodecSplat, representing a 19.8% reduction in total decoding latency. On the GTX 1080 Ti, FeCoSplat-B achieves 0.0964 s versus 0.1240 s, corresponding to a 22.3% reduction in total decoding latency. Furthermore, under CPU-only inference on the AMD EPYC 7Y43, FeCoSplat-B achieves 1.6379 s, compared with 2.3913 s for CodecSplat, reducing total latency by 31.5%. These results suggest that the reduced network complexity of FeCoSplat-B can offer greater practical benefits, particularly when entropy decoding overhead is reduced. Detailed component-wise latency breakdowns and cross-hardware comparisons are provided in Appendix B.4.

We also acknowledge that these experiments use server-class hardware and do not directly validate deployment on resource-constrained devices. Accordingly, we have refined our claims to emphasize lightweight receiver architecture and competitive decoding latency.

The authors have addressed most of my concerns through additional controlled comparisons, ablation studies, and decoding efficiency analyses, which substantially strengthen the experimental evidence. Although some concerns regarding methodological novelty remain, I appreciate the improvements and have raised my rating to Weak Accept.

Thanks for your response. We are glad that our clarifications have addressed most of your concerns, and we greatly appreciate your constructive feedback, which have helped us improve the quality and clarity of our paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their constructive feedback. We are encouraged that the reviewers recognize the strengths of our work, including "the novel design of the rendering-guided feedback and feedback-free decoding", "improved performance over various baselines", and "lightweight receiver for practical deployment".

According to the suggestions provided, we have uploaded a revised manuscript, with all additions and modifications highlighted in blue. To comprehensively address the reviewers' comments and provide a more complete presentation of our work, we have expanded both the main text and the appendix with additional experiments, analyses, and discussions. All section, figure, and table references in our responses refer to the revised manuscript. The key revisions are summarized as follows:

  1. Multi-scale Model Evaluation (R1-FySM, R3-Ts4E). We evaluate three FeCoSplat variants with different model capacities to enable comparisons across image-, Gaussian-, and feature-compression paradigms under comparable model complexity, from a rate–distortion–complexity (RDC) perspective:
    • FeCoSplat-S: Corresponds to FeCoSplat in the original submission, adopting a lightweight DepthSplat-based feature extractor with only 3.45M receiver parameters.
    • FeCoSplat-B: Corresponds to FeCoSplat* in the original submission, adopting a frozen pretrained CodecSplat feature extractor with newly trained compression and reconstruction modules, while retaining the same receiver architecture and parameter count (3.45M) as FeCoSplat-S.
    • FeCoSplat-L: Adopts the same multi-view feature extractor as FeCoSplat-B, with scaled compression and reconstruction modules, resulting in 23.60M receiver parameters, comparable to CodecSplat (24.90M), while requiring fewer sender parameters (154.80M vs. 181.73M).
  2. Additional Experimental Validation (R1-FySM, R2-6vSU, R3-Ts4E). We extend our evaluations to demonstrate the robustness and zero-shot generalization of FeCoSplat across different input-view counts, spatial resolutions, and datasets in Section 4.4 and Appendix B.7. We also introduce controlled ablations to validate the effectiveness of rendering-based feedback, including comparisons with residual feature coding and feedback-enhanced CodecSplat in Appendix B.2.
  3. Experimental Details (R2-6vSU). We have expanded the experimental details in the revised manuscript, including dataset configurations, training protocols, baseline implementations, camera parameter coding, and evaluation settings. We also clarify the pretrained checkpoints, model training strategies, and hardware configurations to improve transparency and reproducibility. Further details are provided in Appendix A.
  4. Decoding Efficiency Analysis (R1-FySM, R2-6vSU). We provide a detailed decoding latency breakdown, including optimized entropy decoding and CPU-only evaluations, to better characterize the relationship between receiver complexity and actual decoding efficiency in Appendix B.4.
  5. Expanded Discussion (R3-Ts4E). We clarify the distinction between our rendering-based feedback mechanism and conventional progressive coding methods in Appendix C.2.
  6. Additional Analysis and Corrections. In the original submission, we adopted diff-gaussian-rasterization as the Gaussian rasterization backend. To ensure a fair comparison, we have switched to gsplat consistent with CodecSplat and updated main experimental results accordingly. We further observe that the choice of rasterization backend may affect the rate--distortion performance, and provide an analysis in Appendix B.5. Additionally, we correct the receiver-side parameter count of CodecSplat to 24.9M.

We hope that our responses and the revisions to the manuscript adequately address the reviewers' concerns and provide a clearer understanding of our contributions. We appreciate the reviewers' time and consideration and welcome any further questions or feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.