acceptodds
Under review as a conference paper at ICLR 2027

RLSplat: Resource-Linear Budget Control for Feed-forward 3D Gaussian Splatting

15 views

Abstract

Feed-forward 3D Gaussian Splatting (FF3DGS) reconstructs 3D scenes from images in a single forward pass, making it well-suited to applications that must operate within limited resource budgets. To further improve its practicality, recent budget-controllable FF3DGS methods allow the number of Gaussians to be adjusted at inference, so that the output meets a given storage limit or rendering-latency constraint. However, these methods overlook the cost of inference itself: to produce a representation with the target number of Gaussians, they first predict a dense set of Gaussians and then prune or select from it, requiring nearly constant latency and peak memory regardless of the budget. We therefore propose RLSplat, a resource-linear budget-controllable FF3DGS framework whose representation size, inference latency, and peak memory all decrease linearly as the Gaussian budget is reduced. Instead of pruning or selection, RLSplat learns a Gaussian allocation policy that distributes the budget across local scene regions, so that exactly the requested number of Gaussians is generated. These Gaussians are then refined through resource-linear Gaussian decoding, in which each Gaussian attends only to nearby Gaussians found via an efficient hierarchical neighbor search, so that the decoding cost decreases linearly with fewer Gaussians. In zero-shot evaluation, RLSplat outperforms prior budget-controllable methods at every budget, surpassing the strongest baseline even with fewer Gaussians, while running up to faster with less peak memory at low budgets.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Reconstruction, Feed-Forward 3DGS

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_DbVAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

RLSplat makes feed-forward 3DGS budget-controllable in inference cost, not just output size: prior methods predict a dense candidate set and prune/select, so latency and memory stay nearly constant. It learns a Gaussian allocation policy that distributes a requested number of Gaussians across anchors, and decodes them with a resource-linear decoder using local cross-attention over a small neighborhood. On zero-shot transfer it achieves the best quality at every budget, near-flat quality as the budget shrinks, and beats budget-fixed methods with far fewer Gaussians, with component ablations.

Critical strengths
  1. Correctly targets resource-linear inference, which is the right problem for deployment-oriented FF3DGS, with an elegant allocation formulation.
  2. Consistent quality gains at every budget, near-flat quality under budget reduction, and thorough component ablations.
Critical weaknesses
  1. Evaluation breadth is limited: a single zero-shot benchmark at low resolution, and in-domain measurements are missing.
  2. A few reproducibility/clarity issues remain: a prior baseline appears under the same name as the proposed method, naming is inconsistent across the paper, and Eq. (3) appears degenerate as printed.
Questions for authors
  1. In-domain quantitative results would be a welcome addition to complement the zero-shot evaluation.
  2. Could the authors quantify the policy-training overhead relative to base training?
2 comments
AuthorsOpenReview ↗

Response to Reviewer DbVA

We thank the reviewer for recognizing resource-linear inference as the right problem for deployment-oriented FF3DGS and for the constructive feedback on evaluation breadth and clarity. We add in-domain RE10K (Table 2) and higher-resolution Mip-NeRF 360 (Appendix I) evaluations, report the policy-training overhead, and revise the manuscript for clarity, including consistent naming and an explanation of Eq. (3). We address each point below.

[W1, Q1] Additional in-domain and higher-resolution experiments

Following the reviewer's suggestion, we have added quantitative evaluations on RE10K (in-domain) and Mip-NeRF 360 (higher-resolution, cross-dataset).

Table A. In-domain evaluation on RE10K (16 input views). Budgets in Gaussians per view (1K = 1,024); best per budget in bold. F⁴Splat† is a low-budget checkpoint we retrained (five multi-scale feature maps instead of three). Full results in Table 2 of the revised manuscript.

Budget/viewMethodLPIPS ↓SSIM ↑PSNR ↑
4KEcoSplat0.1860.81224.44
4KF⁴Splat†0.1380.86025.81
4KRLSplat0.1200.89026.78
2KEcoSplat0.2200.78223.29
2KF⁴Splat†0.1570.84125.34
2KRLSplat0.1260.88626.59
1KEcoSplat0.3000.70420.56
1KF⁴Splat†0.1860.81224.64
1KRLSplat0.1430.87326.13

Table B. Mip-NeRF 360 (16 input views). #GS denotes the total Gaussian count per scene. Best results are bold.

Method#GS ↓LPIPS ↓SSIM ↑PSNR ↑
DepthSplat (pose-required) + ZPressor672K0.4250.46318.29
AdaptiveSplat647K0.5600.38317.79
F⁴Splat64K0.5340.39918.07
RLSplat64K0.3980.49820.87

(1) In-domain evaluation on RE10K. We evaluate all methods with 8, 16, and 24 input views at budgets of 4K, 2K, and 1K Gaussians per view. RLSplat outperforms both baselines on all three metrics across all budgets and input-view counts, and its PSNR advantage over F⁴Splat† grows as the budget tightens, from 0.97 dB at 4K/view to 1.49 dB at 1K/view.

(2) Higher-resolution evaluation on Mip-NeRF 360. We evaluate at 448 × 448 input resolution (vs. 256 × 256 in the original experiments) against baselines that support higher-resolution inference. RLSplat achieves the best results on all three metrics, improving PSNR by 2.80 dB over F⁴Splat at the same Gaussian count and by 2.58 dB over the strongest baseline, DepthSplat + ZPressor, with about 10× fewer Gaussians. The full results and evaluation protocol are provided in Appendix I of the revised manuscript.

[Q2] Policy-training overhead compared to Base training

Table C. Training cost of base training and GSPO policy training (rollout group size C = 5).

StageTraining time (hours)Peak VRAM (GiB)
Base training (40K steps)10h 21m93.9
Policy training (5K steps)1h 58m106.6

GSPO policy training adds 1h 58m (19.0% of the base training time) and increases peak VRAM by 13.5%. Although each step renders C = 5 candidate allocations for the reward, the rollouts are forward-only and 3DGS rasterization is fast and lightweight, so both overheads remain far below 5×. This overhead is incurred once during training and does not affect inference latency or memory.

[W2] A prior baseline appearing under the same name as the proposed method, inconsistent naming, and Eq. (3) appearing degenerate

Baseline sharing the method's name. GSPO refers to Group Sequence Policy Optimization (Zheng et al., 2025), which we adopt as the optimizer for our allocation policy; it is not itself our contribution. Labels such as "GSPO (Ours)" in the original manuscript could suggest otherwise. In the revision, RLSplat denotes our method and GSPO refers only to the optimization algorithm, and all tables and figures follow this convention.

Inconsistent naming. We have standardized method names throughout the manuscript (EcoSplat, F⁴Splat, NoPoSplat, SPFSplat, AnySplat, and RLSplat) and verified that all row labels, legends, and figure annotations match.

Regarding Eq. (3). The ratio in Eq. (3) equals one in value but not in gradient. Each rollout group is sampled from the current policy and used for exactly one update, so the denominator is the current probability with gradients stopped, sg⁡(pn)\operatorname{sg}(p_n). The gradient therefore flows only through the numerator, which yields a length-normalized policy gradient weighted by the advantage in Eq. (4), following GSPO-token (Zheng et al., 2025, Sec. 4.3). We have added this explanation right after Eq. (3).

We have addressed each point based on our best understanding of the comment. If the reviewer could point us to the specific locations, we will correct them immediately.

Thank you for the thoughtful response. All of my concerns are addressed. Therefore, I'll raise my score.

Reviewer_Cu6GOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RLSplat proposes a method to allocate budget-aware 3D Gaussians with linear scaling of compute resources according to budget. It uses GSPO based policy optimization which allocates Gaussians in each anchor based on its complexity. Results on the ACID dataset show state-of-the-art results.

Critical strengths
  • [S1] Using GSPO to allocate Gaussians is novel
  • [S2] Latency and complexity increase linearly as the budget increases to allocate Gaussians
  • [S3] State-of-the-art results on the ACID dataset for different numbers of Gaussians.
  • [S4] Analysis that GSPO ultimately allocates Gaussians based on texture is very interesting.
Critical weaknesses
  • [W1] Limited Evaluation: The major issue is that the evaluation is only done on the ACID dataset; the model was trained on RE10K. Further, the authors do not report quantitative results for RE10K but show qualitative results on it.
  • [W2] Missing Baseline methods: The authors do not compare with AdaptiveSplat [R1], which uses texture-based allocation of 3D Gaussians based on budget. ZPressor [R2] is another relevant baseline that compresses the features before predicting Gaussians to be compared against. Finally, the authors omit a comparison with recent query token-based methods such as TokenGS[3], C3G[4], and GlobalSplat[5].
  • [W3]: The authors do not give any details about the TokenGen module discussed in L241.
  • [W4]: The first scene is discarded for each budget but should be included in the latency calculation since it is a necessary step to predict Gaussians. (L311)
  • [W5] Limited discussion on results: It is unclear why GSPO-based learning is allocating Gaussians based on texture complexity in the scene even though texture was never explicitly used. Other methods like AnySplat allocate based on voxel size, EcoSplat based on frequency and AdaptiveSplat based on textureness because they are explictly optimized for those but it would be interesting to see why GSPO eventually converged to texture based allocation

References:

  • [R1] Singhal, Badrinath, et al. "AdaptiveSplat: Texture Aware Controllable 3D Gaussian Allocation for Feed-Forward Reconstruction." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.
  • [R2] Wang, Weijie, et al. "Zpressor: Bottleneck-aware compression for scalable feed-forward 3dgs." Advances in Neural Information Processing Systems 38 (2026): 113407-113436.
  • [R3] Ren, Jiawei, et al. "TokenGS: Decoupling 3d gaussian prediction from pixels with learnable tokens." arXiv preprint arXiv:2604.15239 (2026).
  • [R4] An, Honggyu, et al. "C3G: Learning compact 3d representations with 2k Gaussians." arXiv preprint arXiv:2512.04021 (2025).
  • [R5] Itkin, Roni, et al. "GlobalSplat: Efficient feed-forward 3D Gaussian splatting via global scene tokens." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.
Questions for authors
  • [Q1] Eqn 3, log(pnsg(pn))log (\frac{p_n}{sg(p_n)}). This ratio will always be 1, making the log always 0. Is this a typo?
  • [Q2] C3G can give a good 3D representation with 2K Gaussians, which is very compact. Can RLSplat also generate a good 3D representation with just 2K Gaussian primitives?
  • [Q3] How does the results change when mininum possible number of Gassians are allocated? Is it comparable to the lowest number of Gaussians of Ecosplat, F4Splat and AdaptiveSplat[R1] for matched Gaussians?
  • [Q4] Additional results on different Gaussian counts (especially when using the lowest possible number of Gaussians and the highest possible number of Gaussians) would be a valuable addition to show the lower and upper bounds of quality that can be achieved with RLSplat
6 comments
AuthorsOpenReview ↗

Response to Reviewer Cu6G - 1

We thank the reviewer for the constructive review and the helpful pointers to related work, and appreciate that the reviewer finds the policy-based allocation novel and the emergent texture-aware behavior interesting. Following the suggestions, we compare against all five suggested methods (AdaptiveSplat, DepthSplat + ZPressor, TokenGS, C3G, and GlobalSplat; Appendices H and I), and add RE10K results (Table 2) and results at both ends of the budget range. We also clarify TokenGen, the latency protocol, and Eq. (3). We address each point below.

[W2, Q2] Additional baselines and compact reconstruction with 2K Gaussians

Table A. Comparison with recent query-based methods on RE10K. All methods use two context views and a total budget of 2,048 Gaussians. RLSplat* is fine-tuned with the budget fixed to 2,048 during policy training. Best results are bold.

Method#GSLPIPS ↓SSIM ↑PSNR ↑
C3G2,0480.27460.685421.54
GlobalSplat2,0480.31180.690221.12
RLSplat2,0480.26150.754622.56
RLSplat*2,0480.23830.772523.15

Table B. Higher-resolution evaluation on Mip-NeRF 360 (4 views). Best results are bold.

Method#GS ↓LPIPS ↓SSIM ↑PSNR ↑
TokenGS (pose-required)256K0.5740.46518.65
AdaptiveSplat162K0.6350.30015.20
F⁴Splat16K0.5720.38217.00
RLSplat16K0.4910.45319.03

Table C. Higher-resolution evaluation on Mip-NeRF 360 (16 views). Best results are bold.

Method#GS ↓LPIPS ↓SSIM ↑PSNR ↑
DepthSplat (pose-required) + ZPressor672K0.4250.46318.29
AdaptiveSplat647K0.5600.38317.79
F⁴Splat64K0.5340.39918.07
RLSplat64K0.3980.49820.87

(1) Comparison with query-based methods at 2K Gaussians. Since query-based methods are trained for a fixed budget, we also include RLSplat*, which is fine-tuned with the budget fixed to 2,048 during the policy training stage. Even without this budget-specific fine-tuning, RLSplat outperforms both C3G and GlobalSplat on all three metrics. With it, RLSplat* further improves PSNR by 0.59 dB, yielding gains of 1.61 dB and 2.03 dB over C3G and GlobalSplat, respectively. Unlike these methods, a single RLSplat model also serves every other budget without retraining. These results show that RLSplat remains effective under a tight 2K Gaussian budget.

(2) Higher-resolution comparisons with additional baselines. We further evaluate on Mip-NeRF 360 at 448 × 448 input resolution with 4 and 16 context views, including AdaptiveSplat, TokenGS, and DepthSplat + ZPressor. With 4 views, RLSplat achieves the highest PSNR and lowest LPIPS, while TokenGS achieves slightly higher SSIM using 16× more Gaussians and ground-truth poses. With 16 views, RLSplat achieves the best results on all three metrics, outperforming the strongest baseline, DepthSplat + ZPressor, by 2.58 dB PSNR while using about 10× fewer Gaussians.

The full results and evaluation protocols are provided in Appendix H (query-based methods) and Appendix I (Mip-NeRF 360) of the revised manuscript.

[W1] Additional in-domain evaluation on RE10K

Table D. In-domain evaluation on RE10K (16 input views). Budgets in Gaussians per view (1K = 1,024); best per budget in bold. F⁴Splat† is a low-budget checkpoint we retrained (five multi-scale feature maps instead of three). Full results in Table 2 of the revised manuscript.

Budget/viewMethodLPIPS ↓SSIM ↑PSNR ↑
4KEcoSplat0.1860.81224.44
4KF⁴Splat†0.1380.86025.81
4KRLSplat0.1200.89026.78
2KEcoSplat0.2200.78223.29
2KF⁴Splat†0.1570.84125.34
2KRLSplat0.1260.88626.59
1KEcoSplat0.3000.70420.56
1KF⁴Splat†0.1860.81224.64
1KRLSplat0.1430.87326.13

Following the reviewer's suggestion, we have added quantitative evaluations on the RE10K test set, complementing the original zero-shot evaluation on ACID. We evaluate all methods with 8, 16, and 24 input views at matched budgets of 4K, 2K, and 1K Gaussians per view. RLSplat outperforms both baselines on all three metrics across all evaluated budgets and input-view counts. With 16 input views, its PSNR advantage over F⁴Splat† increases from 0.97 dB at 4K/view to 1.25 dB at 2K/view and 1.49 dB at 1K/view. These results show that RLSplat's advantage also holds on the in-domain dataset and grows as the Gaussian budget becomes tighter.

AuthorsOpenReview ↗

Response to Reviewer Cu6G - 2

[Q3] Comparison with budget-controllable baselines at the lowest possible budget

Table E. Lowest-budget comparison on RE10K (2 views). Each method is evaluated at the minimum budget within its official training range.

Method#GS ↓LPIPS ↓SSIM ↑PSNR ↑
AdaptiveSplat82,3640.43370.616219.49
EcoSplat8,1920.31000.697220.27
F⁴Splat8,1920.22930.754522.81
RLSplat2,0480.26150.754622.56
RLSplat8,1920.19840.794623.62

Following the reviewer's question, we report each method at the minimum budget within its official training range, and also report RLSplat at the matched budget of 8,192. At the matched budget, RLSplat outperforms all baselines on all three metrics, improving PSNR over F⁴Splat by 0.81 dB. At its own minimum budget of 2,048, 4× fewer than the baselines' minimum, RLSplat matches F⁴Splat's SSIM and stays within 0.25 dB in PSNR, while outperforming EcoSplat and AdaptiveSplat (which uses ~40× more Gaussians) on all three metrics.

[Q4] Quality across the full training budget range

Table F. Quality–budget trade-off on ACID (8 views).

#GaussiansLPIPS ↓SSIM ↑PSNR ↑
2,0480.3680.66423.31
8,1920.2540.73424.82
32,7680.2110.75625.24
131,0720.2090.75725.26

Following the reviewer's suggestion, we evaluate RLSplat across its full training budget range (2K–128K Gaussians) with eight input views. PSNR improves from 23.31 dB at 2K to 25.24 dB at 32K, while increasing the budget to 128K adds only 0.02 dB, indicating that quality saturates around 32K Gaussians. The full quality–budget curves are provided in Appendix J (Fig. 9 and Table 11).

[W3] Clarification of the TokenGen module

TokenGen is detailed in Appendix D, but we agree that its operation should also be summarized in the main text. We have added this summary to Sec. 3.3 in the revised manuscript.

For an anchor ana_n allocated knk_n Gaussians, TokenGen creates knk_n tokens indexed by normalized slot positions ρn,r=(r+1/2)/kn\rho_{n,r}=(r+1/2)/k_n, which tile the unit interval regardless of knk_n. Each slot is encoded with a mip-NeRF-style integrated positional encoding of (ρn,r,kn)(\rho_{n,r},k_n), which averages Fourier features over the slot's bin of width 1/kn1/k_n to avoid aliasing between nearby counts. The encoding, passed through an MLP and modulated by FiLM parameters predicted from ana_n, is added to the anchor feature, so tokens from the same anchor share its context but remain distinct. All tokens are packed into a single sequence of exactly BB tokens without padding.

[W4] Clarification of inference latency measurement (L311)

Our wording at L311 was misleading. The first scene is not discarded: for each budget, we run one unmeasured warm-up pass on it to absorb one-time GPU initialization costs (e.g., CUDA context creation and kernel autotuning), and then measure latency over all test scenes, including the first. We have clarified this in the revised manuscript.

[W5] Why does GSPO learn texture-aware allocation?

Our policy does not use any explicit texture or edge supervision. However, since our allocation policy is trained via GSPO to produce allocations with lower reconstruction loss, it implicitly assigns more Gaussians to textured regions. The revised Fig. 5 compares our allocation with an edge-based alternative, a simple rule-based method that allocates Gaussians according to the Sobel gradient magnitude. This comparison shows two resulting behaviors. Within each view, both concentrate Gaussians on complex structures. However, when the same structure appears in two views, the edge-based baseline allocates heavily to it in both, whereas our policy allocates heavily to it in only one view and spends the saved budget elsewhere.

These observations indicate that texture-aware allocation emerges from optimizing rendering quality rather than from a hand-designed texture heuristic. We have clarified this interpretation in the "Allocation policy" paragraph of Sec. 4.2.

[Q1] Clarification of Eq. (3)

Eq. (3) is intentional, not a typo. As in standard on-policy GRPO/GSPO training, rollouts are sampled from the current policy. We perform a single policy update per rollout group, so the importance ratio evaluates to one in the forward pass. However, this does not imply a zero gradient: the denominator is detached, allowing gradients to flow through the numerator. This stop-gradient construction is also used in GSPO-token [1] (Zheng et al., 2025, Sec. 4.3). Thus, Eq. (3) provides a valid policy-gradient signal despite its forward value being one. We have clarified this point after Eq. (3) in the revised manuscript.

[1] Chujie Zheng et al. Group sequence policy optimization. arXiv preprint arXiv:2507.18071, 2025.

Thanks to the authors for the response.

I really appreciate the authors' efforts to compare against the missing baselines from the paper. These results show that the proposed method outperforms recent baselines.

My unresolved concerns are as follows:

  • [W1] The authors presented in-domain results on RE10K. However, I request that the authors also include a comparison on the standard DL3DV dataset to improve the manuscript. Many methods, such as AnySplat, DepthSplat and AdaptiveSplat use this dataset in their experimental setup.
  • [W2] TokenGS, AdaptiveSplat, DepthSplat and F4Splat evaluation on Mip-NeRF 360 seems to be on out-of-distribution data. Is it the same for RLSplat as well? This clarification is important for a fair comparison.
  • [W5] I agree that per-view edge-based supervision will be suboptimal, as shown in Fig. 5. However, the good performance of the proposed method on high-texture regions remains unclear. Prior methods allocate budget to high-texture regions through explicit analysis: AdaptiveSplat used a wavelet-based texture score and EcoSplat used frequency cues. How does RLSplat achieve better performance using the standard reconstruction loss?
Reviewer_3pT8OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Argues that budget-controllable feed-forward 3DGS today controls only the size of the output, not its cost: EcoSplat and F4Splat both derive their B Gaussians from a dense candidate set, so latency and memory stay nearly flat in B. RLSplat allocates instead. A policy predicts per-anchor probabilities and draws an allocation summing to exactly B, trained with GSPO using negative reconstruction loss as reward; at inference sampling is replaced by floor-plus-largest-remainder. A decoder spawns exactly kn tokens per anchor and refines them by cross-attention over K neighbours found hierarchically, kept valid by Voronoi and offset losses. Trained on RealEstate10K, tested zero-shot on ACID.

Critical strengths
  • The diagnosis is correct and the resource story is measured. 4.6× less memory and 3.3× faster than F4Splat at 32K with 0.7 dB more PSNR. Budget robustness is real too: 12K → 4K per view costs EcoSplat 0.60 dB and F4Splat 0.63 dB, against at most 0.04 dB here.
  • The decoder is well built and honestly ablated. Neighbour sets come from detached initial positions and are reused across layers; the hierarchical search is 27× faster than exact search and, with the Voronoi loss, retrieves identical neighbours for 99.0% of Gaussians versus 60.2% without it. Removing the offset loss costs 0.53 dB, both regularizers 2.51 dB, local refinement 0.96 dB, and training collapses (5.80 dB) without the projection loss.
Critical weaknesses
  • The reinforcement learning does not earn its place. Table 3: base 24.18, uniform split 24.53, Sobel-gradient heuristic 24.76, GSPO policy 24.83 dB. The full pipeline beats a one-line edge detector by 0.07 dB, with no seed variance, after a dedicated 5,000-step policy stage — while the projection loss alone is load-bearing to the point of training failure. The missing contrast is a supervised allocator sampling straight-through from the same probabilities with the same loss, which would settle whether policy gradient is needed at all.
  • Quality is insensitive to the thing being optimised. At 24 views PSNR is 25.91 / 25.95 / 25.94 at 12K / 8K / 4K per view; at 8 views, 25.28 / 25.28 / 25.24. Tripling the budget changes nothing and once hurts. Framed as robustness, it equally means the allocation is not binding: whatever the policy decides, quality is set elsewhere.
  • Evaluation is one dataset wide. RealEstate10K training, ACID zero-shot, 256 × 256 coastal footage: no indoor test, no higher resolution. Figure 5 also shows the policy nearly emptying flat walls, floor and water; that may be right at 1K Gaussians per view, but the number of anchors receiving zero is never reported. Cost covers Gaussian generation only.
Questions for authors
  1. Report seed variance for Table 3 and compare against a differentiable allocator trained with the same loss.
  2. What fraction of anchors receive zero Gaussians, and does that hold indoors?
  3. Why is quality flat between 4K and 12K per view, and where is the knee?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 3pT8 - 1

We thank the reviewer for the careful and quantitative review, and are glad that the reviewer finds our diagnosis correct and the decoder well designed and thoroughly ablated. To address whether the learned allocation is necessary and binding, we add seed variance and a differentiable-allocator baseline (Table 5), extend the budget sweep down to 1K Gaussians per view (Table 1), and add RE10K, Mip-NeRF 360 (Appendix I), and zero-allocation results (Appendix G). We address each point below.

[W1, Q1] Significance of performance gain in Table 3 and comparison with differentiable allocator

Table A. Comparison of allocation strategies. All variants are fine-tuned from the same base checkpoint for 5,000 steps using 4 GPUs (mean ± std over three seeds).

MethodLPIPS ↓SSIM ↑PSNR ↑
Uniform0.2629 ± 0.00090.7229 ± 0.000324.625 ± 0.012
Edge-based0.2635 ± 0.00030.7277 ± 0.000124.795 ± 0.013
Differentiable allocator0.2575 ± 0.00090.7266 ± 0.000724.777 ± 0.016
Ours0.2503 ± 0.00010.7321 ± 0.000524.866 ± 0.012

Following the reviewer’s request, we report the mean and standard deviation over three random seeds and additionally compare ours with a differentiable allocation method. The differentiable allocator modulates the Gaussian opacities with their allocation probabilities, so the reconstruction loss directly updates the allocation policy. As shown in Table A, our method outperforms all alternatives on all three metrics with low variance, indicating that the gain is significant. The gain is clearest in LPIPS: ours reduces LPIPS by 5.0% relative to Edge-based, a difference over 40 times the seed standard deviation, whereas Edge-based does not improve LPIPS over Uniform at all. Our PSNR gain over Edge-based (0.071 dB) is also more than five times the standard deviation. The differentiable alternative also improves over Uniform, but it underperforms our GSPO-based policy on all three metrics. We have added these results to Table 5 and the “Allocation policy” paragraph of Sec. 4.2 in the revised manuscript.

[W2, Q3] Negligible degradation vs. marginal improvement (scaling) between 4K and 12K in Table 1?

Thank you for this perspective. As the reviewer notes, RLSplat's quality changes little between 4K and 12K Gaussians per view. We interpret this as RLSplat already reaching near-saturated quality at 4K. Table 1 covered only this range because the official F⁴Splat weights cannot generate fewer than 4K Gaussians per view (L314–316 of the original manuscript). To locate the knee, we extend the evaluation down to 1K Gaussians per view, retraining a low-budget F⁴Splat† checkpoint (five multi-scale feature maps instead of three).

Table B. Budget-controllable methods on ACID (zero-shot from RE10K) with 16 input views. Budgets are specified in Gaussians per input view. Best results within each budget are bold. The full extended results are provided in Table 1 of the revised manuscript.

Budget/viewMethodLPIPS ↓SSIM ↑PSNR ↑
1KEcoSplat0.4120.57520.03
1KF⁴Splat†0.2760.69024.21
1KRLSplat0.2260.76025.38
2KEcoSplat0.3240.65422.63
2KF⁴Splat†0.2420.72024.72
2KRLSplat0.2050.77325.63
4KEcoSplat0.2820.68923.86
4KF⁴Splat†0.2180.74325.09
4KRLSplat0.1960.77825.75

In this range, quality clearly depends on the budget. RLSplat's PSNR increases by 0.25 dB from 1K to 2K and 0.12 dB from 2K to 4K, compared with only 0.04 dB across 4K–12K, placing the knee at around 2K–4K Gaussians per view. When reducing the budget from 4K to 1K, F⁴Splat†'s PSNR drops by 0.88 dB, whereas RLSplat's drops by only 0.37 dB. The allocation itself is also binding: at the same budget, the learned allocation outperforms both uniform and edge-based allocation beyond seed variance (Table A).

AuthorsOpenReview ↗

Response to Reviewer 3pT8 - 2

[W3-1] Broader evaluations

As requested, we evaluate our method in two additional settings: RE10K (in-domain, mostly indoor scenes) and Mip-NeRF 360 (higher-resolution, cross-dataset).

Table C. In-domain evaluation on RE10K with 16 input views. Budgets are specified in Gaussians per input view. Best results within each budget are bold.

Budget/viewMethodLPIPS ↓SSIM ↑PSNR ↑
4KEcoSplat0.1860.81224.44
4KF⁴Splat†0.1380.86025.81
4KRLSplat0.1200.89026.78
2KEcoSplat0.2200.78223.29
2KF⁴Splat†0.1570.84125.34
2KRLSplat0.1260.88626.59
1KEcoSplat0.3000.70420.56
1KF⁴Splat†0.1860.81224.64
1KRLSplat0.1430.87326.13

Table D. Mip-NeRF 360 (16 input views). #GS denotes the total Gaussian count per scene. Best results are bold.

Method#GS ↓LPIPS ↓SSIM ↑PSNR ↑
DepthSplat (pose-required) + ZPressor672K0.4250.46318.29
AdaptiveSplat647K0.5600.38317.79
F⁴Splat64K0.5340.39918.07
RLSplat64K0.3980.49820.87

(1) In-domain evaluation on RE10K. We evaluate all methods with 8, 16, and 24 input views at budgets of 4K, 2K, and 1K Gaussians per view. Table C presents the results for 16 input views; the complete results are provided in Table 2 of the revised manuscript. RLSplat outperforms both baselines on all three metrics at every evaluated budget, and its PSNR gain over F⁴Splat† grows as the budget tightens, from 0.97 dB at 4K/view to 1.25 dB at 2K/view and 1.49 dB at 1K/view.

(2) Higher-resolution evaluation on Mip-NeRF 360. We additionally evaluate at 448 × 448 resolution, compared with 256 × 256 in the original experiments, and include additional baselines supporting higher-resolution inference. RLSplat achieves the best results on all three metrics. It improves PSNR by 2.80 dB over F⁴Splat at the same Gaussian count, and by 2.58 dB over DepthSplat + ZPressor while using about 10× fewer Gaussians.

These evaluations extend the evidence beyond ACID, showing strong performance both on the training-domain dataset and on a different dataset at higher resolution. The full results and evaluation protocols are provided in Table 2 and Appendix I of the revised manuscript.

[W3-2, Q2] What fraction of anchors receive zero Gaussians in Figure 5?

We report the fraction of anchors allocated zero Gaussians for the indoor and outdoor examples shown in Fig. 5, across different budgets.

Table E. Zero-allocation anchor fractions for the two scenes in Fig. 5.

Budget/viewIndoor sceneOutdoor scene
1K49.81%42.82%
2K27.43%17.52%
4K1.54%0.12%
8K0%0%
16K0%0%

At the budget used in Fig. 5 (1K Gaussians per view), 49.81% of anchors in the indoor example and 42.82% in the outdoor example receive zero Gaussians. As the budget increases, these fractions decrease and reach zero at 8K/view in both examples. We have added these statistics and corresponding visualizations to Appendix G of the revised manuscript.

[W3-3] Cost covers Gaussian generation only

We apologize for the unclear wording at L309–L310 of the original manuscript: "we report the latency and peak VRAM of Gaussian generation, excluding data loading, pose alignment, and rendering." Here, "Gaussian generation" means each method's complete pass from the input images to its final set of 3D Gaussians, i.e., the model's entire inference time. Data loading, pose alignment, and rendering vary with each method's implementation and are unrelated to the performance of the model itself; in particular, pose alignment is performed only at test time, to align the target camera poses.

We have clarified this part in the revised manuscript to avoid any misunderstanding. If the reviewer intended a different point, we would appreciate a clarification so that we can address it directly.

Thanks for your reply, I will raise my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.