- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes ReCap-GS, a 3D Gaussian Splatting framework that maintains a strict Gaussian budget throughout training. It identifies stale primitives through opacity-rank changes after a perturbation-and-recovery phase and recycles their slots for demand-driven refinement.
Critical strengths
- The distinction between strict training budgets and final representation size is clear and practically useful.
- Stale selection is tested against probe-only, random, and post-recovery opacity controls.
- Broad budget sweeps and a common evaluation pipeline support the reconstruction-quality comparisons.
Critical weaknesses
- The large VRAM savings are confounded by CPU image staging; a matched data-staging comparison is missing.
- Stale recovery adds only 0.07 dB in the 300k ablation. Its contribution at the main 50k–200k budgets is not isolated.
- Evaluation covers only seven scenes from one dataset and two seeds, leaving broader generalization unclear.
- Sensitivity to the recovery duration, opacity threshold, and reclamation fraction is not evaluated.
- There is a lack of an overview of the proposed method, making it hard to understand the techniques.
Questions for authors
How much VRAM improvement remains if Improved-GS uses the same CPU image-staging strategy?
2 comments
Response to Reviewer XxA3 (2/2): weaknesses
We thank the reviewer for highlighting the need for broader evaluation and a clearer method overview. We address W3–W5 below; cross-cutting corrections are documented in our public update.
W3. Generalization across seeds and datasets
We extended the main Mip-NeRF360 benchmark from two to three seeds. At 50k/100k/200k, ReCap-GS improves mean PSNR over Improved-GS by +0.41/+0.36/+0.24 dB, winning 20/21, 19/21, and 15/21 scene–seed pairs. The gains diminish at higher budgets (absolute mean differences below 0.06 dB at 500k–2M; Table 1; Appendix C.1).
We also evaluated three seeds across seven budgets (50k–2M) on Tanks&Temples (truck, train) and Deep Blending (drjohnson, playroom), using the same evaluation pipeline:
| ΔPSNR vs. Improved-GS | 50k | 100k | 200k |
|---|---|---|---|
| Tanks&Temples | +0.47 | +0.59 | +0.46 |
| Deep Blending | +0.36 | +0.14 | −0.17 |
All six Tanks&Temples scene–seed pairs favor ReCap-GS at 50k, 100k, and 200k; the gains on Deep Blending are smaller and reverse at 200k. The four-scene mean advantage is +0.41/+0.37/+0.15 dB at 50k/100k/200k, but −0.16 dB at 2M. Thus, the low-budget advantage extends beyond Mip-NeRF360 on these scenes, without establishing universal superiority. Recovery-policy controls were also expanded to all seven Mip-NeRF360 benchmark scenes and these four scenes (Table 6). The flowers/treehill study is ongoing; results will be shared when complete.
W4. Sensitivity to recovery hyperparameters
We performed a one-factor-at-a-time study at 100k on seven Mip-NeRF360 scenes (seed 1; Appendix D.2). Reference: , recovery updates, , cycles at 3k/6k, topology freeze at 25k.
| Variant | ΔPSNR | ΔSSIM | ΔLPIPS |
|---|---|---|---|
| τ=.02 | −.003 | +.0001 | +.0003 |
| τ=.10 | +.007 | +.0001 | +.0004 |
| U=100 | −.049 | −.0010 | +.0012 |
| U=1000 | +.024 | −.0012 | +.0014 |
| ρ=.10 | −.033 | −.0019 | +.0016 |
| ρ=.30 | +.002 | +.0003 | +.0004 |
| One cycle (3k) | −.009 | −.0020 | +.0016 |
| Four cycles (3k/6k/9k/12k) | −.091 | −.0007 | +.0017 |
| Late cycles (12k/24k) | −.558 | −.0128 | +.0126 |
| No recovery | −.089 | −.0055 | +.0038 |
Changes relative to reference; lower LPIPS is better.
Moderate variations in τ, U and ρ shift mean PSNR by −.049 to +.024 dB, without a consistent improvement across metrics. Schedule matters more: one cycle nearly matches PSNR but worsens SSIM/LPIPS; four cycles reduce PSNR; moving both cycles to 12k/24k is worse than removing recovery. The second late cycle finishes at 24.3k, leaving only 700 iterations before topology freeze, although we have not isolated the cause. This study uses one seed and budget, so it does not establish universal hyperparameter robustness.
W5. Method overview and presentation
We agree that the original explanation did not sufficiently connect the components. The expanded Sec. 3 overview (with Figure 1) and Appendix A (including Algorithm 1) now document the training sequence. In brief:
- Allocate under a strict budget. Starting from a deterministically capped SfM initialization, early residual-consensus insertion (until 1.8k) adds Gaussians where several views agree on high residual error; absolute-gradient clone/split refinement then continues until topology freeze. A stability-adaptive ceiling paces growth toward ; once the population is near , budget-preserving slot exchanges reuse low-support slots instead of exceeding .
- Identify stale capacity. At 3k and 6k, all opacities are clamped to at most , without changing geometry or appearance, and optimization continues for views. For primitives that persist through this window, the stale score is the drop in normalized opacity rank from before to after recovery.
- Reclaim and reallocate. The fraction of eligible primitives with the largest rank drops is removed; the freed slots return to the capacity pool and are consumed by subsequent gradient-driven refinement within the same budget. Structural updates use the opacity correction of Rota Bulò et al.; the absolute-gradient criterion follows AbsGS, and late MU-mean is adopted from Improved-GS.
The contribution is the integration of residual-consensus allocation, budget-preserving slot exchanges, and stale-capacity recovery under a training-time population constraint—not a new gradient criterion, opacity correction, or faster renderer. The expanded ablations also clarify that stale recovery's most consistent incremental benefits appear in SSIM/LPIPS, rather than accounting for most of the PSNR gain (Table 5). We hope this overview makes the roles and novelty of the components easier to assess.
Response to Reviewer XxA3 (1/2): questions
We thank the reviewer for the careful assessment. We have added matched-staging VRAM measurements, three-seed component ablations across the main budgets, and evaluations of recovery sensitivity and additional datasets. Cross-cutting methodological corrections, including those affecting the original ablation tables, are described in our public manuscript-update comment. We address the VRAM question and stale recovery here; the remaining weaknesses are addressed in our second comment.
W1 and Question. Peak VRAM with matched image staging
The reviewer is correct that our original VRAM reduction claim was confounded by image staging. We repeated peak-VRAM measurements on seven Mip-NeRF360 scenes (seed 1) with the same GPU- or CPU-resident image strategy for both methods (Table 4):
Mean peak VRAM in MiB over seven scenes.
With CPU-resident images, no systematic VRAM advantage remains: ReCap-GS uses 9%/0%/7% more memory on average at 50k/100k/200k; per-scene differences range from −24% to +31%. With GPU-resident images, ReCap-GS uses 9–13% less peak VRAM on average (less on every scene), consistent with Improved-GS retaining per-image edge maps in GPU memory. This is an implementation difference, not a demonstrated saving from stale recovery.
We have withdrawn the original 65–76% VRAM-reduction claim and replaced it with matched-staging comparisons (Sec. 4.3). A strict Gaussian-population budget constrains the number of Gaussian parameters and their optimizer states, not total VRAM, which also depends on image staging and other implementation details.
W2. Contribution of stale recovery at low budgets
We expanded the component ablation to 50k, 100k, 200k, and 300k (seven Mip-NeRF360 scenes × three seeds; revised Table 5), with a consistent budget-growth schedule. The correction to the original cross-campaign scheduling discrepancy is documented in our public update; it does not affect the main benchmark.
The incremental contribution of stale recovery is measured by Full − (backbone + MU):
Mean differences over 21 scene–seed pairs, calculated before rounding. All ablation variants use the adopted opacity correction of Rota Bulò et al.
The reviewer's observation is confirmed: stale recovery adds about 0.06 dB PSNR at every budget, including 50k–200k. At 300k, backbone+stale exceeds the backbone by 0.069 dB, matching the originally reported 0.07 dB. Stale recovery's gains are more consistent in SSIM (17–21/21 pairs) and LPIPS (15–21/21 pairs). At 50k, stale recovery and late MU-mean contribute similar PSNR improvements (+0.056 vs. +0.052 dB); above 50k, MU-mean contributes more PSNR. The full method achieves the best mean PSNR, SSIM, and LPIPS among the ablated configurations at all four budgets, but its advantage should not be attributed primarily to stale recovery.
We have sharpened this interpretation in Sec. 4.4: ReCap-GS combines strict-budget capacity allocation and stale-capacity recovery, with the latter providing complementary gains most consistently reflected by SSIM/LPIPS. The full component-ablation results are also presented in our response to BSXZ (W2).