- Rating
- 3(+1)
- Confidence
- 2(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
The paper investigates the GPU memory bottleneck during high-resolution 3D Gaussian Splatting (3DGS) training (3K–5K+). It observes that while full-view aggregation is mathematically necessary for parameter updates, intermediate rasterization caches do not need to remain concurrently resident across the entire image. The authors propose BoundedGS, an execution pipeline that decouples rasterization and projection backward passes. By partitioning the image into bounded sub-regions (leaves), BoundedGS releases regional forward and backward rasterization states immediately after accumulating gradients on projected Gaussian attributes. To prevent secondary peaks, it introduces candidate-indexed projection (frustum-level screening), compact parameter gradient accumulation, and CPU-assisted structure rebuilding with pinned prefetch queues. Evaluated on Mip-NeRF 360 and Tanks and Temples, the paper reports an 82.3% reduction in peak rendering working set (from 9.97 GiB to 1.76 GiB at 5K) while maintaining reconstruction fidelity and comparable training time.
Critical strengths
S1: Principled mathematical decoupling and engineering execution. Decomposing the full-frame rasterization backward pass from the projection backward pass is mathematically grounded in the spatial locality of tile-based rasterization. By accumulating intermediate gradients on projected attributes across sequential leaf regions via the chain rule, the method strictly preserves complete-view rasterization semantics without lossy approximations. The underlying GPU systems engineering—including dynamic leaf splitting/merging based on measured load, candidate ID mapping, and direct gradient accumulation—is coherent, well-formulated, and addresses an authentic hardware bottleneck in high-resolution volumetric rendering.
S2: Thorough memory lifetime profiling and component diagnostics. The paper provides detailed diagnostic traces (Figure 3 and Section 4.3) analyzing the overlap and lifetime of intermediate buffers (rasterization caches, projection states, and parameter gradients). Table 2 systematically validates how each component progressively shaves off memory spikes, demonstrating conclusively that the active rasterization working set can be capped to an invariant bound regardless of observation resolution.
Critical weaknesses
W1: Complete absence of direct comparisons against specialized high-resolution baselines. The introduction explicitly highlights InfiniGS (Liu et al., 2026) as prior art tackling memory pressures in high-resolution 3DGS via image blocking and asynchronous optimization. However, in Table 1, the method is evaluated solely against vanilla 3DGS (Original). There is zero empirical comparison—in terms of peak VRAM, wall-clock time, or rendering quality—against InfiniGS or other memory-efficient 3DGS variants. Without benchmarking against direct competitors targeting the same design space, it is impossible to substantiate whether BoundedGS offers a superior Pareto frontier or distinct system advantages over existing blocking paradigms.
W2: Restricted memory accounting and lack of end-to-end VRAM closure. The paper heavily advertises an 82.3% memory reduction (from 9.97 GiB to 1.76 GiB at 5K), but Section 4.1 explicitly defines this metric as the "temporary rendering working set," excluding model parameters, initialized Adam optimizer states, and persistent density statistics. For scenes with 5M–6M Gaussians, model weights and Adam momentums alone occupy multiple gigabytes of VRAM. Furthermore, by offloading structure updates and image decoding to host RAM via pinned memory transfers, part of the memory pressure is shifted to the host system. The paper fails to report the actual physical peak GPU memory allocation (torch.cuda.max_memory_allocated), total host RAM consumption, or transfer bottlenecks across different PCIe bandwidth configurations.
W3: Confounded speedup attribution and missing Gaussian counts. Table 1 indicates that BoundedGS slightly reduces training time at 5K (from 36.7 to 33.9 minutes) despite the overhead of multi-kernel leaf dispatching and synchronization. This speedup appears to stem largely from candidate screening pruning invisible Gaussians prior to expensive spherical harmonics evaluation, rather than the bounded rasterization itself. The experimental evaluation does not decouple the runtime overhead of leaf tiling from the computational savings of candidate culling. Additionally, Table 1 omits the final Gaussian count per scene, leaving open the possibility that slight variations in densification dynamics between the runs contributed to the timing discrepancy.
W4: Substantial presentation gap with no methodological pipeline diagram. The method relies entirely on dense, text-heavy descriptions across Sections 3.1–3.4 and Appendix B to articulate complex multi-stage mechanisms (leaf trees, candidate ID mapping, compact gradients, pinned image queues, asynchronous host streams). The complete absence of an architectural overview or pipeline data-flow figure significantly hurts clarity and makes the system execution unnecessarily difficult to comprehend and reproduce.
Questions for authors
Q1: How does BoundedGS compare quantitatively against InfiniGS (or other tile/block-based memory-efficient 3DGS methods) on native 5K outdoor scenes in terms of final PSNR/SSIM, total training time, and peak memory?
Q2: What is the true end-to-end peak GPU memory allocation (torch.cuda.max_memory_allocated) during full 30K training for both Original and BoundedGS in Table 1? What is the peak Host RAM usage resulting from the CPU-assisted layout reconstruction and prefetch queues?
Q3: Could you provide a runtime ablation isolating the wall-clock time spent on: (a) candidate frustum screening, (b) leaf tree scheduling and rasterization dispatch, and (c) host-to-device transfers and structure updates? What are the exact final Gaussian counts for both methods across the scenes in Table 1?
Q4: How sensitive is training throughput to PCIe bandwidth constraints (e.g., PCIe Gen4 x16 vs. Gen4 x8 or Gen3 x16), given the continuous pinned image streaming and full-model parameter re-uploads during densification steps?
Thank you for separating the effects of fixed tile scheduling, adaptive leaves, and the gradient implementation.
Fixed tiles and early release (Table 2 and Appendix C.2). In the five-scene component study, stage separation and early release reduce the temporary working set from 9.980 to 7.638 GiB before adding Auto Leaf. We also ran a direct leaf-layout control on Bicycle, Flowers, and Stump at 16K. The layouts share the same projection, compact gradients, immediate leaf-state release, and one update after all pixels in the view. Fixed 1,024-pixel leaves achieve peak working sets within 0.003 GiB of Auto Leaf. At this memory level, Auto Leaf shortens the synchronized warm pass by 15.4%, 14.5%, and 14.1%, respectively. Fixed 2,048-pixel leaves run slightly faster than Auto Leaf and increase the working set to 1.747–1.862 GiB, compared with 0.982–1.173 GiB for Auto Leaf.
Source of the first timing change (Appendix A.4, Table 5). The original transition from 12.19 to 11.35 s changed both the state lifetime and the projection-gradient implementation. We added a whole-image control that uses the same raw-parameter projection backward kernel as stage separation. In that matched-kernel test, releasing rasterization state before projection backward lowers the working set from 9.62 to 7.64 GiB, and warm-pass time is 11.16 versus 11.17 s. The whole-image control reproduces the timing improvement when the backward kernel changes. We corrected the speed attribution and identify lower peak memory as the independently measured effect of stage separation.
Execution rule (Section 3.2 and Figure 1). Equation (4) now gives the four terms of the leaf estimate in the main text. They cover intersections, pixel and tile state, backward checkpoints, and output buffers. When a region exceeds the estimate budget, the scheduler bisects it along the dimension with more tiles, splitting horizontally on a tie. It stops when the estimate fits or the leaf is a single 16 × 16 tile. A single tile is processed even if its estimate exceeds the budget. Figure 1 shows how the leaf gradients are accumulated before the complete-view update.
Relation to prior systems and new comparisons (Section 2, Table 2, and Appendices E and G). gsplat includes packed projection and sparse gradients. The tested full-view path still materializes the view's intersection and pixel state together. CLM-GS uses view filtering and host offloading to make room for larger models. InfiniGS combines image blocking with asynchronous optimization. BoundedGS processes image regions sequentially at fixed model parameters, releases each region's temporary state, and applies one update per view. Table 2 reports separate increments for adaptive splitting, indexed projection, and compact gradients, along with same-checkpoint comparisons to upstream 3DGS, DISTWAR, LiteGS, gsplat, and Turbo-GS. Table 9 reports the two 30K Turbo-GS training comparisons. In the separate CLM-GS control, offloading reduces GPU allocation peaks to 6.752 and 6.407 GiB on Bicycle and Stump, below the same-PLY BoundedGS references. This adds about 8.7 GiB of warm-pass host RSS and 19.4–19.6% wall time within CLM-GS (Table 14). The two systems use different update schedules.