acceptodds
Under review as a conference paper at ICLR 2027

Bake It Till You Make It: Ultrafast Spatial Texture-Atlas Splatting

Abstract

Gaussian Splatting (GS) with neural radiance representations achieves high-fidelity color detail, but the per-frame network evaluations impose a heavy rendering cost. This is an obstacle for real-time and resource-constrained applications such as VR, gaming, and mobile rendering, where sustained high frame rates are essential. We present a learned sparse scene representation built on 2D primitives together with a novel baking method that adaptively converts the neural component into a single %global, hardware-accelerated texture atlas, eliminating network inference at runtime. Our 2D surfels carry low-frequency geometry and view-dependent appearance, while view-independent, per-primitive high-frequency texture is encoded with a spatial hash grid and distilled into the texture atlas. A sparsity objective that penalizes per-primitive falloffs yields a substantially more compact representation than prior methods. Our approach renders roughly to faster than 3DGS on common benchmarks, and over an order of magnitude faster on individual scenes, while surpassing the reconstruction quality and speed of the fastest sparsification methods (FastGS and Speedy-Splat). A quality-optimized variant matches the perceptual quality of the strongest baseline while still rendering several times faster. Replacing software with hardware rasterization pushes frame rates beyond FPS with no loss in quality, making high-quality neural scene rendering practical on commodity hardware.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Neural Rendering, Vision, Graphics, Texturing, Splatting

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NcgUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents an ultrafast textured surfel rendering framework that combines neural appearance learning with explicit texture atlas baking. It further introduces a Beta-kernel falloff regularizer and an optimized ray-surfel intersection formulation to reduce primitive count and rendering overhead. Extensive experiments demonstrate impressive rendering speed and competitive visual quality.

Critical strengths

The paper presents a well-engineered rendering framework with substantial practical improvements in rendering efficiency and perceptual quality. The experimental evaluation is comprehensive, covering numerous baselines, detailed ablations, hardware-matched rendering comparisons, and quality–speed trade-offs. The implementation details are also well documented.

Critical weaknesses
  1. Limited novelty of the individual components. The proposed framework largely builds upon existing techniques, including NeST-Splatting, Beta-Splatting, FastGS, neural texture baking, and conventional perspective-correct interpolation. Although their integration is effective, the fundamental methodological novelty remains somewhat limited.

  2. Insufficient isolation of the contributions relative to the closest prior work. Since the implementation builds upon NeST-Splatting, a direct comparison incorporating the same baking and rendering optimizations into that baseline would better establish the contributions of the proposed representation and regularization.

  3. Memory and rendering-quality trade-offs. The substantial frame-rate improvements come with considerably larger texture storage than competing sparse methods. Furthermore, the default renderer disables mipmapping, raising concerns about texture aliasing and temporal stability under challenging viewing conditions.

Questions for authors
  1. Could the authors provide a controlled comparison against NeST-Splatting equipped with the same texture baking, atlas finetuning, and rendering optimizations?
  2. How does the proposed method compare with FastGS and Speedy-Splat under matched rendering quality and GPU memory budgets?
  3. Could the authors evaluate rendering performance and temporal stability across different GPU classes, image resolutions, and camera-motion conditions, particularly with and without mipmapping?
1 comment
AuthorsOpenReview ↗

Reply to Reviewer NcgU

We thank the reviewer for the specific requests. Each of the three questions asks for a measurement, and the revised PDF provides them in the new Appendix M, in blue.

W1, novelty. We agree that the method combines known components, and the paper credits each of them. The contribution we claim is the pipeline that turns a neural appearance model into a static per-primitive texture at no loss: the view-independence constraint on the residual, the bake, and the finetune in which the atlas exceeds its network (Table 3; Table 13 shows that training the network longer does not). To our knowledge, this has not been done for splatting primitives before. Each remaining component is measured in isolation: the kernel, the prior, and the closed-form intersection in Table 2, the density criterion in Table 16, and the whole chain in Table 22.

W2 / Q1, NeST-Splatting with our baking and renderer. We would gladly have run this comparison, but NeST-Splatting decodes appearance per pixel in screen space from the blended features and the view direction (their Eq. 5), so it has no static texel values to bake; making the appearance bakeable is the change our method introduces. The network is also the largest cost of its renderer, so the rendering optimizations alone would not move its frame rate considerably. The closest controlled comparison is our own neural model before baking, which is NeST-style rendering of the same surfels: 111 to 155 FPS against 1479 to 2392 FPS baked, at 0.24 to 0.47 dB lower quality than the finetuned atlas (Tables 3 and 22).

W3 / Q2, storage and matched quality. Table 23 compares at equal total bytes on four scenes: with the atlas sized to the released FastGS model we keep the LPIPS and SSIM lead within 0.12 dB PSNR, at 1.4 to 1.9 times its hardware frame rate. Matching quality exactly across architectures is not possible, so we report configurations slightly above and slightly below FastGS instead. At PSNR parity on the benchmark means we run at 1.2 to 1.8 times FastGS's frame rate, and no FastGS configuration reaches our LPIPS. A sparser variant with 27 to 39 percent fewer primitives is 0.12 to 0.32 dB below our paper model and still ahead of FastGS in LPIPS (Table 23).

Q3, GPU classes, resolutions, motion, mipmapping. Table 24 reports HD (1920 wide) and 4K (3840 wide) on the RTX 5090 on both rasterizers. On the hardware rasterizer we are the fastest of the three methods for every scene at both resolutions, and our margin over FastGS and Speedy-Splat is larger at 4K than at HD. On the software path our fragment is more expensive and we fall behind at 4K, which we now state. Table 25 adds an Apple M3 Pro laptop at three resolutions, where we are fastest or tied at the evaluation resolution on all four scenes.

For temporal stability under camera motion, the demo linked below can be orbited with and without the mip chain, and the split-screen videos of the supplementary material show the paper's renderer in motion against FastGS and Nexels. Appendix E.4 explains why mipmapping is off by default. At the evaluation resolution one pixel spans about one texel, so the atlas is barely minified. Sampling one mip level finer than the pixel footprint gives the same quality as no mipmapping; sampling at the footprint's own level costs 0.32 dB, because the texels were finetuned at the full atlas resolution (mip level 0). An anonymous demo at https://bitymi.github.io/mip-demos/ renders the paper's models with a six-level atlas chain and a toggle between anisotropic filtering and plain full-resolution sampling; the per-pixel mip modes cost 18 to 22 percent of the frame on the laptop and nothing on the 5090. Anisotropic filtering removes the texture's share of the aliasing; the primitives' own aliasing is shared by all splatting methods and would call for Mip-Splatting-style filtering, which we consider outside the scope of this work.

Reviewer_7QhuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a 2D surfel-based novel view synthesis (NVS) method that aims ultrafaster rendering. The main idea is to decouple scene appearance into per-surfel view-dependent color and per-texel view-independent color on each surfel. Together with Beta-kernel falloff regularization, the decoupled scene representation allows representing the scene with sparser surfels while keeping high-frequency detail via per-surfel textures. During training, a multi-resolution 3D hash grid and lightweight MLP capture high-frequency spatial texture residuals. These residuals are then baked into a 2D texture atlas, allowing runtime neural queries to be discarded entirely. The texture atlas is then finetuned to recover baking losses and eliminate hash-grid collisions. Quantitative and qualitative evaluations show that the proposed method outperforms other fast 3DGS methods in terms of rendering speed and primitive count, while maintaining comparable PSNR/SSIM and superior perceptual quality.

Critical strengths
  • Decoupling view-dependent directional color from view-independent spatial texture on surfel primitives is conceptually clean and highly effective. Using a 3D hash grid during training avoids the memory explosion of early per-primitive texture allocation, and baking it into a 2D atlas with texture finetuning removes runtime neural evaluation entirely, which is technically sound.
  • Quantitative evaluations show that the proposed method achieves an ultrafast 2,698–4,413 mean FPS with hardware rasterization, while achieving the best LPIPS score among speed-optimized methods (>1,000 FPS) across all 13 benchmark scenes using far fewer primitives than dense 3DGS.
  • The experimental evaluation and ablation studies are very thoroughly conducted. Every compared baseline is retrained with official code under a unified metric and single-GPU timing protocol. Furthermore, the ablations systematically validate nearly every design choice, including Beta-kernel falloff regularization, closed-form ray-surfel intersection, atlas sizing/quantization, signed vs. positive-only residuals, and controlled checks showing that atlas finetuning outperforms simply training the neural teacher longer.
Critical weaknesses
  • The overall framework is technically sound, but the individual algorithmic components are relatively incremental. The pipeline mostly combines existing components: 2DGS surfels, Beta kernels, spherical-Voronoi color, NeST-Splatting's hash grid, FastGS densification and Speedy-Splat rasterization. Baking neural appearance into textures for rasterization is established, but closely related baking work (MobileNeRF, BakedSDF) is not discussed.
  • The proposed representation has structural limitations on view-dependent textures. Because the baked texture atlas is strictly view-independent, sub-surfel view-dependent variations, e.g., sharp, moving specular highlights, cannot be captured by the atlas. Such high-frequency specularities force the optimization to densify surfels locally, reducing primitive sparsity in highly reflective areas.
  • Evaluations on non-solid / volumetric media are missing. The evaluation focuses exclusively on solid surfaces. Because the proposed method flattens kernels into hard-edged disks to prune overdraw, it remains unclear how well the representation handles semi-transparent, volumetric media such as dense fog, smoke, or complex layered glass structures.
Questions for authors
  • All timings use a flagship RTX 5090. Could the authors report FPS and memory usage on lower-tier/integrated GPUs and mobile devices, compared against FastGS and Compressed 3DGS, where large atlases may become cache- or bandwidth-bound?
  • Because view-dependent color is constant per surfel and the atlas is view-independent, how does the method behave, both in visual quality and local primitive density, on strongly reflective scenes with sharp moving highlights? Does it critically affect the proposed method’s efficiency?
  • Since the falloff regularizer flattens kernels into hard-edged disks, how does the representation handle volumetric or complex semi-transparent media, e.g., fog, smoke, or layered glass, without introducing disk-edge artifacts?
1 comment
AuthorsOpenReview ↗

Reply to Reviewer 7Qhu

We thank the reviewer for the clear summary and the three questions. We have uploaded a revised PDF; the additions are in blue, with the new measurements in Appendix M.

W1, incremental components; MobileNeRF and BakedSDF. We see our contribution in identifying and adapting components of prior work to suit high-speed rendering at high quality. We introduce the constraint that the texture residual be view-independent, which is what makes a hash-grid appearance model bakeable into a static atlas; the hash grid is a training-time scaffold only, unlike in NeST-Splatting, Nexels or Instant-NGP. The finetuned atlas exceeds the network it came from (Table 3; Table 13 shows that training the network longer does not). The Beta kernels serve a different purpose than in Beta-Splatting: we regularize them to reduce overdraw rather than adapt them to appearance. The revision adds MobileNeRF and BakedSDF to the related work; they bake appearance onto an extracted mesh, whereas we bake onto the splatting primitives and finetune the texels through the primitive renderer.

W2 / Q2, sharp moving highlights. By construction the atlas carries no view dependence below the surfel scale. The spherical Voronoi colour is not band-limited like SH, so a highlight moving across a surfel's footprint is representable at the surfel scale (+1.06 dB over SH on bonsai, Table 12). Finer variation falls to densification, as for any primitive with per-primitive colour, and extra surfels remain cheaper than a view-dependent texture lookup. In the worst case the method behaves like a per-surfel-colour method. None of the benchmark scenes has specularities sharper than the surfel scale; the glossy scenes (bonsai, kitchen, room) stay at 4 to 5 times fewer primitives than FastGS. We do not claim mirror-like scenes and state this limitation.

W3 / Q3, volumetric media. The falloff term is a prior, not a constraint: the shape stays trainable, the kernel ranges from a flat disc to a Gaussian-like profile, and a surfel keeps a soft tail where the images need one (Table 26). Flat discs are not opaque; the prior mainly discourages stacks of faint kernels that fit detail with high overdraw. Our evaluation targets the standard benchmarks with surface-like media. Fog, smoke or layered glass is a different setting, where an adaptive flatness prior or 3D kernels with a textured bisecting plane would be the better primitive; both are beyond this paper, and the next upload states this scope in the limitations.

Q1, lower-tier GPUs. Table 25 and Fig. 12 report the same bundles on an Apple M3 Pro laptop through our WebGPU viewer, with FastGS and Speedy-Splat through the same WebGPU sort and blend, at the evaluation resolution, HD (1920 wide) and 4K (3840 wide). At the evaluation resolution we are fastest or tied on all four scenes (1.1 to 1.5 times FastGS, 1.0 to 1.35 times Speedy-Splat). At 4K, we lead on two scenes and are 6 to 18 percent behind Speedy-Splat on truck and garden. There each surfel covers many pixels, and our per-fragment texture fetch costs more than a Gaussian evaluation. We did not observe any bandwidth limitations on the Macbook, suggesting that our method is not bandwidth limited by the textures at this scale. Download sizes are given with the table. We could not include Compressed 3DGS on the laptop in time; on the RTX 5090 its hardware viewer is 2.9 times behind ours (Table 20). As atlases grow, more informed texture compression is the natural next step.

Reviewer_STWDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a sparse textured-surfel representation for fast novel-view synthesis. During training, the method separates view-dependent appearance represented per surfel from view-independent residual appearance represented with a multi-resolution hash grid. The learned residual is then baked into a BC7-compressed 2D texture atlas and fine-tuned directly. The paper additionally introduces a Beta-kernel falloff regularizer, a closed-form rational ray-surfel intersection, and a hardware-rasterization implementation. The method is evaluated on Mip-NeRF 360, Tanks and Temples, and Deep Blending. The reported results show substantially higher frame rates than 3DGS and several speed-optimized or textured baselines, while maintaining competitive PSNR and improving LPIPS relative to sparse primitive-based methods.

Critical strengths
  • Impressive speed-quality trade-off. The method achieves very high frame rates while maintaining competitive perceptual quality. The combination of sparse geometry, explicit texture fetches, and the closed-form intersection is practically valuable for real-time rendering.

  • Good experimental and implementation detail. The paper evaluates multiple datasets and includes ablations for the kernel, falloff regularization, atlas baking, fine-tuning, texture resolution, density control, and hardware rasterization. The appendix also addresses quantization, temporal stability, and several implementation details that are often omitted in similar work.

Critical weaknesses
  • The comparison with the closest surfel methods is not fully fair. The paper includes 2DGS, NeST-Splatting, and Nexels, which is a positive aspect. However, the proposed method receives a custom hardware-rasterization implementation, while the closest surfel baselines are not all evaluated through the same backend. Appendix J ports FastGS, Speedy-Splat, and Compressed 3DGS, but does not provide an equivalent hardware-port comparison for NeST-Splatting and Nexels. Consequently, the reported speed gap over surfel-based methods mixes representation-level improvements with renderer and implementation differences.

  • The storage-efficiency claim should be qualified. The method substantially reduces the number of primitives, but the texture atlas can require tens to hundreds of megabytes. In particular, the paper’s own results indicate that the representation is not consistently smaller than neural surfel baselines. The main advantage is therefore better described as low primitive count and high rendering throughput, rather than universally lower memory or storage cost.

Questions for authors

Can NeST-Splatting and Nexels be evaluated using the same hardware-rasterization backend, resolution, camera set, warm-up protocol, and end-to-end timing procedure as the proposed method?

How much of the speed gain comes separately from the Beta-kernel regularizer, the density-control criterion, the closed-form intersection, and the hardware rasterizer?

Can the authors report a comparison at matched total storage, rather than only matched or reduced primitive counts? This would clarify the trade-off between atlas memory, quality, and speed.

The paper states that flattening the Beta kernels reduces overdraw. Since the kernel value can increase away from the center when the exponent is reduced, can the authors provide statistics showing how the learned scales, opacities, primitive count, and actual blended fragments change under the regularizer?

The paper would be stronger if the main claims were separated into three categories: representation sparsity, rendering throughput, and total deployment cost. The current presentation sometimes makes a reduction in primitive count sound equivalent to a reduction in memory, which is not always supported by the reported atlas sizes.

I also recommend moving the closest surfel comparisons into a more prominent table and clearly labeling which results use the authors’ original renderer and which results use a matched hardware backend. If a common backend is not feasible for all baselines, the paper should narrow the speed claim and report the comparison as an end-to-end system comparison.

The method appears technically sound and practically useful, but its acceptance would be more convincing after clarifying these fairness and storage issues.

1 comment
AuthorsOpenReview ↗

Reply to Reviewer STWD

We thank the reviewer for the careful reading and the concrete suggestions. We have uploaded a revised PDF; all additions are in blue (new Appendix M, two hardware rows in Table 1, one sentence in Sec. 2).

W1 / Q1, common backend for NeST-Splatting and Nexels. We may not have made clear why the port stops at FastGS, Speedy-Splat and Compressed 3DGS. NeST-Splatting and Nexels decode appearance with a network at render time (NeST per pixel from the blended hash-grid features and the view direction, their Eq. 5; Nexels in a world-space second pass). A fixed-function rasterizer has no stage for this, so a port would first require removing the network, which is the change our method makes. The network is also the largest cost of their frame, so the rest of the protocol would not move it. All methods in Table 1 already share resolution, camera set and timing on the software path. As the closest controlled comparison, the revision reports our own neural model before baking, which renders the same surfels with the same per-fragment hash-grid query and MLP: 111 to 155 FPS against 1479 to 2392 FPS after baking at identical primitives (Table 22). Where a common backend is not possible, the next upload describes the comparison as an end-to-end system comparison.

W2 / Q5, storage claim and separating the claims. The paper does not claim lower storage; its claims are primitive count and rendering throughput. The atlas size is stated in Sec. 1 and 2 (55 to 320 MB, more than memory-optimized 3DGS), and Table 1 lists the stored size of every method next to its primitive count. We take the reviewer's suggestion to make the three categories explicit: the next upload names representation sparsity, rendering throughput and total deployment cost separately, so that primitive count and bytes are read apart, and Table 23 gives the trade-off at matched storage.

Q2, speed gain per component. Table 22 lists each change in isolation on the same bakes, with frame rates before and after. Baking the residual into the atlas is worth 13 to 17 times (111 to 155 FPS to 1479 to 2392). The Beta kernel adds 1.36 times (684 to 929), the falloff prior 1.62 times (929 to 1504), the closed-form intersection 1.32 times (1141 to 1504), and the hardware rasterizer 1.82 times (1681 to 3011). The density criterion changes the primitive count by 28 percent but the frame rate by under 10 percent.

Q3, matched total storage. Table 23 re-allocates the atlas so that atlas plus geometry equals the released FastGS model on garden, room, truck and drjohnson, with primitives and training unchanged. At equal bytes we keep the LPIPS and SSIM lead on all four scenes and stay within 0.12 dB PSNR, and the frame time is unchanged within noise. A sparser variant with 27 to 39 percent fewer primitives is listed alongside.

Q4, kernel statistics under the regularizer. Table 26 and Fig. 13 give them on three scenes trained with and without the prior. The prior moves 94 to 96 percent of the primitives to a shape exponent below one (none above three), shrinks the median footprint by 15 to 27 percent, lowers the primitive count by 25 to 30 percent and the blended fragments per pixel by 36 to 41 percent. The opacity parameter is lower with the prior because it is the plateau height of a near-flat disc rather than a Gaussian peak, so blends per pixel is the overdraw statistic to read. The kernel is monotonically decreasing in the radius for every exponent: a smaller exponent widens the plateau but never raises the tail.

Q6, prominent table and renderer labels. Table 1 already groups the textured surfel methods in one block. It now also carries the hardware frame rates of FastGS and Speedy-Splat next to their software rates, as it did for Compressed 3DGS, so the matched-backend comparison is visible in the main table.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.