- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
- 3DGS is limited in its ability to represent high-frequency details, particularly under memory constraints and Rate-Distortion Optimization (RDO).
- The paper proposes a perceptual wrapper that predicts high-frequency residuals for Gaussian splatting. It uses a multi-resolution 3D hash grid with fixed scalar Gaussian noise values, and a very lightweight CNN to predict the residuals.
- This perceptual wrapper improves image fidelity such as FID and human rating Elo scores across various Gaussian splatting models with a small memory overhead.
Critical strengths
- The paper proposes a perceptual wrapper that can be applied to arbitrary 3D scene reconstruction models to predict high-frequency residuals.
- It represents high-frequency texture at the scene level rather than the primitive level, using a 3D hash grid of noise and a CNN to maintain a lightweight structure.
- Qualitative comparisons involving a large number of raters (120) demonstrate perceptual improvement.
Critical weaknesses
-
Grid representations have already been used with Gaussian splatting, for example in 4DGS [4]. Beyond reducing dimensionality and limiting the number of splats for memory efficiency, I do not see what is fundamentally new. Obtaining 2D features from a 3D grid also seems similar in essence to the empty-space skipping used in methods such as TensoRF, rather than a fundamentally different algorithm simply because it uses depth.
-
Does 3D to 2D sampling followed by network-based texture computation preserve view consistency? Because the network operates at the 2D image level, it may generate the appearance of sharper geometry even when the underlying Gaussian splats are much smoother, leading to inaccurate results from other viewpoints. Worse pixel-level metrics such as PSNR, alongside better texture-oriented metrics such as FID, could be evidence that the method generates textures with similar patterns rather than faithfully reconstructing the actual scene.
Questions for authors
-
Dense 3D grid capacity scales cubically with scene extent at a fixed spatial resolution, and larger or more detailed scenes require higher grid resolutions. Even for relatively simple object-centric scenes such as NeRF Synthetic, NeRF [1] uses ten levels of positional encoding for coordinates, presumably from to . For the NeRF Synthetic Lego scene, the official DVGO [2] configuration uses a voxel budget of , while TensoRF [3] specifies for its VM model and 500^3 for its optional CP configuration, for an object-centric scene. This paper, however, uses only eight levels.
-
The actual resolution of these eight levels needs to be identified from the paper, supplement, or code. If the resolutions run from to , the finest resolution is only 128, which seems too small. If a larger base resolution or growth rate brings the finest resolution to around 1024, a dense grid would contain approximately a billion values, or about 4 GB with one fp32 value per entry. Hashing avoids this direct cubic allocation, but the actual hash table sizes and their memory costs still need to be specified. Although the claim of less than 0.01 MB at line 223 concerns the network, the memory overhead is not limited to the network, so this seems overstated.
-
Existing grid-based methods generally use learnable features, whereas this method uses a single scalar value that is randomly initialized and frozen. I question whether such a grid can sufficiently distinguish and represent scene textures while remaining lightweight.
-
The paper argues that smooth Gaussian primitives are inefficient at representing high-frequency texture. By the same reasoning, their depth maps would also be smooth. Is sampling the grid using such smooth depth maps appropriate?
[1] Mildenhall et al. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis [2] Sun et al. Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction [3] Chen et al. TensoRF: Tensorial Radiance Fields [4] Wu et al. 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering