acceptodds
Withdrawn from ICLR 2027

Perceptual Wrapper with 3D-Anchored Gaussian Noise for 3D Gaussian Splatting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

3D Gaussian Splatting (3DGS) delivers impressive real-time rendering of 3D scenes but remains limited in its ability to represent high-frequency textures, especially under memory constraints and Rate-Distortion Optimization (RDO). Recent methods augment the Gaussian primitives with texture maps, improving the rendering quality given a fixed Gaussian count but introduce substantial storage overhead. Towards this end, we present a general, model-agnostic perceptual wrapper that enhances 3DGS renderings via content- and view-adaptive texture synthesis. Our approach employs a lightweight per-scene network conditioned on the base rendering and 3D-anchored pseudo-random Gaussian noise to synthesize realistic textures, effectively offloading high-frequency appearance details from the underlying 3DGS representation. Guided by Wasserstein Distortion (WD), the network learns to match local feature statistics rather than enforcing pixel-wise fidelity, reducing the oversmoothing associated with traditional reconstruction objectives. Extensive subjective and objective evaluations across RDO and non-RDO 3DGS baselines demonstrate that our perceptual wrapper delivers improved perceptual quality with minimal overhead and substantially reduced storage requirements at comparable perceptual quality levels.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3DGS, Texture Synthesis, Perceptual Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_LJR4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • 3DGS is limited in its ability to represent high-frequency details, particularly under memory constraints and Rate-Distortion Optimization (RDO).
  • The paper proposes a perceptual wrapper that predicts high-frequency residuals for Gaussian splatting. It uses a multi-resolution 3D hash grid with fixed scalar Gaussian noise values, and a very lightweight CNN to predict the residuals.
  • This perceptual wrapper improves image fidelity such as FID and human rating Elo scores across various Gaussian splatting models with a small memory overhead.
Critical strengths
  • The paper proposes a perceptual wrapper that can be applied to arbitrary 3D scene reconstruction models to predict high-frequency residuals.
  • It represents high-frequency texture at the scene level rather than the primitive level, using a 3D hash grid of noise and a CNN to maintain a lightweight structure.
  • Qualitative comparisons involving a large number of raters (120) demonstrate perceptual improvement.
Critical weaknesses
  1. Grid representations have already been used with Gaussian splatting, for example in 4DGS [4]. Beyond reducing dimensionality and limiting the number of splats for memory efficiency, I do not see what is fundamentally new. Obtaining 2D features from a 3D grid also seems similar in essence to the empty-space skipping used in methods such as TensoRF, rather than a fundamentally different algorithm simply because it uses depth.

  2. Does 3D to 2D sampling followed by network-based texture computation preserve view consistency? Because the network operates at the 2D image level, it may generate the appearance of sharper geometry even when the underlying Gaussian splats are much smoother, leading to inaccurate results from other viewpoints. Worse pixel-level metrics such as PSNR, alongside better texture-oriented metrics such as FID, could be evidence that the method generates textures with similar patterns rather than faithfully reconstructing the actual scene.

Questions for authors
  1. Dense 3D grid capacity scales cubically with scene extent at a fixed spatial resolution, and larger or more detailed scenes require higher grid resolutions. Even for relatively simple object-centric scenes such as NeRF Synthetic, NeRF [1] uses ten levels of positional encoding for coordinates, presumably from 202^0 to 292^9. For the NeRF Synthetic Lego scene, the official DVGO [2] configuration uses a voxel budget of 1603160^3, while TensoRF [3] specifies 3003300^3 for its VM model and 500^3 for its optional CP configuration, for an object-centric scene. This paper, however, uses only eight levels.

  2. The actual resolution of these eight levels needs to be identified from the paper, supplement, or code. If the resolutions run from 202^0 to 272^7, the finest resolution is only 128, which seems too small. If a larger base resolution or growth rate brings the finest resolution to around 1024, a dense grid would contain approximately a billion values, or about 4 GB with one fp32 value per entry. Hashing avoids this direct cubic allocation, but the actual hash table sizes and their memory costs still need to be specified. Although the claim of less than 0.01 MB at line 223 concerns the network, the memory overhead is not limited to the network, so this seems overstated.

  3. Existing grid-based methods generally use learnable features, whereas this method uses a single scalar value that is randomly initialized and frozen. I question whether such a grid can sufficiently distinguish and represent scene textures while remaining lightweight.

  4. The paper argues that smooth Gaussian primitives are inefficient at representing high-frequency texture. By the same reasoning, their depth maps would also be smooth. Is sampling the grid using such smooth depth maps appropriate?

[1] Mildenhall et al. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis [2] Sun et al. Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction [3] Chen et al. TensoRF: Tensorial Radiance Fields [4] Wu et al. 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering

Reviewer_5q8rOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a model-agnostic perceptual wrapper (PW) for enhancing the perceptual quality of 3D Gaussian Splatting (3DGS), particularly under constrained representation sizes. Instead of explicitly representing high-frequency textures through additional Gaussian primitives or texture maps, the method introduces a lightweight, per-scene convolutional network that synthesizes residual texture details from the base 3DGS rendering and multi-scale 3D-anchored pseudo-random Gaussian noise.

The noise is generated using fixed multi-resolution 3D hash grids queried through depth-based unprojection, with the intention of encouraging spatial consistency across viewpoints. The underlying 3DGS representation and synthesis network are jointly optimized using Wasserstein Distortion (WD) after an initial reconstruction-based warm-up stage.

The method is evaluated on several rate-distortion-optimized and non-rate-distortion-optimized 3DGS backbones. The authors report improved subjective perceptual quality, FID, and DISTS. The experiments also investigate navigation quality, noise anchoring, and the distortion-perception trade-off.

Critical strengths
  1. The paper identifies an important limitation of 3D Gaussian Splatting: its limited ability to represent perceptually plausible high-frequency details, particularly under constrained memory or bitrate budgets.
  2. The proposed Perceptual Wrapper provides an intuitive approach for offloading high-frequency appearance modeling from the underlying Gaussian representation. In particular, combining a lightweight image-space synthesis network with 3D-anchored pseudo-random Gaussian noise under Wasserstein Distortion supervision is conceptually simple and appears effective in improving perceptual quality.
  3. The experimental results demonstrate consistent improvements in perceptual metrics and subjective quality across several RDO and non-RDO 3DGS backbones, while introducing only a small storage overhead. The method also maintains real-time rendering performance, although the runtime overhead itself is non-negligible.
  4. The paper contains a relatively comprehensive set of experiments, including component ablations, 2D- versus 3D-anchored noise, depth-rendering strategies, navigation quality, storage cost, and rendering efficiency.
Critical weaknesses
  1. The Free-Viewpoint Navigation Quality Comparison in Section 4.1 is important because it is one of the main pieces of evidence supporting the claimed cross-view consistency of the proposed 3D-anchored noise. However, important experimental details remain unclear, such as the construction and length of the evaluation trajectories, camera-motion characteristics, and how it differs from training trajectories. More importantly, the paper provides limited analysis of why an image-space synthesis network improves navigation quality. Since the final residual is generated independently for each rendered frame by a 2D convolutional network, 3D-anchored noise provides consistent conditioning but does not by itself guarantee that the synthesized residual is consistent across views. A more direct analysis of cross-view or reprojection consistency would strengthen this claim.
  2. Based on the architecture of the Perceptual Wrapper network, it is trained and inference on a specific image resolution HH and WW. No explanation on how the renderer should adjust for different image resolutions and how to generalize to new image resolutions or aspect ratios, which is often the case in real-time applications.
  3. The related work section mentioned the memory inefficiency of textured gaussians representation. This is a valid concern, but some improvements have been proposed to improve memory efficiency of textured gaussians representation, for example: A$^2$TG (ICLR 2026) and ASAP-Textured Gaussians (Arxiv). It would be better if they were mentioned.
Questions for authors
  • Please provide more details about the Free-Viewpoint Navigation Quality Comparison in Fig. 3(c). How are the evaluation camera trajectories constructed, and what are their duration, frame rate, and camera-motion range?
  • Why does PW improve navigation quality in Fig. 3(c)? Since the final residual is generated by a 2D image-space synthesis network for each rendered frame, how does the method ensure that synthesized high-frequency details remain consistent across views?
  • Would the navigation-quality improvement also hold for different backbones such as 2DGS or TexGS?
  • Does a PW model trained at one image resolution generalize to different rendering resolutions or aspect ratios without retraining or fine-tuning? If so, please provide quantitative or qualitative evidence.
  • Please report the additional training cost introduced by Perceptual Wrapper for completeness.
  • Unrelated texts “(see “CrossViewArtifacts” folder here)” at line 243.

I believe addressing these points will significantly enhance the quality of the paper, and I would be inclined to raise the score. While I acknowledge the contribution of the paper, improving these aspects is essential to further strengthen its overall quality.

Reviewer_fRSJOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

In this paper, authors proposes a perceptual wrapper for 3DGS to help preserve high-frequency texture detials under storage constraints. The method combines a lightweight image-space synthesis network, 3D-anchored Gaussian noise, and Wasserstein Distortion for optimization. Experiments across multiple RDO and non-RDO 3DGS baselines sligtly demonstrate improved quality.

Critical strengths
  1. The proposed perceptual wrapper is a plug-in module and can be inserted into many existing 3DGS models.
  2. Authors provide not only objective (including both non-generative and generative) metrics but also subjective metrics. Experiments are comprehensive.
Critical weaknesses
  1. The paper does not have any novelty: Wasserstein Distortion comes from Perceptual-GS (ECCV 26), image-space texture synthesis network comes from IBGS (NIPS 25). The perceptual wrapper lies in their combination and adaptation to 3DGS. In my opinion, the overall contribution falls substantially short of the novelty and significance expected at a top-tier conference such as ICLR.

  2. The Introduction and Method Overview contain redundant explanations, such as the key challenges of texture synthesis, which largely reiterate well-established concepts in 3D recon and NVS. These discussions could be significantly condensed. Furthermore, the contribution statements appear overstated. The only contribution is the perceptual wrapper itself, which is essentially a plug-in module rather than a new framework.

  3. (a) Sections 3.3 and 3.4 devote substantial space to technical details. Many of them are standard and could be moved to the appendix, allowing the main paper to focus more clearly on the actual contributions. (b) Section 3.5 would benefit from explicitly separating the two training stages and presenting their respective optimization objectives. Moreover, the regularization term L_{Reg} and rate term L_{rate} are insufficiently defined. Their formulations and roles should be clarified in either the main paper or the appendix.

  4. The component ablations (line 466) are insufficient to convincingly establish the effectiveness of each component. The evaluation relies heavily on subjective scores, without adequately reporting standard objective metrics such as PSNR, SSIM, and LPIPS for each component. Subjective evaluations are valuable, but they should be complemented by quantitative metrics (such as PSNR LPIPS SSIM) to provide a more comprehensive understanding of the contribution of each component.

  5. The paper emphasizes improvements in FID as evidence of better novel-view synthesis quality. However, FID measures distribution-level similarity and does not adequately capture reconstruction fidelity for each novel view. Standard NVS metrics such as PSNR, SSIM, and LPIPS deserve greater emphasis. Moreover, the results, including those reported in the appendix, suggest that introducing the perceptual wrapper can lead to noticeable degradation in novel view quality. This trade-off is not sufficiently discussed or justified. Furthermore, the rendering speed decreases substantially after introducing the wrapper (e.g., from 269.5 to 133.8 FPS in Table 2), weakening the claim of negligible computational overhead. The additional training time introduced by the wrapper is also unclear and should be reported.

In summary, the contributions presented in this paper fall short of the high standards required by ICLR. Furthermore, the authors need to reorganize the paper, with a particular focus on improving the writing.

Questions for authors

See above.

Reviewer_pHMFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a perceptual wrapper (PW) for per-scene 3D Gaussian Splatting (3DGS). It renders RGB images and median depth from the underlying 3DGS model, unprojects the depth into per-pixel 3D points, and queries an eight-level hash grid with fixed random entries to obtain multiscale features. A lightweight 2D CNN takes the RGB image concatenated with the eight-channel random features as input and predicts a three-channel residual image. The underlying Gaussians are first trained with their original objective for 15,000 iterations, then jointly optimized with the CNN using Wasserstein Distortion (WD). The authors report objective metrics and human pairwise preferences for static images and navigation sequences across multiple backbones.

Critical strengths

S1. The method has practical value and is evaluated across a broad range of settings. PW preserves each backbone's Gaussian parameterization and is evaluated on several representations. In the evaluated RDO settings, the authors report that PW achieves Elo scores comparable to the high-rate baselines at roughly one-sixth of the bitrate. The authors also conduct comparisons at comparable final model sizes and, separately, comparable final Gaussian counts, helping examine how the representation budget is allocated.

S2. The perceptual evaluation is substantial. The paper reports nearly 10,000 pairwise comparisons from over 100 raters. The evaluation covers static images, free-viewpoint navigation, comparisons with WD-R on the same backbone, and several component ablations. Human preference evaluation is valuable for a method targeting perceptual quality.

Critical weaknesses

W1. The distinction from prior work and the contribution of joint training remain unclear. IBGS already jointly optimizes a 3DGS model and a 2D CNN that predicts residuals to refine the base renderings, but this related work is not discussed. The authors should clarify PW's contribution beyond existing 2D residual frameworks. In addition, the paper lacks a comparison that freezes a fully trained Gaussian representation and trains only the same CNN. It therefore remains unclear how much joint optimization adds over training the post-processing network alone.

W2. The component ablations do not sufficiently explain the sources of improvement and their fidelity costs. Figure 6 compares the synthesis network trained with the original loss, WD alone, and the full method, but mainly reports Elo scores without a corresponding breakdown of PSNR and other metrics. This makes it difficult to assess the fidelity cost of the perceptual gains. In Table 9, removing the noise input also removes the explicit 3D positional conditioning, so the gains cannot yet be attributed specifically to randomness. A positional encoding baseline using the same network would help clarify this mechanism.

Questions for authors
  1. Related work and Table 1: Please discuss IBGS and clarify the contribution beyond existing 2D residual frameworks, taking into account the differences in source-image dependence and storage requirements.

  2. Joint training: Could you add a baseline that fully trains the Gaussians using the original method, freezes them, and then trains only the same PW network? Please use the same PW input formulation, loss, and number of PW training iterations as in the joint training scheme, and compare the resulting quality and total training cost.

  3. Loss and network ablations (Figure 6): Please report PSNR, SSIM, and DISTS for each variant alongside Elo scores to show both the perceptual gains and their fidelity costs.

  4. Random inputs (Table 9; supplementary suggestion): Using the same network and loss, please consider adding a deterministic 3D positional encoding baseline for comparison with the variants using fixed random features and no noise input. This would help assess whether randomly generated 3D features offer additional value over conventional 3D positional encoding.

  5. Local structures (Appendices F and I; supplementary discussion): Can PW improve texture appearance while degrading edges, thin lines, or high-frequency structures that are clear in the base renderings? Please consider providing crops of the same regions from the base renderings, PW outputs, and ground truth to illustrate potential limitations of perceptual synthesis.

  6. Computational cost: Table 2 shows that PW roughly halves the FPS of CAT-3DGS Pro* and HAC++. Please distinguish storage overhead from computational overhead and clarify what is meant by “minimal overhead”. Please also report training time and peak GPU memory with and without PW, the training hardware, the iteration counts for the Gaussian-only and joint training stages, and the image resolution used for inference speed measurements.

Questions 1–3 address the two core weaknesses. Questions 4–5 are supplementary suggestions for understanding the mechanism and its limitations. Question 6 concerns computational cost and the experimental setup.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.