- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this work, the authors propose READ, a lightweight rendering-time refinement module for pre-trained frozen feed-forward 3DGS prediction networks. Specifically, READ queries the input source views by target-to-source reprojection and uses the queried evidence to predict a bounded residual, which is introduced to refine the rendered target view. According to the experiments on DL3DV and Re10k, READ can enhance the rendering quality with low additional computation cost and zero-shot transferring ability.
Critical strengths
-
The idea to refine novel view synthesis with an extra network is reasonable;
-
The presentation is clear and easy to follow, where the experiments also show clear improvements.
Critical weaknesses
- Some ablation experiments are missing. Since the main contribution of this work is the construction of the queried evidence-to-residual network, the necessity of items in the queried evidence should be carefully analyzed, while existing experiments have not clearly illustrate the effectiveness of, e.g., and ;
Questions for authors
-
As mentioned in the weakness section, some extra ablation experiments may be needed;
-
Some discussions about [1] could be included, which is also an enhancement approach for a initial 3DGS object;
[1] Splatformer: Point Transformer for Robust 3D Gaussian Splatting
- The setting of this work is constrained to 4/8 source views-conditioned 3DGS feed-forward prediction frameworks. Is it possible to support continuous partial view inputs like [2] instead of whole source view inputs?
[2] FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenes.
- This method may be incompatible with a 3DGS scene without source images. Is it possible to introduce image generation/editing models to enhance this method?
Response to EMuw: Ablations, Related Work, and Partial-Input Generalization
Thank you for the helpful questions. The revision includes the requested evidence ablations, SplatFormer discussion, and FreeSplat partial-input experiments.
W1/Q1: shallow source features Fi and relative-view information ϕi.
Table 8 compares the requested ablations and controls for evidence processing and source selection on DL3DV. ΔPSNR is measured relative to TokenGS Base:
Removing either Fi or ϕi reduces all three quality metrics. Fi supplies local source-image structure complementary to RGB, while ϕi encodes relative viewing directions to inform source selection. Their gains support complementary roles in representing and selecting source evidence.
The other controls clarify the broader design. GeometryBlend uses fixed RGB fusion and yields little PSNR gain. Uniform/Fixed weighting adds learned evidence processing and substantially improves quality; Full READ further improves all three metrics through target-conditioned source selection. Thus, geometric retrieval provides the observations, while learned processing and selection turn them into effective refinement.
Q2: Relation to SplatFormer.
Section 2.1 now discusses SplatFormer [1]: its point transformer updates Gaussian attributes to improve the scene representation used for rendering. READ keeps the Gaussian scene fixed and retrieves source evidence to correct each target image. Its RGB/depth/alpha interface allows the same refinement network to operate across different reconstruction architectures.
Q3: continuous partial inputs and FreeSplat.
We evaluate FreeSplat-fv [2] on Replica with 2/4/6/8/10 arriving source views on fixed targets, using READ weights trained with TokenGS on DL3DV. Mean PSNR gains over FreeSplat-fv are shown below; Table 21 provides the absolute scores:
Mean gains are positive at every tested prefix, with a small gain at two inputs. READ also improves FreeSplat when newly arriving views update an existing Gaussian scene, providing a 2.253 dB PSNR gain at ten input views (Table 22).
These FreeSplat experiments directly address continuous partial inputs: READ can enhance novel views as source images arrive, after either reconstruction from the available views or incremental scene updates, using the same TokenGS-trained weights throughout. It therefore does not require the full source set in advance. Transfer from TokenGS on DL3DV to FreeSplat on Replica also demonstrates generalization to a new reconstruction model and dataset without retraining READ.
Q4: combining READ with image generation or editing.
Thank you for suggesting this promising direction. READ is designed as an add-on that recovers detail from available source observations with minimal additional latency. Its interface takes source images, their cameras, and rendered scene geometry, providing a natural connection to a source-view generation model. For scenes whose original source images are unavailable, such a model could generate replacement source views conditioned on the existing scene and specified cameras. The generator would provide appearance details, while READ would retain its role of using scene geometry to retrieve and fuse them into target renderings. We highlight this integration and the study of cross-view consistency in generated evidence as future work in Section 4.5.
[1] Chen et al., SplatFormer, ICLR, 2025.
[2] Wang et al., FreeSplat, NeurIPS, 2024.