- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DeepCR, a neural network-based method for assessing 3D Gaussian Splatting renderings. The proposed method leverages a frozen DINOv2 backbone with the developed Pixel Decoder to output quality maps. Experiments across several datasets demonstrate the superior performance of the proposed method.
Critical strengths
-
The paper contains several images, which makes readers easy to follow.
-
The paper is well-written and not hard to understand.
-
The proposed method demonstrates strong performance over other approaches.
Critical weaknesses
-
I don't get the meaning of this task. In general, PSNR SSIM LPIPS are up as metrics for novel view synthesis, as well as to the whole scene. I am not an expert in this field and not familiar with this topic. It is important for novices to understand and follow this work.
-
Have the authors tried only using the pose-aware selection? I think it works well when only with this method. Maybe it dominates the proposed method.
-
Table 1 shows PLCC and SRCC metrics. These two metrics are all against the SSIM map. So, it brings to my original question, why not directly using SSIM as the metric.
-
Table 2 shows the ablation study of the proposed method. It seems the proposed pixel decoder only brings marginal improvements.
-
In Table 5, the proposed method demonstrates strong improvement compared with previsous methods for blender dataset but marginal in real dataset like Mip-NeRF 360. Please explain this phenomenon.
Questions for authors
-
What does Figure 4 mean? There's not a light bar. I cannot get the idea of the figure
-
Any plan to release the source code?
2 comments
Thank you for the rebuttal. However, this rebuttal only addresses part of my concerns. I am still not convinced by the tasks and the motivation. Therefore, I keep my original score.
Response for Reviewer
Thank you for highlighting where the task and experimental results need a clearer explanation. We first clarify why quality must be estimated without an aligned reference, then address the component contributions, reconstruction results, and presentation.
W1/W3. Task motivation and the role of SSIM. The key issue is access to the real image at the rendered viewpoint. Suppose we render a room from a new camera position where no photograph has been captured. PSNR, SSIM, and LPIPS require an aligned real image from that position, even when these metrics are later averaged over a scene. Photographs from other viewpoints provide scene information but cannot directly serve as aligned references.
DeepCR uses these other views to estimate the query's quality and identify local reconstruction deficiencies. This enables assessment before acquiring a photograph at the queried viewpoint and supports quality-guided acquisition. For training and benchmarking, we use an aligned image to construct SSIM targets, but exclude it from the model's reference pool. Table 1 therefore evaluates how well predicted scores agree with ground-truth SSIM when the model cannot access that image. Tables 3-4 further evaluate spatial accuracy and human agreement. We have clarified this distinction in the introduction and Section 3.1.
W2. Contributions of reference selection and prediction. To address your suggestion of using pose selection alone, we applied the same normalized pose selector to CrossScore and retrained it. This tests whether changing the reference evidence can reproduce DeepCR's performance without its prediction network. DeepCR retains higher correlations and lower pixel MAE across all three datasets (Table S7).
We also examine the remaining components within DeepCR's shared base configuration (Table 2). The "w/o pose selection" variant retains multi-level fusion and spatial decoding; removing multi-level features lowers both correlations on all datasets. Thus, pose selection contributes useful evidence, while the representation and prediction network also contribute to the results. Appendix D specifies the base-configuration differences in the complete-predictor comparison.
W4. Contribution of the pixel decoder. The pixel decoder combines neighboring patch features to predict local quality variations at pixel resolution. To assess this spatial contribution beyond image-level correlations, we added a pixel-MAE comparison (Table 3) and separate decoder/auxiliary-loss controls (Table S6). The full configuration lowers MAE relative to removing decoding and auxiliary supervision on all datasets: .1306/.0823/.1119 versus .1332/.0851/.1146 on MFR/RE10K/T&T.
Retaining the decoder without auxiliary supervision gives the lowest MFR MAE; the full model gives the lowest RE10K/T&T MAE and the highest image-level correlations among these variants. This clarifies the joint spatial benefit and the effect of the training loss. Figure 5 complements the quantitative comparison by showing reduced patch-wise discontinuities and finer spatial variation.
W5. Reconstruction gains across datasets. We address this by comparing the same baseline across datasets and examining all reconstruction metrics. In the five-run results, the mean PSNR gain over CrossScore is .9817 dB on Blender and .0777 dB on Mip-NeRF360 (Table 5). DeepCR also improves SSIM and LPIPS over CrossScore, and achieves the best mean values for all three metrics on both datasets.
The size of an acquisition gain depends on the views selected by the competing policies and the evidence available from the candidate pool. Scene geometry and visibility can also influence how additional views affect reconstruction. These are plausible factors in the dataset difference; our repeated experiments quantify the observed benefit under the shared 20-view protocol. Figure 6 illustrates associated local structure and texture improvements, providing a spatial perspective alongside the aggregate scores (Section 4.4).
Q1. Figure 4 and the missing colorbar. Thank you for pointing out the missing colorbar. We have added a vertical bar on the right and clarified the caption: blue indicates lower and red higher displayed map values. Each example shows the query, aligned GT, target SSIM map, and predicted maps. Comparing the predicted spatial patterns with the target reveals where each method captures local quality changes; this is the purpose of the visualization.
Q2. Code release. Yes. We had already planned to make the code publicly available upon publication. Following your question, we have now stated this explicitly in the abstract.