acceptodds
Under review as a conference paper at ICLR 2027

DeepCR: Deep Cross-Reference Reasoning for Assessing 3D Gaussian Splatting Renderings

Abstract

Reliable quality assessment is crucial for identifying viewpoint-dependent artifacts and improving 3D Gaussian Splatting (3DGS) reconstruction, yet aligned ground truth is generally unavailable at inference. While no-reference methods lack scene-specific evidence, existing cross-reference methods insufficiently exploit the geometric relevance and complementary semantics of non-aligned observations. In this paper, we propose DeepCR, a novel cross-reference method for dense quality assessment of 3DGS renderings. In particular, pose-aware reference selection reduces viewpoint disparity and filters ambiguous evidence, while multi-level cross-view reasoning over DINOv2 features integrates local distortion cues with viewpoint-robust semantics. A pixel decoder with auxiliary supervision further recovers fine-grained quality variations from coarse patch representations, yielding a full-resolution structural similarity map and an image-level score. Experiments show that DeepCR achieves strong agreement with ground-truth SSIM scores and outperforms the strongest competing CR-IQA method by 8.5 and 11.7 percentage points in average absolute PLCC and SRCC, respectively. The gains in 3DGS reconstruction achieved through DeepCR-guided active view selection further demonstrate the practical value of its quality predictions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Cross-Reference Reasoning, Image Quality Assessment, 3DGS Renderings

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_LwrNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DeepCR, a neural network-based method for assessing 3D Gaussian Splatting renderings. The proposed method leverages a frozen DINOv2 backbone with the developed Pixel Decoder to output quality maps. Experiments across several datasets demonstrate the superior performance of the proposed method.

Critical strengths
  1. The paper contains several images, which makes readers easy to follow.

  2. The paper is well-written and not hard to understand.

  3. The proposed method demonstrates strong performance over other approaches.

Critical weaknesses
  1. I don't get the meaning of this task. In general, PSNR SSIM LPIPS are up as metrics for novel view synthesis, as well as to the whole scene. I am not an expert in this field and not familiar with this topic. It is important for novices to understand and follow this work.

  2. Have the authors tried only using the pose-aware selection? I think it works well when only with this method. Maybe it dominates the proposed method.

  3. Table 1 shows PLCC and SRCC metrics. These two metrics are all against the SSIM map. So, it brings to my original question, why not directly using SSIM as the metric.

  4. Table 2 shows the ablation study of the proposed method. It seems the proposed pixel decoder only brings marginal improvements.

  5. In Table 5, the proposed method demonstrates strong improvement compared with previsous methods for blender dataset but marginal in real dataset like Mip-NeRF 360. Please explain this phenomenon.

Questions for authors
  1. What does Figure 4 mean? There's not a light bar. I cannot get the idea of the figure

  2. Any plan to release the source code?

2 comments
AuthorsOpenReview ↗

Response for Reviewer

Thank you for highlighting where the task and experimental results need a clearer explanation. We first clarify why quality must be estimated without an aligned reference, then address the component contributions, reconstruction results, and presentation.

W1/W3. Task motivation and the role of SSIM. The key issue is access to the real image at the rendered viewpoint. Suppose we render a room from a new camera position where no photograph has been captured. PSNR, SSIM, and LPIPS require an aligned real image from that position, even when these metrics are later averaged over a scene. Photographs from other viewpoints provide scene information but cannot directly serve as aligned references.

DeepCR uses these other views to estimate the query's quality and identify local reconstruction deficiencies. This enables assessment before acquiring a photograph at the queried viewpoint and supports quality-guided acquisition. For training and benchmarking, we use an aligned image to construct SSIM targets, but exclude it from the model's reference pool. Table 1 therefore evaluates how well predicted scores agree with ground-truth SSIM when the model cannot access that image. Tables 3-4 further evaluate spatial accuracy and human agreement. We have clarified this distinction in the introduction and Section 3.1.

W2. Contributions of reference selection and prediction. To address your suggestion of using pose selection alone, we applied the same normalized pose selector to CrossScore and retrained it. This tests whether changing the reference evidence can reproduce DeepCR's performance without its prediction network. DeepCR retains higher correlations and lower pixel MAE across all three datasets (Table S7).

We also examine the remaining components within DeepCR's shared base configuration (Table 2). The "w/o pose selection" variant retains multi-level fusion and spatial decoding; removing multi-level features lowers both correlations on all datasets. Thus, pose selection contributes useful evidence, while the representation and prediction network also contribute to the results. Appendix D specifies the base-configuration differences in the complete-predictor comparison.

W4. Contribution of the pixel decoder. The pixel decoder combines neighboring patch features to predict local quality variations at pixel resolution. To assess this spatial contribution beyond image-level correlations, we added a pixel-MAE comparison (Table 3) and separate decoder/auxiliary-loss controls (Table S6). The full configuration lowers MAE relative to removing decoding and auxiliary supervision on all datasets: .1306/.0823/.1119 versus .1332/.0851/.1146 on MFR/RE10K/T&T.

Retaining the decoder without auxiliary supervision gives the lowest MFR MAE; the full model gives the lowest RE10K/T&T MAE and the highest image-level correlations among these variants. This clarifies the joint spatial benefit and the effect of the training loss. Figure 5 complements the quantitative comparison by showing reduced patch-wise discontinuities and finer spatial variation.

W5. Reconstruction gains across datasets. We address this by comparing the same baseline across datasets and examining all reconstruction metrics. In the five-run results, the mean PSNR gain over CrossScore is .9817 dB on Blender and .0777 dB on Mip-NeRF360 (Table 5). DeepCR also improves SSIM and LPIPS over CrossScore, and achieves the best mean values for all three metrics on both datasets.

The size of an acquisition gain depends on the views selected by the competing policies and the evidence available from the candidate pool. Scene geometry and visibility can also influence how additional views affect reconstruction. These are plausible factors in the dataset difference; our repeated experiments quantify the observed benefit under the shared 20-view protocol. Figure 6 illustrates associated local structure and texture improvements, providing a spatial perspective alongside the aggregate scores (Section 4.4).

Q1. Figure 4 and the missing colorbar. Thank you for pointing out the missing colorbar. We have added a vertical bar on the right and clarified the caption: blue indicates lower and red higher displayed map values. Each example shows the query, aligned GT, target SSIM map, and predicted maps. Comparing the predicted spatial patterns with the target reveals where each method captures local quality changes; this is the purpose of the visualization.

Q2. Code release. Yes. We had already planned to make the code publicly available upon publication. Following your question, we have now stated this explicitly in the abstract.

Thank you for the rebuttal. However, this rebuttal only addresses part of my concerns. I am still not convinced by the tasks and the motivation. Therefore, I keep my original score.

Reviewer_Z81jOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DeepCR, a cross-reference image quality assessment (CR-IQA) method for evaluating 3D Gaussian Splatting (3DGS) renderings. DeepCR selects relevant reference views using pose-aware reference selection and integrates information from the query and references through multi-level DINOv2 features and cross-attention. A PixelShuffle-based decoder with auxiliary supervision then predicts a dense SSIM quality map without requiring an aligned ground-truth image at inference time. Experiments demonstrate strong IQA performance on MFR, RE10K, and Tanks and Temples, and the authors also evaluate its use for active view selection in 3DGS reconstruction.

Critical strengths

The paper clearly identifies key challenges in CR-IQA, including reference relevance, the integration of local structural information with high-level semantics, and dense quality prediction. The proposed pose-aware reference selection, multi-level feature aggregation, and pixel decoder are logically motivated by these challenges. The overall method follows a coherent progression from problem formulation to architectural design.

Critical weaknesses
  • Despite substantial improvements in IQA performance, the additional gains in downstream 3DGS reconstruction are relatively limited. In particular, Table 5 shows that on Mip-NeRF360, DeepCR improves PSNR by only about 0.08 dB and SSIM by about 0.005 compared with CrossScore. Although more pronounced gains are observed on Blender, the small differences on Mip-NeRF360 raise questions about how much practical benefit the improved IQA performance provides for reconstruction quality. Thus, while the contribution to IQA is clear, the contribution to improving downstream 3DGS reconstruction appears relatively modest.

  • DeepCR's core architecture is closely related to CrossScore. Both methods rely on a DINOv2 backbone, query-reference cross-attention, and dense SSIM map prediction, sharing a similar overall framework and training objective. DeepCR introduces pose-aware reference selection, multi-level feature aggregation, a convolutional pixel decoder, and auxiliary supervision, which are reasonable and effective improvements. However, the central cross-view reasoning mechanism remains largely unchanged. Consequently, despite providing meaningful architectural refinements, the method appears closer to an incremental extension of CrossScore than to a fundamentally new methodological approach.

Questions for authors
  1. Despite the substantial improvement in IQA performance, the gains in downstream 3DGS reconstruction on Mip-NeRF360 are quite limited compared with existing methods. How do the authors interpret this gap, and how do they assess the additional practical benefit of improved IQA performance for reconstruction?

  2. DeepCR appears to share much of its core neural architecture and training objective with CrossScore. What do the authors consider the main methodological novelty of DeepCR, beyond engineering refinements to the existing CrossScore framework?

2 comments
AuthorsOpenReview ↗

Response for Reviewer

Thank you for recognizing the coherence of our design and its IQA improvements. Your questions concern two central points: how those improvements translate into practical use, and what distinguishes the method from CrossScore. We clarify both through the design rationale and the expanded experimental evidence.

W1/Q1. Interpreting IQA and downstream reconstruction gains. We appreciate your focus on the practical meaning of the Mip-NeRF360 results. DeepCR's primary purpose is to estimate rendering quality and its spatial distribution when aligned GT is unavailable. The image-level gains over CrossScore in Table 1, lower pixel errors in Table 3, and human agreement in Table 4 establish its assessment value. We use AVS to examine whether these quality estimates can additionally guide reconstruction without fine-tuning.

The two evaluations measure different stages of this process. IQA correlations assess the agreement of predictions across rendered queries; reconstruction metrics measure the cumulative result of the selected observations and 3DGS optimization. Under our protocol, all methods start with four views and acquire another sixteen under the same schedule and 30,000-iteration budget. Better quality estimates inform those acquisition choices, while final reconstruction also depends on what the available candidate views reveal and how optimization integrates them. We therefore assess the downstream benefit through repeated reconstruction results and spatial examples alongside the IQA metrics.

To strengthen that evidence, we added five paired-seed runs for all methods (Table 5). On Mip-NeRF360, DeepCR/CrossScore achieve PSNR of 20.9870/20.9093, SSIM of .6283/.6226, and LPIPS of .2994/.3046. These show improvements in all three measures, with DeepCR achieving the best mean values on both datasets. The accompanying SSIM/LPIPS improvements and Figure 6's local detail comparisons provide complementary evidence of the selected views' usefulness. Table 5 reports SD across five run-level dataset means to quantify variability, and Section 4.4 specifies the shared acquisition and optimization protocol. These results demonstrate an additional use of the quality estimates alongside their primary assessment role.

W2/Q2. Contribution beyond the shared CrossScore framework. CrossScore uses DINOv2 and cross-attention to predict quality from non-aligned views, but its reference selection overlooks query-reference pose relevance, its single-layer representation has limited capacity to combine quality cues across depths, and token-level prediction limits spatial detail. We designed DeepCR to address these three limitations together.

Specifically, normalized pose ranking prioritizes relevant observations; fusion of layers 4/8/12 on both query and reference sides supplies local detail and higher-level context before cross-view association; and convolutional PixelShuffle decoding with auxiliary supervision predicts fine spatial quality variation. The contribution lies in improving the evidence, representation, and spatial prediction for CR-IQA. The underlying attention mechanism and SSIM supervision are shared.

To further explain the performance gains, we tested whether the same reference selector would suffice for CrossScore. After training with that selector, CrossScore still yields lower correlations and higher pixel MAE on all three datasets (Table S7). We then examine the design choices under a shared DeepCR base configuration: removing pose selection or multi-level features lowers correlations (Table 2), fusion gives the highest SRCC among tested feature depths across datasets (Table S5), and decoder/supervision controls clarify the spatial and training effects (Table S6). The human spatial evaluation in Table 4 further tests the resulting predictions against independent annotations.

These comparisons connect the design rationale to observed benefits. Related Work (Sec. 2.1) now explains how DeepCR's designs address the identified limitations of CrossScore. Appendix D specifies the base-network differences, so Table S7 evaluates complete predictors using the same selector, while shared-base ablations assess component contributions.

Reviewer_Z81jOpenReview ↗

New Official Comment

Thank you for the detailed response and additional experiments. The IQA results are strong, but the improvement in downstream 3DGS reconstruction is still quite small, especially on Mip-NeRF360. I also think the main cross-view reasoning framework is largely the same as CrossScore. The added components are useful, but the novelty still seems limited.

I have one additional concern about Table 1. DeepCR is trained to predict GT SSIM, and the evaluation is also based on GT SSIM. This may favor methods trained with SSIM supervision over methods with different objectives, such as PR-IQA, PuzzleSim. I think evaluating the correlation with GT PSNR and LPIPS would provide a fairer and more complete comparison.

Overall, I appreciate the authors' efforts, but I will keep my rating at 2 (Weak Reject).

Reviewer_RKBVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DeepCR, a cross-reference image quality assessment framework for 3DGS renderings. It combines pose-aware reference selection, multi-level DINOv2 features, and a PixelShuffle decoder to predict dense SSIM maps without aligned ground truth at inference.

Critical strengths
  1. The problem is well-motivated and relevant to 3D reconstruction.

  2. The method achieves strong results across multiple datasets, including cross-dataset evaluations.

  3. The active view selection experiments demonstrate potential practical applications.

Critical weaknesses
  1. The overall framework closely resembles CrossScore, which already uses DINOv2, cross-attention, and SSIM supervision. The proposed improvements mainly involve standard architectural modifications.

  2. The paper lacks detailed analysis of individual DINOv2 layers, the auxiliary supervision branch, and alternative reference selection strategies.

  3. Despite strong IQA results, improvements in active view selection are relatively small, and their statistical significance is unclear.

  4. Pose distance does not always reflect visual overlap, especially under occlusion or large viewpoint changes.

  5. The model mainly learns to predict SSIM. Its ability to capture broader perceptual quality remains unclear.

Questions for authors
  1. What is the main methodological novelty compared with CrossScore?

  2. How does pose-based reference selection perform under severe occlusion or limited visual overlap?

  3. How does the proposed reference selection compare with feature-based or geometry-aware retrieval?

  4. What are the individual contributions of different DINOv2 layers and auxiliary supervision?

  5. Can the authors evaluate pixel-level agreement with human artifact annotations?

  6. Are the active view selection improvements consistent across different scenes and random seeds?

1 comment
AuthorsOpenReview ↗

Response for Reviewer

Thank you for your detailed questions. We address methodological attribution, component effects, reference relevance, human agreement, and downstream repeatability below.

W1/Q1. Methodological contribution. We clarify the rationale linking the modules to the task. CrossScore overlooks query-reference pose relevance, relies on a single feature layer, and predicts quality at token granularity. We designed DeepCR to address these limitations through relevant-view selection, fusion of query/reference layers 4/8/12 before attention, and convolutional PixelShuffle decoding with auxiliary supervision. The contribution concerns how reference evidence is selected, represented, and converted into spatial quality estimates; DINOv2, cross-attention, and SSIM supervision remain shared. To clarify the source of the observed gains, we trained CrossScore with DeepCR's pose selector. DeepCR retains higher correlations and lower pixel MAE on all three datasets (Table S7), showing that selection alone does not explain its performance. We assess individual designs through shared-base controls (Tables 2 and S5-S6). Related Work (Sec. 2.1) now makes this problem-to-design link explicit; Appendix D specifies base-configuration differences in the complete-predictor comparison.

W2/Q4. Feature depths and auxiliary supervision. As you suggested, we trained separate models using layers 4, 8, or 12 to evaluate the benefit of multi-level fusion (Table S5). Fusion achieves the highest SRCC on all three datasets and the highest PLCC on MFR and T&T. Its consistent advantage in quality ranking supports combining feature depths rather than relying on a single layer.

To isolate the contribution of auxiliary supervision, we retained the decoder and removed only the auxiliary loss. Adding this supervision improves both PLCC and SRCC on all three datasets (Table 2) and reduces pixel MAE on RE10K and T&T (Table S6). These controls support auxiliary supervision as an effective training signal for image-level quality prediction, with additional pixel-level benefits on RE10K and T&T.

W4/Q2-Q3. Limited overlap and alternative retrieval. As you note, nearby poses do not by themselves determine visibility. To examine performance when shared content is limited, we evaluated all selectors on identical low-overlap query groups from existing captures (Table S10). Pose selection obtains PLCC/SRCC of .9034/.8856, .9505/.9298, and .9370/.9509 on MFR/RE10K/T&T and remains competitive with feature retrieval. These results address your concern about limited shared content and support selection under low overlap.

Following your suggestion, we compared different retrieval strategies and aligned GT excluded (Table S9; Appendix F.5). Pose selection improves both correlations over random references on all datasets and exceeds feature retrieval in PLCC on all three. Feature retrieval ranks five references by cosine similarity of frozen DINOv2 final-layer 384-D CLS descriptors; pose selection directly uses existing camera metadata, without extracting descriptors for retrieval or requiring depth maps. Geometry retrieval achieves higher T&T correlations but requires additional depth maps. Thus, our selector uses existing reconstruction metadata to retrieve useful references.

W5/Q5. Pixel-level human agreement. As you suggested, we evaluated spatial agreement with human annotations independently of SSIM supervision. On annotated-3DGS-artifacts, we evaluate renderings from 12 scenes at 60/70/80% retained-view ratios. We convert annotations to the quality direction, compute pixel-wise correlations within each image, average within each scene, and then equally across scenes.

DeepCR obtains spatial PLCC/SRCC of .4827/.3792, compared with .4683/.3343 for CrossScore, .4610/.3763 for PuzzleSim, and .1458/.1285 for PR-IQA (Table 4). It also has the highest within-scene image-level correlations (.4787/.5030). The spatial and image-level results together support agreement with human-annotated artifact patterns and quality changes on this benchmark.

W3/Q6. AVS repeatability and scene consistency. To address your concern about stability across seeds, we added five paired-seed runs for all methods, sharing initial views, candidate/evaluation pools, acquisition budgets, and optimization settings. DeepCR has the best mean PSNR, SSIM, and LPIPS on both datasets (Table 5). Each run averages evaluation views within scene and then across scenes; SD is computed across the five dataset means.

This reports repeatability at dataset level. The question of paired statistical significance or improvement in each scene requires the corresponding run- or scene-level differences; the reported mean/SD summaries do not establish either. We therefore base our practical conclusion on the aggregate repeated results and the qualitative examples in Figure 6, with the complete protocol in Section 4.4.

Reviewer_MQBMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper, based on CrossScore, selects reference images according to camera poses, fuses features from different layers of DINOv2, and uses a pixel decoder to predict 3DGS rendering quality, then uses the predicted scores for active viewpoint selection. The quality prediction metrics have improved, but existing comparisons do not clearly explain the individual contributions of each modification relative to CrossScore; the advantage of active viewpoint selection is small, and repeated experiments are also lacking.

Critical strengths

The author selects reference images based on camera pose and integrates features from different layers of DINOv2, showing a relatively clear approach. Improvements in correlation and pixel error across multiple datasets indicate that these changes are effective.

Critical weaknesses
  1. Although CrossScore predicts per-pixel SSIM maps through DINOv2 and cross-attention, the additional multi-layer fusion, convolution, and auxiliary supervision in this paper are relatively common, and the overall approach still leans towards improvements on existing frameworks.
  2. The reference selection experiment in Table 2 only compares with randomly selected images, making it impossible to distinguish the contributions of neighbour selection and distance normalisation. It is recommended to supplement with a control experiment of standard pose neighbour selection. In addition, Table 3 only compares the complete model; the authors should also report the pixel MAE "w/o pixel decoder" to clarify the actual effect of the new decoder.
  3. In Table 1, CrossScore and the Baseline in Table 2 have PLCC/SRCC on MFR of 0.7792/0.7221 and 0.7993/0.7352, respectively. The authors need to explain the differences in configuration and training between the two to assess the actual benefit of the new module.
  4. The PSNR advantage of active viewpoint selection on the two datasets is only 0.0573 and 0.0817 dB, and the paper does not report repeated experimental results. It is recommended to repeat the experiments for this method and the strongest baseline several times, providing mean values and standard deviations to verify the stability of the improvement.
  5. The authors do not clarify how test images are handled when the long edge is not a multiple of 14, nor whether SSIM is calculated before or after scaling. These details may affect the evaluation results and the complete process needs to be provided.
Questions for authors

Look at the weaknesses section

1 comment
AuthorsOpenReview ↗

Response for Reviewer

Thank you for recognizing the IQA improvements. We clarify their sources through the requested controls, repeated AVS evaluation, and technical details. References below point to the revised manuscript.

W1. Contribution relative to CrossScore. CrossScore predicts SSIM maps through DINOv2 and cross-attention, but overlooks query-reference pose relevance, uses a single feature layer with limited representational capacity, and predicts at token granularity. We address these limitations through pose-guided reference selection, fusion of query/reference layers 4/8/12 before attention, and convolutional PixelShuffle decoding with auxiliary supervision. These choices improve the evidence available for comparison, combine local detail with higher-level context, and recover spatial quality variation.

To explain these gains, we added layer and supervision controls (Tables S5-S6) and trained CrossScore with our pose selector (Table S7). DeepCR retains higher correlations and lower pixel MAE on all datasets in this comparison. Together with Table 2, this supports contributions beyond reference selection. Related Work (Sec. 2.1) now maps CrossScore's limitations to these designs and notes the shared backbone, attention, and SSIM objective.

W2. Neighbor selection and no-decoder pixel MAE. As you suggested, we added a neighbor-selection control (Table S8). Our normalized strategy matches or improves every reported metric across all three datasets, retaining the benefit of relevant-view selection demonstrated against random references (Table 2). Beyond predictive accuracy, normalization converts position and orientation distances into dimensionless, query-specific scores, avoiding dependence on arbitrary scene units when combining them. The gains over the neighbor baseline are small on these fixed, discrete camera pools, where different rankings may retrieve similar evidence. These results support our choice of a scale-normalized pose score while maintaining consistently strong quality prediction.

Following your request, we also added the no-decoder pixel MAE to Table 3. Removing decoding and its auxiliary supervision increases MAE on MFR/RE10K/T&T from .1306/.0823/.1119 to .1332/.0851/.1146. To separate these changes, Table S6 retains the decoder while removing only auxiliary supervision. Across these variants, the full configuration achieves the highest image-level correlations on all three datasets and the lowest pixel MAE on RE10K and T&T, supporting the combined use of spatial decoding and auxiliary supervision.

W3. CrossScore versus the ablation Baseline. Thank you for highlighting this distinction. Our Baseline removes pose selection, multi-level fusion, and pixel decoding from DeepCR's base network. CrossScore uses fixed additional positional embeddings and a 384-to-384-to-384 FFN; our base uses learnable embeddings and a 384-to-768-to-384 FFN, with dropout regularization. These affect task adaptation and predictor capacity. Thus, Table 2 measures component effects under a shared base configuration, while Table 1 compares existing methods. We do not attribute the difference between the two baseline rows to our three components. Section 4.3 and Appendix D now explain this distinction; Section 4.1 specifies retraining on the same MFR split with the methods' original objectives and official configurations.

W4. Repeated AVS evaluation. As you suggested, we repeated AVS five times with paired seeds, extending this evaluation to all compared methods. Within each run, methods share initial views, candidate/evaluation pools, the 20-view budget, and optimization. DeepCR achieves the best mean PSNR, SSIM, and LPIPS on both datasets (Table 5). We report mean and SD across five run-level dataset means so that the size of the gains can be assessed alongside variability. These repeated measurements support the practical use of the quality estimates for acquisition; Section 4.4 describes the protocol.

W5. Input dimensions and SSIM computation. We clarify the sequence in Section 3.4 and Appendix B.1. At test time, antialiased bilinear interpolation resizes the short side to 518 while preserving aspect ratio, without additional input cropping or padding. Only complete 14-by-14 patches produce predictions: an H-by-W input yields a 14 floor(H/14)-by-14 floor(W/14) map. For example, 540-by-960 becomes 518-by-920, with a 518-by-910 output. Image-level SSIM is computed from the stored rendering/GT pair before input resizing. For pixel MAE, the precomputed SSIM map is clipped to [0,1], resized with the query's interpolation settings, and cropped from the top left to the predicted extent. We compare corresponding pixels without enlarging predictions or recomputing SSIM from resized RGB images. This defines both the reported full-resolution extent and the evaluation alignment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank the reviewers for their constructive feedback. We have clarified our methodological contribution and added experiments addressing spatial accuracy, reference selection, and downstream utility. Substantial manuscript additions are marked in blue.

  1. Methodological contribution and supporting controls. CrossScore predicts SSIM quality maps by applying cross-attention to DINOv2 features from a query and non-aligned references. However, CrossScore overlooks the pose relationship between reference and query views, relies on a single feature layer with limited representational capacity, and uses token-level prediction with limited ability to resolve fine spatial quality variations. These limitations motivate DeepCR's three complementary designs. We use normalized pose distances to prioritize relevant reference views, fuse query and reference features across layers 4/8/12 before attention to combine local detail with higher-level context, and decode the attended patch grid with convolutions and PixelShuffle to predict quality at pixel resolution. Auxiliary supervision supports spatial learning during training. Thus, although the two methods share DINOv2, cross-attention, and SSIM supervision, DeepCR changes the reference evidence, the representation used to associate views, and the spatial prediction head. To clarify component contributions beyond Tables 1 and 3, we added shared-base ablations (Table 2; Tables S5-S6). CrossScore trained with the same pose selector still yields lower correlations and higher pixel MAE on all three datasets (Table S7), showing that reference selection alone does not explain DeepCR's results. Related Work (Sec. 2.1) now links these limitations to DeepCR's designs; Sec. 4.3 and Appendix D distinguish the matched-selector predictor comparison from shared-base ablations and state their configuration differences.

  2. Spatial accuracy and human-perception validation. Our decoder combines neighboring patch features to recover local quality variation, which calls for pixel-level evaluation alongside image-level scores. We therefore added the no-decoder control to the pixel-MAE evaluation (Table 3). The full configuration reduces error on all three datasets relative to removing both decoding and auxiliary supervision; Table S6 separates their settings. To complement SSIM-based evaluation with direct evidence of perceptual agreement, we also extended the human-annotation evaluation to spatial artifact patterns. DeepCR achieves the highest spatial PLCC/SRCC point estimates (Table 4), complementing its within-scene image-level correlations. Together, these evaluations support more accurate spatial quality prediction and agreement with human-annotated quality variations on this benchmark.

  3. Reference relevance under different retrieval conditions. To evaluate pose selection against alternative retrieval strategies and under limited shared content, we added neighbor-selection controls, feature/depth-assisted retrieval comparisons, and evaluations on identical low-overlap query groups (Tables S8-S10). The neighbor controls yield similar results on the fixed camera pools, while pose selection improves over random references and performs comparably to feature retrieval overall. These results support the effectiveness of selecting relevant reference views across the evaluated benchmarks. Geometry retrieval achieves higher T&T correlations with additional depth maps; pose selection uses existing camera metadata. Its competitive results against feature retrieval within the low-overlap groups further support its practical use when shared content is limited.

  4. Practical utility and reproducibility. To evaluate the practical value and repeatability of quality-guided view acquisition, we added five paired-seed runs for all compared methods, using DeepCR without fine-tuning. We held the initial views, candidate/evaluation pools, acquisition budgets, and optimization settings identical across methods. DeepCR achieves the best mean PSNR, SSIM, and LPIPS on both datasets (Table 5), supporting the use of its quality estimates to guide reconstruction. Standard deviations across run-level dataset means quantify repeatability, complementing our primary IQA evaluation. For reproducibility and map interpretation, we clarified input resizing, complete-patch output support, and SSIM-target alignment (Sections 3.4/4.1 and Appendix B.1), and added a colorbar to Figure 4.

Notably, these results are obtained without access to the query's aligned ground-truth image, which we exclude from every reference pool. In this setting, SSIM cannot be computed directly; DeepCR instead estimates quality from other views of the scene. The spatial, human-annotation, and acquisition results together support the value of these estimates for assessment and reconstruction. Code will be made publicly available upon publication.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.