acceptodds
Under review as a conference paper at ICLR 2027

Learning Which Regions to Trust: Support-Aware Risk Ranking for Selective Novel View Synthesis"

10 views

Abstract

Novel-view synthesis (NVS) produces spatially heterogeneous errors, making it important to identify which regions of a rendered view can be trusted. We study spatially selective NVS, where regions within each view are ranked to minimise error at a prescribed coverage under a declared loss. Cross-view consistency provides strong spatial evidence when source observations are available, but becomes incomplete under occlusion, extrapolation, and limited view overlap. Moreover, the desired reliability ordering depends on the deployment objective: pixel, structural, and perceptual losses induce different regional rankings. We derive a pairwise decomposition of selective ranking error by source-view support, identifying where cross-view evidence is informative and where complementary evidence is needed. Guided by this analysis, we introduce L2R-GS, a target-conditioned learning-to-rank framework that combines external cross-view evidence, internal Gaussian-state evidence, and observability cues according to source support and the declared error objective. Across 11 scenes from the Mip-NeRF 360, Tanks and Temples, and Deep Blending datasets, L2R-GS lowers L1, DSSIM, and LPIPS AURC by 4.9%, 9.8%, and 15.8% relative to the strongest published methods on the same held-out views. On held-out views, pairs involving unsupported regions account for more of the ranking gain as source support weakens. A separate study with human annotations shows that training for perceived artifacts improves the selection of reliable regions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Novel View Synthesis, Uncertainty Estimation, Selective Prediction

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_FDTQOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies spatially selective novel-view synthesis: ranking the regions within a rendered view by how much they can be trusted, so that the least reliable regions can be excluded at a chosen coverage level. The proposed method combines cross-view consistency evidence with internal Gaussian-state evidence, weighting each according to how much reliable cross-view evidence is actually available for a given region, and trains this combination with a pairwise ranking objective. The approach builds on classical selective-prediction and learning-to-rank theory, adapted to a setting where some regions may have no cross-view correspondence at all.

Critical strengths

S1. The paper's theoretical framework — the risk-coverage/AURC formulation from the selective-prediction literature and the pairwise ranking objective from the learning-to-rank literature — is imported from established fields rather than newly invented. That said, the paper's transfer of this framework to Gaussian splatting is a relevant and non-trivial contribution in its own right, particularly in how it handles the cross-view correspondence problem: regions can lack any valid cross-view correspondence at all (rather than merely noisy correspondence), which motivates a three-way supported/unsupported pair taxonomy. Building on this, Proposition 2 goes beyond a design justification and yields a quantitative, falsifiable prediction of how much evidence fusion helps as a function of source-support availability — a prediction that is tested against held-out data and holds up well (r = 0.80–0.96 across targets). This combination of transferred theory with genuine empirical validation is a real strength of the paper.

Critical weaknesses

W1. The method's scope is narrow for a venue like ICLR: the paper demonstrates only a single downstream application (regional retention under a declared loss) and explicitly defers more substantial applications (acting on the ranking, active view selection, cross-regime adaptation, other radiance-field representations) to future work. The scope stays considerably narrower than comparable work that demonstrates the same kind of reliability score across several applications, which limits the significance of the contribution relative to the breadth expected at this venue.

W2. The evaluation omits the metrics standard in the closely related NVS-uncertainty literature (AUSE, Pearson correlation) in favor of AURC alone, without justifying this choice relative to prior work, and does not discuss how its choice of a comparatively large region size (64×64 pixels) relates to the granularity used elsewhere in the literature, despite the paper's own ablation suggesting sensitivity to this choice. Together, these gaps make it difficult to assess how the reported gains would translate under the evaluation conventions most readers coming from the uncertainty-quantification side of this literature would expect.

W3. Beyond the choice of training objective (a pairwise ranking loss, rather than the native objective used by PRIMU), the proposed approach is, in substance, closely similar to a combination of PRIMU and PPU. Its standalone methodological novelty beyond this change of objective and the addition of external cross-view evidence appears limited. Moreover, the gain over the embedded signal cannot yet be attributed to genuinely new evidence rather than to objective adaptation: PPU is used frozen with its original objective, and the ablation that would isolate the effect of retraining the signal under the ranking objective is missing. The ablation provided (Table 2) only re-weights PPU's frozen output and does not address this question.

W4. The paper is simply not well written, and extremely dense throughout the introduction, method, and experiments. This is not a minor stylistic issue: the degree of compression repeatedly prevents the reader from following the core argument and mechanics of the method at all, independent of any assessment of the method's substance.

Questions for authors

Q1. How does the proposed method compare to the baselines under the metrics standard in the NVS-uncertainty literature (AUSE, Pearson correlation between predicted uncertainty and error), rather than only AURC? Without this, it is unclear whether the reported advantage holds under more established evaluation conventions.

Q2. The embedded PPU signal is used frozen, with its own original training objective; only the linear combination layer on top is trained with the pairwise ranking loss. How would an end-to-end retrained version of that signal's underlying model, using the same pairwise ranking objective, perform relative to the full method? This would clarify whether the remaining gap to the embedded baseline reflects genuinely new evidence (external/ observability cues) or an objective-adaptation benefit the baseline itself has not been given access to.

Q3. Why was 64×64 chosen as the default region size, and how do the results relate to the region/patch granularities used in closely related work? The paper's own robustness check already shows a shrinking relative advantage at larger region sizes — would this trend continue at finer, e.g. pixel-level, granularity?

Reviewer_po4DOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies regional selection within novel-view renders under a declared error metric. L2R-GS combines ten frozen external, internal, and observability signals through a target-specific linear ranker with support-dependent interactions. A pairwise analysis motivates the treatment of missing cross-view evidence. Experiments report lower AURC across several reconstruction benchmarks, with additional analyses of support regimes and human-perceived artifacts.

Critical strengths
  • Clear and relevant problem formulation. The distinction between missing correspondence and high rendering error is useful, as is the explicit dependence on the evaluation target.
  • Coherent theoretical analysis. The pairwise regret decomposition and cross-support placement bound provide a sound framework for understanding regional ordering.
  • Substantial empirical work. The paper includes multiple reconstruction runs, scene-holdout evaluation, target swaps, and matched ablations. The disclosure of exploratory analyses and inconclusive results is appreciated.
Critical weaknesses
  1. The mechanistic validation is not fully independent of the observed results. Appendix C.6 states that view normalization was adopted after inspecting the original diagnostic correlations; for L1, the reported correlation increases from 0.23 to 0.80. The revised analysis has a reasonable motivation, but its stronger agreement needs independent confirmation under a frozen protocol. Appendix C.12 also identifies the evidence-role ablations as exploratory. These issues do not invalidate the unchanged raw-AURC comparisons, but they weaken the evidence supporting the explanatory claims.

  2. The distinctive contribution of the support-aware structure remains modest relative to the overall fusion benefit. Table 14 shows that removing both interactions increases AURC by approximately 0.8-1.4%, while boosted trees using the same ten signals achieve lower AURC. The controls establish a contribution from gating, often statistically significant, but do not make its practical importance across support regimes sufficiently clear. The paper should separate the benefits of supervised evidence fusion from those specifically attributable to the proposed support-dependent structure.

  3. Sensitivity to the tie-handling convention is insufficiently characterized. Appendix B.10 orders tied scores by decreasing true loss inside the evaluator. Although consistently applied, this worst-case convention may disproportionately penalize methods that assign identical scores to unsupported regions. Without tie frequencies and an alternative evaluation, it remains unclear how much of the comparative advantage depends on this convention rather than better ordering of distinguishable regions.

Questions for authors
  1. Independent confirmation: Can the normalized support-gain diagnostic be evaluated on genuinely untouched views or scenes, with normalization, binning, and category-rate estimation rules fixed beforehand? Please report both raw and normalized diagnostics and identify regimes where the prediction fails.

  2. Contribution attribution: Please compare the full linear model, the ungated linear model, and the matched boosted-tree model across different support regimes, reporting effect sizes and paired uncertainty estimates. Where do the proposed interactions provide a reproducible advantage?

  3. Tie sensitivity: Please report the frequency of tied scores for each method, particularly on unsupported regions, and repeat the main comparison using expected random tie ordering. Do the principal improvements remain?

  4. Scene-level isolation: Does scene holdout exclude the evaluated scene from signal-bank and hyperparameter selection, as well as coefficient fitting? Please clarify the full nesting procedure and provide strictly nested results if these choices used the held-out scene.

3 comments
AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear reviewer,

We would like to kindly follow up on our rebuttal and the additional experimental results provided. We would appreciate your feedback on whether these results address your main concerns or if any questions remain.

Thank you again for your time and consideration.

Thank you for the response and the additional experiments. The new results strengthen the evaluation. My remaining concerns are the size of the gain attributable to the support-aware design and the strength of the independent evidence for the proposed mechanism. For now, I’m keeping my rating.

Thank you for your time and thoughtful feedback. We appreciate your acknowledgment of the additional experiments and will carefully consider your remaining concerns regarding the support-aware contribution and its supporting evidence.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Follow-up on our Rebuttal

Dear reviewer,

We would like to kindly follow up on our rebuttal and the additional experimental results provided. We would appreciate your feedback on whether these results address your main concerns or if any questions remain.

Thank you again for your time and consideration.

Reviewer_cmcVOpenReview ↗

I will keep my score

Thank you for the additional experiments and clarifications. The results with random tie-breaking address my concern about whether the original tie rule affected the comparisons. The tests on 14 new scenes, which were not used to select the signals or develop and train the model, also make the results more convincing. Training PRIMU separately for each error metric makes the comparison fairer, and the experiments using fewer signals help explain the trade-off between accuracy and processing time.

The additional tests also provide stronger support for the theoretical prediction. I appreciate the clearer explanation of when this prediction works and where it becomes less accurate.

My main reservation is still the limited algorithmic novelty. The comparisons with MIM and NeuMiss are useful, but the proposed design adds only a small improvement over the simpler model using the same ten signals. Please clarify whether these methods used the same signals and training loss, and how their model sizes and settings were chosen.

Overall, the response addresses most of my concerns about the evaluation. My view of the novelty remains largely unchanged, and I maintain my weak acceptance recommendation.

Thank you for your thoughtful feedback and for acknowledging the additional experiments and clarifications. We appreciate your positive assessment of the strengthened evaluation and your continued support. We will carefully consider your remaining concerns regarding algorithmic novelty and the fairness of the MIM and NeuMiss comparisons.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.