- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes RCGA, a learned gate that balances the frozen VIVID backbone's own correspondence against external geometric priors. The gate controls how much the geometric prior reshapes the backbone attention and is supervised by an oracle that compares geometric and backbone correspondence. RCGA improves over VIVID on RealEstate10K and ScanNet and is more robust to corrupted geometry than fixed injection.
Critical strengths
- The motivation and idea are clear. Explicit geometry often adds a small improvement to strong NVS backbones, and judging its value relative to the backbone's own correspondence is a reasonable fix.
- As a lightweight adapter on a frozen backbone, RCGA consistently outperforms VIVID and degrades much more gracefully than fixed injection when the geometry is corrupted. This shows that the gating is useful in practice.
Critical weaknesses
The paper's main claim is contradicted by its experiment. It attributes the limited gain of geometry to reliable and unreliable cues offsetting each other, yet with clean ground-truth geometry, naive injection still barely beats pose-only while RCGA clearly does, so the gain of RCGA may come from its update rule rather than from modeling reliability.
Questions for authors
- With clean ground-truth geometry, naive injection barely improves over pose-only. What causes this, if not unreliable cues? Does RCGA with a constant gate, i.e. the same update rule without reliability modeling, already recover most of the gain?
- How is the scale of the Depth Anything V2 depth aligned with the RealEstate10K poses before reprojection?
5 comments
I would like to thank the authors for their detailed rebuttal and additional experiments. Most of concerns are addressed. I have small question about features chosen as input: what motivates the choice of these seven statistics, e.g., what does add beyond ? Maybe those seven statistics provide redundant information. (No experiments required)
We thank the reviewer for this important question. Our central motivation is to determine whether geometric evidence can provide useful information beyond what the backbone has already learned. Accordingly, the seven statistics are selected to characterize complementary aspects of geometric reliability from three perspectives: the intrinsic correspondence state of the backbone, the characteristics of geometric evidence, and the agreement between the two.
Specifically, H(P_i) and Delta_top capture different aspects of backbone correspondence uncertainty: the former measures the overall dispersion of the correspondence distribution, while the latter measures the discriminability between the top two matching candidates. Similarly, H(P_g) characterizes the concentration of geometric correspondence, whereas C_g provides complementary information about geometric support. Finally, JS(P_i || P_g) measures the distributional discrepancy between backbone and geometric correspondence, while O_ig captures their support overlap. M_src describes the backbone's source-attention allocation, which is particularly relevant to our partition-preserving update.
We have recognized that these statistics are not necessarily statistically independent. To examine their complementary contributions, we conducted a leave-one-out ablation by individually masking each statistic while keeping the field prediction function unchanged (Appendix Table 6). Removing each individual statistic degrades performance, suggesting that none is entirely redundant given the remaining six. Notably, removing either agreement statistic causes a relatively larger degradation, highlighting the importance of explicitly modeling the relationship between backbone and geometric correspondence.
We thank the reviewer again for this helpful suggestion and the constructive discussion throughout the review process.
Thank you for this insightful and constructive commentt, which encouraged us to further investigate and better understand our experimental findings.
Response to Weakness and Question 1
To directly investigate whether RCGA's gains arise from its update rule or reliability modeling, we added a constant-gate PoE baseline to the controlled corruption analysis (Sec. 4.2, Fig. 6). It uses the same update rule as RCGA, differing only in that its reliability is fixed rather than dynamically predicted. The results are as follows:
The results show that RCGA's gains do not come solely from reliability modeling, but also from how geometric evidence is integrated with backbone attention. The contribution of the update rule remains relatively stable across corruption levels, whereas the additional gains from dynamic reliability increase as corruption becomes more severe. Importantly, even when all geometric correspondences are correct, dynamic reliability provides additional benefits.
This also explains why naive injection provides only marginal gains even with clean ground-truth geometry. Geometric correctness does not necessarily imply geometric utility. Even when all geometric correspondences are correct, they may provide redundant information that the backbone has already implicitly learned, rather than additional complementary evidence. A fixed injection strategy cannot distinguish between these cases or adapt the geometric influence to the varying requirements of different spatial queries and denoising stages. As illustrated by the reliability visualizations in Fig. 8, RCGA predicts substantially different geometric influence across spatial queries and denoising stages.
Therefore, it reflects two distinct limitations of native injection:
(1) Under heterogeneous geometric quality, the benefits of reliable correspondences can be offset by the negative effects of corrupted ones.
(2) Even with clean geometry, a fixed gate cannot fully exploit complementary geometric evidence according to its varying utility across queries and denoising states.
Response to Question 2
For RCGA-Depth, we follow VIVID's depth preprocessing procedure. Specifically, the predicted depth is inverted, normalized by its per-image maximum, and standardized using fixed mean and standard deviation statistics. As in VIVID, no additional metric-scale alignment with RealEstate10K camera translations is performed, since the monocular depth predictions are not up to scale with the dataset poses. The resulting depth is therefore used as an approximate geometric prior rather than metrically calibrated geometry, whose inherent uncertainty motivates our reliability-aware integration.
We sincerely thank the reviewer again for raising these important questions. They prompted us to conduct an additional controlled experiment that more clearly disentangles the contributions of geometric integration and reliability calibration, as well as to clarify the geometric assumptions underlying our depth-based correspondence construction. We believe these revisions provide a more precise explanation of the motivation and effectiveness of RCGA.