- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Gate3R addresses the reliability of prior-conditioned feed-forward 3D reconstruction under inaccurate geometric priors. While camera intrinsics, relative pose, and depth can improve reconstruction, corrupted priors may degrade performance below image-only inference. The authors propose a training-free test-time prior-gating method that extracts two signals from a single conditioned forward pass: the prior residual and the attention entropy of a preselected head. These signals are standardized using deployment-batch statistics and combined through tail-weighted fusion. The method then discards the corresponding priors for a fixed fraction of view sets with the highest fused scores and reruns inference. Under controlled synthetic corruptions, Gate3R outperforms both always retaining and always discarding priors on eight of nine single-prior evaluation axes and all 27 multi-prior metric cells, averaged across test datasets.
Critical strengths
1. Practical relevance.
The paper treats whether to trust an input prior as an explicit decision problem, rather than assuming that additional geometric information is always beneficial. This is relevant to reconstruction pipelines that use outputs from other estimators as priors. Gate3R requires no updates to the reconstruction model’s parameters, facilitating integration with existing models.
2. Comprehensive controlled evaluation.
The experiments cover three prior types, five corruption levels, single- and multi-prior settings, and transfer from two to four views. All policies share the same corruption draws across ten random seeds, while attention heads are selected on a held-out dataset. This paired design helps reduce variability in comparisons.
3. Clear average gains over fixed policies.
For two-view MapAnything, Gate3R reduces intrinsics ray-direction error from 2.20° with all priors to 1.79° and achieves a pose AUC@5° of 36.1, compared with 28.4 for retaining all priors and 29.0 for image-only inference. The multi-prior results also show consistent improvements in aggregate.
Critical weaknesses
1. Insufficient Explanation of Attention Entropy. The paper provides limited insights into why attention entropy can indicate conflicts between geometric priors and visual evidence. Although empirical results demonstrate correlations between entropy changes and prior corruption, the underlying mechanism remains unclear. Such changes may simply result from perturbations in prior embeddings rather than genuine geometric inconsistencies. The current visualizations are insufficient to distinguish these possibilities. More interpretable analyses, such as layer-wise attention studies or geometric correspondence visualizations, would strengthen the justification for this approach.
2. Limited Evaluation Across Different Numbers of Views. The experiments are restricted to two-view and four-view reconstruction settings. It remains unclear whether Gate3R can generalize effectively to larger numbers of input views. Increasing the number of views may substantially change attention distributions and the aggregation of prior residuals. Consequently, the selected attention heads and gating strategy may behave differently in larger multi-view settings. Additional experiments with more diverse view counts would better demonstrate the scalability and robustness of the proposed method.
3. Limited Validation Under Real-World Prior Corruptions. The evaluation primarily relies on synthetically corrupted geometric priors, including predefined perturbations to camera intrinsics, poses, and depth. However, real-world prior estimation errors are often more complex and may exhibit spatially varying noise, systematic biases, or correlations across views. It is unclear whether Gate3R can reliably identify such naturally occurring errors. The current corruption protocol may not fully capture the challenges encountered in practical reconstruction pipelines. Experiments using priors estimated by real calibration, visual odometry, or depth estimation methods would strengthen the practical relevance of the approach.
4. Limitations of the Fixed Discard Budget. Gate3R adopts a fixed discard ratio of across different datasets and corruption settings. Although the authors provide budget ablations, their results indicate that the optimal discard ratio varies with the proportion of reliable priors. A fixed budget may unnecessarily discard accurate priors when corruption is rare or retain harmful priors when corruption is widespread. This limitation becomes particularly relevant when the corruption rate is unknown in real-world deployment. An adaptive gating strategy based on estimated prior reliability could potentially improve both reconstruction accuracy and computational efficiency.
5. Ambiguity in Attention Entropy and Head Selection. Attention entropy may reflect intrinsic scene difficulty or visual ambiguity rather than prior corruption alone. The paper does not sufficiently disentangle these factors when interpreting entropy changes as indicators of unreliable priors. Moreover, the attention heads are selected based on their clean-versus-corrupted detection AUC rather than their actual contribution to downstream reconstruction performance. A head with strong corruption detection capability does not necessarily provide the most effective gating decisions, particularly when mildly corrupted priors remain beneficial. Further analysis of these potential confounding factors and alternative head-selection criteria would strengthen the methodological justification.
Questions for authors
All my concerns have been detailed in the weaknesses section.
4 comments
Official Response by Authors – Part 2
Q5: Attention entropy may reflect scene difficulty rather than prior corruption, and heads are selected by detection AUC rather than by their contribution to reconstruction; can these factors be disentangled?
The entropy level does track scene difficulty (error of the image-only reconstruction), but the shift caused by a corruption largely does not, and detection is unchanged among sets of equal difficulty. We select heads by detection AUC because it needs no ground-truth geometry; mild corruptions still rank low under the fused score, so Gate3R keeps mildly corrupted priors, and selecting heads on any other dataset gives almost the same results.
Dear Reviewer Gcpk,
As the discussion period ends, we’re following up on our rebuttal since we haven’t heard back. We value your insights and would appreciate it if you could review our responses and let us know whether they address your concerns.
Best regards, All authors
Thank you for the additional experiments and clarifications. The expanded multi-view evaluation and experiments with real estimated priors substantially address my main empirical concerns. I will raise my rating.
Official Response by Authors – Part 1
We sincerely thank the reviewer for the detailed and constructive comments. We take every comment seriously and hope our response addresses the reviewer's concerns. If any questions remain, we are happy to address them.
The selected head appears to respond to prior–image disagreement. It separates corrupted poses from clean ones of the same rotation angle, whereas removing the prior altogether shifts entropy in the opposite direction or barely at all. Layer-wise, the signal is absent at the injection layer and in Pow3R's pose-blind encoder, growing only in deeper layers where visual evidence meets the prior, with all deployed heads in the network's second half (median head AUC, severe vs. clean, per depth quarter):
We evaluate 8 and 16 views on MapAnything with the two-view heads and ρ = 0.3 unchanged; the frozen heads' detection AUC rises with more views, and Gate3R beats both fixed policies on every axis (All / None / Gate3R):
We evaluate on real priors from AnyCalib (intrinsics), DROID-SLAM (pose), and UniDepthV2 (depth), with heads and scores unchanged, under two deployment modes: (i) a reference-free fixed budget, and (ii) an Adaptive threshold set at the 95% score quantile of clean reference sets from the target domain, which adapts the discard rate to the data. Across all models, ρ = 0.3 beats both fixed policies on pose and depth by catching estimator failures, and Adaptive gains further on pose. AnyCalib intrinsics are already accurate on MapAnything, so keeping all priors is best there, with Adaptive close behind:
When corruption is rarer (clean fractions up to 1/2), ρ = 0.3 still beats both fixed policies on all axes but Pow3R depth. When it is widespread (1/6 clean, the rest at the two most severe levels), ρ = 0.3 discards too few, but a larger budget or an Adaptive threshold beats both fixed policies on most axes. The Adaptive threshold replaces ρ with the 95% quantile of fused scores on clean reference sets from the target domain, so the discard rate follows the data; without such references, ρ = 0.3 remains the default: