- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper presents a viewpose-then-Bayesian framework that leverages pretrained 3D foundation models for panoramic registration. It achieves substantial accuracy improvements, particularly under low-overlap conditions, with reasonable computational cost (~2 seconds per image pair) and without task-specific retraining.
Critical strengths
-
High Accuracy and Computational Efficiency: The method achieves substantial accuracy improvements under low-overlap conditions while maintaining reasonable computational cost (~2 seconds per image pair).
-
Training-Free Generalizability: The framework leverages pretrained 3D foundation models without task-specific retraining and demonstrates generalizability across different models and datasets.
Critical weaknesses
- Insufficient Justification of Method Necessity
- The proposed intra-panorama geometric consistency is not validated against actual pose accuracy. Since the sub-views have known relative extrinsics, it is unclear why the authors use reliability weighting instead of directly optimizing the transformation between two calibrated camera rigs using constrained optimization or bundle adjustment. The advantages over simpler geometric alternatives remain unverified.
- Experimental Fairness and Insufficient Validation of Key Contributions
-
Overlap Definition: Overlap is computed using different geometric information across datasets, but the paper does not clearly establish whether the resulting overlap ratios are equivalent. Moreover, the reported EDM results differ from those in the original paper, without sufficient clarification of the evaluation protocols and dataset splits.
-
Missing Baselines: PanoVGGT is included in Table 2 but omitted from Table 1 (Stanford2D3D) without explanation. The authors should also compare against recent training-based methods, particularly Argus (ECCV 2026), a strong panoramic reconstruction approach. Furthermore, the paper does not clarify whether PanoVGGT was retrained or evaluated using pretrained weights, nor does it specify the corresponding training data and settings.
-
Incomplete Ablation: The paper does not report the accuracy of the raw joint multi-view pose predictions. Consequently, it is difficult to determine whether the substantial performance gains originate from the foundation model's joint inference or the proposed reliability-aware Bayesian pose fusion.
Questions for authors
In addition to the concerns raised in the Weaknesses section, I have the following questions:
-
End-to-End Runtime: Does the reported ~2.03 s/pair include the entire pipeline, including VGGT inference, Bayesian pose estimation, and adaptive geometric verification? Please provide a module-wise runtime breakdown.
-
Effect of on Bayesian Estimation: Since the pose hypotheses share the same underlying view-pose predictions, how is the conditional independence assumption justified? How does increasing affect posterior uncertainty and confidence calibration, particularly when accuracy improvements saturate?
4 comments
Response to Reviewer zRuC (Part 2/3)
W2: Experimental Fairness and Insufficient Validation of Key Contributions?
A2: (i) Overlap Definition and EDM Results. We apologize for the confusion and respectfully clarify that we adopt a unified voxel-based overlap computation across both datasets. Specifically, Stanford2D3D directly provides ground-truth global 3D point maps, whereas Matterport3D does not provide corresponding ground-truth point maps. We therefore back-project its ground-truth depth maps into world-coordinate 3D point clouds using ground-truth camera poses. We then apply identical voxelization and shared-voxel overlap computation to both datasets, ensuring a consistent overlap definition. We have clarified these details in Appendix A.4 of the revised manuscript (marked in red). Since EDM does not publicly release its evaluation splits or dataset construction code, we re-evaluated EDM and other methods under our unified evaluation protocol to ensure fair comparisons, which explains the differences from previously reported scores.
(ii) Missing PanoVGGT/Argus Baselines and Unclear PanoVGGT Settings. For PanoVGGT, we respectfully clarify that we use the official model weights pretrained on Matterport3D, Stanford2D3D, Structured3D, and PanoCity. Since our Stanford2D3D evaluation follows the protocol of EDM and covers the full dataset, we exclude PanoVGGT from Table 1 to avoid training-evaluation overlap. In this rebuttal, we additionally evaluate PanoVGGT and PanoREG on Stanford2D3D Area5 (official test split) under the same evaluation protocol. As shown below, PanoREG consistently outperforms PanoVGGT across all overlap ranges.
| Overlap | Method | AUC@5 ↑ | AUC@10 ↑ | ACC@5 ↑ | ACC@10 ↑ |
|---|---|---|---|---|---|
| 0.1–0.3 (293 pairs) | PanoVGGT | 0.93 | 5.15 | 3.07 | 15.36 |
| PanoREG | 40.27 | 56.62 | 66.55 | 76.79 | |
| 0.3–0.5 (294 pairs) | PanoVGGT | 1.91 | 10.35 | 8.16 | 29.93 |
| PanoREG | 60.62 | 76.34 | 89.80 | 92.52 | |
| ≥0.5 (424 pairs) | PanoVGGT | 3.66 | 15.54 | 11.79 | 42.69 |
| PanoREG | 68.31 | 82.16 | 93.16 | 97.41 |
For Argus, we thank the reviewer for highlighting this interesting, solid work and have additionally evaluated it under the same evaluation protocol. The results are presented below (reported as AUC@5↑ / AUC@10↑ / AUC@20↑) and are also included in the revised manuscript (marked in red).
| Method | Stanford2D3D (0.1–0.3) | Matterport3D (0.1–0.3) | PanoCity |
|---|---|---|---|
| Argus | 40.73 / 62.14 / 77.86 | 40.52 / 61.40 / 73.59 | 58.91 / 73.35 / 85.31 |
| Ours | 56.61 / 74.40 / 84.41 | 46.72 / 66.49 / 79.61 | 69.47 / 83.26 / 90.84 |
(iii) Incomplete Ablation of Raw Joint Inference vs. Bayesian Fusion. We would like to respectfully clarify that Table 3(a) already evaluates raw joint VGGT predictions, with all variants using identical subview poses from the same joint 12-view inference. Specifically, we compare (v1) raw VGGT predictions using two forward-facing subviews, (v2) uniform averaging of all 36 cross-view pose hypotheses, and (v8) our reliability-aware Bayesian fusion over the same hypotheses. Under identical VGGT predictions, PanoREG improves AUC@5 by 129.0% (24.25 55.53) over (v1) and 34.6% (41.25 55.53) over (v2). These controlled comparisons demonstrate that the gains arise not merely from VGGT's inherent multi-view reasoning, but from our geometry-aware hypothesis construction and robust fusion. Further ablations in Tables 3(b) and 3(c) validate the complementary contributions of geometry-induced reliability modeling (+8.5%) and Cauchy-based robust inference (+10.5%).
Response to Reviewer zRuC (Part 3/3)
Q1: Module-wise Runtime?
A3: We clarify that the reported 2.03 s/pair includes the complete end-to-end pipeline. The module-wise breakdown is: image projection (0.76 s), VGGT inference (1.12 s), Bayesian pose estimation (0.034 s), and AGV (0.11 s). We have added this detailed runtime breakdown to Appendix A.5 of the revised manuscript (marked in red).
Q2-1: Justification of Likelihood Factorization.
A4: We thank the reviewer for raising this important point. We would like to respectfully clarify that explicitly modeling the joint likelihood of the pose hypotheses is challenging, as their shared view-pose predictions introduce complex statistical dependencies that are difficult to characterize. A well-established approach to handling such dependencies is composite likelihood (Varin et al., 2011 [1]), which combines individual likelihood contributions into a tractable estimation objective without requiring conditional independence among the observations:
where denotes the -th observation component, and is its associated weight, controlling its contribution to the composite likelihood. Following this principle, we formulate Eq. (7) as a reliability-weighted composite likelihood over the cross-view pose hypotheses:
This formulation allows us to aggregate evidence from all hypotheses for robust pose estimation without requiring them to be statistically independent. We have clarified this point in the revised manuscript (marked in red).
Q2-2: Effect of Increasing on Posterior Uncertainty and Confidence Calibration?
A5: (i) For Posterior Uncertainty: We respectfully clarify that the posterior uncertainty (covariance) is determined by the Hessian evaluated at the MAP estimate. Naively, increasing the number of hypotheses adds more terms to , potentially increasing its curvature and thereby shrinking the posterior uncertainty (covariance). Notably, benefitting from our reliability-weighted () mechanism and heavy-tailed Cauchy likelihood (see Eq. 5) whose second derivative (curvature) mathematically vanishes at large residuals (), unreliable and divergent hypotheses still contribute near-zero curvature to despite increasing , helping mitigate the posterior uncertainty from misleading shrinkage. In contrast, more reliable hypotheses (as increases) can properly contribute greater curvature to , appropriately shrinking the posterior uncertainty .
(ii) For Confidence Calibration: Building on the robustness mechanisms (reliability and heavy-tailed Cauchy) detailed above, the confidence does not blindly inflate simply because increases; rather, it only grows when hypotheses genuinely reach a tight geometric consensus, which is reasonable and expected. As shown in the table below, increasing from 6 to 10 nearly triples the number of pose hypotheses (from to ) on Stanford2D3D (overlap 0.1–0.3). Notably, while accuracy gains begin to saturate from to (AUC@5: 59.60 60.97), the mean confidence remains nearly unchanged (0.0675 0.0677). Meanwhile, the failure detection AUROC remains above 0.95, and the accepted failure rate drops to near zero (). Here, Failure Recall measures the proportion of severe failures (pose error exceeding ) identified for AGV refinement, while Accepted Failure measures the severe failure rate among pairs accepted without refinement. These results demonstrate that increasing improves pose accuracy without apparent overconfidence, even as accuracy gains saturate.
| AUC@5 ↑ | Mean | AUROC ↑ | Failure Recall ↑ | Accepted Failure ↓ | |
|---|---|---|---|---|---|
| 6 | 55.53 | 0.0611 | 0.9545 | 99.2% | 0.06% |
| 8 | 59.60 | 0.0675 | 0.9657 | 100.0% | 0.00% |
| 10 | 60.97 | 0.0677 | 0.9624 | 100.0% | 0.00% |
Reference
[1] An overview of composite likelihood methods. Statistica Sinica, 2011.
Dear Reviewer zRuC,
Thank you again for your time and consideration. With less than 4 hours remaining in the rebuttal period, we would like to kindly ask whether our follow-up response has adequately addressed your concerns. We will remain available until the rebuttal deadline and would be happy to provide any further clarifications if needed.
Best regards,
The Authors of Submission #57629
Response to Reviewer zRuC (Part 1/3)
W1: Justification of Method Necessity.
A1: We thank the reviewer for raising these two concerns.
(i) Does intra-panorama geometric consistency reflect pose accuracy? We directly evaluate the relationship between our geometric reliability scores and ground-truth pose errors on Stanford2D3D (0.1–0.3 overlap). The analysis yields a mean Spearman correlation of -0.73, with 97.71% of panorama pairs exhibiting negative correlations, supporting the use of intra-panorama consistency as a predictive reliability cue. Moreover, incorporating this reliability into the same robust fusion framework improves AUC@5 from 51.16 to 55.53 (Table 3(a) v5 vs. v8).
(ii) Why Reliability Weighting Rather Than Direct Rig Optimization? We respectfully clarify that, beyond merely enforcing rig constraints (viewi↔rig pose) through conventional rig optimization, PanoREG explicitly exploits intra-rig pairwise geometric compatibility (viewi↔viewj, see Eq. 3) to additionally assess the reliability of individual view-pose predictions. This yields complementary reliability cues for suppressing unreliable hypotheses during robust pose estimation. Notably, unlike conventional residual-driven weighting (e.g., IRLS), our geometric reliability is solution-independent and can be assessed prior to optimization, complementing the subsequent Cauchy-based robust consensus in our Bayesian estimator (Eq. 8). Our additional comparisons with direct rig optimization and bundle adjustment on Stanford2D3D (overlap 0.1–0.3), as shown below, further demonstrate the effectiveness of this design.