acceptodds
Under review as a conference paper at ICLR 2027

PanoREG: From Pixel Matching to View Reasoning for Robust Panoramic Registration

Abstract

Conventional panoramic registration typically follows a match-then-RANSAC paradigm, establishing pixel-level association followed by RANSAC-style pose estimation. However, under challenging conditions such as low overlap and visual ambiguity, high-quality correspondences often become scarce, leading to limited robustness. This paper proposes PanoREG, a novel viewpose-then-Bayesian paradigm for robust, training-free panoramic registration. Unlike matching-dependent paradigm, our key idea is to recover the relative pose between two panoramas by associating and reasoning over pose estimates from their decomposed sub-views. Specifically, PanoREG comprises two core steps: calibrated view-pose association and robust Bayesian pose estimation. Calibrated view-pose association constructs multiple panorama-level pose hypotheses from sub-view pose predictions. To this end, we decompose each panorama into calibrated perspective views and employ a 3D foundation model to estimate per-view poses. Cross-panorama view-pose pairs are then associated through the predefined view-to-panorama extrinsics, yielding multiple noisy hypotheses of the panorama-level relative pose. Given these hypotheses, robust Bayesian pose estimation defines an outlier-robust Cauchy likelihood over them, and derives the posterior of the panorama pose via Bayesian inference, enabling robust maximum a posteriori (MAP) pose inference by suppressing highly inconsistent hypotheses. In particular, beyond pose estimation, we further exploit the posterior uncertainty to quantify the confidence of the MAP pose estimate, enabling confidence-controlled adaptive verification, in which reliable MAP estimates are directly accepted, whereas low-confidence cases trigger additional point-level geometric refinement. Extensive experiments on challenging panoramic benchmarks demonstrate that PanoREG substantially outperforms existing state-of-the-art methods, with particularly strong robustness under low-overlap conditions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Panoramic Registration, 3D Foundation Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zRuCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents a viewpose-then-Bayesian framework that leverages pretrained 3D foundation models for panoramic registration. It achieves substantial accuracy improvements, particularly under low-overlap conditions, with reasonable computational cost (~2 seconds per image pair) and without task-specific retraining.

Critical strengths
  1. High Accuracy and Computational Efficiency: The method achieves substantial accuracy improvements under low-overlap conditions while maintaining reasonable computational cost (~2 seconds per image pair).

  2. Training-Free Generalizability: The framework leverages pretrained 3D foundation models without task-specific retraining and demonstrates generalizability across different models and datasets.

Critical weaknesses
  1. Insufficient Justification of Method Necessity
  • The proposed intra-panorama geometric consistency is not validated against actual pose accuracy. Since the sub-views have known relative extrinsics, it is unclear why the authors use reliability weighting instead of directly optimizing the transformation between two calibrated camera rigs using constrained optimization or bundle adjustment. The advantages over simpler geometric alternatives remain unverified.
  1. Experimental Fairness and Insufficient Validation of Key Contributions
  • Overlap Definition: Overlap is computed using different geometric information across datasets, but the paper does not clearly establish whether the resulting overlap ratios are equivalent. Moreover, the reported EDM results differ from those in the original paper, without sufficient clarification of the evaluation protocols and dataset splits.

  • Missing Baselines: PanoVGGT is included in Table 2 but omitted from Table 1 (Stanford2D3D) without explanation. The authors should also compare against recent training-based methods, particularly Argus (ECCV 2026), a strong panoramic reconstruction approach. Furthermore, the paper does not clarify whether PanoVGGT was retrained or evaluated using pretrained weights, nor does it specify the corresponding training data and settings.

  • Incomplete Ablation: The paper does not report the accuracy of the raw joint multi-view pose predictions. Consequently, it is difficult to determine whether the substantial performance gains originate from the foundation model's joint inference or the proposed reliability-aware Bayesian pose fusion.

Questions for authors

In addition to the concerns raised in the Weaknesses section, I have the following questions:

  1. End-to-End Runtime: Does the reported ~2.03 s/pair include the entire pipeline, including VGGT inference, Bayesian pose estimation, and adaptive geometric verification? Please provide a module-wise runtime breakdown.

  2. Effect of KK on Bayesian Estimation: Since the K2K^2 pose hypotheses share the same underlying view-pose predictions, how is the conditional independence assumption justified? How does increasing KK affect posterior uncertainty and confidence calibration, particularly when accuracy improvements saturate?

4 comments
AuthorsOpenReview ↗

Response to Reviewer zRuC (Part 1/3)

W1: Justification of Method Necessity.
A1: We thank the reviewer for raising these two concerns.
(i) Does intra-panorama geometric consistency reflect pose accuracy? We directly evaluate the relationship between our geometric reliability scores and ground-truth pose errors on Stanford2D3D (0.1–0.3 overlap). The analysis yields a mean Spearman correlation of -0.73, with 97.71% of panorama pairs exhibiting negative correlations, supporting the use of intra-panorama consistency as a predictive reliability cue. Moreover, incorporating this reliability into the same robust fusion framework improves AUC@5 from 51.16 to 55.53 (Table 3(a) v5 vs. v8).
(ii) Why Reliability Weighting Rather Than Direct Rig Optimization? We respectfully clarify that, beyond merely enforcing rig constraints (viewi↔rig pose\mathrm{view}_i ↔ \mathrm{rig\ pose}) through conventional rig optimization, PanoREG explicitly exploits intra-rig pairwise geometric compatibility (viewi↔viewj\mathrm{view}_i ↔ \mathrm{view}_j, see Eq. 3) to additionally assess the reliability of individual view-pose predictions. This yields complementary reliability cues for suppressing unreliable hypotheses during robust pose estimation. Notably, unlike conventional residual-driven weighting (e.g., IRLS), our geometric reliability is solution-independent and can be assessed prior to optimization, complementing the subsequent Cauchy-based robust consensus in our Bayesian estimator (Eq. 8). Our additional comparisons with direct rig optimization and bundle adjustment on Stanford2D3D (overlap 0.1–0.3), as shown below, further demonstrate the effectiveness of this design.

MethodAUC@5° ↑AUC@10° ↑AUC@20° ↑Acc@5° ↑Acc@10° ↑
Rig Optimization36.9854.8468.0666.8676.64
Bundle Adjustment52.5566.9877.1277.5284.51
Ours55.5373.5983.8688.3093.09
AuthorsOpenReview ↗

Response to Reviewer zRuC (Part 2/3)

W2: Experimental Fairness and Insufficient Validation of Key Contributions?
A2: (i) Overlap Definition and EDM Results. We apologize for the confusion and respectfully clarify that we adopt a unified voxel-based overlap computation across both datasets. Specifically, Stanford2D3D directly provides ground-truth global 3D point maps, whereas Matterport3D does not provide corresponding ground-truth point maps. We therefore back-project its ground-truth depth maps into world-coordinate 3D point clouds using ground-truth camera poses. We then apply identical voxelization and shared-voxel overlap computation to both datasets, ensuring a consistent overlap definition. We have clarified these details in Appendix A.4 of the revised manuscript (marked in red). Since EDM does not publicly release its evaluation splits or dataset construction code, we re-evaluated EDM and other methods under our unified evaluation protocol to ensure fair comparisons, which explains the differences from previously reported scores.
(ii) Missing PanoVGGT/Argus Baselines and Unclear PanoVGGT Settings. For PanoVGGT, we respectfully clarify that we use the official model weights pretrained on Matterport3D, Stanford2D3D, Structured3D, and PanoCity. Since our Stanford2D3D evaluation follows the protocol of EDM and covers the full dataset, we exclude PanoVGGT from Table 1 to avoid training-evaluation overlap. In this rebuttal, we additionally evaluate PanoVGGT and PanoREG on Stanford2D3D Area5 (official test split) under the same evaluation protocol. As shown below, PanoREG consistently outperforms PanoVGGT across all overlap ranges.

OverlapMethodAUC@5 ↑AUC@10 ↑ACC@5 ↑ACC@10 ↑
0.1–0.3 (293 pairs)PanoVGGT0.935.153.0715.36
PanoREG40.2756.6266.5576.79
0.3–0.5 (294 pairs)PanoVGGT1.9110.358.1629.93
PanoREG60.6276.3489.8092.52
≥0.5 (424 pairs)PanoVGGT3.6615.5411.7942.69
PanoREG68.3182.1693.1697.41

For Argus, we thank the reviewer for highlighting this interesting, solid work and have additionally evaluated it under the same evaluation protocol. The results are presented below (reported as AUC@5↑ / AUC@10↑ / AUC@20↑) and are also included in the revised manuscript (marked in red).

MethodStanford2D3D (0.1–0.3)Matterport3D (0.1–0.3)PanoCity
Argus40.73 / 62.14 / 77.8640.52 / 61.40 / 73.5958.91 / 73.35 / 85.31
Ours56.61 / 74.40 / 84.4146.72 / 66.49 / 79.6169.47 / 83.26 / 90.84

(iii) Incomplete Ablation of Raw Joint Inference vs. Bayesian Fusion. We would like to respectfully clarify that Table 3(a) already evaluates raw joint VGGT predictions, with all variants using identical subview poses from the same joint 12-view inference. Specifically, we compare (v1) raw VGGT predictions using two forward-facing subviews, (v2) uniform averaging of all 36 cross-view pose hypotheses, and (v8) our reliability-aware Bayesian fusion over the same hypotheses. Under identical VGGT predictions, PanoREG improves AUC@5 by 129.0% (24.25 →\rightarrow 55.53) over (v1) and 34.6% (41.25 →\rightarrow 55.53) over (v2). These controlled comparisons demonstrate that the gains arise not merely from VGGT's inherent multi-view reasoning, but from our geometry-aware hypothesis construction and robust fusion. Further ablations in Tables 3(b) and 3(c) validate the complementary contributions of geometry-induced reliability modeling (+8.5%) and Cauchy-based robust inference (+10.5%).

AuthorsOpenReview ↗

Response to Reviewer zRuC (Part 3/3)

Q1: Module-wise Runtime?
A3: We clarify that the reported 2.03 s/pair includes the complete end-to-end pipeline. The module-wise breakdown is: image projection (0.76 s), VGGT inference (1.12 s), Bayesian pose estimation (0.034 s), and AGV (0.11 s). We have added this detailed runtime breakdown to Appendix A.5 of the revised manuscript (marked in red).

Q2-1: Justification of Likelihood Factorization.
A4: We thank the reviewer for raising this important point. We would like to respectfully clarify that explicitly modeling the joint likelihood of the K2K^2 pose hypotheses is challenging, as their shared view-pose predictions introduce complex statistical dependencies that are difficult to characterize. A well-established approach to handling such dependencies is composite likelihood (Varin et al., 2011 [1]), which combines individual likelihood contributions into a tractable estimation objective without requiring conditional independence among the observations: LCL(θ)=∏m=1MP(Dm∣θ)wm,wm≥0,\mathcal{L} _{\mathrm{CL}}(\theta) =\prod _{m=1}^{M} P(\mathcal{D} _m \mid \theta)^{w _m},\quad w _m \geq 0, where Dm\mathcal{D} _m denotes the mm-th observation component, and wm≥0w _m \geq 0 is its associated weight, controlling its contribution to the composite likelihood. Following this principle, we formulate Eq. (7) as a reliability-weighted composite likelihood over the cross-view pose hypotheses: LPanoREG(θ)=∏i=1K∏j=1KP(R^ij,t^ij∣R,t)πij.\mathcal{L} _{\mathrm{PanoREG}}(\theta) =\prod _{i=1}^{K}\prod _{j=1}^{K}P(\hat{R} _{ij}, \hat{t} _{ij} \mid R, t)^{\pi _{ij}}. This formulation allows us to aggregate evidence from all K2K^2 hypotheses for robust pose estimation without requiring them to be statistically independent. We have clarified this point in the revised manuscript (marked in red).

Q2-2: Effect of Increasing KK on Posterior Uncertainty and Confidence Calibration?
A5: (i) For Posterior Uncertainty: We respectfully clarify that the posterior uncertainty (covariance) Σ=(H+λI)−1\Sigma=(H+\lambda I)^{-1} is determined by the Hessian H=∑i,jπij∇2[−log⁡P(R^ij,t^ij∣R,t)]H=\sum _{i,j}\pi _{ij}\nabla^2[-\log P(\hat{R} _{ij}, \hat{t} _{ij}\mid R,t)] evaluated at the MAP estimate. Naively, increasing the number of hypotheses K2K^2 adds more terms to HH, potentially increasing its curvature and thereby shrinking the posterior uncertainty (covariance). Notably, benefitting from our reliability-weighted (πij\pi _{ij}) mechanism and heavy-tailed Cauchy likelihood (see Eq. 5) whose second derivative (curvature) mathematically vanishes at large residuals (∇2[−log⁡P]→0\nabla^2[-\log P]\rightarrow 0), unreliable and divergent hypotheses still contribute near-zero curvature to HH despite increasing KK, helping mitigate the posterior uncertainty from misleading shrinkage. In contrast, more reliable hypotheses (as KK increases) can properly contribute greater curvature to HH, appropriately shrinking the posterior uncertainty Σ\Sigma.
(ii) For Confidence Calibration: Building on the robustness mechanisms (reliability πij\pi _{ij} and heavy-tailed Cauchy) detailed above, the confidence qq does not blindly inflate simply because KK increases; rather, it only grows when hypotheses genuinely reach a tight geometric consensus, which is reasonable and expected. As shown in the table below, increasing KK from 6 to 10 nearly triples the number of pose hypotheses (from 626^2 to 10210^2) on Stanford2D3D (overlap 0.1–0.3). Notably, while accuracy gains begin to saturate from K=8K=8 to K=10K=10 (AUC@5: 59.60 →\rightarrow 60.97), the mean confidence qq remains nearly unchanged (0.0675 →\rightarrow 0.0677). Meanwhile, the failure detection AUROC remains above 0.95, and the accepted failure rate drops to near zero (≤0.06\leq 0.06\\%). Here, Failure Recall measures the proportion of severe failures (pose error exceeding 20∘20^\circ) identified for AGV refinement, while Accepted Failure measures the severe failure rate among pairs accepted without refinement. These results demonstrate that increasing KK improves pose accuracy without apparent overconfidence, even as accuracy gains saturate.

KKAUC@5 ↑Mean qqAUROC ↑Failure Recall ↑Accepted Failure ↓
655.530.06110.954599.2%0.06%
859.600.06750.9657100.0%0.00%
1060.970.06770.9624100.0%0.00%

Reference
[1] An overview of composite likelihood methods. Statistica Sinica, 2011.

Dear Reviewer zRuC,

Thank you again for your time and consideration. With less than 4 hours remaining in the rebuttal period, we would like to kindly ask whether our follow-up response has adequately addressed your concerns. We will remain available until the rebuttal deadline and would be happy to provide any further clarifications if needed.

Best regards,
The Authors of Submission #57629

Reviewer_dwi2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work presents PanoREG, a training-free framework for panoramic relative pose estimation. It reformulates pixel-level matching into view-level pose reasoning. In this approach, no task-specific retraining is required. To obtain better view pose estimates, a frozen 3D foundation model is used for view pose prediction. Experiments demonstrate that the approach achieves superior performance across multiple open datasets.

Critical strengths
  1. A frozen 3D foundation model is used for view pose prediction, which leads no task-specific retraining.
  2. Experiments demonstrate that the approach achieves superior performance across multiple open datasets.
Critical weaknesses
  1. The core concept of decomposing a panorama into multiple sub-views and inputting them into VGGT/PanoVGGT is already implemented by PanoVGGT itself. Since VGGT possesses strong multi-view geometric reasoning capabilities, it is unclear whether the performance improvements result from the foundation model's joint inference or the proposed fusion strategy.

  2. All three datasets (Stanford2D3D, Matterport3D, and PanoCity) consist of indoor scene panoramas. There is no evaluation on outdoor street panoramas, nor any comparison with existing outdoor panorama registration benchmarks. Additionally, no tests have been conducted under conditions such as motion blur, low light, or occlusion.

  3. In Appendix A.7, the authors discuss the failure cases qualitatively but do not provide any figures to illustrate these failures. As a result, it is difficult to understand the algorithm's boundaries.

Questions for authors
  1. The authors should include ablation experiments to isolate and evaluate the individual contributions of VGGT joint multi-view inference and the modules proposed in this paper.

  2. How do the authors ensure the reliability of posterior uncertainty and confidence estimates derived from pose hypotheses?

4 comments
AuthorsOpenReview ↗

Response to Reviewer dwi2 (Part 1/2)

W1&Q1: Are the gains from VGGT/PanoVGGT joint inference or the proposed fusion? Can ablations disentangle their contributions?
A1: We thank the reviewer for the insightful comment.
(i) Methodological Distinction. Despite the powerful geometric reasoning capability of VGGT-like foundation models, their predicted subview poses may remain unreliable under challenging conditions, such as low overlap. Beyond VGGT's learned multi-view reasoning (we note that PanoVGGT directly processes panoramas without subview decomposition), our PanoREG explicitly exploits intrinsic panoramic geometry for viewpose-then-Bayesian multi-view pose reasoning and robust matching through three complementary components: 1) geometry-calibrated view-pose association to construct structured cross-view pose hypotheses, 2) intra-panorama geometric self-consistency to derive analytical reliability priors, and 3) reliability-aware robust Bayesian fusion to suppress inconsistent hypotheses.
(ii) Experimental Validation. To disentangle these contributions from VGGT's inherent joint inference, we emphasize that all variants in Table 3(a) of our manuscript use identical subview pose predictions from the same joint 12-view VGGT inference. Specifically, we compare (v1) raw VGGT predictions using two forward-facing subviews, (v2) uniform averaging of all 36 cross-view pose hypotheses, and (v8) our reliability-aware Bayesian fusion over the same hypotheses. Under identical VGGT predictions, PanoREG improves AUC@5 by 129.0% (24.25 →\rightarrow 55.53) over v1 and 34.6% (41.25 →\rightarrow 55.53) over v2, demonstrating the overall benefits of our geometry-aware inference framework and the additional gains from reliability-aware Bayesian fusion, respectively. Further ablations in Tables 3(b) and 3(c) validate the complementary contributions of geometry-induced reliability modeling (+8.5%) and Cauchy-based robust inference (+10.5%).
These controlled comparisons demonstrate that PanoREG's improvements stem not merely from VGGT's joint inference, but from explicitly constructing and robustly fusing geometry-calibrated cross-view pose hypotheses.

W2: Lack of Outdoor and Challenging-Scene Evaluation?
A2: (i) Outdoor Evaluation. We would like to respectfully clarify that Table 2 already reports results on PanoCity, a public outdoor panoramic benchmark comprising 3,714 evaluation pairs across 100 urban blocks, covering diverse urban environments with streets, buildings, and complex urban structures. On PanoCity, PanoREG achieves an AUC@5 of 69.46, substantially outperforming PanoVGGT (33.63) and EDM (7.90), demonstrating its strong zero-shot generalization to outdoor urban scenes.
(ii) Challenging Conditions. We note that our tested Matterport3D benchmark itself contains many challenging cases involving motion blur and low illumination. Our results in Table 2 already demonstrate the robustness of PanoREG, an AUC@5 of 46.38, substantially outperforming EDM (26.50). In this rebuttal, we additionally construct three challenging subsets from Matterport3D, covering synthetic occlusion, low brightness, and motion blur. PanoREG consistently outperforms competing methods across all three subsets, benefiting from our calibrated view-pose association and robust Bayesian pose estimation with adaptive geometric verification. We have included these additional quantitative results and qualitative visualizations in Appendix A.8 of the revised manuscript (marked in red).

Challenging ConditionMetric ↑SphereGluePanoSplatt3RG2GPanoVGGTEDMPanoREG
OcclusionAUC@58.204.352.231.8712.9049.53
AUC@1014.838.714.949.1125.2170.75
Low LightAUC@546.8048.0412.083.5052.6661.03
AUC@1065.8565.7826.0314.1875.2380.74
Motion BlurAUC@51.874.421.061.4715.0130.14
AUC@103.668.042.906.0828.4751.63
AuthorsOpenReview ↗

Response to Reviewer dwi2 (Part 2/2)

W3: Lack of Failure Case Visualization.
A3: We thank the reviewer for the suggestion and have added representative failure-case visualizations in Appendix A.7 of the revised manuscript (marked in red), highlighting challenging scenarios with extremely limited overlap or textureless regions to illustrate PanoREG's limitations.

Q2: How is the reliability of posterior uncertainty and confidence estimates ensured?
A4: We thank the reviewer for this important question. We clarify the underlying mechanisms that support reliable uncertainty quantification and confidence estimation:
(i) Reliability of Posterior Uncertainty. We note that our Laplace-based uncertainty is derived from a geometry-aware robust posterior, where two complementary mechanisms can effectively enhance the reliability of posterior uncertainty estimation: 1) geometry-induced reliability priors πij\pi_{ij} downweight hypotheses originating from geometrically inconsistent subview predictions, reducing their influence on posterior curvature and the resulting covariance (i.e., uncertainty); 2) the heavy-tailed Cauchy likelihood suppresses large-residual hypotheses, limiting their influence on local posterior curvature. Consequently, the resulting covariance primarily reflects constraints supported by geometrically reliable and mutually consistent hypotheses, rather than being dominated by unreliable predictions, thereby enhancing the reliability of posterior uncertainty.
(ii) Reliability of Confidence Estimates. Building on the robust posterior uncertainty estimation described above, we further derive confidence from the estimated posterior covariance by integrating the local posterior probability mass within prescribed pose tolerances (see Eq. 10). Thus, stronger and more consistent geometric constraints produce a more concentrated posterior and higher confidence, whereas weakly constrained estimates yield lower confidence.
(iii) Empirical Reliability Evaluation: Table 4 shows that Acc@5 increases from 19.60 (q<0.005q<0.005) to 99.51 (q≥0.05q\geq0.05) across confidence levels, demonstrating a strong correlation between estimated confidence and actual pose accuracy. Moreover, the lowest-confidence 25.94% of pairs capture 99.2% of severe failures, demonstrating reliable failure detection.
(iv) Practical Effectiveness. Confidence-guided verification selectively refines only 25.94% of panorama pairs, improving overall AUC@5 from 55.53 to 56.61 (Table 4, η=0.03\eta=0.03), demonstrating the effectiveness of selective refinement without processing all pairs. Notably, on these refined pairs, AUC@5 increases from 31.27 to 35.42 (+13.26%), demonstrating that our confidence estimates effectively identify unreliable predictions where targeted geometric verification yields substantial accuracy gains.

The authors addressed some of my concerns. So I maintain my original ranking.

Dear Reviewer dwi2,

Thank you for your response. We are glad that our clarification has addressed some of your concerns. We remain available to address any further concerns or provide additional clarifications before the rebuttal deadline.

We sincerely appreciate your valuable time, effort, and insightful feedback!

Best regards,
The Authors of Submission #57629

Reviewer_qu2wOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents PanoREG, a method for estimating the relative pose between two panoramic images. It projects each panorama into six perspective subviews, uses a frozen 3D foundation model to predict poses, assesses the reliability of these predictions using known geometric relationships among subviews of the same panorama, and robustly fuses the resulting cross-panorama pose hypotheses. The motivation is clear: pixel-level matching can become unreliable when the panoramas have little overlap.

Critical strengths
  1. The reliability assessment is well motivated. Subviews from the same panorama share a camera center and have known relative orientations. Using these constraints to assess pose predictions is more principled than directly averaging the model outputs.
  2. The low-overlap results are compelling. On Stanford2D3D [3], the reported AUC@5 for the 0.1–0.3 overlap range is substantially higher than that of EDM [2]. The paper also includes ablations of the fusion strategy.
  3. The method has practical value. It does not require retraining the foundation model on panoramic data.

[1] Eder et al., Tangent Images for Mitigating Spherical Distortion, CVPR 2020. [2] Jung et al., EDM: Equirectangular Projection-Oriented Dense Kernelized Feature Matching, CVPR 2025. [3] Armeni et al., 3D Semantic Parsing of Large-Scale Indoor Spaces, CVPR 2016.

Critical weaknesses
  1. The scope of the methodological contribution needs clarification. Projecting panoramas into perspective views has precedent [1], while robust pose averaging and Laplace approximation are established techniques. The authors should state more precisely whether the main contribution lies in constructing cross-view pose hypotheses, modeling their reliability, or combining these components into an inference pipeline. The characterization of the approach as a shift from “match-then-RANSAC” to “viewpose-then-Bayesian” also needs a more specific comparison with prior work to avoid overstating the distinction.
  2. The view coverage may leave blind spots. The six subviews are sampled only along the horizontal direction. The paper does not clearly specify their field of view or evaluate cases involving large pitch or roll differences, or cases in which the shared visible content lies near the top or bottom of the panoramas.
  3. The dataset construction is insufficiently specified. The Stanford2D3D experiments describe the principles of pairing images from the same room and grouping pairs by geometric overlap, but do not specify how valid points, occlusions, geometric tolerances, or the directionality of overlap are handled. These details matter for reproducibility.

[1] Eder et al., Tangent Images for Mitigating Spherical Distortion, CVPR 2020. [2] Jung et al., EDM: Equirectangular Projection-Oriented Dense Kernelized Feature Matching, CVPR 2025. [3] Armeni et al., 3D Semantic Parsing of Large-Scale Indoor Spaces, CVPR 2016.

Questions for authors
  1. What is PanoREG’s core novel contribution relative to existing methods for perspective projection of panoramas and robust pose fusion? Could a more direct comparison clarify the role of each component?
  2. What is the field of view of the six subviews? How does the method perform when shared content is concentrated near the top or bottom of the panoramas, or when the images differ substantially in pitch or roll?
  3. Could the authors specify the detail and rule used to construct benchmark based on Stanford2D3D .
  4. What translation scale is used for the odometry error, reported in meters, in Table 3(d)?
6 comments
AuthorsOpenReview ↗

Response to Reviewer qu2w (Part 1/3)

Q1: The Scope of the Methodological Contribution Needs Clarification.
A1: We thank the reviewer for this insightful comment and would like to respectfully clarify our methodological contributions along four key dimensions:
(i) Paradigm Innovation. Unlike conventional match-then-RANSAC pipelines (e.g., EDM [2]) and panoramic foundation models requiring panorama-specific training (e.g., PanoVGGT [3]), PanoREG establishes a novel viewpose-then-Bayesian paradigm that shifts the fundamental inference unit from pixel correspondences to calibrated cross-view pose hypotheses. This paradigm bridges pretrained perspective 3D foundation models with spherical registration, enabling training-free panoramic pose estimation without task-specific retraining. Table 1 shows that PanoREG achieves an AUC@5 of 55.53 on Stanford2D3D (overlap 0.1–0.3), substantially outperforming conventional match-then-RANSAC methods, including EDM (9.13), SphereGlue (3.34), SPHORB (1.31), and TORB (0.61).
(ii) Geometry-Calibrated Hypothesis Construction and Reliability Modeling. Beyond conventional perspective projection for distortion-reduced pixel matching (e.g., Tangent Images [1]), PanoREG further exploits the intrinsic geometric constraints among projected subviews to construct structured cross-view pose hypotheses (2K2K view poses →\rightarrow K2K^2 relative pose hypotheses). Crucially, these same geometric constraints further enable the derivation of closed-form reliability priors from intra-panorama geometric inconsistencies, providing a training-free assessment of cross-view hypothesis reliability.
(iii) Reliability-Aware Robust Bayesian Inference. Unlike conventional robust pose averaging that primarily relies on residual-based consensus, PanoREG explicitly integrates geometry-induced reliability priors with a heavy-tailed Cauchy likelihood, jointly exploiting intra-panorama geometric self-consistency and cross-panorama hypothesis agreement to suppress unreliable predictions. This complementary geometric reasoning distinguishes our formulation from standard robust averaging and substantially improves estimation robustness (AUC@5: 41.25 →\rightarrow 55.53 in Table 3).
(iv) Uncertainty-Guided Adaptive Geometric Verification. While the Laplace approximation itself is established, our contribution lies in transforming posterior uncertainty into an actionable criterion for adaptive geometric verification. Specifically, PanoREG couples pose-level Bayesian inference with selective point-level verification, triggering refinement only for uncertain panorama pairs. This uncertainty-driven inference-to-verification mechanism further improves AUC@5 from 55.53 to 56.61 (Table 1) with limited computational overhead (0.11 s/pair).

AuthorsOpenReview ↗

Response to Reviewer qu2w (Part 2/3)

Q2: What is the Subview FOV? Robustness to Top/Bottom Region Overlap? Robustness to Large Pitch/Roll Variations?
A2: (i) About Subview Field of View. We clarify that, following prior work [4, 5], PanoREG projects six perspective subviews, each with a 100∘100^\circ FOV in both horizontal and vertical directions, uniformly spaced at 60∘60^\circ azimuth intervals. Importantly, our model also flexibly supports diverse perspective projection schemes, such as increasing the number of subviews (see Table 5 in the appendix) or extending coverage to the remaining vertical FOV regions by incorporating two additional polar subviews (top/bottom regions). As shown in the table below, adding top/bottom polar subviews (2nd row) maintains strong performance on Stanford2D3D (overlap 0.1–0.3), although the performance is slightly lower than that of our default configuration, possibly because polar regions typically provide weaker geometric cues. We have included these results and discussion in Appendix A.5 of the revised manuscript (marked in red).

MethodACC@5 ↑ACC@10 ↑AUC@5 ↑AUC@10 ↑AUC@20 ↑
EDM [2] (SOTA)21.6542.599.1320.9235.73
Eight-View (w/ Polar Views)81.3587.2653.4969.0179.87
Six-View (Original)89.1893.6356.6174.4084.41

(ii) About Robustness to Top/Bottom Overlap: Following your suggestion, we construct a challenging subset of 200 panorama pairs from PanoCity, where over 70% of the shared visual content is concentrated in the top/bottom regions. We evaluate PanoREG with both the original six-view and extended eight-view configurations (including two additional polar subviews), alongside the SOTA method EDM. As shown below, while the pixel matching-based EDM fails completely, our six-view PanoREG remains effective, benefiting from reliability-aware robust Bayesian estimation. Moreover, by capturing additional overlapping content in the polar regions, the eight-view configuration achieves the best performance.

MethodACC@5 ↑ACC@10 ↑AUC@5 ↑AUC@10 ↑AUC@20 ↑
EDM [2] (SOTA)0.000.000.000.000.00
Eight-View (w/ Polar Views)11.5036.004.0214.1136.08
Six-View (Original)8.0033.503.5511.8831.31

(iii) About Robustness to Large Pitch/Roll Variations. Following your suggestion, we further construct challenging test cases by applying 60∘60^\circ pitch and roll rotations to panorama pairs from Stanford2D3D. As shown below, both PanoREG's default six-view and extended eight-view configurations demonstrate strong robustness to large orientation variations, substantially outperforming the prior SOTA method EDM [2].

MethodPitch 60° AUC@5 ↑Roll 60° AUC@5 ↑
EDM [2] (SOTA)0.060.04
Eight-View (w/ Polar Views)10.699.55
Six-View (Original)10.009.36
AuthorsOpenReview ↗

Response to Reviewer qu2w (Part 3/3)

Q3: How is the Stanford2D3D benchmark constructed?
A3: We respectfully clarify that we follow the general evaluation protocol of EDM [2], which uses geometric overlap to construct panorama pairs on Stanford2D3D. The specific implementation details of overlap computation are as follows: (i) About Valid Points: For each panorama, we use its corresponding ground-truth XYZ point map and retain points x∈R3\mathbf{x}\in\mathbb{R}^3 with finite coordinates and ∥x∥2>10−6\|\mathbf{x}\|_2>10^{-6}, and randomly sample up to 200,000 valid points per panorama using a fixed random seed; (ii) About Occlusions: We compute overlap using only the observed 3D surfaces from each panorama's ground-truth XYZ map, thereby accounting for viewpoint-dependent occlusions in the observed geometry; (iii) About Geometric Tolerance: We quantize valid points into 5 cm voxels in the shared world coordinate system and consider voxels with identical indices to be overlapping; (iv) About Overlap Directionality: We define overlap as O(P,Q)=∣VP∩VQ∣min⁡(∣VP∣,∣VQ∣)O(P,Q)=\frac{|V_P\cap V_Q|}{\min(|V_P|,|V_Q|)}, where VPV_P and VQV_Q are the voxel sets of the two panoramas, ensuring symmetry regardless of image ordering; (v) Evaluation Fairness: All methods are evaluated under the same protocol. We have provided these construction details in Appendix A.4 (marked in red) and will publicly release our benchmark.

Q4. About Translation Scale in Odometry Evaluation.
A4: We respectfully clarify that, following the evaluation practices of prior methods [6,7], we use the translation directions predicted by PanoREG as unit-length steps to construct the estimated trajectory, and subsequently apply a global scale factor obtained through Sim(3) alignment with the ground-truth trajectory. This enables the computation of translation errors in metric units, as reported in Table 3(d).

References
[1] Tangent Images for Mitigating Spherical Distortion, CVPR 2020.
[2] EDM: Equirectangular Projection-Oriented Dense Kernelized Feature Matching, CVPR 2025.
[3] PanoVGGT: Feed-Forward 3D Reconstruction from Panoramic Imagery, CVPR 2026.
[4] Urban Visual Modeling and Tracking, Book Chapter, 2015.
[5] Co-registration of Panoramic Mobile Mapping Images and Oblique Aerial Images, The Photogrammetric Record, 2019.
[6] DeepFEPE: Deep Keypoint-Based Camera Pose Estimation with Geometric Constraints, IROS 2020.
[7] PanoPose: Self-supervised Relative Pose Estimation for Panoramic Images, CVPR 2024.

Comments on the paper

1 comment

Dear PCs, SACs, ACs, and all of our reviewers,

We sincerely thank you for your thorough reviews and valuable feedback. Your comments and discussions have greatly helped us improve our work.

We appreciate the reviewers' recognition of our paper's strengths, including the following:

  • Clear Motivation (Reviewer qu2w): Reviewer qu2w found our reliability assessment “well motivated” and “more principled than directly averaging the model outputs”.

  • Training-Free Generalizability and Efficiency (Reviewers qu2w, dwi2, zRuC): Reviewers highlighted "practical value", generalization across models and datasets without task-specific retraining, and “reasonable computational cost”.

  • Strong Low-Overlap Performance (Reviewers qu2w, dwi2, zRuC): Reviewers dwi2 and zRuC recognized superior performance across datasets and substantial gains under low overlap, while Reviewer qu2w described our low-overlap results as "compelling".

We have also carefully addressed the reviewers' concerns throughout the discussion period, with key points summarized below:

  • In response to questions regarding improvements over VGGT’s joint inference (Reviewers dwi2, zRuC): Table 3(a) provides comparisons under consistent settings, showing that PanoREG substantially improves AUC@5 over raw VGGT predictions (24.25 → 55.53, +129.0%) and uniform averaging (41.25 → 55.53, +34.6%), highlighting the clear advantages of our proposed framework.
  • Regarding method necessity over direct rig optimization (Reviewer zRuC): Unlike conventional view-to-rig optimization, PanoREG additionally exploits pairwise view-to-view consistency to assess pose reliability before optimization, significantly outperforming direct rig optimization (55.53 vs. 36.98 AUC@5). The reliability scores also correlate strongly with pose errors (Spearman ρ = −0.73).
  • Subview coverage and viewpoint robustness (Reviewer qu2w): Additional tests with polar views, 60° pitch/roll rotations (AUC@5: 10.00/9.36 vs. EDM’s 0.06/0.04), and zero shared overlap within six-view coverage (3.56 vs. EDM’s 0.00) support robustness across subview configurations and our effective reasoning under limited geometric observations.

Finally, we would like to highlight the four key contributions of our work: (i) A novel viewpose-then-Bayesian paradigm that shifts panoramic registration from fragile pixel matching to robust view-level pose reasoning; (ii) A geometry-grounded probabilistic registration framework integrating calibrated cross-view pose association, intra-panorama reliability priors, robust Cauchy-based Bayesian fusion, and posterior uncertainty-guided adaptive verification; (iii) Training-free and backbone-agnostic generality, enabling zero-shot panoramic registration across diverse frozen 3D foundation models (VGGT, π3\pi^3, VGGT-Ω\Omega); and (iv) Substantial empirical improvements over state-of-the-art methods across indoor/outdoor benchmarks and downstream visual odometry, particularly under extreme low overlap (AUC@5: 56.61 vs. 9.13 for EDM).

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.