acceptodds
Under review as a conference paper at ICLR 2027

CoVisFormer: Co-Visibility-Aware Correspondence Learning for Cross-View Geo-Localization

Abstract

Estimating 3-DoF pose between a ground image and an aerial image is vital for a variety of computer vision applications. However, achieving robust cross-view geo-localization remains challenging because the limited cross-view interaction results in erroneous matches in non-overlapping regions, and fixed rules for inlier selection limit the effective rejection of outliers. To deal with these issues, a novel cross-view geo-localization method is proposed by integrating cross-view interaction with dynamic inlier detection, including a co-visibility-aware cross-view contextual feature learning (C3FL) module and a dynamic inlier detection-based pose regression (DIDP) module. The proposed CoVisFormer enjoys several merits. First, the proposed C3FL module can leverage cross-view interaction to focus matching on overlapping regions. % Second, the DIDP module can adaptively update according to the current input, enabling effective outlier rejection. % Extensive experimental results on two challenging benchmarks show that our proposed method significantly outperforms state-of-the-art cross-view geo-localization methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
cross view geo-localization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VxydOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoVisFormer, a cross-view geo-localization framework for estimating 3-DoF camera pose from ground and aerial images. The method addresses two main issues in existing correspondence-based approaches: false matches in non-overlapping regions and the limited adaptability of fixed-rule inlier selection. To this end, the authors introduce a co-visibility-aware cross-view contextual feature learning (C3FL) module, which uses learnable prompts to facilitate cross-view feature interaction, and a dynamic inlier detection-based pose regression (DIDP) module, which derives adaptive inlier detectors from the current correspondence set for pose estimation. The framework is evaluated on the VIGOR and KITTI benchmarks and achieves strong improvements over existing methods, particularly under challenging unknown-orientation settings. Ablation studies further show that both C3FL and DIDP contribute to the overall performance, and additional experiments support the effectiveness of the prompt-based attention design.

Critical strengths

1.The paper addresses two practically important limitations in correspondence-based cross-view geo-localization, namely false matches in non-overlapping regions and the limited adaptability of fixed-rule inlier selection. The proposed C3FL and DIDP modules are clearly motivated by these issues and form a coherent overall framework. 2. The experimental results are strong and reasonably comprehensive. CoVisFormer achieves substantial improvements over existing methods on both VIGOR and KITTI, particularly under challenging unknown-orientation settings, while the ablation studies provide clear evidence that both C3FL and DIDP contribute to the final performance.

Critical weaknesses
  1. The methodological novelty is somewhat limited. Both C3FL and DIDP are largely built upon familiar prompt/query-based attention and prototype-based set processing mechanisms. Although these components are reasonably adapted to cross-view geo-localization, the paper does not clearly establish how they differ fundamentally from existing latent-token, learnable-query, or learned correspondence filtering approaches. This makes the contribution appear stronger at the task-specific integration level than at the level of a genuinely new learning mechanism.
  2. The central claims of co-visibility awareness and dynamic inlier detection are not supported by sufficiently direct analysis. C3FL is mainly validated through final localization performance and qualitative attention maps, while DIDP is primarily compared against fixed-rule RANSAC-based selection. More direct correspondence-level evidence, such as false-match rates in non-overlapping regions, inlier precision/recall, or comparisons with representative learned outlier rejection methods, would be important to substantiate the claimed mechanisms.
Questions for authors
  1. How does the proposed C3FL differ fundamentally from existing learnable-query, latent-token, or prompt-based cross-attention mechanisms? Clarifying this distinction would help assess the methodological novelty of the paper.
  2. Can the authors provide more direct quantitative evidence that C3FL specifically improves co-visibility modeling, for example by reducing false correspondences in non-overlapping regions or improving correspondence precision/inlier ratio?
  3. How does DIDP differ from existing learning-based correspondence filtering or outlier-rejection approaches? A clearer conceptual comparison, and if available, experimental evidence against representative learned filtering methods, would materially affect the assessment of its novelty and necessity.
1 comment
AuthorsOpenReview ↗

Clarification of the Methodological Novelty and Mechanistic Effectiveness of C3FL and DIDP

We sincerely thank the reviewer for the insightful and constructive suggestions, which have helped us better clarify the methodological contributions of C3FL and DIDP and strengthen the validation of their underlying mechanisms.

Q1: Clarification of the Methodological Novelty of C3FL and DIDP

A1: In existing latent-token and learnable-query approaches, learnable tokens typically interact with feature maps to aggregate contextual information or form compact task-specific representations, with their interactions primarily serving feature extraction or prediction. In contrast, C3FL is specifically designed to establish cross-view interactions under partial co-visibility. It first employs two separate sets of learnable prompts to interact with ground and aerial features independently, and then jointly updates them through cross-view self-attention. The updated prompts are further mapped back to both spatial feature maps via similarity-based aggregation. This design enables prompts to capture complementary patterns across views and propagate cross-view context to spatial features, guiding feature interaction toward co-visible regions and reducing interference from non-overlapping areas. Therefore, C3FL differs from existing prompt-based approaches in its explicit cross-view interaction and spatial feature refinement tailored to co-visibility.

Q2: Insufficient Direct Validation of Co-visibility Awareness and Dynamic Inlier Detection

A2: To address this concern, we have added a quantitative analysis of false correspondences in non-co-visible regions. Specifically, compared with directly using the DINOv2 backbone features without C3FL, our method reduces the false-match rate in non-co-visible regions from 99.94% to 27.96% (same view) and from 99.94% to 87.80% (cross view). Besides, to investigate whether the benefit of DIDP merely comes from replacing hand-crafted RANSAC with a learning-based outlier rejection method, we have conducted an additional comparison using a state-of-the-art learning-based outlier rejection method (PointCN:Learning to Find Good Correspondences). Specifically, we replace the original RANSAC-based outlier rejection in the SOTA baseline with PointCN, and also use PointCN in place of DIDP while retaining our framework. As reported in Table 4 in the manuscript(pp.10), although learning-based outlier rejection already improves upon conventional RANSAC, our framework further benefits from more effective correspondence learning. More importantly, DIDP consistently outperforms PointCN when applied within our framework, demonstrating that its gains do not simply come from adopting a learning-based filtering strategy. Instead, the proposed input-conditioned inlier detection mechanism provides additional benefits by adapting to the correspondence patterns of the current input.

Q3: Methodological Novelty of C3FL over Existing Attention Mechanisms

A3: Please refer to A1.

Q4: Direct Quantitative Validation of C3FL's Co-visibility Modeling

A4: Please refer to A2.

Q5: Methodological Novelty of DIDP over Existing Learned Correspondence Filtering Methods

A5: Please refer to A2.To further isolate the contribution of inlier detection from pose regression, we have added three controlled variants using the same input correspondences: 1.Model(1) : We use the product of the predicted inlier probability and matching confidence, pkckβp_k c_k^{\beta}, to score each correspondence. A fixed threshold is applied to select inliers, followed by a conventional geometric pose solver. 2.Model(2): All input correspondences are treated equally. Their feature representations are directly average-pooled and fed into the same MLP pose regression heads. 3,Model(3): RANSAC is applied to select inliers, whose feature representations are then average-pooled and fed into the same MLP pose regression heads. As shown in Table 5 in the manuscript(pp.10), all three variants perform substantially worse than the full DIDP. In particular, comparing variants (2) and (3) with our full model demonstrates that neither direct feature aggregation nor fixed-rule RANSAC selection can achieve the same performance as our learned inlier detection mechanism, even with the same MLP regression structure. These results highlight the importance of adaptively identifying reliable correspondences for accurate pose regression. We added a visualization in Fig. 7 (pp. 15) to show that dynamically generated detectors respond selectively to different correspondence patterns, demonstrating their ability to capture diverse inlier patterns and reject outliers.

Reviewer_Yov9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents a model called CoVisFormer for estimating the 3-DoF pose between the ground image and the aerial image. In CoVisFormer, a co-visibility-aware cross-view contextual feature learning (C3FL) module is proposed to focus matching on overlapping regions, and a DIDP module is proposed to adaptively reject outliers. Experiments on the KITTI and VIGOR datasets demonstrate the proposed method's state-of-the-art performance.

Critical strengths

The proposed method is novel. Estimating the 3-DoF pose between ground and aerial images is a challenging task, while experiments on two datasets demonstrate strong improvements from each of the proposed modules.

Critical weaknesses
  • The motivation of the method design remains largely unclear. It is generally easy to understand that full attention can lead to wrong matches from irrelevant areas, while it is hard to understand what exactly motivates introducing “prompts” to better enable cross-view interaction and filter out the irrelevant areas. Similarly, it is also hard to follow what motivates the “dynamic” and which part is designed to be dynamic compared to the standard RANSAC.
  • There needs to be more explanation of the prompt-based attention, and more analysis on how it works. It is unclear what the prompts are, do they have specific text information or are they learnable parameters? Moreover, what information do prompts bring to the cross-view matching process? If it is text information, how can it effectively interact with the image modality? Is there any information leakage in the design of the prompts? It is also important to help readers understand why the prompt-based attention work, for example, what makes it effective at removing the wrong matches.
  • Also, in the DIDP module, what prototypes are, more details about them, and how they work were not explained in the paper. In addition, what makes the DIDP adaptively identifies reliable inliers (as claimed in the introduction) is not explained.
  • Introducing more modules into the model usually reduces the efficiency, while the relevant analysis is missing.
Questions for authors

None

1 comment
AuthorsOpenReview ↗

Clarification of Method Motivation, Design, and Efficiency in Response to Reviewer Yov9

We sincerely thank the reviewer for the insightful and constructive comments, which have helped us clarify the motivation, design rationale, working mechanisms, and efficiency of the proposed framework.

Q1: Motivation for Prompt-based Interaction and Dynamic Design

A1: The two designs address different challenges in cross-view pose estimation. For C3FL, large viewpoint differences result in substantial non-overlapping regions, making unrestricted full attention prone to introducing irrelevant information. Learnable prompts guide cross-view interaction toward specific local patterns within co-visible regions, thereby reducing interference from irrelevant regions. For DIDP, correspondence distributions and outlier patterns vary across image pairs. Unlike RANSAC, which relies on random sampling and geometric consensus, DIDP generates input-conditioned detectors from current correspondence features to adaptively estimate correspondence reliability. As shown in Figure 5 in the manuscript (pp.10) , the visualizations illustrate how C3FL focuses on relevant cross-view regions. And how DIDP identifies reliable correspondences is clarified in Figure 7 in the manuscript (pp.15) .

Q2: Clarification of the Prompt-based Attention Mechanism

A2: We clarify that the prompts are randomly initialized learnable vectors, without any textual information or external semantic priors. Each prompt independently interacts with the ground and aerial features to capture consistent patterns across the image pair. By capturing complementary cross-view patterns, the prompts facilitate more comprehensive modeling of co-visible regions and help suppress mismatches from irrelevant areas.

Q3: Clarification of Prototype Design and Adaptive Inlier Identification

A3: We clarify that the prototypes are randomly initialized learnable vectors. Through attention-based interactions with the input correspondence features, different prototypes capture distinct patterns in the current set of matches and generate input-conditioned inlier detectors. These detectors are thus tailored to the correspondence patterns of each input pair, enabling DIDP to adaptively identify reliable inliers across different scenes. As shown in Figure 7 in the manuscript (pp.15), the visualizations further illustrate the ability of DIDP to identify reliable correspondences.

Q4: Analysis of Computational Efficiency

A4: We thank the reviewer for this suggestion. We have added an efficiency comparison between CoVisFormer and FG2 under the same hardware and input settings. Specifically, CoVisFormer introduces a modest computational overhead, increasing the parameter count from 25.3M to 26.4M, FLOPs from 38.6G to 41.2G, and inference time from 31.5 ms to 35.0 ms. These results suggest that the proposed modules introduce limited additional computational cost while improving localization performance.

Reviewer_ckrHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoVisFormer for ground–aerial cross-view 3-DoF localization, combining a co-visibility-aware cross-view contextual feature learning module (C3FL) and a dynamic inlier detection-based pose regression (DIDP) with depth-lifted BEV and local correspondence matching. The paper reports competitive localization results on VIGOR and KITTI under multiple evaluation settings, with particularly favorable performance in terms of mean localization error.

Critical strengths
  • The overall pipeline is clear and the design is reasonable. Both the cross-view context interaction and the correspondence-set-based reliability estimation target practical difficulties in this task; modeling reliability using correspondence coordinates together with matching confidence also has sound geometric intuition.

  • The reported localization performance is appealing. The method improves mean localization error on both benchmarks under multiple settings, with notable gains in some settings.

  • Basic component validation is provided. The module on/off experiments in Table 3 show that C3FL and DIDP, as integrated components, both yield performance gains.

Critical weaknesses

W1. The co-visibility-aware mechanism lacks direct validation

C3FL does enable information exchange between the two views, but cross-view interaction itself is not equivalent to co-visibility awareness. A reduction in final localization error is also insufficient to prove that mismatches in non-co-visible regions are effectively suppressed. The paper positions the suppression of non-co-visible interference as a core motivation, yet the existing experiments mainly report final pose errors. Table 5 shows that the tested prompt-based attention configuration outperforms full attention, but it cannot distinguish whether this advantage comes from more effective co-visibility modeling or from more general feature enhancement, information compression, or regularization. The qualitative example in Figure 1 is also insufficient to establish systematic mechanistic evidence. The gap is therefore not whether the module is effective, but whether the current evidence supports its specific interpretation as ``co-visibility awareness.”

W2. The DIDP ablation does not isolate the contribution of dynamic inlier detection

The ablation in Table 3 is valuable, but when DIDP is removed, the authors replace the entire backend, including inlier scoring, feature aggregation, and MLP pose regression, with Procrustes and RANSAC. This experiment therefore supports the effectiveness of DIDP as a whole, but it cannot clearly attribute the performance gain to dynamic inlier detection. The current results still cannot sufficiently distinguish among three explanations: (i) the reliability weights of correspondences are more accurate; (ii) the learned pose regression backend is better suited to the current task or data distribution; and (iii) the added supervision or joint training improves the front-end correspondence features. These explanations correspond to different technical contributions and require finer-grained controlled experiments to separate.

Questions for authors
  • Can the authors fix the correspondence set and the regression structure, and compare (a) using only matching confidence versus adding predicted inlier probabilities, and (b) an input-conditioned detector versus a fixed-parameter scoring head?

  • Can the authors report a variant that uses the predicted weights for weighted geometric solving, so as to separate the respective benefits of reliability estimation and MLP regression?

  • Can the authors clarify, for each ablation variant, the training losses used, the training stages, and whether the front end is jointly updated, so that these comparisons are interpretable?

  • Since the key ablations are limited to KITTI-Ori180 and report only mean translation error, can the authors report heading error and validate the key designs on VIGOR to clarify the completeness and applicability of the component gains?

1 comment
AuthorsOpenReview ↗

Mechanism Validation and Fine-Grained Ablation Studies in Response to Reviewer ckrH

We sincerely thank Reviewer ckrH for the insightful and constructive suggestions. We have carefully considered each concern and will revise the manuscript and conduct additional experiments accordingly to address the raised issues.

Q1:The co-visibility-aware mechanism lacks direct validation

A1:We thank the reviewer for pointing out the need for more direct evidence of co-visibility awareness. To address this concern, we have added a quantitative analysis of false correspondences in non-co-visible regions. Specifically, compared with directly using the DINOv2 backbone features without C3FL, our method reduces the false-match rate in non-co-visible regions from 99.94% to 27.96% (same view) and from 99.94% to 87.80% (cross view). We have also added a visualization in Figure 4 in the manuscript(pp.10) to further illustrate this effect. Specifically, the cross-view feature responses extracted directly by the DINOv2 backbone exhibit strong activations in non-co-visible regions. In contrast, after applying our C3FL module, the responses become predominantly concentrated in co-visible regions. These results provide direct evidence that C3FL effectively suppresses interference from non-co-visible regions, rather than merely improving the final localization performance.

Q2: The DIDP ablation does not isolate the contribution of dynamic inlier detection

A2: To further isolate the contribution of inlier detection from pose regression, we have added three controlled variants using the same input correspondences: 1.Model(1) : We use the product of the predicted inlier probability and matching confidence, pkckβp_k c_k^{\beta}, to score each correspondence. A fixed threshold is applied to select inliers, followed by a conventional geometric pose solver. 2.Model(2): All input correspondences are treated equally. Their feature representations are directly average-pooled and fed into the same MLP pose regression heads. 3,Model(3): RANSAC is applied to select inliers, whose feature representations are then average-pooled and fed into the same MLP pose regression heads. As shown in Table 5 in the manuscript(pp.10), all three variants perform substantially worse than the full DIDP. In particular, comparing variants (2) and (3) with our full model demonstrates that neither direct feature aggregation nor fixed-rule RANSAC selection can achieve the same performance as our learned inlier detection mechanism, even with the same MLP regression structure. These results highlight the importance of adaptively identifying reliable correspondences for accurate pose regression. We added a visualization in Figure 7 (pp. 15) to show that dynamically generated detectors respond selectively to different correspondence patterns, demonstrating their ability to capture diverse inlier patterns and reject outliers.

Q3: How about (a) using only matching confidence and (b) a fixed-parameter scoring head?

A3: Both alternatives yield inferior results to the full DIDP. Specifically, removing the predicted inlier probabilities increases the mean translation errors by 0.35 m and 0.69 m in the same-area and cross-area splits, respectively. Replacing the prototype-based detector generation with a shared MLP scoring head leads to increases of 0.25 m and 0.44 m, respectively. These results demonstrate the complementary contribution of predicted inlier probabilities and the advantage of input-conditioned detector generation in adapting to diverse correspondence patterns.

Q4: separate the respective benefits of reliability estimation and MLP regression

A4: Please refer to A2.

Q5: How to train ablation variants

A5:All variants keep the image feature extractor frozen. [A] Fixed features + Procrustes-RANSAC pose;[B] Train DIDP only (soft inlier,trans,orient loss); [C] Train C3FL only (matching loss) + Procrustes-RANSAC; [D] Train C3FL (matching), then jointly C3FL+DIDP (matching,soft inlier,trans,orient)

Q6: Experiments on VIGOR

A6: We have extended the component ablations to VIGOR and additionally report the mean heading error for a more comprehensive evaluation of pose estimation. Unlike KITTI-Ori180, which features forward-facing vehicle images with a limited field of view and substantial orientation uncertainty, VIGOR introduces a more challenging cross-view localization setting with arbitrary query locations, and multiple partially overlapping reference images. Moreover, its dense urban scenes involve substantial occlusions and shadows, while the cross-area protocol further evaluates generalization to unseen cities. The consistent gains brought by C3FL and DIDP across both evaluation settings demonstrate that CoVisFormer can effectively learn cross-view contextual representations and identify reliable correspondences despite varying co-visibility and challenging matching conditions. These results further validate the complementary roles of the two modules and their ability to generalize across diverse urban environments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.