acceptodds
Under review as a conference paper at ICLR 2027

How to Reduce Localization Ambiguity? Geometry-Semantic Constrained BEV Representation Learning for Satellite-Ground Localization

Abstract

Satellite-ground localization estimates the planar position and yaw orientation of a ground camera within a geo-referenced satellite image of its surroundings. The predominant approach to this task maps features from ground-view images and satellite references into a shared bird's-eye-view (BEV) space and then establishes spatial correspondences between them. Although effective, this approach still faces ambiguity in feature placement and descriptor matching. When mapping ground-view features into BEV space, insufficient depth constraints allow the same feature to be assigned to different distances along the viewing direction, creating geometric ambiguity in BEV feature placement. Meanwhile, similar appearances at different locations create descriptor matching ambiguity, and existing descriptor learning lacks explicit semantic supervision to distinguish these locations. To reduce these ambiguities, we propose GeoSem-BEV, a geometry-semantic constrained BEV representation learning method. First, we geometrically constrain ground-view BEV feature placement through radial depth supervision for distance assignment and vertical height supervision for height aggregation. Then, shared explicit semantic supervision further promotes consistent semantic predictions across views and helps distinguish different locations with similar semantics. These constraints jointly improve feature placement and descriptor discriminability, thus improving the satellite-ground localization performance of state-of-the-art models by a large margin. Qualitative and quantitative results demonstrate the effectiveness of GeoSem-BEV in enhancing these models. On VIGOR with unknown orientation, GeoSem-BEV reduces mean orientation error relative to the corresponding state-of-the-art method by 37.2% and 38.1% in the cross-area and same-area settings, respectively. The corresponding errors were reduced by 10.8% and 15.6% on DReSS-D. On KITTI-CVL, GeoSem-BEV reduces the same-area mean orientation error by 26.8% relative to the corresponding baseline under orientation noise.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Satellite-Ground Localization, BEV Representation Learning, Cross-View Matching, Geometric and Semantic Supervision, Metric Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KmVUOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies ambiguity in BEV-based satellite-ground localization and proposes GeoSem-BEV, which introduces radial-depth and vertical-height supervision for BEV feature construction, together with explicit semantic supervision and same-class hard-negative mining for descriptor learning. The method is applied to FG², DenseFlow, and ViewBridge, with strong gains on VIGOR and mixed improvements on DReSS-D and KITTI-CVL.

I find the problem relevant and the radial-versus-vertical ambiguity formulation interesting. However, I have concerns about novelty attribution, the reliance on strong external teachers, and the consistency of the claimed gains.

Critical strengths

The distinction between radial assignment along a viewing ray and vertical feature aggregation is well motivated. FG² already constructs a 3D point cloud and learns feature selection along the height dimension, but it does not explicitly supervise which radial location corresponds to the actual scene surface.

The ablation showing that semantic supervision alone hurts performance, while geometry followed by semantic supervision improves it, is interesting and supports the argument that semantic descriptor learning benefits from better geometric placement.

Applying the method to FG², DenseFlow, and ViewBridge is useful and suggests that the proposed supervision is not tied to a single architecture.

Critical weaknesses

1. The geometric novelty appears somewhat incremental.

The progression from FG² to GeoSem-BEV is conceptually similar to the progression from implicit BEV lifting to BEVDepth-style explicit depth supervision. BEVDepth already established that explicitly supervising depth can substantially improve BEV construction. The main novelty here therefore seems to be the identification of radial ambiguity specifically in satellite-ground localization, rather than explicit depth supervision itself.

2. The contribution of the proposed formulation is confounded by strong external teachers.

GeoSem-BEV uses DA²/SphereViT for radial-depth supervision, Depth Anything V2 for satellite-side height targets, and SAM3 for semantic pseudo-labels. DA² itself reports strong zero-shot panoramic depth performance, and SAM3 is a powerful concept segmentation model. It is therefore unclear how much of the gain comes from the proposed learning mechanism versus the quality of these external priors.

3. A matched-teacher or direct-use baseline is missing.

Loc² already lifts image-plane correspondences using monocular depth at inference. A natural control would be Loc² using the same DA² depth, optionally together with SAM3-based correspondence filtering or reweighting. Without such a comparison, the current experiments do not isolate the benefit of training-time distillation from simply having access to stronger depth and semantic information.

4. The claimed framework-agnostic improvement is not fully consistent on stronger baselines.

On DReSS-D, GS-ViewBridge improves mean orientation error substantially but degrades same-area localization from 2.940/1.490 m to 3.186/1.707 m in mean/median error. Similar translation-orientation trade-offs appear on KITTI-CVL. This weakens the claim that the proposed geometry consistently improves BEV feature placement.

5. The ablation does not sufficiently isolate the central radial-depth contribution.

Since the paper argues that radial ambiguity is distinct from vertical-height ambiguity, a direct radial-only versus height-only comparison is important. The current geometry ablation combines several supervision terms, making it difficult to determine whether radial supervision is truly the key source of improvement.

Questions for authors

Please refer to the Weaknesses section.

9 comments
AuthorsOpenReview ↗

Response to W1: Specific contribution of geometric modeling

Thank you for your comment. The original manuscript did not clearly explain the novelty of GeoSem-BEV or sufficiently describe its technical distinctions from existing depth-supervised BEV methods. In the revised Abstract, Introduction, Related Work (Sections 2.2 and 2.3), and Method (Sections 3.2 and 3.3), we have added detailed comparisons and explanations of the mechanisms. We also independently ablate the architecture and supervision to verify their respective contributions.

First, regarding geometric modeling, autonomous-driving multi-view perception methods such as BEVDepth use LiDAR-projected ground truth for 3D object detection under known and relatively stable vehicle-camera geometry. Satellite-ground localization faces extreme viewpoint differences between street-level panoramas and overhead satellite imagery. Existing cross-view BEV methods either rely on a 2D ground-plane assumption or impose height constraints only along the vertical direction, leaving the same ground-view feature assignable to ambiguous 3D locations along its viewing ray. We introduce Geometry-Guided BEV Construction (GGBC), which expands geometric constraints to 3D space by simultaneously constraining radial distance and vertical height on a shared metric 3D query grid. During training, it learns the 3D spatial assignment of ground-image features in BEV and their aggregation weights at different heights. The accompanying radial-depth and height supervision guide distance assignment and height aggregation over the same 3D candidate locations and can further improve matching performance in some settings.

Second, regarding semantic representation learning, existing BEV matching methods mainly rely on appearance and pose supervision. Their descriptors implicitly mix semantic cues from different categories with appearance and high-level structural information, lacking an explicit semantic learning mechanism to control matching ambiguity caused by similar appearances. We use SAM3-generated targets for buildings, vegetation, and paved surfaces as supervision signals. A shared semantic head extracts explicit category-level semantic predictions from BEV descriptors, with direct category supervision (CE) and cross-view semantic consistency constraints (JS). These are complemented by same-class hard-negative learning to align semantics across views while distinguishing highly confusing candidates that belong to the same class but occupy different locations.

Additional experiments verify these designs separately. Table 6 in Appendix A.5 shows that, on VIGOR cross-area with unknown orientation, GGBC trained using only the base matching and pose objectives outperforms uniform aggregation and our reproduced FG²†, demonstrating the effectiveness of the 3D geometric construction itself. The fixed-architecture radial-depth/height ablations on VIGOR cross-area with unknown orientation in Table 2 verify the incremental and complementary effects of the two accompanying supervision terms. The fixed-architecture controls on KITTI-CVL same-area/test1 in the same table and Table 11 in Appendix A.9 separately verify the localization improvements from combined CE/JS supervision and same-class HN.

AuthorsOpenReview ↗

Response to W2: Teacher quality and the learning mechanism

Thank you for your comment. GGBC learns the spatial assignment of ground-image features in BEV through radial-distance prediction and height aggregation.

To examine the influence of external teachers, Table 7 in Appendix A.5 keeps the student architecture and other supervision fixed and replaces only the radial-depth teacher. Together with the control without radial-depth supervision in Table 2, the results show that radial targets from both DA²/SphereViT and UniK3D lower all four localization errors on VIGOR cross-area with unknown orientation. This indicates that the construction and learning mechanism can use depth labels from different teachers. Table 9 in Appendix A.7 further replaces Depth Anything V2 Metric radial targets with LiDAR distances. Both groups compute radial losses only at their commonly valid sparse locations, retaining the same architecture and other training conditions. The two supervision sources yield similar localization performance, supporting the feasibility of using teacher predictions as training targets under this common-coverage condition.

The contribution of GGBC itself is examined through experiments with the added supervision disabled. Table 6 in Appendix A.5 shows that GGBC trained using only the base matching and pose objectives has lower errors on all four metrics than uniform aggregation and our reproduced FG²†. Table 12 in Appendix A.10 also shows that GS-ViewBridge without added supervision still lowers mean translation and orientation errors relative to the original ViewBridge. Table 8 in Appendix A.6 further shows that, on KITTI same-area, the base-objective configuration achieves the lowest radial MAE and translation errors, indicating that GGBC can learn effective geometric predictions and spatial assignments from the localization task itself.

For SAM3, our core design uses its strong concept-segmentation capability to automatically add pixel-level masks for categories such as buildings, vegetation, and paved surfaces to existing datasets, reducing manual annotation costs. We convert these masks with explicit category information into semantic supervision signals. Table 2 keeps the teacher sources, student architecture, and geometric supervision fixed while independently switching CE, JS, and same-class HN. Adding CE or JS alone improves all four errors. Adding CE+JS lowers mean translation and orientation errors, while adding same-class HN alone also improves all four errors. With the same category-label source, CE guides descriptors to learn category information, JS constrains cross-view semantic consistency, and same-class HN trains the model to distinguish candidates of the same class at different locations. These independent switches demonstrate the incremental effects of the learning objectives. Our method does not require running the additional SAM3 or depth-teacher models at inference.

AuthorsOpenReview ↗

Response to W3: Direct use of depth and SAM3 in Loc²

Thank you for your suggestion. Table 10 in Appendix A.8 adds a comparison of Loc² directly using depth and SAM3. In this KITTI experiment, both Loc² and GS-FG² use depth models from the Depth Anything V2 Metric VKITTI family. Both Loc² variants retain the same official weights, depth model, sampling of 1,024 correspondence pairs, and original pose-solving pipeline. The only addition is SAM3 class-conflict filtering. A candidate's matching score is set to zero when both endpoint classes are valid, both confidences are at least 0.5, and the classes differ.

On KITTI-CVL same-area/test1 with ±10° orientation noise and no RANSAC, all four errors increase after adding the filtering. We have not observed a localization improvement from introducing SAM3 in this experimental setting. GS-FG² has lower errors on all four metrics than both Loc² variants. We will continue exploring ways to use semantic information in Loc² and conduct further comparisons.

Reviewer_ifEBOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GeoSem-BEV, a geometry-semantic constrained representation learning approach for satellite-ground localization. The authors identify two sources of ambiguity in BEV-based feature matching: geometric ambiguity caused by uncertain feature placement along viewing rays and descriptor ambiguity caused by visually similar regions at different locations. To address these issues, the method introduces radial-depth and height supervision for BEV feature construction, together with shared semantic supervision, cross-view consistency, and same-class hard-negative learning for descriptor matching. The proposed approach is integrated into FG², ViewBridge, and DenseFlow and evaluated on VIGOR, DReSS-D, and KITTI-CVL, showing notable improvements, particularly in unknown-orientation localization.

Critical strengths
  1. The paper addresses a meaningful problem in satellite-ground localization. The distinction between radial-depth ambiguity and height aggregation provides a clear geometric motivation, while the discussion of visually similar regions highlights another practical difficulty in cross-view matching. These observations make the proposed supervision strategy reasonably well motivated.

  2. The proposed approach is relatively flexible and practical. Rather than introducing an entirely new localization pipeline, the authors incorporate geometric and semantic constraints into several existing BEV matching frameworks. Using pretrained models to generate supervision targets during training also avoids requiring external depth or semantic models at inference time.

  3. The experimental evaluation provides encouraging evidence for the effectiveness of the approach. Improvements are demonstrated across multiple localization frameworks and datasets, with particularly strong results on VIGOR under unknown orientation. The geometry-only ablation and additional correspondence-level evaluation also provide useful evidence beyond the final localization metrics.

Critical weaknesses
  1. Although the two sources of localization ambiguity are well motivated, the proposed method mainly extends existing geometry-aware BEV construction and semantic descriptor learning techniques. The current ablations also make it difficult to identify the contribution of each component. In particular, radial and height supervision are not evaluated separately, while the semantic objectives are introduced together with hard-negative learning. Some variants also change the feature aggregation or sampling mechanisms, making it unclear how much of the improvement comes from the proposed supervision itself.

  2. The main FG² comparisons are not fully controlled. Table 1 compares the proposed model against the originally reported FG² results under a different inference protocol, while the reproduced baseline in Table 2 is substantially stronger. This makes the large improvements highlighted in the main results difficult to interpret. Although the controlled ablation still shows meaningful gains, the evaluation does not fully establish the improvement over stronger implementations of the same baseline.

  3. The geometric supervision relies on pseudo-depth and relative-height targets whose consistency is not fully established. Ground-view depth is scaled using an assumed camera height, while satellite-view relative depth is normalized and mapped to fixed height bins. These choices may provide useful supervision, but their reliability across different scene geometries is unclear. The additional correspondence evaluation supports the method to some extent, yet it still depends on learned descriptor matching rather than directly measuring the accuracy of BEV feature placement.

  4. The improvements are not equally consistent across datasets and evaluation metrics. Some localization results deteriorate on DReSS-D and KITTI-CVL, while improvements in mean error are not always reflected in the median results on VIGOR. The paper provides limited analysis of when the proposed constraints help or fail, and the effects of training variability and additional computational costs are not sufficiently examined. This weakens the broader claim that the method offers a generally reliable improvement across BEV localization frameworks.

Questions for authors
  1. The current ablations introduce several changes together. It would be helpful to separately evaluate radial-depth supervision, height supervision, semantic CE, JS consistency, and same-class hard-negative learning while keeping the underlying architecture fixed. In particular, Sem-FG² uses uniform height aggregation, and GS-ViewBridge modifies the sampling and gating mechanisms. How much of the improvement remains when these architectural changes are controlled for? A comparison between same-class and standard hard-negative mining would also help clarify the role of semantic information.

  2. Could the authors clarify how the SAM3-generated semantic masks are converted into ground-view BEV targets? In particular, how are occluded regions, uncertain projections, and overlapping semantic classes handled? It is also unclear whether the semantic classes used to select same-class hard negatives come from the fixed teacher labels or the student's predictions. These details would help explain the supervision process and improve reproducibility.

  3. How sensitive is the method to the assumptions used to construct geometric pseudo-targets? In particular, how is satellite relative depth mapped to the height direction, and how is the assumed camera height handled across different datasets? It would be informative to examine alternative depth teachers or scale perturbations, and, where possible, provide direct evaluations of radial-depth or height prediction accuracy.

  4. The FG² results in Table 1 and the reproduced baseline in Table 2 appear to use different evaluation protocols. Could the authors clarify the differences in RANSAC settings, correspondence sampling, and pose estimation? ViewBridge also reports a stronger FG²†-RANSAC reproduction. Understanding the differences between these implementations, ideally through a comparison under matched training and inference settings, would make the reported gains easier to assess.

  5. The results on DReSS-D and KITTI-CVL suggest that the method does not always improve translation and orientation accuracy together. Have the authors examined which types of scenes or matching failures account for these regressions? Additional error-distribution analysis and repeated runs would help assess the consistency of the improvements. It would also be useful to report the added inference overhead and offline teacher preprocessing cost for each adapted framework.

9 comments
AuthorsOpenReview ↗

Response to W1 and Q1:Component ablations with fixed architectures

Thank you for your suggestions. Sections 3.2 and 3.3 of the revised paper clarify the mechanisms of 3D geometric construction and explicit semantic learning. GGBC extends geometric constraints to 3D space by simultaneously constraining radial distance and vertical height on a shared metric 3D query grid. Radial assignment and height aggregation jointly determine the ground-view BEV representation, guided by accompanying radial and height supervision at the same 3D candidates. For semantic learning, existing approaches implicitly mix category-specific semantic cues with appearance and higher-level structural information in their descriptors, without explicit semantic learning. We use SAM3-generated targets for buildings, vegetation, and paved surfaces. A shared semantic head extracts category-level predictions from ground and satellite BEV descriptors, with direct category supervision (CE) and cross-view semantic consistency (JS). Same-class hard-negative learning distinguishes candidates of the same category at different locations, improving spatial discrimination.

To separate architecture and supervision, Appendix A.5, Table 6 compares the original FG² builder, uniform aggregation, and GGBC with all added supervision disabled, retaining only the base matching and pose objectives. GGBC lowers all four pose errors relative to both alternatives, showing that the construction module itself can learn effective spatial assignments through the task objectives.

For radial and height supervision, the VIGOR experiments in Table 2 fix GGBC, CE/JS, HN, and pose solving, switching only the two geometric supervision terms. Adding radial supervision to height supervision lowers all four errors. Adding height supervision to radial supervision lowers the translation mean and median and the orientation mean. These controls examine the incremental and complementary effects of the two terms.

For semantic supervision and HN, the KITTI experiments in Table 2 fix GGBC and geometric supervision and independently enable CE, JS, and same-class HN. CE or JS alone improves all four errors. CE+JS lowers both mean translation and orientation errors relative to geometric supervision alone, with lower orientation errors of 0.688° / 0.471° (mean/median). Enabling same-class HN with CE/JS disabled also improves all four errors. Appendix A.9, Table 11 further compares standard and same-class HN. Category-based candidate selection lowers the translation mean and median and the orientation mean, with a slight increase in orientation median, supporting its contribution to average localization accuracy.

For GS-ViewBridge's gating and sampling and GS-DenseFlow's construction and weighting, Appendix A.10, Table 12 retains each complete architecture and switches only added supervision. Supervision lowers GS-ViewBridge's orientation mean, while disabling it gives lower translation errors and orientation median. For GS-DenseFlow, supervision lowers all four errors. Thus, its effect depends on the underlying framework. Section 4.5 discusses this difference in relation to ViewBridge's contextual matching: the base matching and pose objectives may already guide GGBC effectively, while additional supervision may restrict how spatial weights and descriptors adapt to matching. We will further investigate these mechanisms and the interaction between base objectives and auxiliary supervision, improve GGBC and its supervision, and report the findings in future work.

AuthorsOpenReview ↗

Response to W2 and Q4 : FG² baselines and pose estimation protocols

Thank you for your suggestions. Table 1 of the original manuscript used published FG² results, while Table 2 used our reproduction. Revised Table 1 now includes both the published FG² and our reproduced FG²†. Table 2 uses the same reproduced baseline results, and GS-FG²'s relative improvements are consistently calculated against this baseline under matched settings.

For the reproduction, we added ImageNet normalization for frozen DINOv2 after image resizing and tensor conversion, consistently across training, validation, and testing. Our FG²† and GS-FG² use the same preprocessing, training settings, and seed-controlled rotation augmentation. Appendix B.5 documents these configurations.

Correspondence sampling and pose solving are also matched. Training and direct Weighted Procrustes use 1,024 correspondences. RANSAC uses 8,192 candidates, 20 matching iterations, 100 hypotheses, a three-point minimal set, and four refinement steps, with inlier and soft-inlier thresholds of 2.5 m and 5 m. Table 1 and Section 4.3 show that GS-FG² reduces mean translation and orientation errors relative to our reproduction under the same RANSAC protocol. Both means also decrease with direct Weighted Procrustes, showing that the improvement is retained with both solvers.

For the external FG² reproduction reported by ViewBridge, the code for that reproduction has not been released, so we cannot currently establish its specific differences from our implementation. Appendix B.5 specifies our preprocessing, training, and solver settings to support verification and reproduction.

AuthorsOpenReview ↗

Response to W3 and Q3:Geometric pseudo-target construction and sensitivity

Thank you for your suggestions. Appendix B.3 clarifies the construction and use of both geometric targets. On the satellite side, Depth Anything V2 predicts relative depth. Min-max normalization maps it to continuous height-bin indices from 0 to 10, followed by an 11-bin Gaussian probability target with a standard deviation of 0.75 bins. Ground-truth poses transform these targets into ground BEV coordinates, corresponding to the same height candidates used by GGBC and guiding vertical aggregation through relative height relationships. In GS-FG², the height loss acts directly on feature contribution weights generated by radial likelihoods, so both supervision terms guide spatial assignment at the 3D candidates.

Ground-view radial targets are handled separately by dataset. VIGOR and DReSS-D use SphereViT from DA² to predict relative radial depth. Metric scaling uses an assumed camera height of 2.5 m and the 95th percentile of vertical distances associated with downward rays. KITTI uses Depth Anything V2 Metric VKITTI ViT-S to predict metric axial depth, which is converted to radial distance using the camera intrinsics.

To examine sensitivity to teacher-generated geometric pseudo-targets, Appendix A.5, Table 7 replaces the depth teacher while fixing the student architecture, other supervision, and RANSAC protocol. Together with the control without radial supervision in Table 2, these results show that radial targets from both SphereViT and UniK3D lower all four pose errors on VIGOR cross-area with unknown orientation. Thus, approximate radial targets from different teachers provide effective training signals in this setting. Appendix A.7, Table 9 further replaces teacher radial targets with LiDAR distances on KITTI. Both groups compute radial losses only at sparse locations valid for both sources, retaining the same architecture and other training conditions. Their similar localization performance shows that both teacher-generated pseudo-targets and dataset-provided ground-truth depth can serve as GGBC training targets.

To measure GGBC's radial prediction accuracy, Appendix A.6, Table 8 evaluates radial MAE against calibrated LiDAR distances and reports localization results from the same models. Training with only the base matching and pose objectives gives the lowest radial MAE and translation errors, showing that GGBC can learn accurate radial predictions and effective BEV representations from localization objectives in some simpler settings. Joint radial and height supervision gives lower radial MAE than either term alone, demonstrating their complementary effects on radial prediction. Section 4.5 discusses this observation: when base objectives already learn effective spatial assignments, additional constraints from fitting approximate teacher targets may affect localization performance.

Reviewer_SwtnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose GeoSem-BEV, which combines geometric supervision and semantic descriptor learning to reduce localization ambiguity in satellite-ground matching. The proposed method achieves substantial improvements on VIGOR and is evaluated across multiple BEV localization frameworks.

Critical strengths

The improvements on VIGOR are substantial, and the evaluation covers multiple frameworks and datasets. The additional correspondence analysis provides useful evidence of improved matching accuracy.

Critical weaknesses
  1. Limited methodological novelty. Depth-supervised BEV construction, semantic cross-view matching, and hard-negative mining are all well-established techniques. The proposed method largely combines these existing components without introducing a clearly distinct learning mechanism. Its technical contribution beyond straightforward integration remains insufficiently justified.

  2. The performance gains cannot be reliably attributed to the proposed supervision. Appendix B.6 shows that GS-ViewBridge and GS-DenseFlow introduce architectural and feature-weighting modifications beyond the additional losses. Without architecture-matched controls, the actual contribution of the proposed supervision is unclear. Furthermore, Table 2 does not disentangle semantic supervision from hard-negative mining, leaving the claimed semantic contribution unverified.

  3. The reported improvements are highly dependent on baseline selection. On VIGOR cross-area, the published FG² baseline reports 10.020 m / 31.410°, whereas the reproduced FG2 achieves 5.751 m / 18.908°. Consequently, GS-FG2's improvements shrink from 53.5% / 55.9% to 18.9% / 26.8% when compared against the reproduced baseline. This substantial discrepancy requires a clearer explanation and strictly controlled comparisons.

  4. The claimed generalizability is not consistently supported. Despite strong VIGOR results, GS-ViewBridge increases mean translation error by 8.37% on DReSS-D same-area and 17.06% on KITTI-CVL same-area, while increasing mean orientation error by 15.40% on KITTI-CVL cross-area. These degradations challenge the claim that the proposed constraints consistently reduce localization ambiguity across different settings.

Questions for authors

Please see the weaknesses. My final score depends on the author's rebuttal.

6 comments
AuthorsOpenReview ↗

Response to W1: Methodological novelty

Thank you for your comment. The original manuscript did not clearly explain the novelty of GeoSem-BEV. In the revised Abstract, Introduction, Related Work (Sections 2.2 and 2.3), and Method (Sections 3.2 and 3.3), we have added detailed comparisons and explanations of the mechanisms to clarify the technical novelty and significance of this work. We also independently ablate the architecture and supervision to verify their respective contributions.

First, regarding geometric modeling, existing cross-view BEV methods either rely on a 2D ground-plane assumption or impose height constraints only along the vertical direction. The same ground-view feature can therefore still be assigned to ambiguous 3D locations along its viewing ray. We change how existing methods construct BEV features from ground images by introducing Geometry-Guided BEV Construction (GGBC). GGBC expands geometric constraints to 3D space by simultaneously constraining radial distance and vertical height on a shared metric 3D query grid. During training, it learns the 3D spatial assignment of ground-image features in BEV and their aggregation weights at different heights, constructing BEV representations useful for cross-view matching. The accompanying radial-depth and height supervision guide distance assignment and height aggregation over the same 3D candidate locations and can further improve matching performance in some settings.

Second, regarding semantic representation learning, existing BEV matching methods mainly rely on appearance and pose supervision. Their descriptors implicitly mix semantic cues from different categories with appearance and high-level structural information, lacking an explicit semantic learning mechanism to control matching ambiguity caused by similar appearances. We design an explicit semantic learning mechanism to improve descriptor discriminability. Using SAM3-generated targets for buildings, vegetation, and paved surfaces, a shared semantic head extracts explicit category-level semantic predictions from ground- and satellite-view BEV descriptors. Direct category supervision (CE) and cross-view semantic consistency constraints (JS) explicitly guide the semantic content encoded by these descriptors. Building on this, unlike general hard-negative mining based solely on feature similarity or inter-class comparison, our same-class hard-negative learning specifically selects and penalizes highly confusing, spatially incorrect candidates within the same semantic class. This preserves fine-grained spatial discrimination while aligning semantics across views.

Additional experiments verify these designs separately. Table 6 in Appendix A.5 shows that, on VIGOR cross-area with unknown orientation, GGBC trained using only the base matching and pose objectives outperforms uniform aggregation and our reproduced FG², demonstrating the effectiveness of the 3D geometric construction itself. The fixed-architecture radial-depth/height ablations on VIGOR cross-area with unknown orientation in Table 2 verify the incremental effects of the two accompanying supervision terms. The fixed-architecture controls on KITTI-CVL same-area/test1 in the same table separately verify the localization improvements from combined CE/JS supervision and same-class HN. Table 11 in Appendix A.9 further verifies the gains of same-class over standard hard-negative mining.

AuthorsOpenReview ↗

Response to W2: Contribution of supervision with the architecture fixed

Thank you for pointing this out. The original manuscript overemphasized the added supervision and did not sufficiently explain the contribution of the architecture itself. The changes to BEV construction, feature weighting, and correspondence sampling are all important components of our method. GGBC uses predicted radial-distance probabilities to determine feature contributions at 3D candidate locations, then aggregates them over height to construct the ground-view BEV. Radial-depth supervision guides distance prediction along viewing rays, while height supervision guides feature contributions at different heights. In GS-FG², both supervision terms constrain the contribution weights generated by radial predictions. GS-ViewBridge and GS-DenseFlow additionally predict a height distribution and combine it with radial likelihoods for BEV construction. These explanations have been added to Section 3.2 and Appendix B.6.

To examine the contribution of supervision, Table 12 in Appendix A.10 provides controls that retain the complete architecture, feature weighting, sampling, and pose-solving pipeline while disabling only the added supervision. On VIGOR cross-area with unknown orientation, GS-ViewBridge without added supervision has lower mean translation error and lower median translation and orientation errors, while enabling supervision lowers mean orientation error. This indicates that GGBC can learn spatial assignments useful for matching through the base matching and pose objectives in this setting. For GS-DenseFlow, enabling the added supervision lowers all four errors, demonstrating the contribution of the accompanying supervision in this framework.

For semantic supervision and HN, Table 2 provides independent supervision switches on KITTI-CVL same-area/test1 with the architecture fixed. With geometric supervision retained and HN disabled, adding CE+JS lowers mean translation and orientation errors. With geometric supervision retained and CE and JS disabled, adding same-class HN also lowers these two means. These controls separately examine category and cross-view consistency learning, and spatial discrimination between locations of the same class.

Table 8 in Appendix A.6 further shows that disabling all added supervision while retaining GGBC and the base task objectives yields the lowest translation errors and radial MAE among the configurations in that table. This supports GGBC's ability to learn effective geometry and BEV representations through matching and pose objectives, demonstrating the contribution of the geometric construction itself.

AuthorsOpenReview ↗

Response to W3: FG² comparison under matched configurations

Thank you for pointing this out. Table 1 in the revised manuscript now includes our reproduced FG²†, and the relative improvements of GS-FG² are calculated against this baseline under matched configurations. The VIGOR ablations in Table 2 use the same reproduced baseline results. The sources of the published and reproduced results are distinguished in the table captions.

Regarding the reproduction settings, we have added the ImageNet normalization required by frozen DINOv2 after image resizing and tensor conversion. It is applied consistently during training, validation, and testing so that the inputs satisfy the preprocessing requirements of the pretrained feature extractor. Our reproduced FG²† and GS-FG² share this preprocessing, the same training settings, seed-controlled rotation augmentation, correspondence sampling, and solver parameters. These configurations are listed in Appendix B.5.

Under these shared conditions, Table 1 shows that GS-FG² lowers both mean translation and orientation errors relative to our reproduced FG²†. Section 4.3 also reports a comparison using direct Weighted Procrustes, where both means are lower as well. This supports the effectiveness of the improvement under both pose-solving methods.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.